2026 OpenClaw Frontend QA in Practice:
Playwright Shards, Merged Reports, Flake Retries, and PR Summaries on a Remote Mac

Treat .openclaw/reports/$GIT_SHA/ as the contract root on the remote Mac. Each shard writes raw/shard-$i/blob/. After copy, run merge-reports into merged/html and merged/junit.xml. Store flake_stats.json beside them so OpenClaw templates can read deterministic paths without scanning the tree.

mkdir -p ".openclaw/reports/${GIT_SHA}/raw/shard-${SHARD}/blob"
PLAYWRIGHT_SHARD=${SHARD} PLAYWRIGHT_TOTAL=${TOTAL} npx playwright test --reporter=blob
# fan-in host:
npx playwright merge-reports ".openclaw/reports/${GIT_SHA}/raw/shard-"*"/blob" \
  --reporter=html,junit

Citeable facts: junit gates on GitHub read one file path. Blob reporter is the supported merge input in modern Playwright. WebKit cold starts routinely add tens of seconds on first launch.

Keep browsers and Playwright entirely on the Mac. Let the OpenClaw gateway ingest only structured artifacts: merged junit counts, top failing test titles, shard wall skew, and signed artifact URLs. Do not stream raw traces through the model unless triage needs them; link out like the trace summary playbook. POST summaries with Idempotency-Key: ${GIT_SHA}:${CI_PIPELINE_ID}:pw and embed <!-- openclaw-pw:${GIT_SHA} --> so comment updates stay idempotent.

• Gateway validates JSON schema version pw_summary/v1 before touching Git.
• Secrets never leave the runner; OpenClaw receives presigned GET URLs with short TTL.
• When auth fails mid-run, reuse the observability fields from the token auth article instead of guessing.

Let p be the per-attempt failure probability for a flaky case and ε your residual risk budget after automated retries, still assuming independence as a first pass. Solve p^(r+1) ≤ ε so r_max = min(3, ceil(ln(ε)/ln(p) - 1)). Example: p = 0.2 and ε = 0.01 needs r+1 ≥ 3 because 0.2^2 > 0.01 while 0.2^3 ≤ 0.01, giving two retries after the first attempt. For suites, cap total flake reruns with flake_budget = floor(0.15 * total_tests) so one noisy module cannot consume the whole pool.

Tag known flakes, run them in a slower quarantine project, and let merge reports show both first-attempt and retry columns so reviewers trust the green.

• Fixture timeout on WebKit: raise expect timeouts only after you raise actionTimeout and confirm the Mac is not thermal-throttling parallel shards.
• Navigation timeout to preview URLs: align cold-start probes with your deployment hook smoke cadence so tests start after the edge returns 200.
• Merge step stuck: verify every shard uploaded a complete blob; partial copies produce silent merge hangs—checksum each folder before fan-in.

1. Export GIT_SHA, TOTAL, and per-job SHARD; refuse to start if the tuple is incomplete.
2. Install browsers once per machine image; cache PLAYWRIGHT_BROWSERS_PATH outside the workspace copy.
3. Run tests with blob reporter paths under .openclaw/reports/$GIT_SHA/raw/shard-$SHARD/blob.
4. On the fan-in runner, merge blobs, emit HTML and junit, then compute flake deltas from junit timestamps.
5. Build pr_playwright_summary.md with sections for failures, flakes, shard skew, and artifact links.
6. POST through OpenClaw with the idempotency key and update the existing PR comment marker when the SHA matches.