Skip to main content
Status: plan (review first) · Added: 2026-06-02 · Origin: HR-44/WENO dogfood (could not browser-test locally; root-caused to build-env + seed gaps) This plan defines what it takes to make browser testing and autoheal a trustworthy, ongoing capability — running on real data, against the branch’s own code, on every PR. The surprising headline: ~70% already exists. This is mostly about removing three blockers, not building from scratch.

​
1. What already exists (don’t rebuild)

  • .github/workflows/e2e-full.yml — full Playwright E2E on the DGX self-hosted runner (ARM64), opt-in per-PR via the e2e-full label + scheduled. Default PRs run only test:e2e:smoke (fast).
  • An autoheal loop is already wired into that workflow: on E2E failure it computes regressions (scripts/specs/find-regressions.ts), then runs the encore-test-triage skill per regressed test (≤5), gated by an AUTO_HEAL_ENABLED=1 kill-switch and a healer-skip PR label; scheduled runs file a healer-candidates issue. Emits playwright-report/healer-events.jsonl.
  • The browser-autoheal / heal-e2e skills — the dev-loop counterpart (drive in Playwright, capture console/pageerror/network≥400, classify, fix-to-green; never auto-merge).
  • Playwright bundled chromium works in the runner (npx playwright install chromium).
  • A local Supabase stack (encoreos Docker project) + the ability to serve edge functions with PROLIANT_FIXTURE_TRANSPORT=1 (proven this session).
Conclusion: the pipeline and the healer are real. They are just (a) pointed at the wrong/shared database, (b) starved of seed data for real flows, and (c) opt-in rather than dependable.

​
2. The three blockers (why it isn’t trustworthy yet)

​
G1 — Keystone: the app can’t be pointed at a test database

vite.config.ts resolveStaticPublicEnv() hardcodes VITE_SUPABASE_URL from a Vercel-project-ID → dev2/prod2 map and ignores any VITE_SUPABASE_URL env override. The define: block bakes that URL into the bundle. Consequences:
  • .env.local (local stack) is ignored by every build and the dev server.
  • e2e-full.yml passes E2E_VITE_SUPABASE_URL as a build env — but it’s a no-op; with no VERCEL_PROJECT_ID on the runner, resolveSupabaseTarget() falls through to dev, so CI E2E silently runs against shared dev2. (Confirm by inspecting a built bundle in CI; fix regardless.)
Fix: make resolveStaticPublicEnv() honor an explicit override — when isE2E or a Supabase-branch URL is injected (Vercel preview), use the provided VITE_SUPABASE_URL/key; otherwise the project-ID map. Hard production-safety guard: a non-prod URL must be impossible to bake into a production/Vercel-prod build — extend tests/integration/environment-isolation.test.ts + scripts/utils/check-no-prod-refs to assert both directions. This single change unblocks local-hermetic, the (currently dead) E2E secret, and Supabase branching.

​
G2 — Environment model: shared dev2 vs per-PR isolation

Today E2E hits dev2 — shared, mutable, not the PR’s code. That makes autoheal flaky (concurrent state) and unable to test pre-merge branch changes (the F1/F2/F3 fixes wouldn’t be live there). Options in §3.

​
G3 — Seed coverage: flows can’t reach their terminal screen

config.toml [db.seed] loads only base org/users/sites/schedules; module seeds (04_hr–12) are excluded for “schema/enum drift.” So the Proliant wizard (needs active integration + calendar + run-state) and WENO prescribe (needs org creds) have no data to drive. E2E currently papers over this with page.route mocks — which is exactly why it never caught that the real backend was broken. Fix: a maintained seed:e2e path (resolve the enum drift, or a dedicated E2E seed module) that plants the per-flow fixtures, runnable against whichever target DB §2 chooses. Reuse the integration-test seeders (seedProliantStage2Fixtures, etc.).

​
G4 — Vendor-call determinism (+ F4a auth seed, already fixed)

Proliant/WENO make external calls. The test DB’s served functions must either run with PROLIANT_FIXTURE_TRANSPORT=1 (deterministic, recommended) or hold sandbox creds (needs BAA). Decision needed per env. (Auth-seed NULL-token 500 — F4a — is fixed in supabase/seeds/base/02_users.sql.)
You raised this, and it’s the right primary target. A Supabase preview branch is an ephemeral project created per Git branch, with the branch’s migrations applied + seed run + functions deployed. Wire the Vercel preview deploy to the matching branch and autoheal drives the preview URL. Why it wins for “ongoing”:
  • Hermetic + per-PR — no shared-state flakiness (fixes dev2’s core problem).
  • Tests the branch’s own code — migrations + edge functions + seed are the PR’s (F1/F2/F3 would be live and catchable).
  • Cloud parity — same runtime as prod, no local-stack management.
What it still requires (don’t skip):
  • G1 keystone — the preview build/app must target the branch URL, not dev2. Supabase’s Vercel integration injects the branch URL; vite.config must honor it (today it won’t).
  • Enable branching — Supabase dashboard + GitHub integration + a [branching] block; pick persistent vs ephemeral, and a cost/quota guardrail (branches cost money + spin-up latency ~minutes → per-PR, not per-commit).
  • G3 seed on branch creation; G4 fixture-transport as branch function env.
  • PHI guardrail — preview branches must seed only synthetic data (no prod PHI), enforced.
Tiering (recommended):

​
4. Interactive MCP — recommendation: no, not for the pipeline

You asked whether to use the interactive Playwright/cursor-ide-browser MCP. Recommendation: automated Playwright (headless chromium) + the existing healer is the ongoing pipeline; interactive MCP is a debugging aide only.
  • Automated runner — deterministic, gateable, CI-native, already wired (e2e-full + healer). This is what “ongoing” requires.
  • Interactive MCP — human/agent-in-the-loop, non-deterministic, can’t gate a PR, and has its own env fragility (this session: the MCP needs Chrome at /opt/google/chrome, which isn’t installed; the runner’s bundled chromium-1223 is fine). Keep it for ad-hoc “open this screen / triage this failure,” not for the recurring loop.
  • If you want interactive MCP available for debugging, fix it once: install Chrome on the box or set the MCP launch to --browser chromium to reuse Playwright’s binary.

​
5. Phased worklist (maps blockers → tasks)

Phase 1 — Keystone (unblocks everything).
  1. vite.config.ts: honor injected VITE_SUPABASE_URL/key when isE2E or a preview-branch URL is present; keep the project-ID map as the default. (small change)
  2. Production-safety guard: extend environment-isolation + check-no-prod-refs to prove a non-prod URL can never bake into a prod build, and that prod can never bake a preview URL. (the careful part)
  3. Verify the built bundle’s baked URL in CI (assert it matches the intended target).
Phase 2 — Seed. 4. seed:e2e script reusing the integration seeders (Proliant Stage-2, WENO creds, plus the F4a-fixed users). Resolve the 04_hr–12 enum-drift exclusion or carve a dedicated E2E seed module. Phase 3 — Environment. 5. Enable Supabase branching + Vercel preview wiring; set PROLIANT_FIXTURE_TRANSPORT=1 as branch function env; synthetic-only PHI guard + cost guardrail. Phase 4 — Wire the loop. 6. Render-smoke autoheal on every PR (cheap) against the preview branch; full-flow seeded autoheal on e2e-full/merge-queue; keep the kill-switch. 7. Tie to the AC gate: a green autoheal flow promotes its @verifies AC (spec → UI-verified → verified), mirroring the integration path built this session. Phase 5 — Inner loop (optional). 8. Document E2E_LOCAL=1 local-hermetic mode (G1 local branch) + heal-e2e <core> for fast local iteration.

​
6. Decisions (taken 2026-06-02)

  1. Branching cost/scope → per-PR ephemeral branches (hermetic; accept the $ + spin-up cost), with a per-PR concurrency guardrail.
  2. Vendor calls on previews → fixture-transport (PROLIANT_FIXTURE_TRANSPORT=1, deterministic); live sandbox deferred until BAA/creds exist.
  3. Pilot flows → Proliant payroll-run wizard + WENO prescribe (this session’s specimens).
  4. Autoheal autonomy → kill-switch default off (propose-only PRs); revisit after it proves stable.
  5. Gate strength → render-smoke advisory first; promote to blocking once stable.

​
Progress

  • Phase 1 (G1 keystone) — DONE 2026-06-02. scripts/build/supabase-public-env.ts (chooseSupabaseEnv) is the guarded override; wired into vite.config.ts. Two-direction prod-safety guard unit-tested in tests/unit/build/supabase-public-env.test.ts (6/6) and verified at the build level (E2E build with an injected VITE_SUPABASE_URL bakes the local URL; a prod-context build refuses the override). This also un-breaks the e2e-full.yml E2E_VITE_SUPABASE_URL secret (was a no-op).
  • Phase 2 (local-hermetic seed + first real browser pass) — DONE 2026-06-02. npm run seed:e2e plants test-org-alpha’s Proliant flow (active integration + custom_fields.company_id + calendar + run-state + company mapping + proliant_integration_enabled flag + admin default org). Verified via a live browser pass: dev server pointed at the LOCAL stack via the keystone (E2E_SUPABASE_OVERRIDE=1, confirmed — login hit 127.0.0.1:54321, zero *.supabase.co), admin auto-landed in test-org-alpha, and /hr/payroll/runs/proliant rendered the wizard configReady (Step 1 of 7) — not the “not configured” empty state. Integration page un-gated; WENO card clean; no REAL_BUG. First real browser test of the flow on the PR’s own code, reproducible via seed:e2e + the keystone.
    • Cosmetic/pre-existing nit surfaced: <button>-in-<button> hydration warning in the shared platform WizardShell timeline (StepCircle/ TimelineLayout) — affects all wizards, not HR-44; low priority.
  • Branching confirmed (env-model note): per-PR Supabase preview branches are live (PR #705 has one) but frontend-pinned to dev2 by design (runbook VERCEL_SUPABASE_ENV_ALIGNMENT.md); they validate migrations, they are not the preview frontend’s backend. Hence local-hermetic is the chosen browser-test substrate (no design reversal).
  • Phase 4a (committed local-hermetic lane) — DONE 2026-06-02. npm run test:e2e:local (playwright.local.config.ts + proliant-local.spec.ts
    • global-setup.local.ts) is a repeatable gate: webServer builds the app local-target via the keystone, globalSetup runs seed:e2e, and the spec logs in + asserts the Proliant wizard renders configReady and the integration page is un-gated. 2 passed against the local backend; skips cleanly when the stack is down. Render-smoke scope (no AC backlink — full start→submit drive is the next tier).
    • Surfaced + fixed a real CSP gap: the preview bundle’s connect-src allowed only *.supabase.co, so a local-targeted preview was CSP-blocked from the local Supabase origin (login silently failed). vite.config now allows 127.0.0.1:54321/localhost:54321 under PLAYWRIGHT_E2E only (never prod).
  • Phase 4b (autoheal wiring) — DONE pending CI validation 2026-06-02. .github/workflows/e2e-local-hermetic.yml runs npm run test:e2e:local on the DGX runner (label e2e-local / manual dispatch), skip-safe when the stack is down, with the same propose-only encore-test-triage healer + kill-switch as e2e-full.yml. The local config now emits a JSON report for the healer. Mirrors the proven e2e-full shape; needs one real DGX CI run to validate the local-stack-on-runner assumption (only run locally so far).
  • Full start→submit AC-8 UI proof — decided (b) network-mock; clean local lane deferred. A real local-hermetic AC-8 browser drive can’t inject the x-proliant-fixtures header the integration tests use, so the choice was (a) Proliant sandbox creds (BAA) vs (b) page.route mocking the function — we chose (b) (deterministic, no BAA). The full mocked drive already exists in tests/e2e/hr/proliant-payroll-run.spec.ts (login → mocked function/REST → asserts “payroll posted”), but wiring it into the local lane was reverted: it imports @chromatic-com/playwright (snapshot-after-each errors in this lane) and gates on a mocked “Run Payroll” heading that fights the real local backend. Follow-up: a chromatic-free local full-drive spec (reuse the mock fixtures, @playwright/test import, real local login). Until then: the drive logic is proven by that mocked spec (default lane) and the real server-side AC-8 by payroll-run.int.test.ts (which the verify-acs gate promotes); the local lane is the UI reachability (render-smoke) proof. No @verifies AC-8 on a Playwright spec — verify-acs runs vitest, not Playwright.
  • Minor follow-ups: cosmetic <button>-in-<button> hydration warning in the shared platform WizardShell timeline (all wizards); WENO cred seeding for a local WENO browser pass.
  • Phase 3 (frontend→branch) intentionally deferred (would reverse the dev2 pinning).

​
7. Risks / guardrails

  • Prod-safety of the URL override (G1) is the highest risk — a bug could point a prod build at a test DB or vice-versa. Two-direction guard tests are mandatory before this ships.
  • PHI on preview branches — synthetic seed only; enforce in the seed + a check.
  • Cost/flake of branching — guardrail per-PR concurrency; reuse reuseExistingServer-style short-circuits.
  • Healer scope creep — keep the ≤5-test cap + kill-switch; healer proposes PRs, never auto-merges (already the case).