familysync/.planning/milestones/v1.1-phases/07-mobile-test-harness/07-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
07	mobile-test-harness	draft	false	false	2026-06-10

Phase 07 — Validation Strategy

Per-phase validation contract for feedback sampling during execution. This phase's deliverable IS the test infrastructure, so "validation" here means proving the harness itself detects real defects (see Harness Self-Validation).

---

Test Infrastructure

Property	Value
Framework	@playwright/test 1.60.0 (new dev dep in apps/pwa)
Config file	apps/pwa/playwright.config.ts (none today — Wave 0 creates it)
Quick run command	pnpm --filter @familysync/pwa exec playwright test --project=pixel
Full suite command	pnpm --filter @familysync/pwa exec playwright test
CI invocation	CI=true pnpm --filter @familysync/pwa exec playwright test
Estimated runtime	~30–60s full suite (two profiles, host stack already up)

Existing Vitest unit suite (apps/pwa, *.test.ts) remains the per-commit unit gate; this phase adds a separate E2E suite (e2e/*.spec.ts). The two must not share a glob — Wave 0 adds exclude: ['e2e/**'] to vitest.config.ts.

---

Sampling Rate

• After every task commit: Run pnpm --filter @familysync/pwa exec playwright test --project=pixel (Chromium-only, faster feedback)
• After every plan wave: Run pnpm --filter @familysync/pwa exec playwright test (both iPhone/WebKit + Pixel/Chromium profiles)
• Before /gsd-verify-work: Full suite green on both profiles
• Max feedback latency: ~60 seconds (full suite, host stack running)

---

Per-Task Verification Map

Task IDs resolve when PLAN.md files are written; rows below are keyed by requirement + target file so the planner can attach <automated> verify blocks. Every Phase-7 task must map to one of these or declare a Wave 0 dependency.

Plan/Wave	Requirement	Behavior verified	Test Type	Automated Command	File (Wave 0)	Status
W0	TEST-01/02	@playwright/test installed; config matrix (iPhone WebKit + Pixel Chromium), serviceWorkers: 'block', env baseURL	config	pnpm --filter @familysync/pwa exec playwright test --list	apps/pwa/playwright.config.ts	⬜ pending
W0	TEST-02	global-setup polls /health then deterministically seeds (calendar id 10 INSERT IGNORE FK guard + list + items, reset-per-run)	infra	run produces seeded rows; spec reads populated views	apps/pwa/e2e/global-setup.ts	⬜ pending
W1	TEST-01	Tap targets ≥ 44px (BottomTabBar, FAB, Retry, settings); no horizontal overflow on /calendar /lists	E2E	pnpm --filter @familysync/pwa exec playwright test	apps/pwa/e2e/layout.spec.ts	⬜ pending
W1	TEST-01	Calendar populated + empty + error states (UI-SPEC Rules 4/5)	E2E	same	apps/pwa/e2e/calendar.spec.ts	⬜ pending
W1	TEST-01	Lists populated + empty states (UI-SPEC Rules 4/5)	E2E	same	apps/pwa/e2e/lists.spec.ts	⬜ pending
W1	TEST-02	Authenticated PWA reached via DEV_AUTH_BYPASS (no manual login, no OIDC mock); trace shows no SW-sourced responses	E2E	CI=true pnpm --filter @familysync/pwa exec playwright test	spec preconditions + trace	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Harness Self-Validation (the harness must prove it works)

The deliverable is test infrastructure — a green suite alone does not prove it would catch a real defect. The phase is validated only when the assertions provably fail on injected defects:

1. Tap-target failure injection — temporarily shrink a touch target (e.g. BottomTabBar minHeight: 20px via page.addStyleTag); the ≥44px assertion MUST fail. Restore → passes. Proves boundingBox() measures the rendered element, not the CSS source.
2. Overflow failure injection — add body { width: 2000px } via page.addStyleTag before the overflow assertion; it MUST fail. Remove → passes. Proves the scrollWidth > clientWidth check is live.
3. SW-block verification — with trace: 'on-first-retry', inspect one trace and confirm zero responses sourced from (ServiceWorker) (D-02 / Pitfall 15).
4. Auth-bypass verification (manual) — with DEV_AUTH_BYPASS disabled the API redirects to Authelia and specs MUST fail on missing authed content; with it enabled they pass (D-01 / SC #2).

---

Wave 0 Requirements

• apps/pwa/playwright.config.ts — project matrix (iPhone WebKit + Pixel Chromium), globalSetup, optional vite-only webServer with reuseExistingServer: !process.env.CI, env-driven baseURL, artifact/trace config
• apps/pwa/e2e/global-setup.ts — /health readiness poll + deterministic DB seed (calendar id 10 INSERT IGNORE guard + list + items via mysql2, reset-per-run)
• apps/pwa/vitest.config.ts — add exclude: ['e2e/**'] to prevent the *.spec.ts glob collision
• apps/pwa/package.json — add @playwright/test devDependency + test:e2e script
• Browser install: pnpm --filter @familysync/pwa exec playwright install --with-deps webkit chromium
• apps/pwa/tsconfig/typecheck — ensure e2e/** passes the tsc --noEmit gate

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Auth-bypass off → Authelia redirect	TEST-02 / SC #2	Requires flipping DEV_AUTH_BYPASS off on the dev stack (env change outside the spec)	Disable bypass, run suite, confirm specs fail on missing authed content; re-enable, confirm green
Real prod-service-worker behavior	(out of scope)	Harness blocks the SW by design (D-02); prod SW is a device gate	Human/device check — not automated here
iOS-Safari standalone-PWA (Home-Screen install, standalone OIDC redirect, iOS push)	(out of scope)	Cannot be driven by Playwright/WebKit emulation	Human/device gate per project CLAUDE.md exception

---

Validation Sign-Off

• All tasks have <automated> verify or a Wave 0 dependency
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references (config, global-setup, browser install)
• No watch-mode flags in committed commands
• Feedback latency < 60s
• nyquist_compliant: true set in frontmatter (by planner once task map is complete)

Approval: pending