familysync/.planning/milestones/v1.1-phases/12-initial-setup-wizard/12-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created

phase	slug	status	nyquist_compliant	wave_0_complete	created
12	initial-setup-wizard	ready	true	false	2026-06-15

Phase 12 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	Vitest (API: apps/api/tests/; PWA: apps/pwa) + Playwright (e2e)
Config file	apps/api/vitest.config.ts, apps/pwa/vitest config, apps/pwa/playwright.config.ts
Quick run command	pnpm --filter @familysync/api test -- setup
Full suite command	pnpm --filter @familysync/api test && pnpm --filter @familysync/pwa test
Estimated runtime	~30–60 seconds (API + PWA unit)

---

Sampling Rate

• After every task commit: Run pnpm --filter @familysync/api test -- setup (API tasks) or pnpm --filter @familysync/pwa test -- App (PWA tasks)
• After every plan wave: Run pnpm --filter @familysync/api test && pnpm --filter @familysync/pwa test
• Before /gsd-verify-work: Full suite + pnpm test:e2e green
• Max feedback latency: 60 seconds

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
12-01-01	01	1	SETUP-03 (schema)	T-12-02	Migration MODIFY (not DROP) + backfill; no orphaned rows	integration	cd apps/api && pnpm exec drizzle-kit migrate && pnpm typecheck	✅	⬜ pending
12-01-02	01	1	SETUP-03	T-12-01	Secrets to stdout only — never DB/file/log	unit (script)	node scripts/generate-secrets.mjs | grep -E ...	✅	⬜ pending
12-01-03	01	1	— (scaffold)	—	Import targets only	typecheck	cd apps/api && pnpm typecheck	✅	⬜ pending
12-01-04	01	1	SETUP-01/02/03/04	T-12-04	RED 423-guard test before happy path	unit (scaffold)	cd apps/api && pnpm test -- setup	✅ W0	⬜ pending
12-02-01	02	2	SETUP-04	T-12-04	Per-call 423; no startup cache	unit	cd apps/api && pnpm test -- setup	✅	⬜ pending
12-02-02	02	2	SETUP-01/02	T-12-05/06/08	noEchoHook; VAPID priv env-only; https issuer; no new crypto	integration	cd apps/api && pnpm test -- setup && pnpm typecheck	✅	⬜ pending
12-02-03	02	2	SETUP-01	T-12-09	Pre-auth mount before OIDC guard; env-OR-app_config boot	integration	cd apps/api && pnpm typecheck && pnpm test	✅	⬜ pending
12-03-01	03	2	SETUP-01 (D-08)	T-12-10/11/12	Claim by oidc_iss IS NULL+claimed=false; no email key; admin gated	unit	cd apps/api && pnpm test -- user && pnpm typecheck	✅	⬜ pending
12-04-01	04	3	SETUP-01/02	T-12-13	UI-SPEC: no generate-secrets step	doc grep	grep -Eq "oidc_issuer|/api/setup/config" 12-UI-SPEC.md	✅	⬜ pending
12-04-02	04	3	SETUP-01/02	T-12-14/15	No dangerouslySetInnerHTML; password input	typecheck+build	cd apps/pwa && pnpm typecheck && pnpm build	✅	⬜ pending
12-04-03	04	3	SETUP-01	—	Redirect gate; no flash	unit	cd apps/pwa && pnpm test -- App && pnpm typecheck	✅	⬜ pending
12-04-04	04	3	SETUP-01/02	T-12-14	End-to-end wizard flow (playwright-cli desktop)	e2e / human	playwright-cli drive /setup (see plan)	✅	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Wave 0 Requirements

• apps/api/tests/routes/setup.test.ts — scaffolds SETUP-01/02/03/04 incl. the RED 423-guard test (Pitfall 8) — created in Plan 01 Task 4
• apps/api/tests/auth/user.test.ts — D-08 first-login-claims scaffold — extended in Plan 01 Task 4
• apps/api/src/routes/setup.ts — stub Hono router (import target) — Plan 01 Task 3
• apps/api/src/lib/setupGuard.ts — stub isSetupLocked (import target) — Plan 01 Task 3

Existing Vitest + Playwright infrastructure covers all other phase requirements.

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Live Authelia OIDC discovery round-trip	SETUP-02	No live Authelia in test env; mock the discovery fetch in unit tests	If a live Authelia is available, validate /api/setup/validate/oidc against the real issuer; otherwise rely on the fetch-mock unit test
Live Fastmail CalDAV PROPFIND	SETUP-02	Requires a real Fastmail app password; unit tests mock createFastmailClient	Optional live check with a known-good app password during the playwright-cli smoke (Plan 04 Task 4)
iOS-Safari standalone behavior	—	Not in scope this phase; wizard is desktop-driven	N/A — desktop Chromium via playwright-cli covers the wizard per CLAUDE.md

All automatable phase behaviors have automated verification; the above need live services or are out of scope.

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify (the only checkpoint, 12-04-04, follows three automated PWA tasks)
• Wave 0 covers all MISSING references (setup.test.ts, user.test.ts, setup.ts stub, setupGuard.ts stub — all in Plan 01)
• No watch-mode flags
• Feedback latency < 60s
• nyquist_compliant: true set in frontmatter

Approval: approved 2026-06-15