diff --git a/.planning/phases/12-initial-setup-wizard/12-UAT.md b/.planning/phases/12-initial-setup-wizard/12-UAT.md index d16727d..8802a41 100644 --- a/.planning/phases/12-initial-setup-wizard/12-UAT.md +++ b/.planning/phases/12-initial-setup-wizard/12-UAT.md @@ -1,24 +1,14 @@ --- -status: testing +status: diagnosed phase: 12-initial-setup-wizard source: [12-VERIFICATION.md] started: 2026-06-15T19:22:00Z -updated: 2026-06-15T19:22:00Z +updated: 2026-06-15T19:55:00Z --- ## Current Test -number: 1 -name: Complete the setup wizard end-to-end against a real Fastmail account -expected: | - With the rebuilt API container running and the dev DB unconfigured (no setup_complete, - no member_credentials), navigating to http://localhost:5173/ redirects to /setup. The - operator advances Welcome → Instance (App URL / OIDC issuer / OIDC client ID / VAPID - public key → Save & Validate shows DB, OIDC, and VAPID validation rows all passing - against a reachable Authelia) → Calendar (Fastmail email + app password → CalDAV PROPFIND - validation succeeds) → Complete. POST /api/setup/complete returns 200 and the "Setup - complete" terminal surface renders. -awaiting: user response +[testing complete — 6 issues logged across 3 tests] ## Tests @@ -28,28 +18,116 @@ expected: | rows all green (needs a reachable Authelia + correct VAPID env); Calendar step validates a real Fastmail app password via live CalDAV PROPFIND; POST /api/setup/complete returns 200; "Setup complete" terminal screen appears. -result: [pending] +result: issue +reported: "Happy path works (CalDAV PROPFIND validates, complete returns 200, 'Setup complete' renders). But along the way: extraneous copy on Instance step (gap 1), invalid VAPID key validates green (gap 2), DB 'verified' row has no on-screen referent (gap 3), and going Back from the Fastmail step loses all entered Instance config (gap 4)." +severity: major ### 2. Step-2 validation rows reflect real backend results expected: | With a reachable Authelia and correct VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY env, the OIDC and VAPID validation rows pass. With a wrong/swapped VAPID key, the VAPID row fails and the Continue button stays disabled (the gap-closure guard — db AND oidc AND vapid). -result: [pending] +result: issue +reported: "for the VAPID Public Key - I put BH123 (clearly not right) and it somehow validated. Is that expected" +severity: major ### 3. Setup endpoints lock once complete (423) expected: | After completing setup, re-navigating to /setup shows the "setup already complete" surface, and POST to any /api/setup/* mutating route returns HTTP 423 Locked. -result: [pending] +result: issue +reported: "manually going to /setup showed me the wizard again as if I didnt do it. Not good. Additionally after I got through the wizard the first time and into /calendar, the 'Setup your calendar/Setup now' banner on the top was still in my face and it should not have been" +severity: major ## Summary total: 3 passed: 0 -issues: 0 -pending: 3 +issues: 3 +pending: 0 skipped: 0 blocked: 0 +gaps: 6 ## Gaps + +- truth: "Instance step intro copy describes only what to enter, without an implementation aside" + status: failed + reason: "User reported: the sentence 'These are written to the database — not your environment file' should be dropped" + severity: cosmetic + test: 1 + root_cause: "" + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Instance step intro

(line ~556) includes an extraneous DB-vs-env-file aside" + missing: + - "Remove the 'These are written to the database — not your environment file.' sentence" + debug_session: "" + +- truth: "A wrong/invalid VAPID public key entered in the wizard fails validation" + status: failed + reason: "User reported: entered 'BH123' (clearly invalid) as the VAPID public key and the VAPID row still validated green" + severity: major + test: 2 + root_cause: "POST /api/setup/validate/vapid (apps/api/src/routes/setup.ts:202) validates process.env.VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY via webpush.setVapidDetails and ignores the form-entered vapid_public_key entirely; the form value is only Zod min(1).max(512) checked before being persisted to app_config. So any non-empty string passes while the env pair is valid." + artifacts: + - path: "apps/api/src/routes/setup.ts" + issue: "validate/vapid checks env keys, never compares against the user-entered vapid_public_key stored in app_config" + missing: + - "Validation must assert the wizard-entered vapid_public_key equals process.env.VAPID_PUBLIC_KEY (or otherwise forms a valid pair with VAPID_PRIVATE_KEY), so a wrong key fails the row and gates Continue" + debug_session: "" + +- truth: "The DB validation row has a visible on-screen referent so 'verified' makes sense to the operator" + status: failed + reason: "User reported: 'why does it tell me the database connection is verified? I did not enter it' — clarified: show the DB name as a read-only greyed-out field underneath the APP URL field; with that referent present, keeping the 'db connection verified/failed' message is fine" + severity: minor + test: 1 + root_cause: "DB connection is configured via Docker env (DB_HOST/PORT/USER/PASSWORD), not collected in the wizard; validate/db runs a real SELECT 1 but the Instance step shows no field for it, so the 'verified' row appears to reference input the operator never provided" + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Instance step renders a DB validation row with no corresponding (read-only) field showing what is being validated" + missing: + - "Add a read-only, greyed-out/disabled field showing the env-derived DB name, positioned directly underneath the APP URL field on the Instance step; keep the existing DB connection verified/failed validation row as-is" + - "Expose the non-secret DB name to the wizard (e.g. via the setup status/config GET endpoint) so the read-only field can be populated" + debug_session: "" + +- truth: "Wizard field values persist when navigating back to a previous step" + status: failed + reason: "User reported: advanced from Instance config to the Fastmail step, went back to retry validation, and lost all entered configuration values" + severity: minor + test: 1 + root_cause: "Each wizard step is rendered conditionally ({step === N && }) and holds its field values in its own local useState (Step2Config, SetupPage.tsx:439-442). Navigating forward unmounts the step and destroys its state; navigating Back remounts it with empty defaults, so prior input is lost." + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Per-step components own their form state and unmount on navigation (no state lifted to the SetupPage parent that owns `step`)" + missing: + - "Lift Instance/Calendar field values into SetupPage (or persist to sessionStorage) and pass them down as props so Back navigation preserves entered values" + debug_session: "" + +- truth: "After setup is complete, manually visiting /setup shows the 'setup already complete' surface (not the wizard)" + status: failed + reason: "User reported: manually going to /setup showed the wizard again as if setup was never done" + severity: major + test: 3 + root_cause: "App.tsx:139 renders } /> with no alreadyLocked prop and no setupComplete check. The '*' gate only redirects OTHER routes TO /setup when incomplete; there is no reverse guard, so when setupComplete===true, /setup still mounts the full wizard (alreadyLocked defaults to false). Backend still 423s mutations, so this is a frontend gating gap." + artifacts: + - path: "apps/pwa/src/App.tsx" + issue: "/setup route never passes alreadyLocked / never redirects away when setupComplete is true" + missing: + - "Gate the /setup route on setupComplete: pass alreadyLocked={setupComplete === true} (so SetupPage shows its 'already complete' terminal), or Navigate to /calendar when setupComplete is true; respect the setupLoading state to avoid a flash" + debug_session: "" + +- truth: "After completing the wizard (incl. Fastmail credential), the /calendar 'Set up your calendar' banner does NOT show for the operator" + status: failed + reason: "User reported: after finishing the wizard and landing on /calendar, the 'Set up your calendar / Set up now' banner was still showing despite having entered Fastmail credentials in the wizard" + severity: major + test: 3 + root_cause: "PRELIMINARY (needs diagnosis): SetupBanner (SetupBanner.tsx:45) shows when me.user.needsProviderSetup===true and only clears via a credential save that invalidates ['me']. The wizard's Step 3 credential (POST /api/setup/credential) is written against the pre-auth UNCLAIMED user and does not flow through ['me'] invalidation; needsProviderSetup is computed per-authenticated-user from /api/me, so the wizard-stored credential may not be linked to the operator's OIDC identity (claiming gap) — or ['me'] is simply not refetched after wizard completion." + artifacts: + - path: "apps/pwa/src/components/SetupBanner.tsx" + issue: "Banner gated solely on needsProviderSetup with success-only dismissal; not reconciled with a wizard-completed credential" + - path: "apps/api/src/routes/setup.ts" + issue: "Wizard credential (POST /api/setup/credential) stores against an unclaimed user; verify it links to / clears needsProviderSetup for the operator who later authenticates via OIDC" + missing: + - "Diagnose whether the wizard-stored Fastmail credential is linked to the operator's authenticated identity; ensure needsProviderSetup is false for that member after wizard completion (claiming/linking) AND that ['me'] is invalidated/refetched on entry to the app so the banner does not show" + debug_session: ""