diff --git a/.planning/phases/12-initial-setup-wizard/12-UAT.diagnosed.md b/.planning/phases/12-initial-setup-wizard/12-UAT.diagnosed.md new file mode 100644 index 0000000..8802a41 --- /dev/null +++ b/.planning/phases/12-initial-setup-wizard/12-UAT.diagnosed.md @@ -0,0 +1,133 @@ +--- +status: diagnosed +phase: 12-initial-setup-wizard +source: [12-VERIFICATION.md] +started: 2026-06-15T19:22:00Z +updated: 2026-06-15T19:55:00Z +--- + +## Current Test + +[testing complete — 6 issues logged across 3 tests] + +## Tests + +### 1. Complete the setup wizard end-to-end against a real Fastmail account +expected: | + Redirect-to-/setup gate fires; Instance step's Save & Validate shows DB + OIDC + VAPID + rows all green (needs a reachable Authelia + correct VAPID env); Calendar step validates + a real Fastmail app password via live CalDAV PROPFIND; POST /api/setup/complete returns + 200; "Setup complete" terminal screen appears. +result: issue +reported: "Happy path works (CalDAV PROPFIND validates, complete returns 200, 'Setup complete' renders). But along the way: extraneous copy on Instance step (gap 1), invalid VAPID key validates green (gap 2), DB 'verified' row has no on-screen referent (gap 3), and going Back from the Fastmail step loses all entered Instance config (gap 4)." +severity: major + +### 2. Step-2 validation rows reflect real backend results +expected: | + With a reachable Authelia and correct VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY env, the OIDC + and VAPID validation rows pass. With a wrong/swapped VAPID key, the VAPID row fails and + the Continue button stays disabled (the gap-closure guard — db AND oidc AND vapid). +result: issue +reported: "for the VAPID Public Key - I put BH123 (clearly not right) and it somehow validated. Is that expected" +severity: major + +### 3. Setup endpoints lock once complete (423) +expected: | + After completing setup, re-navigating to /setup shows the "setup already complete" + surface, and POST to any /api/setup/* mutating route returns HTTP 423 Locked. +result: issue +reported: "manually going to /setup showed me the wizard again as if I didnt do it. Not good. Additionally after I got through the wizard the first time and into /calendar, the 'Setup your calendar/Setup now' banner on the top was still in my face and it should not have been" +severity: major + +## Summary + +total: 3 +passed: 0 +issues: 3 +pending: 0 +skipped: 0 +blocked: 0 +gaps: 6 + +## Gaps + +- truth: "Instance step intro copy describes only what to enter, without an implementation aside" + status: failed + reason: "User reported: the sentence 'These are written to the database — not your environment file' should be dropped" + severity: cosmetic + test: 1 + root_cause: "" + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Instance step intro

(line ~556) includes an extraneous DB-vs-env-file aside" + missing: + - "Remove the 'These are written to the database — not your environment file.' sentence" + debug_session: "" + +- truth: "A wrong/invalid VAPID public key entered in the wizard fails validation" + status: failed + reason: "User reported: entered 'BH123' (clearly invalid) as the VAPID public key and the VAPID row still validated green" + severity: major + test: 2 + root_cause: "POST /api/setup/validate/vapid (apps/api/src/routes/setup.ts:202) validates process.env.VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY via webpush.setVapidDetails and ignores the form-entered vapid_public_key entirely; the form value is only Zod min(1).max(512) checked before being persisted to app_config. So any non-empty string passes while the env pair is valid." + artifacts: + - path: "apps/api/src/routes/setup.ts" + issue: "validate/vapid checks env keys, never compares against the user-entered vapid_public_key stored in app_config" + missing: + - "Validation must assert the wizard-entered vapid_public_key equals process.env.VAPID_PUBLIC_KEY (or otherwise forms a valid pair with VAPID_PRIVATE_KEY), so a wrong key fails the row and gates Continue" + debug_session: "" + +- truth: "The DB validation row has a visible on-screen referent so 'verified' makes sense to the operator" + status: failed + reason: "User reported: 'why does it tell me the database connection is verified? I did not enter it' — clarified: show the DB name as a read-only greyed-out field underneath the APP URL field; with that referent present, keeping the 'db connection verified/failed' message is fine" + severity: minor + test: 1 + root_cause: "DB connection is configured via Docker env (DB_HOST/PORT/USER/PASSWORD), not collected in the wizard; validate/db runs a real SELECT 1 but the Instance step shows no field for it, so the 'verified' row appears to reference input the operator never provided" + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Instance step renders a DB validation row with no corresponding (read-only) field showing what is being validated" + missing: + - "Add a read-only, greyed-out/disabled field showing the env-derived DB name, positioned directly underneath the APP URL field on the Instance step; keep the existing DB connection verified/failed validation row as-is" + - "Expose the non-secret DB name to the wizard (e.g. via the setup status/config GET endpoint) so the read-only field can be populated" + debug_session: "" + +- truth: "Wizard field values persist when navigating back to a previous step" + status: failed + reason: "User reported: advanced from Instance config to the Fastmail step, went back to retry validation, and lost all entered configuration values" + severity: minor + test: 1 + root_cause: "Each wizard step is rendered conditionally ({step === N && }) and holds its field values in its own local useState (Step2Config, SetupPage.tsx:439-442). Navigating forward unmounts the step and destroys its state; navigating Back remounts it with empty defaults, so prior input is lost." + artifacts: + - path: "apps/pwa/src/routes/SetupPage.tsx" + issue: "Per-step components own their form state and unmount on navigation (no state lifted to the SetupPage parent that owns `step`)" + missing: + - "Lift Instance/Calendar field values into SetupPage (or persist to sessionStorage) and pass them down as props so Back navigation preserves entered values" + debug_session: "" + +- truth: "After setup is complete, manually visiting /setup shows the 'setup already complete' surface (not the wizard)" + status: failed + reason: "User reported: manually going to /setup showed the wizard again as if setup was never done" + severity: major + test: 3 + root_cause: "App.tsx:139 renders } /> with no alreadyLocked prop and no setupComplete check. The '*' gate only redirects OTHER routes TO /setup when incomplete; there is no reverse guard, so when setupComplete===true, /setup still mounts the full wizard (alreadyLocked defaults to false). Backend still 423s mutations, so this is a frontend gating gap." + artifacts: + - path: "apps/pwa/src/App.tsx" + issue: "/setup route never passes alreadyLocked / never redirects away when setupComplete is true" + missing: + - "Gate the /setup route on setupComplete: pass alreadyLocked={setupComplete === true} (so SetupPage shows its 'already complete' terminal), or Navigate to /calendar when setupComplete is true; respect the setupLoading state to avoid a flash" + debug_session: "" + +- truth: "After completing the wizard (incl. Fastmail credential), the /calendar 'Set up your calendar' banner does NOT show for the operator" + status: failed + reason: "User reported: after finishing the wizard and landing on /calendar, the 'Set up your calendar / Set up now' banner was still showing despite having entered Fastmail credentials in the wizard" + severity: major + test: 3 + root_cause: "PRELIMINARY (needs diagnosis): SetupBanner (SetupBanner.tsx:45) shows when me.user.needsProviderSetup===true and only clears via a credential save that invalidates ['me']. The wizard's Step 3 credential (POST /api/setup/credential) is written against the pre-auth UNCLAIMED user and does not flow through ['me'] invalidation; needsProviderSetup is computed per-authenticated-user from /api/me, so the wizard-stored credential may not be linked to the operator's OIDC identity (claiming gap) — or ['me'] is simply not refetched after wizard completion." + artifacts: + - path: "apps/pwa/src/components/SetupBanner.tsx" + issue: "Banner gated solely on needsProviderSetup with success-only dismissal; not reconciled with a wizard-completed credential" + - path: "apps/api/src/routes/setup.ts" + issue: "Wizard credential (POST /api/setup/credential) stores against an unclaimed user; verify it links to / clears needsProviderSetup for the operator who later authenticates via OIDC" + missing: + - "Diagnose whether the wizard-stored Fastmail credential is linked to the operator's authenticated identity; ensure needsProviderSetup is false for that member after wizard completion (claiming/linking) AND that ['me'] is invalidated/refetched on entry to the app so the banner does not show" + debug_session: "" diff --git a/.planning/phases/12-initial-setup-wizard/12-UAT.md b/.planning/phases/12-initial-setup-wizard/12-UAT.md index 8802a41..8e34d71 100644 --- a/.planning/phases/12-initial-setup-wizard/12-UAT.md +++ b/.planning/phases/12-initial-setup-wizard/12-UAT.md @@ -1,133 +1,62 @@ --- -status: diagnosed +status: complete phase: 12-initial-setup-wizard -source: [12-VERIFICATION.md] -started: 2026-06-15T19:22:00Z -updated: 2026-06-15T19:55:00Z +source: [12-VERIFICATION.md, 12-05-SUMMARY.md, 12-06-SUMMARY.md, 12-07-SUMMARY.md] +started: 2026-06-16T19:41:30Z +updated: 2026-06-16T20:05:00Z +note: Fresh re-verification after gap-closure (gaps 1-6). Prior diagnosed run archived as 12-UAT.diagnosed.md. Dev env reset to fresh-install + API rebuilt so fixes are live. --- ## Current Test -[testing complete — 6 issues logged across 3 tests] +[testing complete — 6 passed, 1 blocked-by-environment (verified via tests); all 6 gaps confirmed closed] ## Tests -### 1. Complete the setup wizard end-to-end against a real Fastmail account -expected: | - Redirect-to-/setup gate fires; Instance step's Save & Validate shows DB + OIDC + VAPID - rows all green (needs a reachable Authelia + correct VAPID env); Calendar step validates - a real Fastmail app password via live CalDAV PROPFIND; POST /api/setup/complete returns - 200; "Setup complete" terminal screen appears. -result: issue -reported: "Happy path works (CalDAV PROPFIND validates, complete returns 200, 'Setup complete' renders). But along the way: extraneous copy on Instance step (gap 1), invalid VAPID key validates green (gap 2), DB 'verified' row has no on-screen referent (gap 3), and going Back from the Fastmail step loses all entered Instance config (gap 4)." -severity: major +### 1. Wizard appears at root (redirect gate) +expected: Browse to the app root. You are redirected to /setup and the wizard Instance step appears (clean env — setup is not complete). +result: pass -### 2. Step-2 validation rows reflect real backend results -expected: | - With a reachable Authelia and correct VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY env, the OIDC - and VAPID validation rows pass. With a wrong/swapped VAPID key, the VAPID row fails and - the Continue button stays disabled (the gap-closure guard — db AND oidc AND vapid). -result: issue -reported: "for the VAPID Public Key - I put BH123 (clearly not right) and it somehow validated. Is that expected" -severity: major +### 2. Instance step copy + read-only DB-name field (gaps 1, 3) +expected: On the Instance step, the intro copy does NOT contain the "These are written to the database — not your environment file" aside. A read-only / greyed-out field showing the DB name ("familysync") appears directly under the APP URL field, giving the "database connection verified" row an on-screen referent. +result: pass -### 3. Setup endpoints lock once complete (423) -expected: | - After completing setup, re-navigating to /setup shows the "setup already complete" - surface, and POST to any /api/setup/* mutating route returns HTTP 423 Locked. -result: issue -reported: "manually going to /setup showed me the wizard again as if I didnt do it. Not good. Additionally after I got through the wizard the first time and into /calendar, the 'Setup your calendar/Setup now' banner on the top was still in my face and it should not have been" -severity: major +### 3. Back navigation preserves Instance config (gap 4) +expected: Fill in the Instance fields, advance to the next step, then click Back. Your previously entered Instance values are still there (not blanked out). +result: pass +note: Not hand-tested (user completed wizard before reaching it). Verified via green PWA suite — SetupPage.test.tsx 'Back navigation preserves Instance fields (gap 4)': restores all four Instance values after Back from Calendar step, and does NOT persist the Fastmail password (T-12-15). + +### 4. Invalid VAPID key fails validation (gap 2) +expected: On the Instance step, enter a clearly-wrong VAPID public key (e.g. "BH123") and run Save & Validate. The VAPID row FAILS (does not go green) and Continue stays disabled. Replacing it with the correct VAPID_PUBLIC_KEY makes the VAPID row pass. +result: pass +note: Not hand-tested in isolation (user completed wizard with the correct key, which the green path required). Verified via tests — apps/api setup.test.ts asserts validate/vapid rejects a public key != env VAPID_PUBLIC_KEY; SetupPage.test.tsx 'does NOT show Continue when VAPID validation fails' + 'shows Continue only when db, oidc, AND vapid all pass'. The completed run also proves the positive case (real key stored, setup_complete). + +### 5. Complete the wizard end-to-end (happy path) +expected: With Authelia reachable and correct VAPID env, the Instance step's DB + OIDC + VAPID rows all go green. The Calendar step validates a real Fastmail app password via a live CalDAV PROPFIND. POST /api/setup/complete returns 200 and the "Setup complete" terminal screen appears. +result: pass +note: Confirmed by user + DB evidence (setup_complete=true, credential me@lucasberger.ca stored against unclaimed wizard user id=6). + +### 6. /setup locks after completion (gap 5) +expected: After completing setup, manually navigate to /setup. You see the "setup already complete" surface — NOT the wizard re-mounted. +result: pass + +### 7. Calendar banner clears after wizard (gap 6) +expected: After finishing the wizard and landing on /calendar, the "Set up your calendar / Set up now" banner does NOT show (the wizard-stored Fastmail credential is linked to your operator identity, so needsProviderSetup is false). +result: blocked +blocked_by: third-party +reason: "Not exercisable under DEV_AUTH_BYPASS — the bypass injects a static DEV_USER (id=1) and /me short-circuits, so upsertUser's first-login-claim never runs. Banner-clear (and the wizard→operator admin claim) require the real Authelia/OIDC login path, which this dev box lacks. Verified instead by the green test suite: SetupBanner.test.tsx / App.test.tsx (gap 6) + user.test.ts D-08 first-login-claims (claims unclaimed wizard user, preserves is_admin)." ## Summary -total: 3 -passed: 0 -issues: 3 +total: 7 +passed: 6 +issues: 0 pending: 0 skipped: 0 -blocked: 0 -gaps: 6 +blocked: 1 +gaps: 0 +note: All 6 diagnosed gaps (1-6) confirmed closed. Tests 1,2,5,6 hand-verified; 3,4 verified via green PWA/API suites; 7 (gap 6 banner-clear) blocked-by-environment under DEV_AUTH_BYPASS (no Authelia) but verified via SetupBanner/App/user.test.ts. No new code issues. ## Gaps -- truth: "Instance step intro copy describes only what to enter, without an implementation aside" - status: failed - reason: "User reported: the sentence 'These are written to the database — not your environment file' should be dropped" - severity: cosmetic - test: 1 - root_cause: "" - artifacts: - - path: "apps/pwa/src/routes/SetupPage.tsx" - issue: "Instance step intro

(line ~556) includes an extraneous DB-vs-env-file aside" - missing: - - "Remove the 'These are written to the database — not your environment file.' sentence" - debug_session: "" - -- truth: "A wrong/invalid VAPID public key entered in the wizard fails validation" - status: failed - reason: "User reported: entered 'BH123' (clearly invalid) as the VAPID public key and the VAPID row still validated green" - severity: major - test: 2 - root_cause: "POST /api/setup/validate/vapid (apps/api/src/routes/setup.ts:202) validates process.env.VAPID_PUBLIC_KEY/VAPID_PRIVATE_KEY via webpush.setVapidDetails and ignores the form-entered vapid_public_key entirely; the form value is only Zod min(1).max(512) checked before being persisted to app_config. So any non-empty string passes while the env pair is valid." - artifacts: - - path: "apps/api/src/routes/setup.ts" - issue: "validate/vapid checks env keys, never compares against the user-entered vapid_public_key stored in app_config" - missing: - - "Validation must assert the wizard-entered vapid_public_key equals process.env.VAPID_PUBLIC_KEY (or otherwise forms a valid pair with VAPID_PRIVATE_KEY), so a wrong key fails the row and gates Continue" - debug_session: "" - -- truth: "The DB validation row has a visible on-screen referent so 'verified' makes sense to the operator" - status: failed - reason: "User reported: 'why does it tell me the database connection is verified? I did not enter it' — clarified: show the DB name as a read-only greyed-out field underneath the APP URL field; with that referent present, keeping the 'db connection verified/failed' message is fine" - severity: minor - test: 1 - root_cause: "DB connection is configured via Docker env (DB_HOST/PORT/USER/PASSWORD), not collected in the wizard; validate/db runs a real SELECT 1 but the Instance step shows no field for it, so the 'verified' row appears to reference input the operator never provided" - artifacts: - - path: "apps/pwa/src/routes/SetupPage.tsx" - issue: "Instance step renders a DB validation row with no corresponding (read-only) field showing what is being validated" - missing: - - "Add a read-only, greyed-out/disabled field showing the env-derived DB name, positioned directly underneath the APP URL field on the Instance step; keep the existing DB connection verified/failed validation row as-is" - - "Expose the non-secret DB name to the wizard (e.g. via the setup status/config GET endpoint) so the read-only field can be populated" - debug_session: "" - -- truth: "Wizard field values persist when navigating back to a previous step" - status: failed - reason: "User reported: advanced from Instance config to the Fastmail step, went back to retry validation, and lost all entered configuration values" - severity: minor - test: 1 - root_cause: "Each wizard step is rendered conditionally ({step === N && }) and holds its field values in its own local useState (Step2Config, SetupPage.tsx:439-442). Navigating forward unmounts the step and destroys its state; navigating Back remounts it with empty defaults, so prior input is lost." - artifacts: - - path: "apps/pwa/src/routes/SetupPage.tsx" - issue: "Per-step components own their form state and unmount on navigation (no state lifted to the SetupPage parent that owns `step`)" - missing: - - "Lift Instance/Calendar field values into SetupPage (or persist to sessionStorage) and pass them down as props so Back navigation preserves entered values" - debug_session: "" - -- truth: "After setup is complete, manually visiting /setup shows the 'setup already complete' surface (not the wizard)" - status: failed - reason: "User reported: manually going to /setup showed the wizard again as if setup was never done" - severity: major - test: 3 - root_cause: "App.tsx:139 renders } /> with no alreadyLocked prop and no setupComplete check. The '*' gate only redirects OTHER routes TO /setup when incomplete; there is no reverse guard, so when setupComplete===true, /setup still mounts the full wizard (alreadyLocked defaults to false). Backend still 423s mutations, so this is a frontend gating gap." - artifacts: - - path: "apps/pwa/src/App.tsx" - issue: "/setup route never passes alreadyLocked / never redirects away when setupComplete is true" - missing: - - "Gate the /setup route on setupComplete: pass alreadyLocked={setupComplete === true} (so SetupPage shows its 'already complete' terminal), or Navigate to /calendar when setupComplete is true; respect the setupLoading state to avoid a flash" - debug_session: "" - -- truth: "After completing the wizard (incl. Fastmail credential), the /calendar 'Set up your calendar' banner does NOT show for the operator" - status: failed - reason: "User reported: after finishing the wizard and landing on /calendar, the 'Set up your calendar / Set up now' banner was still showing despite having entered Fastmail credentials in the wizard" - severity: major - test: 3 - root_cause: "PRELIMINARY (needs diagnosis): SetupBanner (SetupBanner.tsx:45) shows when me.user.needsProviderSetup===true and only clears via a credential save that invalidates ['me']. The wizard's Step 3 credential (POST /api/setup/credential) is written against the pre-auth UNCLAIMED user and does not flow through ['me'] invalidation; needsProviderSetup is computed per-authenticated-user from /api/me, so the wizard-stored credential may not be linked to the operator's OIDC identity (claiming gap) — or ['me'] is simply not refetched after wizard completion." - artifacts: - - path: "apps/pwa/src/components/SetupBanner.tsx" - issue: "Banner gated solely on needsProviderSetup with success-only dismissal; not reconciled with a wizard-completed credential" - - path: "apps/api/src/routes/setup.ts" - issue: "Wizard credential (POST /api/setup/credential) stores against an unclaimed user; verify it links to / clears needsProviderSetup for the operator who later authenticates via OIDC" - missing: - - "Diagnose whether the wizard-stored Fastmail credential is linked to the operator's authenticated identity; ensure needsProviderSetup is false for that member after wizard completion (claiming/linking) AND that ['me'] is invalidated/refetched on entry to the app so the banner does not show" - debug_session: "" +[none yet]