familysync/.planning/RETROSPECTIVE.md

Project Retrospective

A living document updated after each milestone. Lessons feed forward into future planning.

Milestone: v1.0 — MVP

Shipped: 2026-06-10 Phases: 6 | Plans: 42 | Sessions: not tracked

What Was Built

• Unified color-coded Fastmail calendar (shared + personal) with full event CRUD written back via CalDAV — read views, recurrence/DST expansion, all-day, and an enqueue-only outbox write path.
• Installable React PWA behind Authelia OIDC, taken live over Pangolin/Newt and verified end-to-end on desktop and iOS.
• Shared collaborative lists with real-time SSE co-edit sync, and VAPID Web Push for reminders / event-change / list alerts.

What Worked

• Dev-auth bypass to build behind a deferred gate (D-14): Phases 2–3 were built against a documented bypass while live Authelia/Pangolin infra wasn't ready, with no rework when Gate 2 finally ran live in Phase 3.
• Wave-based parallel plans within phases kept large phases (Phase 3 = 12 plans) moving.
• Enqueue-only outbox with optimistic 202 cleanly separated request handling from the slow CalDAV write, and made create-before-delete ordering + etag/412 handling tractable.

What Was Inefficient

• A long tail of bugs only reproduced under live conditions (Newt MTU blackhole, OIDC state-cookie race, write-path timezone/identity/join/cache bugs, all-day off-by-one, color collisions, silent Android notifications, session-cookie expiry). Building behind the bypass too long delayed their discovery — they all surfaced at once during live bring-up.
• Background workers silently failed: node-cron 4.2.1 skipped every scheduled tick in the long-lived API process, so reminders/poller/outbox never fired on schedule — caught late, during Phase 5 UAT, not by tests.
• Repeated mobile-only defects could only be found by the operator on real devices because the test harness is desktop-Chromium and the prod PWA is behind OIDC (→ backlog 999.12).

Patterns Established

• setInterval, not node-cron, for in-process schedulers (node-cron silently no-ops in a long-lived process). Do not reintroduce node-cron.
• drizzle-kit generate+migrate, never push, on MariaDB — push emits a false destructive (truncate) diff against populated MariaDB.
• iOS-Safari standalone behavior is a human/device checkpoint, not a playwright-cli check — keep those as explicit manual gates.
• Run tsc --noEmit (both apps) in the post-merge gate — esbuild strips types so vitest stays green while tsc fails.

Key Lessons

1. Bring the real external topology (auth + tunnel) up early and behind a small reversible config, rather than deferring all live verification — the live-only bug class is large and clusters at first contact.
2. Long-running Node schedulers need an integration-level "does it actually fire on a tick" check; unit tests pass while the scheduler silently does nothing.
3. Push has hard platform footguns (iOS revokes after 3 silent pushes; standalone install mandatory; VAPID key truncation = silent Apple 403) — encode them as guards from day one, not after a missed notification.

Cost Observations

• Model mix: not tracked
• Sessions: not tracked
• Notable: TDD red→green discipline is visible in commit history, but per-commit gate_status: trailers were never emitted across the milestone — the ship-time TDD audit had nothing structured to aggregate. Wire gate_status trailers in v1.x if the audit is wanted.

---

Milestone: v1.1 — Operability & Polish

Shipped: 2026-06-18 Phases: 14 (7–20) | Plans: 57 | Sessions: not tracked

What Was Built

• A self-hosted Gitea CI/CD pipeline: PR-gating lint (real ESLint flat config) / typecheck / MariaDB-backed API integration / a mobile + desktop Playwright regression harness, plus dependency-audit / gitleaks / eslint-plugin-security / dev↔prod image-hygiene gates and a Docker publish on merge.
• In-app operability: role-gated admin (credential rotation, shared-calendar designation, member editor), a validated first-run setup wizard, per-event reminders with a variable-lead scheduler, auto timezone detection, and ~1–2s event write-back.
• A first-class local-auth (no-OIDC) mode coexisting with Authelia OIDC, removing the hard dependency on a deployed Authelia.

What Worked

• Backlog → phase promotion pipeline: most of v1.1 (999.4/10/11/12/13/14/15/16) was captured as backlog during v1.0, then promoted cleanly into scoped phases — the deferred-idea capture paid off directly.
• Runner-probe-first for self-hosted CI (PITFALL 12): probing node/pnpm/Docker/registry access on the Gitea runner before authoring any test/build steps surfaced every fork answer (Docker-executor, ubuntu-latest, artifact-fork, REGISTRY_PAT naming) up front and avoided blind CI iteration.
• Zero-dependency in-process solutions: the EventEmitter outbox-drain signal (CAL-15) hit the latency goal with no new infra; the project later removed Redis entirely as unused.
• TDD discipline on the admin/auth chain (Phases 10/11/12/19) kept the role boundary and credential-handling correct, with route-level 403/423/409 guards asserted in tests.

What Was Inefficient

• Dev user can't exercise calendar features end-to-end: DEV_AUTH_BYPASS user 1 has no member_credentials/calendars, so per-event reminders (Phase 11) could only be verified via tests + a route-mocked smoke, not hands-on by the operator (→ backlog 999.19). Recurring dev-testability friction.
• Gitea-specific quirks cost cycles: secrets with the GITEA_ prefix are silently dropped (→ REGISTRY_PAT); actions/upload-artifact@v4 is broken on Gitea (needs the ChristopherHX fork); actions/cache@v4 timed out; skipped jobs may not emit a commit-status (drove the always-running gate aggregate). None are documented as GitHub-incompatible up front.
• Scope grew mid-milestone: the milestone planned as 7–17 but accreted 18/19/20 via /gsd-phase, and the ROADMAP header wasn't kept in sync — the phase-detail sections for 18–20 ended up appended after the Backlog. Keep the roadmap header + section ordering current when inserting late phases.

Patterns Established

• Runner-probe-first for any new self-hosted-CI capability — never author steps against an unprobed runner.
• Always-running gate aggregate (if: always(), passes on success-or-skipped) is the only safe required-check surface when path-filtering jobs — never mark a path-filtered job itself required (deadlock).
• In-process EventEmitter over Redis for single-process work (the outbox drain); reserve external infra for genuinely cross-process needs.
• Local auth is a first-class mode, not a fallback — identity stays OIDC-iss+sub (never email); a local user is linked to an OIDC identity via an explicit claim flow (D-10/D-19).
• Confine dev-only affordances at build + boot: bake NODE_ENV=production into the prod image and refuse-to-boot if DEV_AUTH_BYPASS is set — defense-in-depth beyond the runtime guard.

Key Lessons

1. Capturing deferred ideas as structured backlog entries during one milestone makes the next milestone's roadmap nearly write-itself — invest in the capture.
2. Self-hosted GitHub-Actions-compatible runners are not drop-in GitHub — probe the runtime, the action ecosystem (forks), and the status/secret semantics before designing the pipeline.
3. Dev-environment testability is a feature: if the dev user can't exercise the real flows, every feature regresses to test-only verification and the operator can't UAT — fix the dev seed/provider story early (999.19).

Cost Observations

• Model mix: not tracked
• Sessions: not tracked
• Notable: 14 phases shipped in ~8 days (2026-06-10 → 2026-06-18) with heavy parallelization across independent tracks (CI chain vs admin chain vs polish) once the harness landed.

---

Cross-Milestone Trends

Process Evolution

Milestone	Sessions	Phases	Key Change
v1.0	n/a	6	Established GSD plan→execute→verify→ship→complete loop; dev-auth bypass for gated infra; milestone-branch + Gitea PR shipping
v1.1	n/a	14	Self-hosted Gitea CI/CD as the merge gate; per-phase branch + PR shipping; backlog→phase promotion pipeline; parallel independent tracks

Cumulative Quality

Milestone	Tests	Coverage	Zero-Dep Additions
v1.0	PWA 191 + API broker/events 114 green	not measured	n/a
v1.1	PWA ~249 + API ~347 green	not measured	outboxTrigger.ts EventEmitter (CAL-15); Redis later removed entirely as unused

Top Lessons (Verified Across Milestones)

1. Capture deferred ideas as structured backlog during the milestone — v1.1's roadmap came almost entirely from v1.0-era backlog entries.
2. iOS-Safari standalone / on-device push stays a human gate across both milestones — automated harnesses (desktop + mobile-emulated) cover layout/flows, never the device-only behavior.
3. setInterval + in-process signals over external schedulers/brokers for this single-process app — node-cron silently no-ops (v1.0), Redis went unused (v1.1).