familysync/.planning/RETROSPECTIVE.md

# Project Retrospective

_A living document updated after each milestone. Lessons feed forward into future planning._

## Milestone: v1.0 — MVP

**Shipped:** 2026-06-10
**Phases:** 6 | **Plans:** 42 | **Sessions:** not tracked

### What Was Built

- Unified color-coded Fastmail calendar (shared + personal) with full event CRUD written back via CalDAV — read views, recurrence/DST expansion, all-day, and an enqueue-only outbox write path.
- Installable React PWA behind Authelia OIDC, taken live over Pangolin/Newt and verified end-to-end on desktop and iOS.
- Shared collaborative lists with real-time SSE co-edit sync, and VAPID Web Push for reminders / event-change / list alerts.

### What Worked

- **Dev-auth bypass to build behind a deferred gate (D-14):** Phases 2–3 were built against a documented bypass while live Authelia/Pangolin infra wasn't ready, with no rework when Gate 2 finally ran live in Phase 3.
- **Wave-based parallel plans** within phases kept large phases (Phase 3 = 12 plans) moving.
- **Enqueue-only outbox with optimistic 202** cleanly separated request handling from the slow CalDAV write, and made create-before-delete ordering + etag/412 handling tractable.

### What Was Inefficient

- **A long tail of bugs only reproduced under live conditions** (Newt MTU blackhole, OIDC state-cookie race, write-path timezone/identity/join/cache bugs, all-day off-by-one, color collisions, silent Android notifications, session-cookie expiry). Building behind the bypass too long delayed their discovery — they all surfaced at once during live bring-up.
- **Background workers silently failed:** node-cron 4.2.1 skipped _every_ scheduled tick in the long-lived API process, so reminders/poller/outbox never fired on schedule — caught late, during Phase 5 UAT, not by tests.
- **Repeated mobile-only defects could only be found by the operator on real devices** because the test harness is desktop-Chromium and the prod PWA is behind OIDC (→ backlog 999.12).

### Patterns Established

- **`setInterval`, not node-cron, for in-process schedulers** (node-cron silently no-ops in a long-lived process). Do not reintroduce node-cron.
- **drizzle-kit `generate`+`migrate`, never `push`, on MariaDB** — `push` emits a false destructive (truncate) diff against populated MariaDB.
- **iOS-Safari standalone behavior is a human/device checkpoint**, not a playwright-cli check — keep those as explicit manual gates.
- **Run `tsc --noEmit` (both apps) in the post-merge gate** — esbuild strips types so vitest stays green while tsc fails.

### Key Lessons

1. Bring the real external topology (auth + tunnel) up _early_ and behind a small reversible config, rather than deferring all live verification — the live-only bug class is large and clusters at first contact.
2. Long-running Node schedulers need an integration-level "does it actually fire on a tick" check; unit tests pass while the scheduler silently does nothing.
3. Push has hard platform footguns (iOS revokes after 3 silent pushes; standalone install mandatory; VAPID key truncation = silent Apple 403) — encode them as guards from day one, not after a missed notification.

### Cost Observations

- Model mix: not tracked
- Sessions: not tracked
- Notable: TDD red→green discipline is visible in commit history, but per-commit `gate_status:` trailers were never emitted across the milestone — the ship-time TDD audit had nothing structured to aggregate. Wire gate_status trailers in v1.x if the audit is wanted.

---

## Milestone: v1.1 — Operability & Polish

**Shipped:** 2026-06-18
**Phases:** 14 (7–20) | **Plans:** 57 | **Sessions:** not tracked

### What Was Built

- A self-hosted Gitea CI/CD pipeline: PR-gating lint (real ESLint flat config) / typecheck / MariaDB-backed API integration / a mobile + desktop Playwright regression harness, plus dependency-audit / gitleaks / eslint-plugin-security / dev↔prod image-hygiene gates and a Docker publish on merge.
- In-app operability: role-gated admin (credential rotation, shared-calendar designation, member editor), a validated first-run setup wizard, per-event reminders with a variable-lead scheduler, auto timezone detection, and ~1–2s event write-back.
- A first-class local-auth (no-OIDC) mode coexisting with Authelia OIDC, removing the hard dependency on a deployed Authelia.

### What Worked

- **Backlog → phase promotion pipeline:** most of v1.1 (999.4/10/11/12/13/14/15/16) was captured as backlog during v1.0, then promoted cleanly into scoped phases — the deferred-idea capture paid off directly.
- **Runner-probe-first for self-hosted CI (PITFALL 12):** probing `node`/`pnpm`/Docker/registry access on the Gitea runner *before* authoring any test/build steps surfaced every fork answer (Docker-executor, `ubuntu-latest`, artifact-fork, `REGISTRY_PAT` naming) up front and avoided blind CI iteration.
- **Zero-dependency in-process solutions:** the EventEmitter outbox-drain signal (CAL-15) hit the latency goal with no new infra; the project later removed Redis entirely as unused.
- **TDD discipline on the admin/auth chain** (Phases 10/11/12/19) kept the role boundary and credential-handling correct, with route-level 403/423/409 guards asserted in tests.

### What Was Inefficient

- **Dev user can't exercise calendar features end-to-end:** `DEV_AUTH_BYPASS` user 1 has no `member_credentials`/calendars, so per-event reminders (Phase 11) could only be verified via tests + a route-mocked smoke, not hands-on by the operator (→ backlog 999.19). Recurring dev-testability friction.
- **Gitea-specific quirks cost cycles:** secrets with the `GITEA_` prefix are silently dropped (→ `REGISTRY_PAT`); `actions/upload-artifact@v4` is broken on Gitea (needs the `ChristopherHX` fork); `actions/cache@v4` timed out; skipped jobs may not emit a commit-status (drove the always-running `gate` aggregate). None are documented as GitHub-incompatible up front.
- **Scope grew mid-milestone:** the milestone planned as 7–17 but accreted 18/19/20 via `/gsd-phase`, and the ROADMAP header wasn't kept in sync — the phase-detail sections for 18–20 ended up appended after the Backlog. Keep the roadmap header + section ordering current when inserting late phases.

### Patterns Established

- **Runner-probe-first** for any new self-hosted-CI capability — never author steps against an unprobed runner.
- **Always-running `gate` aggregate** (`if: always()`, passes on success-or-skipped) is the only safe required-check surface when path-filtering jobs — never mark a path-filtered job itself required (deadlock).
- **In-process EventEmitter over Redis** for single-process work (the outbox drain); reserve external infra for genuinely cross-process needs.
- **Local auth is a first-class mode**, not a fallback — identity stays OIDC-`iss+sub` (never email); a local user is *linked* to an OIDC identity via an explicit claim flow (D-10/D-19).
- **Confine dev-only affordances at build + boot:** bake `NODE_ENV=production` into the prod image and refuse-to-boot if `DEV_AUTH_BYPASS` is set — defense-in-depth beyond the runtime guard.

### Key Lessons

1. Capturing deferred ideas as structured backlog entries during one milestone makes the next milestone's roadmap nearly write-itself — invest in the capture.
2. Self-hosted GitHub-Actions-compatible runners are *not* drop-in GitHub — probe the runtime, the action ecosystem (forks), and the status/secret semantics before designing the pipeline.
3. Dev-environment testability is a feature: if the dev user can't exercise the real flows, every feature regresses to test-only verification and the operator can't UAT — fix the dev seed/provider story early (999.19).

### Cost Observations

- Model mix: not tracked
- Sessions: not tracked
- Notable: 14 phases shipped in ~8 days (2026-06-10 → 2026-06-18) with heavy parallelization across independent tracks (CI chain vs admin chain vs polish) once the harness landed.

---

## Cross-Milestone Trends

### Process Evolution

| Milestone | Sessions | Phases | Key Change                                                                                                                    |
| --------- | -------- | ------ | ----------------------------------------------------------------------------------------------------------------------------- |
| v1.0      | n/a      | 6      | Established GSD plan→execute→verify→ship→complete loop; dev-auth bypass for gated infra; milestone-branch + Gitea PR shipping |
| v1.1      | n/a      | 14     | Self-hosted Gitea CI/CD as the merge gate; per-phase branch + PR shipping; backlog→phase promotion pipeline; parallel independent tracks |

### Cumulative Quality

| Milestone | Tests                                 | Coverage     | Zero-Dep Additions                          |
| --------- | ------------------------------------- | ------------ | ------------------------------------------- |
| v1.0      | PWA 191 + API broker/events 114 green | not measured | n/a                                         |
| v1.1      | PWA ~249 + API ~347 green             | not measured | `outboxTrigger.ts` EventEmitter (CAL-15); Redis later removed entirely as unused |

### Top Lessons (Verified Across Milestones)

1. **Capture deferred ideas as structured backlog during the milestone** — v1.1's roadmap came almost entirely from v1.0-era backlog entries.
2. **iOS-Safari standalone / on-device push stays a human gate** across both milestones — automated harnesses (desktop + mobile-emulated) cover layout/flows, never the device-only behavior.
3. **`setInterval` + in-process signals over external schedulers/brokers** for this single-process app — node-cron silently no-ops (v1.0), Redis went unused (v1.1).