familysync/.planning/research/SUMMARY.md

# Project Research Summary

**Project:** FamilySync — v1.2 "Multi-Provider, Theming & Zero-Setup"
**Domain:** Self-hosted family calendar + lists PWA (opening beyond Fastmail to a second provider, on a shipped v1.1)
**Researched:** 2026-06-19
**Confidence:** HIGH for architecture/pitfalls (direct codebase inspection + shipped v1.0/v1.1 lessons); MEDIUM for Google Calendar API specifics (official docs via Context7); LOW for third-party UX patterns (cross-checked websearch)

## Executive Summary

v1.2 opens FamilySync beyond Fastmail. The keystone is a hand-rolled `CalendarProvider` TypeScript interface: the existing tsdav/ical.js CalDAV broker is refactored behind it **as a thin delegation wrapper with zero internal rewrite**, then Google Calendar (Google Calendar API v3 + OAuth2, not CalDAV) plugs in as a second implementation, and an in-memory mock provider plugs in as a third (dev-bypass + hermetic CI). On top of that seam: multiple reminders per event (1→N VALARMs / Google `overrides`), PWA dark mode (Light/Dark/System), and zero-manual-setup DB bootstrap (programmatic `migrate()` at boot). The milestone closes by applying CI-surfaced dependency updates.

The core stack is unchanged. **Two new runtime packages only** — `google-auth-library@10.7.0` and `@googleapis/calendar@15.0.0` (scoped, NOT the 50 MB `googleapis` monolith), both in `apps/api`. Dark mode, multiple reminders, migrate-on-boot, and dependency updates add **zero new dependencies** (Zustand `persist`, `ical.js` multi-VALARM, `drizzle-orm/mysql2/migrator`, and existing `pnpm` tooling already cover them).

Three non-negotiable preservation rules drive the design: (1) the provider refactor must **not regress the live Fastmail path** — all v1.1 outbox/poller/scheduler integration tests must pass unchanged against the new interface; (2) the `reminder_lead_minutes`→`reminder_leads` JSON migration must **migrate existing data** before dropping the old column; (3) Google OAuth tokens are **per-member secrets**, AES-256-GCM encrypted, never in `app_config`, with the `member_credentials` unique constraint widened to `UNIQUE(user_id, provider_type)`.

## Key Findings

### Recommended Stack

No stack change beyond two scoped Google packages (both `apps/api`):

- **`google-auth-library@10.7.0`** — OAuth2 authorization-code flow + offline refresh-token management; ships its own types; `OAuth2Client` auto-refreshes expired access tokens via the `tokens` event.
- **`@googleapis/calendar@15.0.0`** — Google Calendar API v3 typed client (depends only on `googleapis-common`). **Reject the monolithic `googleapis`** (170+ clients, ~50 MB).
- **Provider abstraction** — hand-rolled TS interface in `apps/api/src/broker/`; no normalization library exists worth a dependency.
- **Dark mode** — pure CSS `[data-theme="dark"]` + Zustand `persist` (built into 5.0.14) + an inline `<script>` in `index.html`. No theming library.
- **Migrate-on-boot** — `drizzle-orm/mysql2/migrator` (already transitive); single connection, `multipleStatements: true`. **`drizzle-kit push` stays banned** on MariaDB 11.
- **Dependency updates** — existing `pnpm outdated`/`audit`; selective, pin-aware, one ecosystem at a time.

### Expected Features

**Table stakes:**
- Google events in the unified color-coded view (read sync), recurrence + all-day correct; per-calendar color extends the Phase 17 routing to Google calendar IDs.
- Cross-provider event create/edit/delete with a calendar picker that defaults to the member's most-used calendar and shows recognizable names (not provider IDs/emails).
- Self-service onboarding: Fastmail = app-password form with live CalDAV validation (reuse Phase 10 `CredentialSheet`); Google = single "Connect Google Calendar" OAuth button.
- One-tap reconnect on token expiry, surfaced as a **banner in the calendar view** (not buried in settings) — non-negotiable for the non-technical Apple member.
- Up to 5 reminders per event (add/remove rows, pre-populate all on edit), serialized to VALARMs / Google `overrides`, preserving other-client alarms.
- Light/Dark/System theme toggle in member Settings (per-device localStorage), no flash-of-wrong-theme, clean across every route incl. Schedule-X.
- Migrate-on-boot before `serve()`, idempotent, with a DB-readiness wait and fatal-on-failure.
- Mock provider seeded with recurring/all-day/timed/past/future + reminder events, wired to `DEV_AUTH_BYPASS` for hermetic CI.

**Anti-features (explicitly OUT):** two-way Fastmail↔Google mirroring, CalDAV-for-Google, service-account Google auth, tokens in cookies/localStorage, reminder templates/snooze, per-member server-synced theme, custom theme editor, app self-creating the database, Google per-event `colorId`, `googleapis` monolith.

### Architecture Approach

Grounded in direct codebase inspection. Central seam: `broker/provider.ts` (`CalendarProvider` interface) + `broker/providerFactory.ts` (`createProvider(cred)` dispatching on `member_credentials.provider_type`: `caldav`/`google`/`mock`).

- **CalDavProvider** wraps existing `client.ts`/`sync.ts`/`write.ts` **verbatim** — delegation, not refactor (Anti-Pattern 1).
- **poller.ts / outboxWorker.ts** become provider-agnostic: `createProvider(cred)` → `provider.listCalendars()`/`syncCalendar()`/`dispatchOutboxRow()`. The `isDraining` guard, `drainRequested` flag, optimistic-202, create-before-delete, and `uid:dtstartMs` dedup all stay in the worker (provider-agnostic) and must be unchanged.
- **GoogleCalendarProvider** — new; inline token refresh (check `expires_at` before each call, refresh within 5 min of expiry, retry-once on 401). Google OAuth callback mounted **pre-auth** (like Authelia `/callback`), at a distinct path with a distinct state cookie, redirect URI built from `EXTERNAL_BASE_URL` (never `req.host`).
- **Token storage** — research surfaces two options: a separate `provider_tokens` table (ARCHITECTURE.md) vs. extending `member_credentials` with `UNIQUE(user_id, provider_type)` (STACK/PITFALLS). Either way the unique constraint must widen to composite; `client_id`/`client_secret`/refresh tokens are env/encrypted, never in `app_config`. **Decide in the provider-abstraction phase.**
- **Multiple reminders** — `calendar_events.reminder_lead_minutes INT` → `reminder_leads JSON`, with a `JSON_ARRAY(...)` data-migration step before drop; `buildVeventString` loops N VALARMs; scheduler dedup key → `uid:dtstartMs:lead`; Google cap = 5 (reject 6+ before write).
- **Dark mode** — `themeStore.ts` (Zustand persist) + inline FOUC script in `index.html` + `[data-theme="dark"]` block mapping **all** `--sx-color-*` Schedule-X vars + dynamic `theme-color` meta for iOS.
- **Migrate-on-boot** — `runMigrationsIfNeeded()` in the `isMainModule()` guard, after the boot guards, before broker workers + `serve()`; `migration_lock` sentinel for concurrent-startup safety.
- **Mock provider** — `broker/mockProvider.ts`, in-memory, activated via `DEV_AUTH_BYPASS`/`provider_type='mock'`; resets on restart (intentionally stateless).

### Recommended Phase Structure (dependency-ordered, continues from v1.1 Phase 20 → starts at Phase 21)

ARCHITECTURE.md's authoritative ordering:

1. **Phase 21 — Zero-Setup DB Bootstrap.** `runMigrationsIfNeeded()` + `migration_lock` sentinel. No deps; unlocks every later phase to assume auto-migrate. Risk: low.
2. **Phase 22 — Provider Abstraction (CalDAV only, Fastmail unchanged).** Interface + factory + `CalDavProvider` wrapper; poller/outbox use `createProvider`. Dep: 21. **Risk: HIGH** — must not regress Fastmail; golden-path integration tests pass unchanged. Includes the `UNIQUE(user_id, provider_type)` migration.
3. **Phase 23 — Multiple Reminders.** `reminder_leads` JSON + data-migration; multi-VALARM; scheduler multi-lead + dedup-key change; form add/remove. Dep: 22. Risk: MEDIUM (breaking migration).
4. **Phase 24 — Mock Provider (dev/CI).** Seeded in-memory provider; factory `mock` case; Playwright exercises calendar CRUD. Dep: 22. Risk: low.
5. **Phase 25 — Google Calendar Provider.** Token storage + `googleProvider.ts` + OAuth authorize/callback routes + pre-auth mount. Deps: 22, 23 (reminder serialization), 21 (tokens table). Risk: MEDIUM (OAuth edge cases, recurrence/sync model).
6. **Phase 26 — Self-Service Onboarding UI.** `/api/me/providers` GET/DELETE, `ProviderConnectSheet`, reconnect banner. Dep: 25. Risk: low.
7. **Phase 27 — PWA Dark Mode.** FOUC script, dark tokens, `themeStore`, `ThemeToggle`. No deps — **parallelizable with 22–26**. Risk: low.
8. **Phase 28 (or folded) — Dependency Updates (DEP-01).** Last; per-ecosystem batches with CI green after each. Risk: low but cascade-prone (ESLint 9.x pin, Schedule-X hold, Drizzle coupling).

`THEME-01` (27) and `SETUP-05` (21) are independent of the multi-provider chain. The roadmapper should confirm whether dark mode and dependency updates are standalone phases or folded.

### Critical Pitfalls (phase-mapped)

1. **Provider refactor regresses the live Fastmail path** (P22) — wrong `provider_type` branch silently stops all write-back; keep `caldav` the default, change zero existing tests.
2. **Google refresh tokens expire after 7 days in "Testing" publishing status** (P25) — use the non-restricted `calendar.readonly` + `calendar.events` scopes + Production status to avoid Google verification; add both accounts as test users; store grant timestamp.
3. **Google recurrence ≠ iCalendar RRULE** (P25) — fetch with `singleEvents: false`, fetch modified/`cancelled` instances separately, map cancellations to EXDATE, reconstruct a VCALENDAR string for the existing `expand.ts`.
4. **Google `syncToken` 410 Gone halts sync + full re-sync overwrites un-drained local writes** (P25) — catch 410 → drop token → enqueue full re-sync; drain outbox (or skip pending UIDs) before upsert.
5. **Google all-day/timezone differs** (P25) — normalize `start.dateTime` to UTC `...Z` (never a TZID without VTIMEZONE); map `reminders.useDefault:true` → null ("no FamilySync reminder").
6. **Multiple VALARM dedup + Google 5-cap** (P23/P25) — dedup key `uid:dtstartMs:lead`; preserve-vs-replace driven by a `remindersChanged` flag; reject 6+ reminders before a Google write.
7. **Drizzle auto-migrate on MariaDB 11 — three failure modes** (P21) — journal-hash mismatch, false destructive diff (only if `push` is misused — use the `migrator` API), concurrent-startup race (the `migration_lock` sentinel); log each applied migration.
8. **Dark mode FOWT + Schedule-X coverage + SW cache** (P27) — inline pre-paint script; override **all** `--sx-color-*`; keep theme CSS in hashed build output.
9. **OAuth through Pangolin — redirect_uri_mismatch + state-cookie collision** (P25/26) — distinct callback path outside the Authelia middleware scope, distinct `google.oauth.state` cookie, redirect URI from `EXTERNAL_BASE_URL`.
10. **Token storage — encryption-key reuse + `UNIQUE(user_id)` blocks multi-provider** (P22/25) — widen to `UNIQUE(user_id, provider_type)`; persist only the encrypted refresh token, cache access tokens in memory; minimum scopes.
11. **Dependency-update breaking cascade** (P28) — ESLint held at 9.x (eslint-plugin-react), Schedule-X minor-only until dark mode locked, Drizzle orm+kit bumped together; never bulk `pnpm up`/`audit --fix`.

## Research Flags (deeper investigation during planning)

- **Provider phase:** token-storage shape (`provider_tokens` table vs. extended `member_credentials`) — pick one in P22; verify the `migrator` API behavior against the installed `drizzle-orm` version before shipping auto-migrate.
- **Google phase:** confirm the exact scope tier needed for write-back without Google verification; end-to-end OAuth test over the Pangolin hostname (not localhost); iOS-Safari standalone callback is load-bearing (human gate, like Phase 3).
- **Reminders phase:** confirm the `remindersChanged`/preserve-on-edit contract end-to-end; index the reminder column for the scheduler scan.
- **Dark mode:** enumerate every `--sx-color-*` from `@schedule-x/theme-default`; verify it's in the Workbox precache (hashed).

## Open Questions to Resolve in Requirements / Roadmap

1. Are **dark mode (27)** and **dependency updates (28)** standalone phases or folded into adjacent work? (Both are independent of the provider chain.)
2. Token storage: separate `provider_tokens` table vs. extended `member_credentials` — resolve in P22.
3. Google all-day "9 AM local" reminder semantics can't be replicated (Google fires midnight-minus-lead) — accept as a documented per-provider difference.

## Confidence

| Domain | Confidence | Notes |
| ------ | ---------- | ----- |
| Stack | HIGH | Two scoped Google packages verified 2026-06-19; everything else reuses shipped deps |
| Features | MEDIUM | Google API via official docs; UX patterns cross-checked but LOW-confidence sources |
| Architecture | HIGH | Grounded in direct v1.1 source; component boundaries + build order sound |
| Pitfalls | HIGH | From codebase review + Google API behavior + v1.0/v1.1 migration/CI lessons, each phase-mapped |
| Google OAuth through Pangolin | MEDIUM | Redirect-URI + state-cookie mechanics sound; needs live end-to-end validation |

## Sources

See `STACK.md`, `FEATURES.md`, `ARCHITECTURE.md`, and `PITFALLS.md` in this directory for full citations (Google Calendar API docs, google-auth-library, Drizzle migrator, Schedule-X theming, pnpm audit/outdated, family-calendar UX surveys, and direct codebase inspection of `apps/api/src/broker/`).

---

_Synthesized for: FamilySync — v1.2 Multi-Provider, Theming & Zero-Setup_
_Researched: 2026-06-19_