luckberg/familysync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
67430bb670d00b95d6dae47023c11c9c15474ea2
familysync/.planning/research/SUMMARY.md
T

13 KiB
Raw Blame History

Project Research Summary

Project: FamilySync — v1.2 "Multi-Provider, Theming & Zero-Setup" Domain: Self-hosted family calendar + lists PWA (opening beyond Fastmail to a second provider, on a shipped v1.1) Researched: 2026-06-19 Confidence: HIGH for architecture/pitfalls (direct codebase inspection + shipped v1.0/v1.1 lessons); MEDIUM for Google Calendar API specifics (official docs via Context7); LOW for third-party UX patterns (cross-checked websearch)

Executive Summary

v1.2 opens FamilySync beyond Fastmail. The keystone is a hand-rolled CalendarProvider TypeScript interface: the existing tsdav/ical.js CalDAV broker is refactored behind it as a thin delegation wrapper with zero internal rewrite, then Google Calendar (Google Calendar API v3 + OAuth2, not CalDAV) plugs in as a second implementation, and an in-memory mock provider plugs in as a third (dev-bypass + hermetic CI). On top of that seam: multiple reminders per event (1→N VALARMs / Google overrides), PWA dark mode (Light/Dark/System), and zero-manual-setup DB bootstrap (programmatic migrate() at boot). The milestone closes by applying CI-surfaced dependency updates.

The core stack is unchanged. Two new runtime packages only — google-auth-library@10.7.0 and @googleapis/calendar@15.0.0 (scoped, NOT the 50 MB googleapis monolith), both in apps/api. Dark mode, multiple reminders, migrate-on-boot, and dependency updates add zero new dependencies (Zustand persist, ical.js multi-VALARM, drizzle-orm/mysql2/migrator, and existing pnpm tooling already cover them).

Three non-negotiable preservation rules drive the design: (1) the provider refactor must not regress the live Fastmail path — all v1.1 outbox/poller/scheduler integration tests must pass unchanged against the new interface; (2) the reminder_lead_minutes→reminder_leads JSON migration must migrate existing data before dropping the old column; (3) Google OAuth tokens are per-member secrets, AES-256-GCM encrypted, never in app_config, with the member_credentials unique constraint widened to UNIQUE(user_id, provider_type).

Key Findings

Recommended Stack

No stack change beyond two scoped Google packages (both apps/api):

  • google-auth-library@10.7.0 — OAuth2 authorization-code flow + offline refresh-token management; ships its own types; OAuth2Client auto-refreshes expired access tokens via the tokens event.
  • @googleapis/calendar@15.0.0 — Google Calendar API v3 typed client (depends only on googleapis-common). Reject the monolithic googleapis (170+ clients, ~50 MB).
  • Provider abstraction — hand-rolled TS interface in apps/api/src/broker/; no normalization library exists worth a dependency.
  • Dark mode — pure CSS [data-theme="dark"] + Zustand persist (built into 5.0.14) + an inline <script> in index.html. No theming library.
  • Migrate-on-boot — drizzle-orm/mysql2/migrator (already transitive); single connection, multipleStatements: true. drizzle-kit push stays banned on MariaDB 11.
  • Dependency updates — existing pnpm outdated/audit; selective, pin-aware, one ecosystem at a time.

Expected Features

Table stakes:

  • Google events in the unified color-coded view (read sync), recurrence + all-day correct; per-calendar color extends the Phase 17 routing to Google calendar IDs.
  • Cross-provider event create/edit/delete with a calendar picker that defaults to the member's most-used calendar and shows recognizable names (not provider IDs/emails).
  • Self-service onboarding: Fastmail = app-password form with live CalDAV validation (reuse Phase 10 CredentialSheet); Google = single "Connect Google Calendar" OAuth button.
  • One-tap reconnect on token expiry, surfaced as a banner in the calendar view (not buried in settings) — non-negotiable for the non-technical Apple member.
  • Up to 5 reminders per event (add/remove rows, pre-populate all on edit), serialized to VALARMs / Google overrides, preserving other-client alarms.
  • Light/Dark/System theme toggle in member Settings (per-device localStorage), no flash-of-wrong-theme, clean across every route incl. Schedule-X.
  • Migrate-on-boot before serve(), idempotent, with a DB-readiness wait and fatal-on-failure.
  • Mock provider seeded with recurring/all-day/timed/past/future + reminder events, wired to DEV_AUTH_BYPASS for hermetic CI.

Anti-features (explicitly OUT): two-way Fastmail↔Google mirroring, CalDAV-for-Google, service-account Google auth, tokens in cookies/localStorage, reminder templates/snooze, per-member server-synced theme, custom theme editor, app self-creating the database, Google per-event colorId, googleapis monolith.

Architecture Approach

Grounded in direct codebase inspection. Central seam: broker/provider.ts (CalendarProvider interface) + broker/providerFactory.ts (createProvider(cred) dispatching on member_credentials.provider_type: caldav/google/mock).

  • CalDavProvider wraps existing client.ts/sync.ts/write.ts verbatim — delegation, not refactor (Anti-Pattern 1).
  • poller.ts / outboxWorker.ts become provider-agnostic: createProvider(cred) → provider.listCalendars()/syncCalendar()/dispatchOutboxRow(). The isDraining guard, drainRequested flag, optimistic-202, create-before-delete, and uid:dtstartMs dedup all stay in the worker (provider-agnostic) and must be unchanged.
  • GoogleCalendarProvider — new; inline token refresh (check expires_at before each call, refresh within 5 min of expiry, retry-once on 401). Google OAuth callback mounted pre-auth (like Authelia /callback), at a distinct path with a distinct state cookie, redirect URI built from EXTERNAL_BASE_URL (never req.host).
  • Token storage — research surfaces two options: a separate provider_tokens table (ARCHITECTURE.md) vs. extending member_credentials with UNIQUE(user_id, provider_type) (STACK/PITFALLS). Either way the unique constraint must widen to composite; client_id/client_secret/refresh tokens are env/encrypted, never in app_config. Decide in the provider-abstraction phase.
  • Multiple reminders — calendar_events.reminder_lead_minutes INT → reminder_leads JSON, with a JSON_ARRAY(...) data-migration step before drop; buildVeventString loops N VALARMs; scheduler dedup key → uid:dtstartMs:lead; Google cap = 5 (reject 6+ before write).
  • Dark mode — themeStore.ts (Zustand persist) + inline FOUC script in index.html + [data-theme="dark"] block mapping all --sx-color-* Schedule-X vars + dynamic theme-color meta for iOS.
  • Migrate-on-boot — runMigrationsIfNeeded() in the isMainModule() guard, after the boot guards, before broker workers + serve(); migration_lock sentinel for concurrent-startup safety.
  • Mock provider — broker/mockProvider.ts, in-memory, activated via DEV_AUTH_BYPASS/provider_type='mock'; resets on restart (intentionally stateless).

Recommended Phase Structure (dependency-ordered, continues from v1.1 Phase 20 → starts at Phase 21)

ARCHITECTURE.md's authoritative ordering:

  1. Phase 21 — Zero-Setup DB Bootstrap. runMigrationsIfNeeded() + migration_lock sentinel. No deps; unlocks every later phase to assume auto-migrate. Risk: low.
  2. Phase 22 — Provider Abstraction (CalDAV only, Fastmail unchanged). Interface + factory + CalDavProvider wrapper; poller/outbox use createProvider. Dep: 21. Risk: HIGH — must not regress Fastmail; golden-path integration tests pass unchanged. Includes the UNIQUE(user_id, provider_type) migration.
  3. Phase 23 — Multiple Reminders. reminder_leads JSON + data-migration; multi-VALARM; scheduler multi-lead + dedup-key change; form add/remove. Dep: 22. Risk: MEDIUM (breaking migration).
  4. Phase 24 — Mock Provider (dev/CI). Seeded in-memory provider; factory mock case; Playwright exercises calendar CRUD. Dep: 22. Risk: low.
  5. Phase 25 — Google Calendar Provider. Token storage + googleProvider.ts + OAuth authorize/callback routes + pre-auth mount. Deps: 22, 23 (reminder serialization), 21 (tokens table). Risk: MEDIUM (OAuth edge cases, recurrence/sync model).
  6. Phase 26 — Self-Service Onboarding UI. /api/me/providers GET/DELETE, ProviderConnectSheet, reconnect banner. Dep: 25. Risk: low.
  7. Phase 27 — PWA Dark Mode. FOUC script, dark tokens, themeStore, ThemeToggle. No deps — parallelizable with 22–26. Risk: low.
  8. Phase 28 (or folded) — Dependency Updates (DEP-01). Last; per-ecosystem batches with CI green after each. Risk: low but cascade-prone (ESLint 9.x pin, Schedule-X hold, Drizzle coupling).

THEME-01 (27) and SETUP-05 (21) are independent of the multi-provider chain. The roadmapper should confirm whether dark mode and dependency updates are standalone phases or folded.

Critical Pitfalls (phase-mapped)

  1. Provider refactor regresses the live Fastmail path (P22) — wrong provider_type branch silently stops all write-back; keep caldav the default, change zero existing tests.
  2. Google refresh tokens expire after 7 days in "Testing" publishing status (P25) — use the non-restricted calendar.readonly + calendar.events scopes + Production status to avoid Google verification; add both accounts as test users; store grant timestamp.
  3. Google recurrence ≠ iCalendar RRULE (P25) — fetch with singleEvents: false, fetch modified/cancelled instances separately, map cancellations to EXDATE, reconstruct a VCALENDAR string for the existing expand.ts.
  4. Google syncToken 410 Gone halts sync + full re-sync overwrites un-drained local writes (P25) — catch 410 → drop token → enqueue full re-sync; drain outbox (or skip pending UIDs) before upsert.
  5. Google all-day/timezone differs (P25) — normalize start.dateTime to UTC ...Z (never a TZID without VTIMEZONE); map reminders.useDefault:true → null ("no FamilySync reminder").
  6. Multiple VALARM dedup + Google 5-cap (P23/P25) — dedup key uid:dtstartMs:lead; preserve-vs-replace driven by a remindersChanged flag; reject 6+ reminders before a Google write.
  7. Drizzle auto-migrate on MariaDB 11 — three failure modes (P21) — journal-hash mismatch, false destructive diff (only if push is misused — use the migrator API), concurrent-startup race (the migration_lock sentinel); log each applied migration.
  8. Dark mode FOWT + Schedule-X coverage + SW cache (P27) — inline pre-paint script; override all --sx-color-*; keep theme CSS in hashed build output.
  9. OAuth through Pangolin — redirect_uri_mismatch + state-cookie collision (P25/26) — distinct callback path outside the Authelia middleware scope, distinct google.oauth.state cookie, redirect URI from EXTERNAL_BASE_URL.
  10. Token storage — encryption-key reuse + UNIQUE(user_id) blocks multi-provider (P22/25) — widen to UNIQUE(user_id, provider_type); persist only the encrypted refresh token, cache access tokens in memory; minimum scopes.
  11. Dependency-update breaking cascade (P28) — ESLint held at 9.x (eslint-plugin-react), Schedule-X minor-only until dark mode locked, Drizzle orm+kit bumped together; never bulk pnpm up/audit --fix.

Research Flags (deeper investigation during planning)

  • Provider phase: token-storage shape (provider_tokens table vs. extended member_credentials) — pick one in P22; verify the migrator API behavior against the installed drizzle-orm version before shipping auto-migrate.
  • Google phase: confirm the exact scope tier needed for write-back without Google verification; end-to-end OAuth test over the Pangolin hostname (not localhost); iOS-Safari standalone callback is load-bearing (human gate, like Phase 3).
  • Reminders phase: confirm the remindersChanged/preserve-on-edit contract end-to-end; index the reminder column for the scheduler scan.
  • Dark mode: enumerate every --sx-color-* from @schedule-x/theme-default; verify it's in the Workbox precache (hashed).

Open Questions to Resolve in Requirements / Roadmap

  1. Are dark mode (27) and dependency updates (28) standalone phases or folded into adjacent work? (Both are independent of the provider chain.)
  2. Token storage: separate provider_tokens table vs. extended member_credentials — resolve in P22.
  3. Google all-day "9 AM local" reminder semantics can't be replicated (Google fires midnight-minus-lead) — accept as a documented per-provider difference.

Confidence

Domain Confidence Notes
Stack HIGH Two scoped Google packages verified 2026-06-19; everything else reuses shipped deps
Features MEDIUM Google API via official docs; UX patterns cross-checked but LOW-confidence sources
Architecture HIGH Grounded in direct v1.1 source; component boundaries + build order sound
Pitfalls HIGH From codebase review + Google API behavior + v1.0/v1.1 migration/CI lessons, each phase-mapped
Google OAuth through Pangolin MEDIUM Redirect-URI + state-cookie mechanics sound; needs live end-to-end validation

Sources

See STACK.md, FEATURES.md, ARCHITECTURE.md, and PITFALLS.md in this directory for full citations (Google Calendar API docs, google-auth-library, Drizzle migrator, Schedule-X theming, pnpm audit/outdated, family-calendar UX surveys, and direct codebase inspection of apps/api/src/broker/).

Synthesized for: FamilySync — v1.2 Multi-Provider, Theming & Zero-Setup Researched: 2026-06-19

Reference in New Issue View Git Blame Copy Permalink