familysync/.planning/codebase/ARCHITECTURE.md

Architecture

Analysis Date: 2026-06-09

System Overview

┌─────────────────────────────────────────────────────────────┐
│                   PWA Frontend (React 19)                   │
│ CalendarShell + Schedule-X calendar + EventForm + UI state  │
│ TanStack Query (server state) + Zustand (UI-only state)    │
│ `apps/pwa/src/`                                             │
└────────┬──────────────────────────────────────────────────┬─┘
         │                                                  │
         │ fetch (with credentials)                        │ SSE
         │ (OIDC session cookie)                           │
         ▼                                                  ▼
┌──────────────────────────────────────────────────────────────┐
│     Backend API (Hono + Node.js) — Port 3000                 │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Auth Layer (OIDC + Authelia)                             │ │
│ │ `apps/api/src/auth/middleware.ts`, `devBypass.ts`        │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Route Handlers — Read from MariaDB cache only            │ │
│ │ GET /api/events — windowed occurrences via expand.ts    │ │
│ │ GET /api/me — current user profile + color              │ │
│ │ POST /api/events/create, PATCH /:uid/edit — enqueue     │ │
│ │ DELETE /:uid — enqueue delete to outbox                 │ │
│ │ GET /api/sse/heartbeat — SSE smoke test                 │ │
│ │ `apps/api/src/routes/`                                  │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ DB Layer (Drizzle ORM + mysql2)                          │ │
│ │ Schema: users, member_credentials, calendars,           │ │
│ │ calendar_events, calendar_outbox                         │ │
│ │ `apps/api/src/db/`                                      │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Background Broker (CalDAV sync & write-back)             │ │
│ │ - Poller (5-min): PROPFIND → ctag change detect         │ │
│ │ - Sync (per-cal): REPORT → ical.js → MariaDB upsert    │ │
│ │ - OutboxWorker (15-sec): drain pending writes to        │ │
│ │   Fastmail (PUT/DELETE via tsdav)                       │ │
│ │ `apps/api/src/broker/`                                  │ │
│ └──────────────────────────────────────────────────────────┘ │
└────────┬──────────────────────────────────────────────────┬──┘
         │                                                  │
         └─ Fastmail CalDAV + app passwords ────────────────┘
           (tsdav client, encrypted credentials)
           (PROPFIND, REPORT, PUT, DELETE)

         MariaDB (persistent cache)
         (read on every request, written by broker)

Component Responsibilities

Component	Responsibility	File
CalendarShell	React entry point; wires TanStack Query + Schedule-X + Zustand + event modals	apps/pwa/src/components/CalendarShell.tsx
AppNav	Navigation header/sidebar; responsive (phone/tablet layout)	apps/pwa/src/components/AppNav.tsx
EventDetailPopover	Displays event details; driven by Zustand.openEventId	apps/pwa/src/components/EventDetailPopover.tsx
EventForm	Create/edit event modal; builds payloads for POST/PATCH endpoints	apps/pwa/src/components/EventForm.tsx
SyncStateToast	Toast polling /api/events/sync-status?uid= for optimistic-accept feedback	apps/pwa/src/components/SyncStateToast.tsx
Zustand calendarStore	UI-only state: selectedView, openEventId, eventFormOpen, calendarRange, deleteDialog state	apps/pwa/src/store/calendarStore.ts
TanStack Query hooks	Server state: fetchMe (user profile), fetchEvents (windowed occurrences), fetchSyncStatus	apps/pwa/src/api/client.ts + CalendarShell.tsx
hydrateEvents	Transforms raw CalendarOccurrence[] to Schedule-X CalendarType[] with Temporal dates	apps/pwa/src/lib/hydrateEvents.ts
buildCalendarConfig	Builds Schedule-X calendar configuration (views, plugins, columns)	apps/pwa/src/lib/calendarConfig.ts
OIDC middleware	Protects /api/* routes; 302-redirects unauthenticated requests to Authelia	apps/api/src/auth/middleware.ts
devAuthBypass	Dev-only passthrough auth for local development without Authelia	apps/api/src/auth/devBypass.ts
upsertUser	Identity upsert by (oidc_iss, oidc_sub); auto-assigns color from palette	apps/api/src/auth/user.ts
eventsRouter	GET /api/events (windowed reads), POST/PATCH/DELETE (enqueue outbox), GET /writable-calendars	apps/api/src/routes/events.ts
meRouter	GET /api/me — returns authenticated user profile (id, displayName, color)	apps/api/src/routes/me.ts
sseRouter	GET /api/sse/heartbeat — SSE smoke test for Pangolin proxy validation	apps/api/src/routes/sse.ts
CalDAV Broker	Decrypts credentials, manages tsdav clients, syncs calendars, drains outbox	apps/api/src/broker/*.ts
Poller	5-min background job: PROPFIND → ctag comparison → skip or syncCalendar	apps/api/src/broker/poller.ts
syncCalendar	REPORT → ical.js parse → VEVENT upsert; prunes deleted events	apps/api/src/broker/sync.ts
OutboxWorker	15-sec background job: drain pending outbox rows, build VEVENTs, PUT/DELETE to Fastmail	apps/api/src/broker/outboxWorker.ts
expandOccurrences	Server-side RRULE expansion via ical.js; produces concrete occurrences for UI	apps/api/src/broker/expand.ts

Pattern Overview

Overall: Three-tier monolith (PWA frontend, Node.js backend, MariaDB), split between apps/pwa (React) and apps/api (Hono + broker).

Request-response pattern:

• Frontend reads from MariaDB cache via REST endpoints (GET only)
• Frontend enqueues writes to transactional outbox (POST/PATCH/DELETE return 202 immediately)
• Background broker drains outbox, calls Fastmail CalDAV, updates cache
• Real-time updates via SSE (Phase 4) and/or polling (SyncStateToast for write feedback)

Data ownership pattern:

• Poller owns calendar collection discovery + change detection (D-13 ctag polling)
• syncCalendar owns per-calendar event cache (REPORT → parse → upsert)
• OutboxWorker owns write-back to Fastmail (D-05 transactional outbox)
• Routes own read authorization and ownership checks (T-03-06..T-03-11)

Key Characteristics:

• Events endpoint shares MariaDB cache — no direct Fastmail I/O from routes (T-03-02 broker boundary)
• Write operations use optimistic-accept pattern: 202 + immediate UI response, success confirmed via polling
• All server state in TanStack Query; UI state only in Zustand (clear separation)
• User identity keyed on (oidc_iss, oidc_sub) not email (D-10); color auto-assigned (D-06)
• All-day events stored as DATE, timed events as TIMESTAMP UTC (D-13 schema contract)
• Recurring events expanded server-side (D-09); client receives concrete occurrences only

Layers

Presentation (React PWA):

• Purpose: Display calendar, handle user interactions, manage UI state (view selection, modals, popovers)
• Location: apps/pwa/src/
• Contains: Components (CalendarShell, EventForm, EventDetailPopover, AppNav, SyncStateToast), UI hooks (CalendarShell's useQuery for data, Zustand for view state)
• Depends on: Schedule-X (calendar library), @tanstack/react-query (server state), Zustand (UI state), TanStack utilities
• Used by: Browser tab (Vite dev proxy or production Pangolin tunnel)

API / Route Layer:

• Purpose: Validate requests, enforce authorization (T-03-06..T-03-11), read from cache, enqueue writes
• Location: apps/api/src/routes/
• Contains: Route handlers (events.ts, me.ts, health.ts, sse.ts); Zod schemas for input validation
• Depends on: Hono framework, Drizzle ORM, @hono/zod-validator, auth middleware
• Used by: PWA frontend (fetch with OIDC cookie), load balancer redirects
• Architecture invariant: Routes never import tsdav or call Fastmail directly (T-03-02)

Database / ORM Layer:

• Purpose: Type-safe query building, schema definition, migrations
• Location: apps/api/src/db/
• Contains: Drizzle schema (users, member_credentials, calendars, calendar_events, calendar_outbox), mysql2 client
• Depends on: mysql2 driver, Drizzle ORM
• Used by: All route handlers, broker modules

Broker / Background Worker Layer:

• Purpose: Keep MariaDB calendar cache in sync with Fastmail; drain transactional outbox
• Location: apps/api/src/broker/
• Contains: Poller (5-min cron), syncCalendar (REPORT parse), OutboxWorker (15-sec drain), supporting utilities
• Depends on: tsdav (CalDAV client), ical.js (VEVENT parsing), node-cron (scheduling), Drizzle ORM
• Used by: Scheduled background jobs (started in index.ts only when module is main)
• Data sources: member_credentials (encrypted), calendars, calendar_events (cache), calendar_outbox (pending writes)

Auth / Session Layer:

• Purpose: OIDC authentication via Authelia, user identity upsert, session cookies
• Location: apps/api/src/auth/
• Contains: Middleware (oidcAuthMiddleware, processOAuthCallback from @hono/oidc-auth), upsertUser color assignment, dev bypass
• Depends on: @hono/oidc-auth, Drizzle ORM for user upsert
• Used by: Hono middleware stack, route handlers via getAuth(c) or c.get('user')

Data Flow

Primary Request Path (GET /api/events)

1. Client request — CalendarShell's eventsQuery fires when meQuery succeeds
2. OIDC guard (apps/api/src/auth/middleware.ts:oidcAuthMiddleware) — 302-redirect if unauthenticated; session cookie checked
3. Route handler (apps/api/src/routes/events.ts:eventsRouter.get('/')) — validate start/end dates, resolve userId via getAuth + upsertUser
4. SQL pre-filter — Select from calendar_events JOIN calendars JOIN users; WHERE matches:
   • Ownership: current user's own calendars OR shared-family calendar (isShared=true)
   • Date window: recurring masters (hasRrule=1) OR non-recurring timed (dtstartUtc in range) OR all-day (dtstartDate in range)
5. Expansion (apps/api/src/broker/expand.ts:expandOccurrences) — For each row, parse rawVevent with ical.js, expand RRULE into occurrences, emit CalendarOccurrence[] with stable IDs
6. Response — JSON { occurrences: CalendarOccurrence[] }
7. Client hydration (apps/pwa/src/lib/hydrateEvents.ts) — Convert occurrence ISO strings to Temporal.ZonedDateTime for Schedule-X
8. Schedule-X render — eventsService.set() updates calendar model; re-render with color routing (isShared ? 'shared' : String(ownerUserId))

State Management:

• TanStack Query caches result with key ['events', start, end]; staleTime 5 min
• Zustand calendarRange (start/end) drives query key → navigation re-fetches
• SyncStateToast polls /api/events/sync-status?uid= to show write-back progress

Write Path (POST /api/events/create)

1. User interaction — EventForm.onSubmit calls POST /api/events/create with CreateEventPayload
2. OIDC guard — Session verified
3. Route validation (apps/api/src/routes/events.ts:eventsRouter.post('/create')) — Zod validates payload (title, start, end, location, description, recurrence)
4. Calendar ownership check — If calendarUrl supplied, verify it's owned by currentUser OR isShared; else default to user's first calendar
5. Outbox enqueue — INSERT into calendar_outbox with status='pending', operation='create', uid=randomUUID
6. 202 response — Return immediately with { uid } (optimistic-accept, D-05)
7. UI toast — Zustand setLastSyncedUid; SyncStateToast polls sync-status for this uid
8. Background drain — OutboxWorker (15-sec cron):
   • SELECT outbox WHERE status='pending' AND next_attempt_at <= NOW()
   • Decrypt credential from member_credentials
   • Call /broker/write.ts:createCalendarEvent — builds VEVENT from payload, PUT to Fastmail
   • On 2xx: mark done, trigger targeted sync (syncCalendar) to refetch the calendar
   • On 412 conflict: mark failed (no retry), trigger sync (UI sees server state)
   • On 5xx/408/429: exponential backoff, mark dead after 5 attempts
   • On 400/401/403: mark failed immediately
9. Cache update — syncCalendar upserts calendar_events from REPORT; GET /api/events now includes the new event
10. Client refetch — SyncStateToast sees status='done'; TanStack Query invalidateQueries refetches events

Calendar Sync (Background Poller → syncCalendar)

1. Poller fires — node-cron 5-min schedule calls runPoll()
2. Load credentials — SELECT member_credentials; decrypt each app password (T-03-04 — never log plaintext)
3. Per-credential: Create tsdav client, PROPFIND to discover calendars
4. Per-calendar:
   • Look up known ctag from calendar_events join
   • If ctag unchanged and not null: SKIP (no DB write, no Fastmail round-trip)
   • If ctag changed or null: call syncCalendar
5. syncCalendar (apps/api/src/broker/sync.ts):
   • Upsert calendars row with new ctag/syncToken
   • REPORT (calendar-query) → tsdav.fetchCalendarObjects() → array of { data, etag, url }
   • For each: Parse with ICAL.parse(), extract VEVENT, build dtstartUtc/dtstartDate per schema contract (D-13)
   • Upsert calendar_events with onDuplicateKeyUpdate (idempotency key: calendarId + uid)
   • Prune deletes: DELETE events whose uid is no longer on server (BUG B: scope by (userId, url) for shared account)

Ownership Model (D-03, D-16):

• Shared Fastmail account: both members' credentials fetch the same calendar collections
• Stored as (userId, url) composite unique key so each member caches the same calendar separately
• eventsRouter ownership check: calendar.userId = currentUserId OR isShared=true (writable set)
• poller lookup: AND(userId, url) to fetch the right member's cached version

Key Abstractions

CalendarOccurrence:

• Purpose: Single concrete event occurrence ready for UI (expanded from RRULE if needed)
• Examples: apps/api/src/broker/expand.ts:CalendarOccurrence, apps/pwa/src/api/client.ts:CalendarOccurrence
• Pattern: Backend expands RRULE into N occurrences; each has stable id = ${uid}::${dtstart_iso}, allowing Schedule-X dedup and Zustand.openEventId routing

Transactional Outbox (D-05):

• Purpose: Decouple client request (202 response) from Fastmail write (async worker)
• Examples: apps/api/src/db/schema.ts:calendarOutbox
• Pattern: Write endpoint INSERTs pending row; worker POLLs and drains; status machine (pending → done/failed/dead) controls retry + backoff

Wrapped Schema Contract (D-13):

• Purpose: Guarantee correct DATE vs TIMESTAMP storage for all-day vs timed events
• Examples: apps/api/src/db/schema.ts (dtstartUtc, dtstartDate, allDay); apps/api/src/broker/sync.ts (storage logic); apps/api/src/routes/events.ts (window predicate)
• Pattern: All-day events NEVER coerce to midnight-UTC (Pitfall 2); timed events always UTC; query pre-filters both branches

RRULE Expansion (D-09):

• Purpose: Expand recurring masters server-side so client receives concrete occurrences only
• Examples: apps/api/src/broker/expand.ts:expandOccurrences, apps/pwa/src/lib/hydrateEvents.ts (no expansion on client)
• Pattern: Route calls expandOccurrences for each cached VEVENT; ical.js handles RRULE parsing, EXDATE exclusion, VTIMEZONE DST adjustment

Encrypted Credentials:

• Purpose: Store Fastmail app passwords at rest without exposing plaintext
• Examples: apps/api/src/db/schema.ts:memberCredentials.encryptedPassword, apps/api/src/broker/crypto.ts:decryptPassword
• Pattern: AES-256-GCM with per-message nonce; stored as JSON { iv, authTag, ciphertext }; decrypted only immediately before tsdav client creation (T-03-04)

Entry Points

Browser → PWA:

• Location: apps/pwa/src/main.tsx (Vite SPA entry), apps/pwa/src/App.tsx (root component = CalendarShell)
• Triggers: User navigates to / (domain root) or clicks Home
• Responsibilities: Hydrate React app, mount CalendarShell, wire TanStack Query + Zustand

PWA → API:

• Location: apps/pwa/src/api/client.ts (fetch functions)
• Triggers: CalendarShell useQuery hooks on mount and navigation
• Responsibilities: Fetch events, me profile, sync status; handle OIDC redirects via maybeRedirectToLogin

Unauthenticated User → OIDC:

• Location: apps/api/src/auth/middleware.ts (oidcAuthMiddleware)
• Triggers: Unauthenticated fetch to /api/* endpoint
• Responsibilities: 302-redirect to Authelia /authorize; await callback at /callback; set session JWT cookie

OIDC Callback → API Login:

• Location: apps/api/src/index.ts:app.get('/callback') and apps/api/src/auth/middleware.ts:processOAuthCallback
• Triggers: Authelia POST to /callback after authorization-code exchange
• Responsibilities: Exchange code for token, validate nonce, set JWT cookie with refresh token, redirect to /api/login

API Login → SPA Boot:

• Location: apps/api/src/index.ts:app.get('/api/login')
• Triggers: Top-level navigation after callback redirects here (or direct /api/login hit by PWA)
• Responsibilities: Verify session cookie valid, 302-redirect to / so SPA boots authenticated

Background Poller:

• Location: apps/api/src/broker/poller.ts:startBrokerPoller, called from apps/api/src/index.ts in isMainModule() guard
• Triggers: 5-min node-cron schedule starting at API boot
• Responsibilities: Load all credentials, PROPFIND calendars, compare ctag, call syncCalendar if changed

Outbox Worker:

• Location: apps/api/src/broker/outboxWorker.ts:startOutboxWorker, called from apps/api/src/index.ts in isMainModule() guard
• Triggers: 15-sec node-cron schedule starting at API boot
• Responsibilities: Poll outbox WHERE status='pending', drain to Fastmail via write.ts, update status, trigger refetch

Architectural Constraints

• Threading: Single-threaded event loop (Node.js). Broker poller and outbox worker run in the same process; scheduled tasks do not block request handling.
• Global state: None in routes (all state passed via c context). Broker modules keep DB client as singleton. tsdav clients created per-credential per-poll (not cached).
• Circular imports: None detected. Routes import from routes only; broker imports from db + auth; auth imports from db; no cycles.
• Request handling: Synchronous route completion (routes do not wait for broker background tasks). Writes are optimistic-accept (202); client polls for confirmation.
• Session cookies: Signed JWT stored in httpOnly cookie; refresh token included in JWT payload; @hono/oidc-auth handles rotation every 15 min by default.
• Shared Fastmail account: Both members' credentials fetch the same calendar collections. Ownership tracked per-user via (userId, url) composite key to avoid cross-member cache contamination (BUG B fix).
• Database transactions: Explicit tx() used for edit-as-move (D-04) — delete + create pair atomic. All other operations single-statement (upserts via onDuplicateKeyUpdate).

Anti-Patterns

Direct Fastmail calls from routes

What happens: Routes call tsdav or make fetch requests directly to Fastmail CalDAV endpoints Why it's wrong: Routes would block on network I/O; Fastmail errors would fail the request immediately instead of retrying via outbox; credential decryption happens on every request instead of once per poller cycle; no centralized write ordering (concurrent POSTs can collide) Do this instead: Routes enqueue outbox rows (202) and let broker handle Fastmail I/O. See apps/api/src/routes/events.ts:eventsRouter.post('/create') and apps/api/src/routes/events.ts:eventsRouter.delete('/:uid') — both INSERT outbox, never call tsdav.

Storing displayName as identity key

What happens: User row lookup is by email or displayName instead of OIDC issuer+subject Why it's wrong: Email changes (user migrates providers); displayName is user-editable and can collide (two Lucases). If Authelia email claim changes mid-login, the user gets a duplicate row. Do this instead: Key by (oidc_iss, oidc_sub) composite, never email. See apps/api/src/auth/user.ts:upsertUser — identity lookup is always by (oidcIss, oidcSub), then displayName is updated as a display hint on re-upsert.

Caching tsdav clients across polls

What happens: Broker reuses the same tsdav client instance for multiple credential sessions Why it's wrong: DAVClient maintains HTTP connection state; reusing across credential changes can cross-contaminate requests or leak auth headers. Do this instead: Create a fresh client per credential per poll. See apps/api/src/broker/poller.ts:runPoll — each credential iteration calls createFastmailClient() fresh.

Windowed event query without pre-filter for recurring masters

What happens: SQL query only selects non-recurring events in the date window; recurring masters are not included Why it's wrong: A weekly meeting created 3 years ago has dtstartUtc < window start, so it's filtered out. But it has RRULE so it has occurrences in the window (RESEARCH.md Pitfall 5). Do this instead: OR-combine three sub-predicates: (1) non-recurring timed in window, (2) non-recurring all-day in window, (3) recurring masters with dtstartUtc < windowEnd. See apps/api/src/routes/events.ts lines 173–200 for the full predicate.

Storing all-day events as midnight-UTC datetime

What happens: All-day event is stored as '2026-06-01T00:00:00Z' (datetime) instead of '2026-06-01' (date) Why it's wrong: When the viewer is in a different timezone (e.g., UTC-04:00), the date column renders as 2026-05-31 (one day off). Timezone conversion applies to DATETIME but not DATE. Do this instead: Store all-day events in the DATE column only; timed events in TIMESTAMP UTC. See apps/api/src/db/schema.ts (dtstartUtc vs dtstartDate) and apps/api/src/broker/sync.ts lines 100–112 for the schema contract enforcement.

Relying on 200 response to mean write success

What happens: Route marks an event as written and notifies the client success before verifying the outbox row completed Why it's wrong: Client UI state gets out of sync with server; if the outbox worker later fails, the client never knows. Do this instead: Return 202 Accepted immediately, then client polls /api/events/sync-status?uid= to track the outbox status. See apps/api/src/routes/events.ts:eventsRouter.post('/create') returns 202, and apps/pwa/src/components/SyncStateToast.tsx polls until done/failed/dead.

---

Architecture analysis: 2026-06-09