familysync/.planning/codebase/ARCHITECTURE.md

<!-- refreshed: 2026-06-09 -->

# Architecture

**Analysis Date:** 2026-06-09

## System Overview

```text
┌─────────────────────────────────────────────────────────────┐
│                   PWA Frontend (React 19)                   │
│ CalendarShell + Schedule-X calendar + EventForm + UI state  │
│ TanStack Query (server state) + Zustand (UI-only state)    │
│ `apps/pwa/src/`                                             │
└────────┬──────────────────────────────────────────────────┬─┘
         │                                                  │
         │ fetch (with credentials)                        │ SSE
         │ (OIDC session cookie)                           │
         ▼                                                  ▼
┌──────────────────────────────────────────────────────────────┐
│     Backend API (Hono + Node.js) — Port 3000                 │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Auth Layer (OIDC + Authelia)                             │ │
│ │ `apps/api/src/auth/middleware.ts`, `devBypass.ts`        │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Route Handlers — Read from MariaDB cache only            │ │
│ │ GET /api/events — windowed occurrences via expand.ts    │ │
│ │ GET /api/me — current user profile + color              │ │
│ │ POST /api/events/create, PATCH /:uid/edit — enqueue     │ │
│ │ DELETE /:uid — enqueue delete to outbox                 │ │
│ │ GET /api/sse/heartbeat — SSE smoke test                 │ │
│ │ `apps/api/src/routes/`                                  │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ DB Layer (Drizzle ORM + mysql2)                          │ │
│ │ Schema: users, member_credentials, calendars,           │ │
│ │ calendar_events, calendar_outbox                         │ │
│ │ `apps/api/src/db/`                                      │ │
│ └──────────────────────────────────────────────────────────┘ │
│ ┌──────────────────────────────────────────────────────────┐ │
│ │ Background Broker (CalDAV sync & write-back)             │ │
│ │ - Poller (5-min): PROPFIND → ctag change detect         │ │
│ │ - Sync (per-cal): REPORT → ical.js → MariaDB upsert    │ │
│ │ - OutboxWorker (15-sec): drain pending writes to        │ │
│ │   Fastmail (PUT/DELETE via tsdav)                       │ │
│ │ `apps/api/src/broker/`                                  │ │
│ └──────────────────────────────────────────────────────────┘ │
└────────┬──────────────────────────────────────────────────┬──┘
         │                                                  │
         └─ Fastmail CalDAV + app passwords ────────────────┘
           (tsdav client, encrypted credentials)
           (PROPFIND, REPORT, PUT, DELETE)

         MariaDB (persistent cache)
         (read on every request, written by broker)
```

## Component Responsibilities

| Component                 | Responsibility                                                                                | File                                             |
| ------------------------- | --------------------------------------------------------------------------------------------- | ------------------------------------------------ |
| **CalendarShell**         | React entry point; wires TanStack Query + Schedule-X + Zustand + event modals                 | `apps/pwa/src/components/CalendarShell.tsx`      |
| **AppNav**                | Navigation header/sidebar; responsive (phone/tablet layout)                                   | `apps/pwa/src/components/AppNav.tsx`             |
| **EventDetailPopover**    | Displays event details; driven by Zustand.openEventId                                         | `apps/pwa/src/components/EventDetailPopover.tsx` |
| **EventForm**             | Create/edit event modal; builds payloads for POST/PATCH endpoints                             | `apps/pwa/src/components/EventForm.tsx`          |
| **SyncStateToast**        | Toast polling `/api/events/sync-status?uid=` for optimistic-accept feedback                   | `apps/pwa/src/components/SyncStateToast.tsx`     |
| **Zustand calendarStore** | UI-only state: selectedView, openEventId, eventFormOpen, calendarRange, deleteDialog state    | `apps/pwa/src/store/calendarStore.ts`            |
| **TanStack Query hooks**  | Server state: fetchMe (user profile), fetchEvents (windowed occurrences), fetchSyncStatus     | `apps/pwa/src/api/client.ts` + CalendarShell.tsx |
| **hydrateEvents**         | Transforms raw CalendarOccurrence[] to Schedule-X CalendarType[] with Temporal dates          | `apps/pwa/src/lib/hydrateEvents.ts`              |
| **buildCalendarConfig**   | Builds Schedule-X calendar configuration (views, plugins, columns)                            | `apps/pwa/src/lib/calendarConfig.ts`             |
| **OIDC middleware**       | Protects /api/\* routes; 302-redirects unauthenticated requests to Authelia                   | `apps/api/src/auth/middleware.ts`                |
| **devAuthBypass**         | Dev-only passthrough auth for local development without Authelia                              | `apps/api/src/auth/devBypass.ts`                 |
| **upsertUser**            | Identity upsert by (oidc_iss, oidc_sub); auto-assigns color from palette                      | `apps/api/src/auth/user.ts`                      |
| **eventsRouter**          | GET /api/events (windowed reads), POST/PATCH/DELETE (enqueue outbox), GET /writable-calendars | `apps/api/src/routes/events.ts`                  |
| **meRouter**              | GET /api/me — returns authenticated user profile (id, displayName, color)                     | `apps/api/src/routes/me.ts`                      |
| **sseRouter**             | GET /api/sse/heartbeat — SSE smoke test for Pangolin proxy validation                         | `apps/api/src/routes/sse.ts`                     |
| **CalDAV Broker**         | Decrypts credentials, manages tsdav clients, syncs calendars, drains outbox                   | `apps/api/src/broker/*.ts`                       |
| **Poller**                | 5-min background job: PROPFIND → ctag comparison → skip or syncCalendar                       | `apps/api/src/broker/poller.ts`                  |
| **syncCalendar**          | REPORT → ical.js parse → VEVENT upsert; prunes deleted events                                 | `apps/api/src/broker/sync.ts`                    |
| **OutboxWorker**          | 15-sec background job: drain pending outbox rows, build VEVENTs, PUT/DELETE to Fastmail       | `apps/api/src/broker/outboxWorker.ts`            |
| **expandOccurrences**     | Server-side RRULE expansion via ical.js; produces concrete occurrences for UI                 | `apps/api/src/broker/expand.ts`                  |

## Pattern Overview

**Overall:** Three-tier monolith (PWA frontend, Node.js backend, MariaDB), split between `apps/pwa` (React) and `apps/api` (Hono + broker).

**Request-response pattern:**

- Frontend reads from MariaDB cache via REST endpoints (GET only)
- Frontend enqueues writes to transactional outbox (POST/PATCH/DELETE return 202 immediately)
- Background broker drains outbox, calls Fastmail CalDAV, updates cache
- Real-time updates via SSE (Phase 4) and/or polling (SyncStateToast for write feedback)

**Data ownership pattern:**

- Poller owns calendar collection discovery + change detection (D-13 ctag polling)
- syncCalendar owns per-calendar event cache (REPORT → parse → upsert)
- OutboxWorker owns write-back to Fastmail (D-05 transactional outbox)
- Routes own read authorization and ownership checks (T-03-06..T-03-11)

**Key Characteristics:**

- Events endpoint shares MariaDB cache — no direct Fastmail I/O from routes (T-03-02 broker boundary)
- Write operations use optimistic-accept pattern: 202 + immediate UI response, success confirmed via polling
- All server state in TanStack Query; UI state only in Zustand (clear separation)
- User identity keyed on (oidc_iss, oidc_sub) not email (D-10); color auto-assigned (D-06)
- All-day events stored as DATE, timed events as TIMESTAMP UTC (D-13 schema contract)
- Recurring events expanded server-side (D-09); client receives concrete occurrences only

## Layers

**Presentation (React PWA):**

- Purpose: Display calendar, handle user interactions, manage UI state (view selection, modals, popovers)
- Location: `apps/pwa/src/`
- Contains: Components (CalendarShell, EventForm, EventDetailPopover, AppNav, SyncStateToast), UI hooks (CalendarShell's useQuery for data, Zustand for view state)
- Depends on: Schedule-X (calendar library), @tanstack/react-query (server state), Zustand (UI state), TanStack utilities
- Used by: Browser tab (Vite dev proxy or production Pangolin tunnel)

**API / Route Layer:**

- Purpose: Validate requests, enforce authorization (T-03-06..T-03-11), read from cache, enqueue writes
- Location: `apps/api/src/routes/`
- Contains: Route handlers (events.ts, me.ts, health.ts, sse.ts); Zod schemas for input validation
- Depends on: Hono framework, Drizzle ORM, @hono/zod-validator, auth middleware
- Used by: PWA frontend (fetch with OIDC cookie), load balancer redirects
- Architecture invariant: Routes **never** import tsdav or call Fastmail directly (T-03-02)

**Database / ORM Layer:**

- Purpose: Type-safe query building, schema definition, migrations
- Location: `apps/api/src/db/`
- Contains: Drizzle schema (users, member_credentials, calendars, calendar_events, calendar_outbox), mysql2 client
- Depends on: mysql2 driver, Drizzle ORM
- Used by: All route handlers, broker modules

**Broker / Background Worker Layer:**

- Purpose: Keep MariaDB calendar cache in sync with Fastmail; drain transactional outbox
- Location: `apps/api/src/broker/`
- Contains: Poller (5-min cron), syncCalendar (REPORT parse), OutboxWorker (15-sec drain), supporting utilities
- Depends on: tsdav (CalDAV client), ical.js (VEVENT parsing), node-cron (scheduling), Drizzle ORM
- Used by: Scheduled background jobs (started in index.ts only when module is main)
- Data sources: member_credentials (encrypted), calendars, calendar_events (cache), calendar_outbox (pending writes)

**Auth / Session Layer:**

- Purpose: OIDC authentication via Authelia, user identity upsert, session cookies
- Location: `apps/api/src/auth/`
- Contains: Middleware (oidcAuthMiddleware, processOAuthCallback from @hono/oidc-auth), upsertUser color assignment, dev bypass
- Depends on: @hono/oidc-auth, Drizzle ORM for user upsert
- Used by: Hono middleware stack, route handlers via getAuth(c) or c.get('user')

## Data Flow

### Primary Request Path (GET /api/events)

1. **Client request** — CalendarShell's eventsQuery fires when meQuery succeeds
2. **OIDC guard** (`apps/api/src/auth/middleware.ts:oidcAuthMiddleware`) — 302-redirect if unauthenticated; session cookie checked
3. **Route handler** (`apps/api/src/routes/events.ts:eventsRouter.get('/')`) — validate start/end dates, resolve userId via getAuth + upsertUser
4. **SQL pre-filter** — Select from calendar_events JOIN calendars JOIN users; WHERE matches:
   - Ownership: current user's own calendars OR shared-family calendar (isShared=true)
   - Date window: recurring masters (hasRrule=1) OR non-recurring timed (dtstartUtc in range) OR all-day (dtstartDate in range)
5. **Expansion** (`apps/api/src/broker/expand.ts:expandOccurrences`) — For each row, parse rawVevent with ical.js, expand RRULE into occurrences, emit CalendarOccurrence[] with stable IDs
6. **Response** — JSON { occurrences: CalendarOccurrence[] }
7. **Client hydration** (`apps/pwa/src/lib/hydrateEvents.ts`) — Convert occurrence ISO strings to Temporal.ZonedDateTime for Schedule-X
8. **Schedule-X render** — eventsService.set() updates calendar model; re-render with color routing (isShared ? 'shared' : String(ownerUserId))

**State Management:**

- TanStack Query caches result with key ['events', start, end]; staleTime 5 min
- Zustand calendarRange (start/end) drives query key → navigation re-fetches
- SyncStateToast polls `/api/events/sync-status?uid=` to show write-back progress

### Write Path (POST /api/events/create)

1. **User interaction** — EventForm.onSubmit calls POST /api/events/create with CreateEventPayload
2. **OIDC guard** — Session verified
3. **Route validation** (`apps/api/src/routes/events.ts:eventsRouter.post('/create')`) — Zod validates payload (title, start, end, location, description, recurrence)
4. **Calendar ownership check** — If calendarUrl supplied, verify it's owned by currentUser OR isShared; else default to user's first calendar
5. **Outbox enqueue** — INSERT into calendar_outbox with status='pending', operation='create', uid=randomUUID
6. **202 response** — Return immediately with { uid } (optimistic-accept, D-05)
7. **UI toast** — Zustand setLastSyncedUid; SyncStateToast polls sync-status for this uid
8. **Background drain** — OutboxWorker (15-sec cron):
   - SELECT outbox WHERE status='pending' AND next_attempt_at <= NOW()
   - Decrypt credential from member_credentials
   - Call `/broker/write.ts:createCalendarEvent` — builds VEVENT from payload, PUT to Fastmail
   - On 2xx: mark done, trigger targeted sync (syncCalendar) to refetch the calendar
   - On 412 conflict: mark failed (no retry), trigger sync (UI sees server state)
   - On 5xx/408/429: exponential backoff, mark dead after 5 attempts
   - On 400/401/403: mark failed immediately
9. **Cache update** — syncCalendar upserts calendar_events from REPORT; GET /api/events now includes the new event
10. **Client refetch** — SyncStateToast sees status='done'; TanStack Query invalidateQueries refetches events

### Calendar Sync (Background Poller → syncCalendar)

1. **Poller fires** — node-cron 5-min schedule calls runPoll()
2. **Load credentials** — SELECT member_credentials; decrypt each app password (T-03-04 — never log plaintext)
3. **Per-credential**: Create tsdav client, PROPFIND to discover calendars
4. **Per-calendar**:
   - Look up known ctag from calendar_events join
   - If ctag unchanged and not null: SKIP (no DB write, no Fastmail round-trip)
   - If ctag changed or null: call syncCalendar
5. **syncCalendar** (`apps/api/src/broker/sync.ts`):
   - Upsert calendars row with new ctag/syncToken
   - REPORT (calendar-query) → tsdav.fetchCalendarObjects() → array of { data, etag, url }
   - For each: Parse with ICAL.parse(), extract VEVENT, build dtstartUtc/dtstartDate per schema contract (D-13)
   - Upsert calendar_events with onDuplicateKeyUpdate (idempotency key: calendarId + uid)
   - Prune deletes: DELETE events whose uid is no longer on server (BUG B: scope by (userId, url) for shared account)

**Ownership Model (D-03, D-16):**

- Shared Fastmail account: both members' credentials fetch the same calendar collections
- Stored as (userId, url) composite unique key so each member caches the same calendar separately
- eventsRouter ownership check: calendar.userId = currentUserId OR isShared=true (writable set)
- poller lookup: AND(userId, url) to fetch the right member's cached version

## Key Abstractions

**CalendarOccurrence:**

- Purpose: Single concrete event occurrence ready for UI (expanded from RRULE if needed)
- Examples: `apps/api/src/broker/expand.ts:CalendarOccurrence`, `apps/pwa/src/api/client.ts:CalendarOccurrence`
- Pattern: Backend expands RRULE into N occurrences; each has stable id = `${uid}::${dtstart_iso}`, allowing Schedule-X dedup and Zustand.openEventId routing

**Transactional Outbox (D-05):**

- Purpose: Decouple client request (202 response) from Fastmail write (async worker)
- Examples: `apps/api/src/db/schema.ts:calendarOutbox`
- Pattern: Write endpoint INSERTs pending row; worker POLLs and drains; status machine (pending → done/failed/dead) controls retry + backoff

**Wrapped Schema Contract (D-13):**

- Purpose: Guarantee correct DATE vs TIMESTAMP storage for all-day vs timed events
- Examples: `apps/api/src/db/schema.ts` (dtstartUtc, dtstartDate, allDay); `apps/api/src/broker/sync.ts` (storage logic); `apps/api/src/routes/events.ts` (window predicate)
- Pattern: All-day events NEVER coerce to midnight-UTC (Pitfall 2); timed events always UTC; query pre-filters both branches

**RRULE Expansion (D-09):**

- Purpose: Expand recurring masters server-side so client receives concrete occurrences only
- Examples: `apps/api/src/broker/expand.ts:expandOccurrences`, `apps/pwa/src/lib/hydrateEvents.ts` (no expansion on client)
- Pattern: Route calls expandOccurrences for each cached VEVENT; ical.js handles RRULE parsing, EXDATE exclusion, VTIMEZONE DST adjustment

**Encrypted Credentials:**

- Purpose: Store Fastmail app passwords at rest without exposing plaintext
- Examples: `apps/api/src/db/schema.ts:memberCredentials.encryptedPassword`, `apps/api/src/broker/crypto.ts:decryptPassword`
- Pattern: AES-256-GCM with per-message nonce; stored as JSON { iv, authTag, ciphertext }; decrypted only immediately before tsdav client creation (T-03-04)

## Entry Points

**Browser → PWA:**

- Location: `apps/pwa/src/main.tsx` (Vite SPA entry), `apps/pwa/src/App.tsx` (root component = CalendarShell)
- Triggers: User navigates to / (domain root) or clicks Home
- Responsibilities: Hydrate React app, mount CalendarShell, wire TanStack Query + Zustand

**PWA → API:**

- Location: `apps/pwa/src/api/client.ts` (fetch functions)
- Triggers: CalendarShell useQuery hooks on mount and navigation
- Responsibilities: Fetch events, me profile, sync status; handle OIDC redirects via maybeRedirectToLogin

**Unauthenticated User → OIDC:**

- Location: `apps/api/src/auth/middleware.ts` (oidcAuthMiddleware)
- Triggers: Unauthenticated fetch to /api/\* endpoint
- Responsibilities: 302-redirect to Authelia /authorize; await callback at /callback; set session JWT cookie

**OIDC Callback → API Login:**

- Location: `apps/api/src/index.ts:app.get('/callback')` and `apps/api/src/auth/middleware.ts:processOAuthCallback`
- Triggers: Authelia POST to /callback after authorization-code exchange
- Responsibilities: Exchange code for token, validate nonce, set JWT cookie with refresh token, redirect to /api/login

**API Login → SPA Boot:**

- Location: `apps/api/src/index.ts:app.get('/api/login')`
- Triggers: Top-level navigation after callback redirects here (or direct /api/login hit by PWA)
- Responsibilities: Verify session cookie valid, 302-redirect to / so SPA boots authenticated

**Background Poller:**

- Location: `apps/api/src/broker/poller.ts:startBrokerPoller`, called from `apps/api/src/index.ts` in isMainModule() guard
- Triggers: 5-min node-cron schedule starting at API boot
- Responsibilities: Load all credentials, PROPFIND calendars, compare ctag, call syncCalendar if changed

**Outbox Worker:**

- Location: `apps/api/src/broker/outboxWorker.ts:startOutboxWorker`, called from `apps/api/src/index.ts` in isMainModule() guard
- Triggers: 15-sec node-cron schedule starting at API boot
- Responsibilities: Poll outbox WHERE status='pending', drain to Fastmail via write.ts, update status, trigger refetch

## Architectural Constraints

- **Threading:** Single-threaded event loop (Node.js). Broker poller and outbox worker run in the same process; scheduled tasks do not block request handling.
- **Global state:** None in routes (all state passed via c context). Broker modules keep DB client as singleton. tsdav clients created per-credential per-poll (not cached).
- **Circular imports:** None detected. Routes import from routes only; broker imports from db + auth; auth imports from db; no cycles.
- **Request handling:** Synchronous route completion (routes do not wait for broker background tasks). Writes are optimistic-accept (202); client polls for confirmation.
- **Session cookies:** Signed JWT stored in httpOnly cookie; refresh token included in JWT payload; @hono/oidc-auth handles rotation every 15 min by default.
- **Shared Fastmail account:** Both members' credentials fetch the same calendar collections. Ownership tracked per-user via (userId, url) composite key to avoid cross-member cache contamination (BUG B fix).
- **Database transactions:** Explicit tx() used for edit-as-move (D-04) — delete + create pair atomic. All other operations single-statement (upserts via onDuplicateKeyUpdate).

## Anti-Patterns

### Direct Fastmail calls from routes

**What happens:** Routes call tsdav or make fetch requests directly to Fastmail CalDAV endpoints
**Why it's wrong:** Routes would block on network I/O; Fastmail errors would fail the request immediately instead of retrying via outbox; credential decryption happens on every request instead of once per poller cycle; no centralized write ordering (concurrent POSTs can collide)
**Do this instead:** Routes enqueue outbox rows (202) and let broker handle Fastmail I/O. See `apps/api/src/routes/events.ts:eventsRouter.post('/create')` and `apps/api/src/routes/events.ts:eventsRouter.delete('/:uid')` — both INSERT outbox, never call tsdav.

### Storing displayName as identity key

**What happens:** User row lookup is by email or displayName instead of OIDC issuer+subject
**Why it's wrong:** Email changes (user migrates providers); displayName is user-editable and can collide (two Lucases). If Authelia email claim changes mid-login, the user gets a duplicate row.
**Do this instead:** Key by (oidc_iss, oidc_sub) composite, never email. See `apps/api/src/auth/user.ts:upsertUser` — identity lookup is always by (oidcIss, oidcSub), then displayName is updated as a display hint on re-upsert.

### Caching tsdav clients across polls

**What happens:** Broker reuses the same tsdav client instance for multiple credential sessions
**Why it's wrong:** DAVClient maintains HTTP connection state; reusing across credential changes can cross-contaminate requests or leak auth headers.
**Do this instead:** Create a fresh client per credential per poll. See `apps/api/src/broker/poller.ts:runPoll` — each credential iteration calls `createFastmailClient()` fresh.

### Windowed event query without pre-filter for recurring masters

**What happens:** SQL query only selects non-recurring events in the date window; recurring masters are not included
**Why it's wrong:** A weekly meeting created 3 years ago has dtstartUtc < window start, so it's filtered out. But it has RRULE so it has occurrences in the window (RESEARCH.md Pitfall 5).
**Do this instead:** OR-combine three sub-predicates: (1) non-recurring timed in window, (2) non-recurring all-day in window, (3) recurring masters with dtstartUtc < windowEnd. See `apps/api/src/routes/events.ts` lines 173–200 for the full predicate.

### Storing all-day events as midnight-UTC datetime

**What happens:** All-day event is stored as '2026-06-01T00:00:00Z' (datetime) instead of '2026-06-01' (date)
**Why it's wrong:** When the viewer is in a different timezone (e.g., UTC-04:00), the date column renders as 2026-05-31 (one day off). Timezone conversion applies to DATETIME but not DATE.
**Do this instead:** Store all-day events in the DATE column only; timed events in TIMESTAMP UTC. See `apps/api/src/db/schema.ts` (dtstartUtc vs dtstartDate) and `apps/api/src/broker/sync.ts` lines 100–112 for the schema contract enforcement.

### Relying on 200 response to mean write success

**What happens:** Route marks an event as written and notifies the client success before verifying the outbox row completed
**Why it's wrong:** Client UI state gets out of sync with server; if the outbox worker later fails, the client never knows.
**Do this instead:** Return 202 Accepted immediately, then client polls `/api/events/sync-status?uid=` to track the outbox status. See `apps/api/src/routes/events.ts:eventsRouter.post('/create')` returns 202, and `apps/pwa/src/components/SyncStateToast.tsx` polls until done/failed/dead.

---

_Architecture analysis: 2026-06-09_