Lucas BergerandClaude Opus 4.8
6e1c9ca924
- STACK.md: google-auth-library@10.7.0 + @googleapis/calendar@15.0.0 scoped packages (vs monolithic googleapis), OAuth2 flow, token storage, Google Calendar API event/reminder model - FEATURES.md: 6 feature categories (multi-provider, self-service onboarding, multiple reminders, dark mode, zero-setup DB, dev/CI stub), dependency graph, feature prioritization - ARCHITECTURE.md: CalendarProvider interface, provider factory, CalDavProvider wrapper, GoogleCalendarProvider, MockProvider, provider_tokens table schema, multi-reminder JSON column, OAuth callback routing, 7-component data flows - PITFALLS.md: 11 critical/medium pitfalls (refresh token 7-day expiry in testing status, Google recurrence mismatch, syncToken 410, timezone handling, provider abstraction regression, VALARM dedup key, auto-migrate failures, dark mode FOWT, OAuth callback through tunnel, token encryption, ESLint 10 breaking changes) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
592 lines
32 KiB
Markdown
592 lines
32 KiB
Markdown
# Architecture Research
|
|
|
|
**Domain:** v1.2 Integration Architecture — Multi-Provider, Theming & Zero-Setup
|
|
**Researched:** 2026-06-19
|
|
**Confidence:** HIGH (based on direct codebase inspection)
|
|
|
|
---
|
|
|
|
## Context: What This Document Covers
|
|
|
|
This file is scoped to how the v1.2 features integrate with the existing v1.1 codebase. It does not re-describe the existing architecture (see `docs/ARCHITECTURE.md`). Every component listed here is either NEW or MODIFIED; existing components that are untouched are mentioned only as integration anchor points.
|
|
|
|
---
|
|
|
|
## Feature Integration Map
|
|
|
|
### 1. Provider Abstraction
|
|
|
|
**Goal:** Refactor the existing Fastmail/CalDAV broker behind a `CalendarProvider` interface so Google Calendar (and future providers) plug in without touching the poller, outbox worker, or reminder scheduler.
|
|
|
|
#### New Component: `broker/provider.ts` (NEW)
|
|
|
|
This is the central seam. Define a TypeScript interface that both the CalDAV broker and the Google provider implement:
|
|
|
|
```typescript
|
|
export interface CalendarProvider {
|
|
/** Discover and return all calendars this credential can access */
|
|
listCalendars(): Promise<ProviderCalendar[]>;
|
|
|
|
/** Sync events for one calendar into the DB (upsert). Returns detected changes. */
|
|
syncCalendar(
|
|
calendar: ProviderCalendar,
|
|
onChanges?: (changes: CalendarChange[]) => void
|
|
): Promise<void>;
|
|
|
|
/** Write back one pending outbox row. Returns HTTP-like status (200=success, 412=conflict, etc.) */
|
|
dispatchOutboxRow(row: OutboxRow): Promise<DispatchResult>;
|
|
|
|
/** Serialize N reminder leads into the provider's native format (VALARMs or Google overrides) */
|
|
serializeReminders(leads: number[], allDay: boolean): unknown;
|
|
|
|
/** Parse the provider's reminder representation back into lead-minutes array */
|
|
parseReminders(nativePayload: unknown): number[];
|
|
}
|
|
|
|
export interface ProviderCalendar {
|
|
url: string; // CalDAV URL or Google calendarId -- the DB key
|
|
displayName: string | null;
|
|
color: string | null;
|
|
changeKey: string | null; // ctag (CalDAV) or syncToken (Google)
|
|
}
|
|
```
|
|
|
|
**Key constraint:** The `dispatchOutboxRow` method signature must absorb all the logic currently split across `outboxWorker.ts` `dispatchRow()` + `write.ts`. The outbox worker becomes provider-agnostic: it calls `provider.dispatchOutboxRow(row)` and handles the result status uniformly.
|
|
|
|
#### Where Provider Selection Lives
|
|
|
|
Provider selection is **per-member-credential**, not per-calendar. The existing `member_credentials.provider_type` column (`'caldav'` default) is already the discriminator. The poller and outbox worker load all `member_credentials` rows, read `provider_type`, and instantiate the right provider implementation.
|
|
|
|
A factory function in `broker/providerFactory.ts` (NEW) maps `provider_type` to a `CalendarProvider`:
|
|
|
|
```typescript
|
|
export function createProvider(cred: MemberCredential): CalendarProvider {
|
|
switch (cred.providerType) {
|
|
case 'caldav': return new CalDavProvider(cred);
|
|
case 'google': return new GoogleCalendarProvider(cred);
|
|
case 'mock': return new MockCalendarProvider(); // dev-bypass / tests
|
|
default: throw new Error(`Unknown provider type: ${cred.providerType}`);
|
|
}
|
|
}
|
|
```
|
|
|
|
#### Modified: `broker/poller.ts`
|
|
|
|
Currently calls `createFastmailClient` + `syncCalendar` directly. Must change to:
|
|
1. Load each `memberCredentials` row with its `provider_type`.
|
|
2. Call `createProvider(cred)` to get a `CalendarProvider`.
|
|
3. Call `provider.listCalendars()` and compare `changeKey` (was `ctag`).
|
|
4. On change, call `provider.syncCalendar(calendar, onChanges)`.
|
|
|
|
The existing ctag/syncToken logic in `runPoll` moves into each provider's `listCalendars()` implementation. The poller loop structure stays identical.
|
|
|
|
**Risk mitigation for Fastmail path:** Wrap the new `CalDavProvider` around the existing `client.ts` + `sync.ts` + `write.ts` logic verbatim -- do not refactor the internals of those files in the same phase. The provider interface is a thin delegation layer. Existing tests for `runPoll` can be updated to mock `createProvider` instead of `createFastmailClient`.
|
|
|
|
#### Modified: `broker/outboxWorker.ts`
|
|
|
|
`loadClientForUser` becomes `loadProviderForUser` (renamed, same shape). `dispatchRow` delegates to `provider.dispatchOutboxRow(row)` for the actual CalDAV/Google write. The retry/backoff/dead-letter state machine and the `isDraining` concurrency guard stay in the outbox worker -- those are provider-agnostic concerns.
|
|
|
|
The `buildVeventString` + RRULE/VALARM logic currently inlined in `dispatchRow` moves into `CalDavProvider.dispatchOutboxRow`. The outbox worker only knows about `DispatchResult`.
|
|
|
|
#### Modified: `broker/reminderScheduler.ts`
|
|
|
|
The reminder scheduler queries `calendarEvents` from MariaDB and calls `dispatchPush` -- it is **already provider-agnostic**. No structural change needed. The only change: the `reminderLeadMinutes` column on `calendar_events` becomes `JSON` (see Feature 4 below) and the scheduler query updates accordingly.
|
|
|
|
#### Normalized Internal Event Model
|
|
|
|
The `calendarEvents` table row is the internal canonical form that all providers must write to. Providers translate their native format into this DB shape on sync:
|
|
|
|
| Provider field | Internal DB column |
|
|
|---|---|
|
|
| VEVENT SUMMARY | `title` |
|
|
| DTSTART (UTC) | `dtstart_utc` or `dtstart_date` + `all_day` |
|
|
| RRULE | `has_rrule`, `raw_vevent` (full blob) |
|
|
| VALARM DURATION (single) | `reminder_leads` JSON array |
|
|
| Google `summary` | `title` |
|
|
| Google `start.dateTime` | `dtstart_utc` |
|
|
| Google `recurrence[RRULE]` | `has_rrule`, stored in `raw_vevent` as provider blob |
|
|
| Google reminder overrides | `reminder_leads` JSON array |
|
|
|
|
The `raw_vevent` column contains the full native blob for CalDAV (used for RRULE/VALARM preservation on edit). For Google, this column stores the full Google event JSON blob. Providers know their own blob format; the DB treats it as opaque text.
|
|
|
|
---
|
|
|
|
### 2. Google Calendar Provider
|
|
|
|
#### OAuth2 Token Storage
|
|
|
|
**New table: `provider_tokens`** -- separate from `member_credentials` because Google issues two separate secrets (access_token + refresh_token) with different TTLs, and refresh is async.
|
|
|
|
```sql
|
|
CREATE TABLE provider_tokens (
|
|
id INT AUTO_INCREMENT PRIMARY KEY,
|
|
user_id INT NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
|
provider_type VARCHAR(64) NOT NULL, -- 'google'
|
|
access_token TEXT NOT NULL, -- AES-256-GCM encrypted
|
|
refresh_token TEXT, -- AES-256-GCM encrypted; nullable
|
|
expires_at TIMESTAMP NOT NULL,
|
|
scope VARCHAR(1024),
|
|
created_at TIMESTAMP DEFAULT NOW() NOT NULL,
|
|
updated_at TIMESTAMP DEFAULT NOW() ON UPDATE NOW(),
|
|
UNIQUE KEY uniq_provider_token_user (user_id, provider_type)
|
|
);
|
|
```
|
|
|
|
`member_credentials` already has `UNIQUE(user_id)` enforcing one credential per member, which breaks multi-provider. For v1.2 (one member, one provider at a time), this constraint is fine -- a member has either CalDAV or Google, not both simultaneously. If multi-provider-per-member is ever needed (v1.3+), the unique constraint needs relaxation and the schema revisit is a separate migration.
|
|
|
|
**Recommendation:** Use `member_credentials` as the "this member has provider X connected" flag (providerType + a harmless encrypted sentinel), and `provider_tokens` for the actual Google OAuth token lifecycle. This avoids a schema change to `member_credentials` while cleanly separating concerns.
|
|
|
|
#### Token Refresh Worker
|
|
|
|
Rather than a dedicated worker, piggyback on the `CalendarProvider` contract: `GoogleCalendarProvider` checks `expires_at` before every API call and refreshes inline if the token is expired or within 5 minutes of expiry. This is the standard pattern for single-process deployments. No separate refresh `setInterval` is needed at this scale (2 users).
|
|
|
|
If a Google API call returns 401, the provider refreshes the token, retries once, then propagates the error as a `hardFail` dispatch result.
|
|
|
|
#### Google Read-Sync and Write-Back Mapping
|
|
|
|
| CalDAV outbox model | Google Calendar API equivalent |
|
|
|---|---|
|
|
| PUT .ics to collection URL | `POST/PATCH /calendar/v3/calendars/{calId}/events` |
|
|
| DELETE object URL | `DELETE /calendar/v3/calendars/{calId}/events/{eventId}` |
|
|
| ctag PROPFIND | `GET /calendar/v3/calendars/{calId}` -> `nextSyncToken` |
|
|
| REPORT (fetch all) | `GET /calendar/v3/calendars/{calId}/events?syncToken=...` |
|
|
| VEVENT `uid` | Google `event.id` (stored as `uid` in `calendar_outbox`) |
|
|
| VEVENT `etag` | Google `event.etag` (used for conditional update) |
|
|
|
|
The outbox `calendarUrl` column stores either a CalDAV collection URL (CalDAV provider) or a Google `calendarId` string (Google provider). The column name stays `calendarUrl`; the `GoogleCalendarProvider` interprets it as a calendar ID.
|
|
|
|
#### Google OAuth Callback Routes (MODIFIED: `routes/` + `index.ts`)
|
|
|
|
Google OAuth for calendar access is **not** the same as app login. The flow:
|
|
|
|
1. Member clicks "Connect Google Calendar" on the self-service onboarding page.
|
|
2. PWA calls `GET /api/calendar-connect/google/authorize` (NEW route, auth-guarded).
|
|
3. The route generates an OAuth authorization URL with `scope=https://www.googleapis.com/auth/calendar` + `access_type=offline` + `state=<signed-jwt-with-userId>`.
|
|
4. PWA redirects the browser to that URL.
|
|
5. Google redirects back to `/api/calendar-connect/google/callback` (NEW route, pre-auth -- same pattern as `/callback` for Authelia OIDC).
|
|
6. The callback exchanges the code for tokens, decrypts the state JWT to identify the member, and upserts `provider_tokens` + `member_credentials`.
|
|
|
|
**Routing concern:** The Google OAuth callback must be mounted **before** the OIDC auth guard in `index.ts` (same as `/callback` is mounted before `oidcAuthMiddleware`). Unlike the Authelia callback which establishes app login, the Google callback is for calendar access only -- the member is already logged into the app. The callback must still verify the signed state JWT to confirm the initiating user and prevent CSRF. After token storage, it redirects back to the onboarding page.
|
|
|
|
New routes file: `routes/calendarConnect.ts` (NEW) -- handles `/api/calendar-connect/google/authorize` (auth-guarded, GET) and `/api/calendar-connect/google/callback` (pre-auth, GET). Mount pre-auth in `index.ts`:
|
|
|
|
```typescript
|
|
// Must be before OIDC guard, like /callback
|
|
app.get('/api/calendar-connect/google/callback', googleOAuthCallbackHandler);
|
|
// Auth-guarded
|
|
app.route('/api/calendar-connect', calendarConnectRouter);
|
|
```
|
|
|
|
The Google `client_id` and `client_secret` are env vars (`GOOGLE_CLIENT_ID`, `GOOGLE_CLIENT_SECRET`) -- never in the DB (same SC-3 rule as VAPID_PRIVATE_KEY).
|
|
|
|
---
|
|
|
|
### 3. Self-Service Per-Provider Onboarding
|
|
|
|
#### UI Changes (MODIFIED: PWA)
|
|
|
|
A new "Connect Calendar" page/sheet in the PWA, accessible from Settings or the setup flow. Conditionally renders:
|
|
|
|
- **Fastmail path:** Password input field -> `POST /api/me/credential` (already exists). UI copies from `CredentialSheet.tsx` -- this component already implements the self-service flow.
|
|
- **Google path:** "Connect Google Calendar" button -> calls `GET /api/calendar-connect/google/authorize` -> browser redirect to Google.
|
|
|
|
New PWA component: `components/ProviderConnectSheet.tsx` (NEW) -- wraps both flows with provider-aware branching. `CredentialSheet.tsx` can be refactored to delegate to this, or kept as-is for the Fastmail path (lower risk).
|
|
|
|
#### API Changes
|
|
|
|
`GET /api/me/providers` (NEW) -- returns which providers the current member has connected (`{ providers: [{ type: 'caldav', connected: true }, { type: 'google', connected: false }] }`). PWA uses this to render the onboarding state.
|
|
|
|
`DELETE /api/me/providers/:type` (NEW) -- disconnect a provider (deletes `member_credentials` + `provider_tokens` rows, removes synced calendars + events from DB).
|
|
|
|
The existing `POST /api/me/credential` handles the Fastmail/CalDAV path unchanged. The admin `POST /api/admin/credentials` path also stays as-is for admin-initiated credential rotation.
|
|
|
|
#### Auth Guard Coexistence
|
|
|
|
The Google OAuth callback is the only new pre-auth route. All other onboarding routes are auth-guarded (`/api/me/*` pattern). No changes to the existing auth middleware stack.
|
|
|
|
---
|
|
|
|
### 4. Multiple Reminders Per Event
|
|
|
|
This is an end-to-end schema and data-flow change.
|
|
|
|
#### Schema Change
|
|
|
|
Replace `calendar_events.reminder_lead_minutes INT` with `calendar_events.reminder_leads JSON` (array of integers). A `NULL` value means no reminders. An empty array `[]` is equivalent to null (no reminders set). This is a **breaking migration** for the existing single-reminder column.
|
|
|
|
Migration strategy:
|
|
- Add `reminder_leads JSON` column.
|
|
- Migrate existing data: `UPDATE calendar_events SET reminder_leads = JSON_ARRAY(reminder_lead_minutes) WHERE reminder_lead_minutes IS NOT NULL`.
|
|
- Drop `reminder_lead_minutes` column in the same migration.
|
|
- Update Drizzle schema: `reminderLeads: json('reminder_leads').$type<number[] | null>()`.
|
|
|
|
The `calendarOutbox` payload schema (`outboxPayloadSchema`) changes `reminderLeadMinutes: z.number().int()...` to `reminderLeads: z.array(z.number().int().min(0).max(10080)).nullable().optional()`.
|
|
|
|
#### Data Flow Changes
|
|
|
|
**EventForm -> API (MODIFIED: PWA + `routes/events.ts`):**
|
|
- EventForm renders a list of reminder pickers (add/remove). Each picker = one lead time.
|
|
- Payload changes: `reminderLeadMinutes: number | null` -> `reminderLeads: number[] | null`.
|
|
|
|
**outboxWorker.ts (MODIFIED):**
|
|
- `buildVeventString` call changes: `reminderLeadMinutes` parameter -> `reminderLeads: number[]`.
|
|
- `vevent.ts` must emit multiple `VALARM` sub-components for each lead.
|
|
- The preserve-on-no-change path: `extractValarms` already returns `ICAL.Component[]`; this still works verbatim.
|
|
|
|
**vevent.ts (MODIFIED):**
|
|
- `buildVeventString` params: `reminderLeadMinutes?: number | null` -> `reminderLeads?: number[] | null`.
|
|
- Build one `VALARM` per entry in the array.
|
|
- `buildTimedValarm` and `buildAllDayValarm` helpers are called in a loop.
|
|
|
|
**reminderScheduler.ts (MODIFIED):**
|
|
- Query changes: instead of `WHERE reminder_lead_minutes IS NOT NULL`, query `WHERE reminder_leads IS NOT NULL AND JSON_LENGTH(reminder_leads) > 0`.
|
|
- `reminder_leads` is a JSON array; the scheduler must iterate all leads for an event and compute one fire time per lead.
|
|
- Dedup key: `${uid}:${dtstartMs}:${leadMinutes}` -- per-lead dedup so each reminder fires independently.
|
|
|
|
**Google provider serialization:**
|
|
- `serializeReminders(leads, allDay)` -> `{ overrides: leads.map(m => ({ method: 'popup', minutes: m })), useDefault: false }`.
|
|
- `parseReminders(googleEvent.reminders)` -> `googleEvent.reminders.overrides?.map(o => o.minutes) ?? []`.
|
|
|
|
**sync.ts (MODIFIED for CalDAV path):**
|
|
- When syncing a VEVENT, extract all VALARMs using `extractValarms`, classify each, and store the array of lead minutes as `reminder_leads` JSON.
|
|
|
|
---
|
|
|
|
### 5. PWA Dark Mode / Theming
|
|
|
|
#### Where Theme State Lives
|
|
|
|
Theme preference: `localStorage` key `'theme'` with values `'light' | 'dark' | 'system'`. Resolved theme (what's actually applied): derived from preference + `window.matchMedia('(prefers-color-scheme: dark)')`.
|
|
|
|
**New Zustand store:** `store/themeStore.ts` (NEW) -- holds `{ preference, resolvedTheme }` and a `setPreference()` action that writes to localStorage and applies `data-theme` to `document.documentElement`.
|
|
|
|
An effect in the store (or in `main.tsx` before React hydration) sets `data-theme` on `<html>` immediately from localStorage to prevent FOUC. The token file already has the `[data-theme='light']` selector; adding `[data-theme='dark']` with dark token values is the only CSS change.
|
|
|
|
#### Avoiding FOUC
|
|
|
|
The FOUC problem with theme switching is that React hydration runs after the browser paints. Solution: inject an inline `<script>` in `index.html` before any stylesheet or React bundle that reads `localStorage['theme']` and sets `document.documentElement.setAttribute('data-theme', resolved)`. This runs synchronously at parse time, before CSS is applied.
|
|
|
|
```html
|
|
<script>
|
|
(function(){
|
|
var p = localStorage.getItem('theme') || 'system';
|
|
var resolved = p === 'system'
|
|
? (window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light')
|
|
: p;
|
|
document.documentElement.setAttribute('data-theme', resolved);
|
|
})();
|
|
</script>
|
|
```
|
|
|
|
This script goes in `apps/pwa/index.html` (MODIFIED) before the CSS `<link>` tags.
|
|
|
|
#### tokens.css Changes (MODIFIED)
|
|
|
|
Add `[data-theme='dark']` rule with dark palette values. The stub comment already exists in `tokens.css`. The `--sx-color-*` Schedule-X overrides must also be present inside the dark rule. The `calendarStore` color palette (`--color-member-0` through `--color-member-5`) must be defined in both light and dark rules -- same hue, adjusted brightness.
|
|
|
|
#### Theme Toggle UI
|
|
|
|
New component: `components/ThemeToggle.tsx` (NEW) -- renders a light/dark/system switch. Placed in `SettingsSheet.tsx` (MODIFIED).
|
|
|
|
---
|
|
|
|
### 6. Zero-Manual-Setup DB Bootstrap
|
|
|
|
#### Where Migrate-on-Boot Runs
|
|
|
|
In `index.ts` inside the `isMainModule()` guard, **before** `startBrokerPoller()` and **before** `serve()` -- but **after** the boot guards (`assertNotDevBypassInProduction`, `assertLocalSessionSecretSet`).
|
|
|
|
```typescript
|
|
if (isMainModule()) {
|
|
assertNotDevBypassInProduction();
|
|
assertLocalSessionSecretSet();
|
|
await runMigrationsIfNeeded(); // NEW -- runs drizzle migrate programmatically
|
|
// ... VAPID setup, startBrokerPoller, startOutboxWorker, serve ...
|
|
}
|
|
```
|
|
|
|
Use drizzle-orm's programmatic `migrate(db, { migrationsFolder: './src/db/migrations' })`. This is already the established pattern for drizzle-kit; the change is calling it automatically at boot instead of requiring a manual `docker exec` step.
|
|
|
|
#### Interplay with Setup Wizard
|
|
|
|
The setup wizard (`/api/setup/*`) is mounted pre-auth and checks `isSetupLocked()` on every invocation. Migrate-on-boot runs BEFORE the server starts accepting requests, so by the time the wizard receives its first request, the schema is guaranteed to exist. No race condition.
|
|
|
|
If `DB_HOST`/`DB_NAME`/`DB_USER`/`DB_PASSWORD` env vars are set but the database is empty (first boot of a fresh MariaDB instance), migrate-on-boot creates all tables. The setup wizard then runs to configure OIDC/VAPID/admin account. This is the intended "bring-your-own MariaDB" flow.
|
|
|
|
#### Boot Failure Behavior
|
|
|
|
If migration fails (DB unreachable, bad creds), `runMigrationsIfNeeded` must throw -- the server should not start with a potentially partial schema. The MariaDB healthcheck in `docker-compose.yml` already gates API startup on DB readiness, so typically this won't be reached with an unreachable DB, but the defense-in-depth catch is still correct.
|
|
|
|
---
|
|
|
|
### 7. Dev-User Full-App Exercise (Mock Provider)
|
|
|
|
#### New Component: `broker/mockProvider.ts` (NEW)
|
|
|
|
Implements `CalendarProvider` with an in-memory store -- a `Map<string, ProviderCalendar[]>` for calendars and `Map<string, CalendarEvent[]>` for events. Pre-seeded with a set of realistic test events (recurring, all-day, timed, with reminders) that cover the scheduler's test surface.
|
|
|
|
**Activation:** When `DEV_AUTH_BYPASS=true` AND `provider_type` for the dev user is `'mock'` (or when no credential row exists for the dev user), `createProvider` returns `MockCalendarProvider`. The dev bypass already injects a `DEV_USER` -- the mock provider boots alongside it with no DB credential row required.
|
|
|
|
**Playwright harness integration:** The existing `devBypass.ts` already gates test-only behavior on `DEV_AUTH_BYPASS`. The mock provider is the same gate. Playwright tests that exercise calendar CRUD will hit the mock provider, which returns success responses and updates its in-memory state. The outbox worker calls `provider.dispatchOutboxRow()` which the mock handles without network I/O.
|
|
|
|
**Seed data bootstrapping:** On first call to `listCalendars()`, the mock creates:
|
|
- One personal calendar for DEV_USER.
|
|
- One shared calendar.
|
|
- ~5 pre-seeded events (one recurring weekly, one all-day, one with reminder, one past, one future).
|
|
|
|
This covers all code paths the scheduler, outbox worker, and event expansion touch.
|
|
|
|
---
|
|
|
|
## Component Map: New vs Modified
|
|
|
|
### New Components
|
|
|
|
| Component | Type | Purpose |
|
|
|---|---|---|
|
|
| `broker/provider.ts` | Interface + types | `CalendarProvider` interface, `ProviderCalendar`, `DispatchResult` |
|
|
| `broker/providerFactory.ts` | Factory | `createProvider(cred)` -> `CalendarProvider` |
|
|
| `broker/calDavProvider.ts` | Class | Wraps existing `client.ts` + `sync.ts` + `write.ts` |
|
|
| `broker/googleProvider.ts` | Class | Google Calendar API v3 read/write + token refresh |
|
|
| `broker/mockProvider.ts` | Class | In-memory provider for dev-bypass + Playwright tests |
|
|
| `routes/calendarConnect.ts` | Route file | Google OAuth authorize + callback handlers |
|
|
| `db/migrations/XXXX_v1.2_multi_provider.sql` | Migration | `provider_tokens` table, `reminder_leads` JSON column, drop `reminder_lead_minutes` |
|
|
| `pwa/src/store/themeStore.ts` | Zustand store | Theme preference + resolved theme |
|
|
| `pwa/src/components/ThemeToggle.tsx` | React component | Light/dark/system switch |
|
|
| `pwa/src/components/ProviderConnectSheet.tsx` | React component | Provider onboarding UI (Fastmail + Google paths) |
|
|
|
|
### Modified Components
|
|
|
|
| Component | Change |
|
|
|---|---|
|
|
| `broker/poller.ts` | `createFastmailClient` -> `createProvider(cred)`; `syncCalendar` -> `provider.syncCalendar()` |
|
|
| `broker/outboxWorker.ts` | `loadClientForUser` -> `loadProviderForUser`; `dispatchRow` -> `provider.dispatchOutboxRow(row)` |
|
|
| `broker/reminderScheduler.ts` | Query + dedup logic for multi-lead JSON; fire one push per lead per event |
|
|
| `broker/vevent.ts` | `buildVeventString` accepts `reminderLeads: number[]` instead of single `reminderLeadMinutes` |
|
|
| `broker/credentialSync.ts` | `validateEncryptAndStoreCredential` becomes provider-aware (CalDAV path unchanged) |
|
|
| `broker/sync.ts` | Parse multiple VALARMs into `reminder_leads` array on sync |
|
|
| `db/schema.ts` | `calendarEvents.reminderLeadMinutes` -> `calendarEvents.reminderLeads`; new `providerTokens` table |
|
|
| `routes/events.ts` | Payload schema: `reminderLeadMinutes` -> `reminderLeads` array |
|
|
| `routes/me.ts` | New `/api/me/providers` GET + DELETE endpoints |
|
|
| `index.ts` | Add `runMigrationsIfNeeded()` call; mount `/api/calendar-connect/google/callback` pre-auth |
|
|
| `pwa/src/components/EventForm.tsx` | Multi-reminder picker (add/remove); payload field rename |
|
|
| `pwa/src/components/SettingsSheet.tsx` | Add ThemeToggle; add provider connection status |
|
|
| `pwa/src/styles/tokens.css` | Add `[data-theme='dark']` rule with dark palette |
|
|
| `pwa/src/index.html` | FOUC-prevention inline script before CSS link tags |
|
|
|
|
---
|
|
|
|
## Data Flow Changes
|
|
|
|
### Multi-Provider Poller
|
|
|
|
```
|
|
startBrokerPoller()
|
|
-> runPoll()
|
|
-> SELECT * FROM member_credentials
|
|
-> for each cred:
|
|
createProvider(cred) <- NEW dispatch via providerFactory
|
|
-> provider.listCalendars() <- CalDAV: PROPFIND / Google: GET /calendars
|
|
-> compare changeKey vs stored ctag/syncToken
|
|
-> on change: provider.syncCalendar(cal, onChanges)
|
|
-> writes calendarEvents rows (provider-specific parse -> common DB shape)
|
|
-> calls dispatchEventChange for push notifications
|
|
```
|
|
|
|
### Multi-Provider Outbox Drain
|
|
|
|
```
|
|
runOutboxDrain()
|
|
-> SELECT pending rows FROM calendar_outbox
|
|
-> for each row:
|
|
loadProviderForUser(row.userId) <- replaces loadClientForUser
|
|
-> SELECT member_credentials WHERE userId = row.userId
|
|
-> createProvider(cred)
|
|
-> provider.dispatchOutboxRow(row)
|
|
CalDAV: buildVeventString + PUT/DELETE via tsdav
|
|
Google: build Google event JSON + POST/PATCH/DELETE via googleapis
|
|
-> handle DispatchResult (success/conflict/hardFail/transient)
|
|
-> triggerTargetedResync via provider
|
|
```
|
|
|
|
### Multiple Reminders: Form -> DB -> Scheduler -> Push
|
|
|
|
```
|
|
EventForm
|
|
reminderLeads: number[] <- user adds/removes reminders
|
|
-> POST /api/events/create { reminderLeads: [30, 1440] }
|
|
-> routes/events.ts validates array
|
|
-> INSERT calendar_outbox { payload: JSON.stringify({ reminderLeads: [30, 1440] }) }
|
|
-> 202 Accepted
|
|
-> outboxWorker picks up row
|
|
-> provider.dispatchOutboxRow
|
|
CalDAV: buildVeventString({ reminderLeads: [30, 1440] })
|
|
-> two VALARM sub-components in VCALENDAR
|
|
Google: { reminders: { useDefault: false, overrides: [{ method: 'popup', minutes: 30 }, ...] } }
|
|
-> on success: syncCalendar -> parses VALARMs/overrides back -> stores [30, 1440] in reminder_leads
|
|
|
|
reminderScheduler (every 1 min)
|
|
-> SELECT uid, title, dtstart_utc, all_day, reminder_leads FROM calendar_events
|
|
CROSS JOIN push_subscriptions
|
|
WHERE reminder_leads IS NOT NULL AND JSON_LENGTH(reminder_leads) > 0
|
|
-> for each event:
|
|
leads = JSON.parse(reminder_leads) // e.g. [30, 1440]
|
|
for each lead in leads:
|
|
compute fireTime = dtstartUtc - lead minutes
|
|
dedup key: `${uid}:${dtstartMs}:${lead}`
|
|
if fireTime in (catchUpStart, now] and not in sentReminders:
|
|
dispatchPush(sub, notification)
|
|
sentReminders.set(dedupKey, dtstartMs)
|
|
```
|
|
|
|
### Google OAuth + Calendar Connect
|
|
|
|
```
|
|
PWA (ProviderConnectSheet, logged-in user)
|
|
-> GET /api/calendar-connect/google/authorize (auth-guarded)
|
|
-> generate state JWT signed with LOCAL_SESSION_SECRET { userId, nonce }
|
|
-> store nonce in linkNonceStore (same pattern as OIDC link flow)
|
|
-> return { authorizationUrl: 'https://accounts.google.com/...' }
|
|
-> PWA redirects browser to authorizationUrl
|
|
|
|
Google -> /api/calendar-connect/google/callback (pre-auth, before OIDC guard)
|
|
-> verify state JWT
|
|
-> consumeLinkNonce(nonce)
|
|
-> exchange code for { access_token, refresh_token, expires_in }
|
|
-> encrypt tokens (AES-256-GCM)
|
|
-> upsert provider_tokens (user_id, 'google', tokens)
|
|
-> upsert member_credentials (user_id, provider_type='google', placeholder)
|
|
-> trigger initial sync via GoogleCalendarProvider
|
|
-> redirect to /settings?provider=google&status=connected
|
|
```
|
|
|
|
### Migrate-on-Boot Sequence
|
|
|
|
```
|
|
isMainModule() guard:
|
|
1. assertNotDevBypassInProduction()
|
|
2. assertLocalSessionSecretSet()
|
|
3. await runMigrationsIfNeeded() <- NEW: drizzle migrate(db, { migrationsFolder })
|
|
-> connects to DB
|
|
-> runs all pending migrations in order
|
|
-> no-op on subsequent boots when schema is current
|
|
4. webpush.setVapidDetails(...)
|
|
5. startBrokerPoller()
|
|
6. startOutboxWorker() + initOutboxTrigger()
|
|
7. startReminderScheduler()
|
|
8. serve({ fetch: app.fetch, port: 3000 })
|
|
```
|
|
|
|
---
|
|
|
|
## Dependency-Ordered Build Sequence
|
|
|
|
The phases below are ordered by dependency. Each phase is safe to start only after the listed predecessors are complete.
|
|
|
|
```
|
|
Phase 21: Zero-Setup DB Bootstrap
|
|
Deliverable: runMigrationsIfNeeded() in index.ts isMainModule() guard
|
|
Dependencies: none (touches only index.ts startup + db/migrations)
|
|
Unlocks: all subsequent phases can assume schema bootstraps automatically
|
|
Risk: low
|
|
|
|
Phase 22: Provider Abstraction (CalDAV only, Fastmail path unchanged)
|
|
Deliverable: broker/provider.ts interface, providerFactory.ts, calDavProvider.ts,
|
|
poller.ts + outboxWorker.ts updated to use createProvider
|
|
Dependencies: Phase 21 (schema must auto-migrate before integration tests run)
|
|
Unlocks: Phase 23 (vevent.ts depends on interface), Phase 25 (Google needs interface),
|
|
Phase 24 (mock needs factory)
|
|
Risk: HIGH -- must not regress the working Fastmail path; existing integration tests
|
|
must pass with CalDavProvider wrapper in place
|
|
|
|
Phase 23: Multiple Reminders Per Event
|
|
Deliverable: DB migration (reminder_leads JSON), vevent.ts multi-VALARM,
|
|
outboxWorker.ts payload schema, reminderScheduler.ts multi-lead scan,
|
|
sync.ts multi-VALARM parse, routes/events.ts payload, EventForm.tsx
|
|
Dependencies: Phase 22 (CalDavProvider.dispatchOutboxRow must accept reminderLeads)
|
|
Note: Google reminder serialization/parsing is implemented in Phase 25
|
|
Risk: MEDIUM -- breaking migration; existing data must be migrated
|
|
|
|
Phase 24: Dev-User Full-App Exercise (Mock Provider)
|
|
Deliverable: broker/mockProvider.ts with seed data, providerFactory.ts 'mock' case,
|
|
devBypass.ts sets provider_type='mock' for DEV_USER,
|
|
Playwright harness updated to exercise calendar CRUD
|
|
Dependencies: Phase 22 (CalendarProvider interface + factory)
|
|
Unlocks: Phase 25 (unit tests can mock GoogleCalendarProvider)
|
|
Risk: low -- dev-only path
|
|
|
|
Phase 25: Google Calendar Provider
|
|
Deliverable: db/schema.ts provider_tokens table (migration), broker/googleProvider.ts,
|
|
routes/calendarConnect.ts, index.ts pre-auth callback mount
|
|
Dependencies: Phase 22 (CalendarProvider interface), Phase 23 (reminder serialization API),
|
|
Phase 21 (provider_tokens table exists after migrate-on-boot)
|
|
Risk: MEDIUM -- Google API OAuth flow has edge cases; token refresh inline logic
|
|
|
|
Phase 26: Self-Service Provider Onboarding UI
|
|
Deliverable: GET/DELETE /api/me/providers routes, ProviderConnectSheet.tsx,
|
|
SettingsSheet.tsx wiring
|
|
Dependencies: Phase 25 (Google OAuth callback must exist before UI redirects to it)
|
|
Note: Fastmail/CalDAV path (CredentialSheet.tsx) already works; this phase adds Google
|
|
Risk: low
|
|
|
|
Phase 27: PWA Dark Mode / Theming
|
|
Deliverable: index.html FOUC-prevention script, tokens.css dark rule,
|
|
themeStore.ts, ThemeToggle.tsx, SettingsSheet.tsx
|
|
Dependencies: none (pure frontend, no API changes)
|
|
Can run in parallel with Phases 22-26 if desired
|
|
Risk: low
|
|
```
|
|
|
|
---
|
|
|
|
## Integration Points Summary
|
|
|
|
| Feature | Integration Point | Risk Level |
|
|
|---|---|---|
|
|
| Provider abstraction | `broker/poller.ts` runPoll loop | Medium |
|
|
| Provider abstraction | `broker/outboxWorker.ts` dispatchRow | High |
|
|
| Google OAuth callback | `index.ts` pre-auth route registration | Low |
|
|
| Google tokens | New `provider_tokens` table | Low |
|
|
| Multi-reminder | `calendar_events` schema (column rename to JSON) | High |
|
|
| Multi-reminder | `reminderScheduler.ts` query + dedup key | Medium |
|
|
| Multi-reminder | `EventForm.tsx` UI + routes/events.ts payload | Medium |
|
|
| Migrate-on-boot | `index.ts` `isMainModule()` guard | Low |
|
|
| Dark mode FOUC | `index.html` inline script | Low |
|
|
| Mock provider | `devBypass.ts` + `providerFactory.ts` | Low |
|
|
|
|
---
|
|
|
|
## Anti-Patterns to Avoid
|
|
|
|
### Anti-Pattern 1: Refactoring CalDAV Internals While Adding the Interface
|
|
|
|
Do not inline-refactor `sync.ts`, `write.ts`, `vevent.ts` internals as part of the provider abstraction phase. Wrap them as-is in `CalDavProvider`. Refactoring internals and adding the interface in the same phase doubles the risk surface. Internal cleanup is a separate concern if ever needed.
|
|
|
|
### Anti-Pattern 2: Storing Google OAuth Tokens in `app_config`
|
|
|
|
The `app_config` table is a global key/value store for non-sensitive config. Google OAuth tokens are per-member secrets with a refresh lifecycle. They belong in `provider_tokens` with AES-256-GCM encryption and cascade-delete on user removal. Never put them in `app_config`.
|
|
|
|
### Anti-Pattern 3: Using the Google OAuth Callback to Establish App Login
|
|
|
|
The Google OAuth callback (`/api/calendar-connect/google/callback`) is for calendar access only. The app login session is established separately via Authelia OIDC or local auth. Do not attempt to merge these two OAuth flows -- they serve different purposes and must remain independent auth paths.
|
|
|
|
### Anti-Pattern 4: Schema Migration Without Data Migration
|
|
|
|
The rename from `reminder_lead_minutes` (INT) to `reminder_leads` (JSON) is a breaking change. The migration MUST include a `UPDATE calendar_events SET reminder_leads = JSON_ARRAY(reminder_lead_minutes) WHERE reminder_lead_minutes IS NOT NULL` step before dropping the old column. Without this, existing event reminders are silently lost.
|
|
|
|
### Anti-Pattern 5: Per-Provider `isDraining` Flags
|
|
|
|
The outbox worker's `isDraining` module-level flag is documented as single-process-only. This remains valid for v1.2 (still single-process). Do not introduce per-provider drain locks -- the existing guard covers all rows from all providers in the same drain cycle.
|
|
|
|
---
|
|
|
|
## Sources
|
|
|
|
- Direct codebase inspection: `apps/api/src/broker/`, `apps/api/src/db/schema.ts`, `apps/api/src/index.ts`, `apps/api/src/auth/`, `apps/pwa/src/styles/tokens.css`
|
|
- `docs/ARCHITECTURE.md` -- existing component diagram and data-flow walkthroughs
|
|
- `.planning/PROJECT.md` -- v1.2 milestone scope and key decisions
|
|
|
|
---
|
|
*Architecture research for: FamilySync v1.2 Multi-Provider integration*
|
|
*Researched: 2026-06-19*
|