docs: complete v1.2 research (stack, features, architecture, pitfalls)

- STACK.md: google-auth-library@10.7.0 + @googleapis/calendar@15.0.0 scoped packages (vs monolithic googleapis), OAuth2 flow, token storage, Google Calendar API event/reminder model
- FEATURES.md: 6 feature categories (multi-provider, self-service onboarding, multiple reminders, dark mode, zero-setup DB, dev/CI stub), dependency graph, feature prioritization
- ARCHITECTURE.md: CalendarProvider interface, provider factory, CalDavProvider wrapper, GoogleCalendarProvider, MockProvider, provider_tokens table schema, multi-reminder JSON column, OAuth callback routing, 7-component data flows
- PITFALLS.md: 11 critical/medium pitfalls (refresh token 7-day expiry in testing status, Google recurrence mismatch, syncToken 410, timezone handling, provider abstraction regression, VALARM dedup key, auto-migrate failures, dark mode FOWT, OAuth callback through tunnel, token encryption, ESLint 10 breaking changes)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Lucas Berger
2026-06-19 09:27:35 -04:00
co-authored by Claude Opus 4.8
parent 303484d0ab
commit 6e1c9ca924
4 changed files with 1446 additions and 1055 deletions
+369 -3
View File
@@ -1,11 +1,350 @@
# Stack Research
**Domain:** Self-hosted family calendar + shared-lists PWA on Fastmail
**Researched:** 2026-06-03 (v1.0) / 2026-06-10 (v1.1 additions)
**Researched:** 2026-06-03 (v1.0) / 2026-06-10 (v1.1 additions) / 2026-06-19 (v1.2 additions)
**Confidence:** MEDIUM-HIGH (calendar sharing cross-account caveat: LOW; rest HIGH)
---
## v1.2 Stack Additions — Multi-Provider, Theming & Zero-Setup
> Covers ONLY net-new libraries and patterns for v1.2. The existing stack (Hono, Drizzle,
> mysql2, tsdav, ical.js, web-push, @hono/oidc-auth, TanStack Query, Zustand, vite-plugin-pwa,
> @playwright/test) is shipped and proven — do not re-evaluate it.
### Net-New npm Packages (two only)
| Package | Version | Scope | Purpose |
|---------|---------|-------|---------|
| `google-auth-library` | 10.7.0 | `apps/api` | OAuth2 authorization-code flow + offline refresh token management |
| `@googleapis/calendar` | 15.0.0 | `apps/api` | Google Calendar API v3 typed REST client |
Everything else (dark mode, multiple VALARMs, programmatic migrate, CI updates) uses existing dependencies.
---
### 1. Google Calendar Integration
#### Library decision: `google-auth-library` + `@googleapis/calendar`
Do NOT use the monolithic `googleapis` package. It bundles 170+ API clients (~50 MB in the
Docker image) for Calendar-only use. The scoped split gives the same typed API surface at a
fraction of the footprint.
| Alternative | Rejection reason |
|-------------|-----------------|
| `googleapis` (monolithic) | 170+ bundled clients; bloats Docker image layer with unused code |
| Raw `fetch` + REST | Must hand-roll token refresh, retry logic, typed request/response schemas |
| `@microfox/google-calendar` | Third-party wrapper, last publish 9 months ago, adds indirection over official packages |
`@googleapis/calendar@15.0.0` depends only on `googleapis-common@^8.0.0` (auto-installed as a
transitive dep). `google-auth-library@10.7.0` ships its own TypeScript types — no `@types/` package
needed.
#### OAuth2 Authorization-Code Flow (backend-only)
The flow is fully backend-driven, matching the existing `@hono/oidc-auth` pattern for Authelia:
1. Backend generates the Google consent URL:
```typescript
const url = oauth2Client.generateAuthUrl({
access_type: 'offline',
scope: ['https://www.googleapis.com/auth/calendar.events'],
state: signedStateJwt, // CSRF protection, same pattern as OIDC link flow
});
```
2. User clicks "Connect Google" → redirected to Google consent screen.
3. Google redirects back to `/api/providers/google/callback`.
4. Backend exchanges the code:
```typescript
const { tokens } = await oauth2Client.getToken(code);
// tokens.refresh_token is ONLY present on first authorization with access_type:'offline'
// Subsequent exchanges return only access_token. Persist refresh_token immediately.
```
5. Store the token object `{ refresh_token, access_token, expiry_date }` encrypted
(AES-256-GCM, same crypto as Fastmail app passwords) in `member_credentials`
with `provider_type = 'google'`.
6. On each API call, hydrate the client:
```typescript
oauth2Client.setCredentials({ refresh_token: storedToken });
// google-auth-library auto-refreshes when access_token is expired
oauth2Client.on('tokens', (tokens) => {
// Persist newly issued access_token + expiry_date back to DB
// to avoid unnecessary refresh calls on next request
});
```
**Required scopes:**
- `https://www.googleapis.com/auth/calendar.events` — create/edit/delete events on any calendar
- `https://www.googleapis.com/auth/calendar.readonly` — read-only if write is not needed per calendar
#### Schema change for token storage
The existing `member_credentials` table has `fastmail_email` and `encrypted_password` columns.
For Google, `encrypted_password` stores the JSON token blob and `fastmail_email` stores the
Google account email. In v1.2, add a `provider_account_id VARCHAR(256)` column (additive migration)
that stores the account identifier in a provider-neutral name — avoids abusing `fastmail_email`
for a non-Fastmail email.
#### Google Calendar API event model vs. iCalendar
**Recurring events**: Google's `recurrence` field is an array of RFC 5545 RRULE strings — the same
format ical.js already handles for Fastmail:
```json
{ "recurrence": ["RRULE:FREQ=WEEKLY;BYDAY=MO"] }
```
However, Google uses RFC 3339 with explicit `timeZone` for timed events (not UTC-Z like the
existing CalDAV write path), and `date` fields (`"YYYY-MM-DD"`) for all-day events.
| Operation | Google API call |
|-----------|----------------|
| List instances (expanded) | `events.list({ singleEvents: true })` |
| List parent recurring events | `events.list({ singleEvents: false })` (default) |
| Edit one occurrence | GET instance (has `recurringEventId`), then PATCH |
| Delete one occurrence | `events.delete({ eventId: instanceId })` — only that instance |
| Delete whole series | `events.delete({ eventId: recurringEventId })` |
| Edit this + following | Set UNTIL on RRULE of original → insert new series from that point |
**Reminders**: Google uses a flat `reminders.overrides` array — structurally simpler than VALARM:
```json
{
"reminders": {
"useDefault": false,
"overrides": [
{ "method": "popup", "minutes": 15 },
{ "method": "popup", "minutes": 60 }
]
}
}
```
| Dimension | Google Calendar | iCalendar VALARM |
|-----------|----------------|-----------------|
| Trigger type | Relative minutes only | Relative DURATION or absolute DATE-TIME |
| All-day trigger | Minutes before midnight of event start | Absolute UTC instant (9 AM local in app) |
| Multiple reminders | Yes — array of overrides | Yes — multiple VALARM subcomponents |
| Methods | `popup` + `email` | `DISPLAY`, `AUDIO`, `EMAIL` |
Mapping strategy: read `overrides[*].minutes` where `method='popup'` → `reminderLeadMinutes[]`
array; ignore `email` method (app handles push, not email). On write: map `reminderLeadMinutes[]`
→ `overrides` array with `method: 'popup'`, set `useDefault: false`.
All-day event timing: Google fires at midnight minus lead minutes. The app's "9 AM local" semantic
from Fastmail cannot be replicated — document this as a provider difference; accept Google's
midnight-relative behavior for Google events.
#### Installation
```bash
pnpm add --filter @familysync/api google-auth-library @googleapis/calendar
```
---
### 2. Provider Abstraction — Hand-Rolled TypeScript Interface
No cross-provider calendar normalization library exists worth taking as a dependency. The two
providers have well-understood shapes; a hand-rolled interface in `apps/api/src/broker/` is the
right call — stays under project control, zero external dep, typed exactly to app needs.
```typescript
// apps/api/src/broker/providerTypes.ts
export interface NormalizedEvent {
uid: string; // stable cross-provider event ID
calendarId: string; // provider-internal calendar identifier
summary: string;
allDay: boolean;
dtstart: Date | string; // Date for timed, 'YYYY-MM-DD' for all-day
dtend: Date | string;
location?: string;
description?: string;
rruleString?: string; // bare RRULE value if recurring master
reminderLeadMinutes?: number | null; // legacy single (backward compat)
reminderLeadMinutesMultiple?: number[]; // v1.2 multiple reminders
rawPayload?: string; // CalDAV: raw iCalendar string; Google: JSON string
}
export interface ProviderCalendar {
id: string;
displayName: string;
color?: string;
isShared: boolean;
}
export interface CalendarProvider {
readonly providerType: 'caldav' | 'google';
discoverCalendars(): Promise<ProviderCalendar[]>;
syncEvents(calendarId: string, since?: Date): Promise<NormalizedEvent[]>;
createEvent(calendarId: string, event: Omit<NormalizedEvent, 'uid' | 'calendarId'>): Promise<string>;
updateEvent(calendarId: string, event: NormalizedEvent): Promise<void>;
deleteEvent(calendarId: string, uid: string): Promise<void>;
}
```
**Fastmail adapter**: wraps the existing `broker/sync.ts`, `broker/write.ts`, `broker/poller.ts`
behind this interface. Refactor, not rewrite.
**Google adapter**: new `broker/googleCalendarProvider.ts` implementing `CalendarProvider` using
`@googleapis/calendar` + `google-auth-library`. Fetches credentials from `member_credentials`
where `provider_type = 'google'`, re-hydrates `OAuth2Client` per call.
---
### 3. Multiple Reminders Per Event — No New Dependency
`ical.js` already supports multiple VALARM subcomponents via repeated `vevent.addSubcomponent(alarm)`
calls. The existing `buildVeventString` already processes a `valarms` array (preserve-on-edit path).
The v1.2 change is purely a data-model and serialization update:
1. **Schema**: Add `reminder_lead_minutes_json TEXT` column to `calendar_events` (nullable JSON
array e.g. `[15, 60]`). Keep `reminder_lead_minutes INT` for backward compat; treat single-value
as `[value]`.
2. **vevent.ts**: Change `buildVeventString` to accept `reminderLeadMinutes: number[]`; loop
`buildTimedValarm(lead)` for each, call `vevent.addSubcomponent()` per alarm.
3. **classifyValarms**: The `length > 1` branch currently returns `{ kind: 'custom' }`. v1.2
should return `{ kind: 'multi-preset', leads: number[] }` when all alarms are relative DURATION
triggers with preset lead values.
4. **Google adapter**: Map `reminders.overrides` bidirectionally — multiple `{ method: 'popup', minutes: N }` entries.
No new npm dependency.
---
### 4. PWA Dark Mode / Theming — Pure CSS + Existing Zustand
No theming library needed. The token layer is already structured for this:
- `tokens.css` has `[data-theme='light']` with all semantic tokens defined.
- Schedule-X `--sx-color-*` vars are already mapped to project tokens in `tokens.css` — so
adding a `[data-theme="dark"]` block that overrides `--color-surface`, `--color-text-primary`,
etc. automatically cascades into Schedule-X with no Schedule-X config change.
- The dark stub comment `[data-theme="dark"] { ... }` is already in `tokens.css` (Phase 17
groundwork). Fill it in.
**Implementation — no new package:**
```typescript
// apps/pwa/src/store/themeStore.ts
import { create } from 'zustand';
import { persist } from 'zustand/middleware'; // built-in, already in Zustand 5.x
type ThemeMode = 'light' | 'dark' | 'system';
interface ThemeStore {
mode: ThemeMode;
setMode: (m: ThemeMode) => void;
}
export const useThemeStore = create<ThemeStore>()(
persist(
(set) => ({ mode: 'system', setMode: (mode) => set({ mode }) }),
{ name: 'familysync-theme' },
),
);
```
**DOM application** (called on mount and on mode change):
```typescript
function resolveTheme(mode: ThemeMode): 'light' | 'dark' {
if (mode !== 'system') return mode;
return window.matchMedia('(prefers-color-scheme: dark)').matches ? 'dark' : 'light';
}
document.documentElement.dataset.theme = resolveTheme(store.mode);
```
**System preference listener:**
```typescript
window.matchMedia('(prefers-color-scheme: dark)')
.addEventListener('change', () => {
if (useThemeStore.getState().mode === 'system') {
document.documentElement.dataset.theme = resolveTheme('system');
}
});
```
**Flash prevention**: Add an inline `<script>` in `index.html` (before the React bundle) that
reads `localStorage['familysync-theme']` and sets `document.documentElement.dataset.theme`
synchronously. This is the standard flash-of-wrong-theme prevention pattern; no library needed.
Zustand `persist` middleware is already built into Zustand 5.0.14. No additional package.
---
### 5. Zero-Manual-Setup DB Bootstrap — Programmatic `drizzle-orm/mysql2/migrator`
`drizzle-orm/mysql2/migrator` is already part of `drizzle-orm@0.45.2`. No new package.
The migrate API requires a **single connection** (not the runtime pool):
```typescript
// apps/api/src/db/migrate.ts
import mysql from 'mysql2/promise';
import { drizzle } from 'drizzle-orm/mysql2';
import { migrate } from 'drizzle-orm/mysql2/migrator';
export async function runMigrations(): Promise<void> {
// migrate() must use a single connection, not the runtime pool
const connection = await mysql.createConnection({
host: process.env.DB_HOST ?? 'localhost',
port: Number(process.env.DB_PORT ?? 3306),
user: process.env.DB_USER ?? 'familysync',
password: process.env.DB_PASSWORD ?? '',
database: process.env.DB_NAME ?? 'familysync',
multipleStatements: true, // required: migration files contain multiple DDL statements
});
const db = drizzle(connection);
try {
await migrate(db, { migrationsFolder: './src/db/migrations' });
console.log('[db] migrations applied');
} finally {
await connection.end();
}
}
```
Call `await runMigrations()` in `apps/api/src/index.ts` **before** `serve()` and before
starting the broker poller. The existing `db` pool (from `db/client.ts`) is separate and
unchanged for all runtime queries.
**Idempotency**: Drizzle tracks applied migrations in a `__drizzle_migrations` table it
creates automatically. Running `migrate()` on a container restart is a no-op for already-applied
files. Safe to call unconditionally at every boot.
**MariaDB 11**: `multipleStatements: true` is required because drizzle-kit generates migration
files with multiple DDL statements separated by semicolons. MariaDB 11 is wire-compatible with
MySQL and this flag works identically.
**BANNED**: `db:push` (`drizzle-kit push`) remains banned on MariaDB 11. It schedules
destructive schema diffs. Only `drizzle-kit generate` (dev) + `migrate()` at runtime (prod).
---
### 6. CI Dependency Updates — Existing `pnpm` Tooling Only
No new tooling package. The workflow:
1. `pnpm outdated --recursive` — table of current / wanted / latest across all workspaces.
2. `pnpm update --interactive --latest -r` — selective upgrade; review each before accepting.
3. `pnpm audit --fix=update` (pnpm v11+) — bump packages to fix security findings rather than
adding overrides.
4. Run full local CI gates (`pnpm run lint`, `pnpm run typecheck`, `pnpm test`) before committing.
| Category | Action |
|----------|--------|
| Patch / minor | `pnpm update -r` within semver range |
| Major with API changes | Evaluate per-package; check changelog |
| HIGH/CRITICAL security | Prioritize; `--fix=update` where possible |
| `@playwright/test` | Pin to the Playwright binary installed in CI runner; bumping requires browser reinstall |
---
## v1.1 Stack Additions — Operability & Polish
This section covers ONLY what is new for v1.1. The rest of the file (below) documents the v1.0 stack, which is unchanged.
@@ -218,6 +557,9 @@ npm install drizzle-orm mysql2 ioredis
npm install tsdav ical.js rrule
npm install web-push zod openid-client
# v1.2 additions (apps/api)
npm install google-auth-library @googleapis/calendar
# Frontend
npm install react react-dom @tanstack/react-query zustand
npm install -D vite vite-plugin-pwa
@@ -376,6 +718,8 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
| CalDAV | JMAP | JMAP calendars not available on Fastmail as of 2026 |
| @playwright/test | playwright-cli alone | playwright-cli lacks storageState save/restore and device presets needed for CI; both coexist |
| PAT for Gitea registry | secrets.GITEA_TOKEN / built-in token | Gitea does not inject a built-in token with container-registry push scope; PAT required |
| @googleapis/calendar | googleapis (monolithic) | Monolithic bundles 170+ clients; scoped package is Calendar-only; same typed API, fraction of size |
| Hand-rolled CalendarProvider interface | cross-provider normalization library | No maintained library exists; hand-rolled interface stays under project control |
---
@@ -394,6 +738,10 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
| `mysqladmin ping` health check with MariaDB 11 | mysqladmin not shipped in mariadb:11 image; silently blocks CI | `healthcheck.sh --connect --innodb_initialized` |
| `runs-on: ubuntu-latest` on Gitea self-hosted runner | Label only resolves on GitHub's hosted infrastructure | `runs-on: self-hosted` (or the runner's registered label) |
| Any validation library for setup wizard | zod + mysql2 + web-push + Node 22 fetch cover all checks natively | Use existing stack |
| `googleapis` (monolithic npm package) | Bundles 170+ API clients; ~50 MB for Calendar-only use | `@googleapis/calendar` + `google-auth-library` |
| `next-themes` or any theming library | Adds indirection over CSS data-theme + Zustand persist (already in stack) | CSS `[data-theme="dark"]` + Zustand `persist` middleware |
| `drizzle-kit push` in production / MariaDB 11 | Schedules destructive schema diffs; banned on MariaDB 11 | `drizzle-kit generate` + `migrate()` at runtime |
| FCM/Firebase for Google Calendar push | Google Calendar push = REST polling / webhooks; unrelated to VAPID push | Use existing `web-push` for app notifications; Google Calendar webhooks are separate |
---
@@ -402,12 +750,17 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
| Package | Compatible With | Notes |
| ---------------------------- | ---------------------- | ------------------------------------------------------------------------------------------ |
| drizzle-orm@0.45.x | mysql2@3.x | Use `drizzle-orm/mysql2` import path; mysql2@3.x uses Promises API by default |
| drizzle-orm/mysql2/migrator | drizzle-orm@0.45.x, mysql2@3.x | Single connection required (not pool); `multipleStatements: true` for MariaDB 11 |
| vite-plugin-pwa@1.3.x | Vite@8.x, Workbox@7.x | vite-plugin-pwa 0.16+ requires Node 16+; 1.x tracks Vite 6+ |
| @hono/oidc-auth@1.8.x | hono@4.x, oauth4webapi | Peer-depends on hono 4.x |
| ical.js@2.x | rrule@2.8.x | Use together: ical.js parses the RRULE string, pass to `new RRule(RRule.parseString(...))` |
| web-push@3.6.x | Node.js 18+ | VAPID uses Web Crypto; works in Node.js 18+ natively |
| @playwright/test@1.60.x | Node.js 18+ | Install Chromium only in CI (`npx playwright install --with-deps chromium`) |
| mariadb:11 service container | GitHub/Gitea Actions | Health check must use `healthcheck.sh`; `mysqladmin` removed in 11.x |
| google-auth-library@10.7.0 | Node.js 18+, TypeScript 5.x | Ships own types; no @types/ needed; OAuth2Client auto-refreshes expired access tokens |
| @googleapis/calendar@15.0.0 | googleapis-common@^8.0.0 | Auto-installed as transitive dep; do not pin googleapis-common separately |
| zustand/middleware `persist` | zustand@5.0.x | Built-in middleware; no separate import package; works with localStorage in PWA |
| Schedule-X `--sx-color-*` | tokens.css `[data-theme]` | All Schedule-X color vars already map to project tokens; dark overrides cascade automatically |
---
@@ -423,6 +776,10 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
5. **Playwright mobile tests and DEV_AUTH_BYPASS in CI:** The mobile test harness depends on `DEV_AUTH_BYPASS=true` being available in CI. This means the CI run starts the API with that flag — confirm it is only set in the test environment, never in the production image/deploy step.
6. **Google OAuth2 consent screen verification:** Google requires app verification for production OAuth apps requesting Calendar scopes. For a self-hosted household app, the project must be in "testing" mode (max 100 users) or published. For a two-person household, testing mode (unverified) is sufficient; add both Google accounts as test users in Google Cloud Console. No app review needed.
7. **Google Calendar webhook push vs. polling:** The Google Calendar API supports webhook push notifications (via `events.watch()`) that POST to a public URL when calendars change. This is more efficient than polling but requires a verified public HTTPS endpoint. The existing ctag-poller pattern (5-min interval) is simpler and sufficient for a two-person household — evaluate webhooks only if polling latency becomes a problem.
---
## Sources
@@ -436,6 +793,7 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
- [rrule npm](https://www.npmjs.com/package/rrule) — Version 2.8.1 confirmed
- [Hono](https://hono.dev/) — Version 4.12.23; Node.js adapter confirmed
- [Drizzle ORM MySQL](https://orm.drizzle.team/docs/get-started-mysql) — MariaDB via mysql2 confirmed
- [Drizzle ORM Migrations API](https://mintlify.wiki/drizzle-team/drizzle-orm/api/core/migrations) — `migrate(db, { migrationsFolder })` with mysql2, single connection required
- [vite-plugin-pwa](https://vite-pwa-org.netlify.app/) — Version 1.3.0; Workbox 7 integration
- [web-push npm](https://www.npmjs.com/package/web-push) — Version 3.6.7
- [Meet Declarative Web Push — WebKit](https://webkit.org/blog/16535/meet-declarative-web-push/) — Safari 18.4+, iOS 18.4+ confirmed
@@ -452,8 +810,16 @@ Redis (`ioredis`) is only needed if multiple Node containers run behind a load b
- [MariaDB 11 health check fix](https://github.com/mage-os/github-actions/issues/365) — mysqladmin removed in mariadb:11.4; healthcheck.sh required
- [MySQL in GitHub Actions (ovirium.com)](https://ovirium.com/blog/how-to-make-mysql-work-in-your-github-actions/) — Service container pattern; ports, env, options syntax (GitHub-compatible = Gitea-compatible)
- [DEV Community — Docker-in-Docker with Gitea Actions](https://dev.to/tmlr/the-definitive-guide-to-safe-docker-in-docker-with-gitea-actions-331l) — DinD vs socket-mount tradeoffs; socket-mount recommended for homelab
- [@googleapis/calendar npm](https://www.npmjs.com/package/@googleapis/calendar) — Version 15.0.0, verified 2026-06-19; depends only on googleapis-common
- [google-auth-library npm](https://www.npmjs.com/package/google-auth-library) — Version 10.7.0, verified 2026-06-19; ships own TypeScript types
- [Google Auth Library Node.js — Context7 / googleapis GitHub](https://github.com/googleapis/google-auth-library-nodejs) — OAuth2Client.generateAuthUrl, getToken, setCredentials, tokens event
- [Google Calendar API — Recurring Events](https://developers.google.com/workspace/calendar/api/guides/recurringevents) — RRULE recurrence array, instance fields, singleEvents param, series delete
- [Google Calendar API — Reminders](https://developers.google.com/workspace/calendar/concepts/reminders) — overrides array, method types, useDefault=false requirement
- [pnpm audit CLI](https://pnpm.io/cli/audit) — --fix=update (pnpm v11+) vs --fix=override default
- [pnpm outdated CLI](https://pnpm.io/cli/outdated) — --recursive workspace support
- [Schedule-X Theme Docs](https://schedule-x.dev/docs/calendar/theme) — --sx-color-* CSS variable list; no built-in dark mode
---
_Stack research for: FamilySync — self-hosted family calendar + shared-lists PWA on Fastmail_
_Researched: 2026-06-03 (v1.0 baseline) / 2026-06-10 (v1.1 Operability & Polish additions)_
_Stack research for: FamilySync — self-hosted family calendar + shared-lists PWA_
_Researched: 2026-06-03 (v1.0 baseline) / 2026-06-10 (v1.1 Operability & Polish) / 2026-06-19 (v1.2 Multi-Provider, Theming & Zero-Setup)_