Lucas BergerandClaude Opus 4.8
24bc8d2c32
CI / changes (pull_request) Successful in 6s
CI / api (pull_request) Successful in 2m9s
CI / fast-checks (pull_request) Successful in 2m30s
CI / security (pull_request) Successful in 59s
CI / harness (pull_request) Failing after 12m0s
CI / gate (pull_request) Failing after 2s
Reformats 4 phase-17 files (SettingsSheet.tsx, tokens.css, vite.config.ts, pwa-assets.config.ts) plus 11 pre-existing non-conformant docs/READMEs that the repo-wide format:check also flags. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
255 lines
24 KiB
Markdown
255 lines
24 KiB
Markdown
<!-- generated-by: gsd-doc-writer -->
|
|
|
|
# FamilySync Architecture
|
|
|
|
FamilySync is a self-hosted family organization hub — a unified, color-coded calendar and shared collaborative lists — delivered as a React PWA. The system is a two-container Docker Compose stack (API + MariaDB) running on Unraid, exposed through a Pangolin/Newt tunnel with Authelia providing OIDC authentication.
|
|
|
|
---
|
|
|
|
## System Overview
|
|
|
|
FamilySync follows a layered architecture with Fastmail CalDAV as the external calendar source of truth. The React PWA talks exclusively to a single Hono API backend. The backend handles authentication (local username/password and/or Authelia OIDC), calendar read/write via CalDAV (Fastmail), list persistence (MariaDB), and real-time push delivery.
|
|
|
|
```mermaid
|
|
graph TD
|
|
subgraph "Client (Browser / Home Screen PWA)"
|
|
PWA["React 19 PWA\n(Vite + vite-plugin-pwa)"]
|
|
SW["Service Worker\n(Workbox precache + push)"]
|
|
end
|
|
|
|
subgraph "API (apps/api — Hono on Node 22)"
|
|
AUTH["Auth Layer\n(local session + OIDC middleware)"]
|
|
ROUTES["API Routes\n/events /lists /me /push /sse\n/admin /setup /auth"]
|
|
BROKER["CalDAV Broker\n(tsdav + ical.js + rrule)"]
|
|
OUTBOX["Outbox Worker\n(15s drain loop)"]
|
|
POLLER["CalDAV Poller\n(5-min setInterval)"]
|
|
REMINDER["Reminder Scheduler\n(1-min setInterval)"]
|
|
SSE_LIB["List Emitter\n(in-process EventEmitter)"]
|
|
PUSH_LIB["Push Dispatcher\n(web-push VAPID)"]
|
|
end
|
|
|
|
subgraph "Persistence"
|
|
DB["MariaDB 11\n(Drizzle ORM)"]
|
|
end
|
|
|
|
subgraph "External Services"
|
|
AUTHELIA["Authelia\n(OIDC / OAuth2 IdP — optional)"]
|
|
FASTMAIL["Fastmail CalDAV\ncaldav.fastmail.com"]
|
|
PUSH_SVC["Browser Push Services\n(APNs / FCM)"]
|
|
end
|
|
|
|
PWA -- "HTTPS (same-origin via Pangolin)" --> ROUTES
|
|
SW -- "push events" --> PWA
|
|
ROUTES --> AUTH
|
|
AUTH -- "authorization-code + PKCE (when oidcEnabled)" --> AUTHELIA
|
|
ROUTES --> BROKER
|
|
ROUTES --> SSE_LIB
|
|
ROUTES --> DB
|
|
BROKER -- "PROPFIND / REPORT / PUT / DELETE" --> FASTMAIL
|
|
POLLER --> BROKER
|
|
OUTBOX --> BROKER
|
|
BROKER --> DB
|
|
REMINDER --> DB
|
|
REMINDER --> PUSH_LIB
|
|
OUTBOX --> PUSH_LIB
|
|
SSE_LIB -- "text/event-stream" --> PWA
|
|
PUSH_LIB -- "VAPID push" --> PUSH_SVC
|
|
PUSH_SVC --> SW
|
|
```
|
|
|
|
---
|
|
|
|
## Directory Structure
|
|
|
|
```text
|
|
familysync/
|
|
├── apps/
|
|
│ ├── api/ # Hono backend (Node 22 + TypeScript)
|
|
│ │ └── src/
|
|
│ │ ├── index.ts # App entry: mounts routes, starts background workers
|
|
│ │ ├── routes/ # HTTP route handlers
|
|
│ │ ├── auth/ # OIDC middleware + local auth + dev-bypass + session persistence
|
|
│ │ ├── broker/ # CalDAV integration layer
|
|
│ │ ├── db/ # Drizzle schema, client, migrations
|
|
│ │ └── lib/ # Shared dispatchers and utilities
|
|
│ └── pwa/ # React 19 PWA (Vite + vite-plugin-pwa)
|
|
│ └── src/
|
|
│ ├── App.tsx # BrowserRouter shell with persistent nav chrome
|
|
│ ├── routes/ # Page-level React components
|
|
│ ├── components/ # Shared UI components
|
|
│ ├── api/ # Typed fetch wrappers (client.ts, listsClient.ts)
|
|
│ ├── hooks/ # Custom hooks (useListSSE, usePushSubscription)
|
|
│ ├── store/ # Zustand UI-state stores
|
|
│ ├── lib/ # Pure utilities (event date/time, login redirect)
|
|
│ └── sw.ts # Custom Workbox service worker
|
|
├── docker-compose.yml # Production stack
|
|
└── docker-compose.dev.yml # Dev overrides (port-binds, volume mounts)
|
|
```
|
|
|
|
### Directory Rationale
|
|
|
|
| Directory | Purpose |
|
|
| ---------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
|
|
| `apps/api/src/routes/` | One file per resource — `events.ts`, `lists.ts`, `me.ts`, `push.ts`, `sse.ts`, `health.ts`, `admin.ts`, `setup.ts`, `authMode.ts`, `localAuth.ts` |
|
|
| `apps/api/src/broker/` | All CalDAV I/O: `client.ts` (tsdav factory), `sync.ts` (REPORT→DB), `poller.ts` (5-min ctag check), `outboxWorker.ts` (async write-back), `expand.ts` (RRULE expansion), `write.ts` (PUT/DELETE), `vevent.ts` (ICS builder), `crypto.ts` (AES-256-GCM for app passwords) |
|
|
| `apps/api/src/auth/` | `middleware.ts` (re-exports `@hono/oidc-auth`), `devBypass.ts` (DEV_AUTH_BYPASS inject), `localAuthMiddleware.ts` (local-session cookie → user), `localCredentials.ts` (scrypt hash/verify), `localSession.ts` (JWT cookie issue/verify/clear), `oidcConfig.ts` (env+DB fallback for OIDC config), `linkNonceStore.ts` (single-use OIDC-link nonces), `linkOidc.ts` (bind OIDC identity to local user), `persistSessionCookie.ts` (session lifetime extension), `user.ts` (upsert on first OIDC login) |
|
|
| `apps/api/src/db/` | `schema.ts` (Drizzle `mysqlTable` definitions), `client.ts` (mysql2 pool), `migrations/` (drizzle-kit output) |
|
|
| `apps/api/src/lib/` | Stateless helpers: `listEmitter.ts` (EventEmitter fan-out), `listChangeDispatcher.ts`, `eventChangeDispatcher.ts`, `pushDispatcher.ts` (VAPID send), `pushCoalescer.ts`, `listAccess.ts`, `rank.ts` (fractional indexing), `bootGuards.ts` (startup safety assertions), `requireAdmin.ts` (admin-role guard), `setupGuard.ts` (isSetupLocked check) |
|
|
| `apps/pwa/src/api/` | Thin typed fetch wrappers — `client.ts` (events, me, sync-status, auth-mode, local login/logout), `listsClient.ts` (lists and items) |
|
|
| `apps/pwa/src/store/` | `calendarStore.ts` and `listsStore.ts` — Zustand UI-only state (no server data) |
|
|
| `apps/pwa/src/hooks/` | `useListSSE.ts` (bounded-backoff EventSource), `usePushSubscription.ts` (VAPID subscribe) |
|
|
|
|
---
|
|
|
|
## Key Abstractions
|
|
|
|
| Abstraction | File | Description |
|
|
| ------------------------------------------------------ | ------------------------------------------ | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
|
| `app` (Hono) | `apps/api/src/index.ts` | Root Hono app; mounts all routes and serves the PWA static build |
|
|
| Drizzle schema | `apps/api/src/db/schema.ts` | Single source of truth for all table definitions (`users`, `memberCredentials`, `localCredentials`, `calendars`, `calendarEvents`, `calendarOutbox`, `lists`, `listShares`, `listItems`, `pushSubscriptions`, `appConfig`) |
|
|
| `syncCalendar` | `apps/api/src/broker/sync.ts` | REPORT → ical.js parse → `onDuplicateKeyUpdate` upsert into MariaDB |
|
|
| `expandOccurrences` | `apps/api/src/broker/expand.ts` | Server-side RRULE expansion using `ical.js` + `rrule`; never runs in the browser |
|
|
| `CalendarOccurrence` | `apps/api/src/broker/expand.ts` | Wire type for a single concrete event occurrence; mirrored in the PWA's `api/client.ts` |
|
|
| `calendarOutbox` table | `apps/api/src/db/schema.ts` | Transactional outbox pattern — CalDAV writes are enqueued here and drained asynchronously |
|
|
| `runOutboxDrain` | `apps/api/src/broker/outboxWorker.ts` | Drains pending outbox rows every 15s; handles retry backoff, 412 conflict, dead-lettering, and edit-as-move ordering |
|
|
| `publishListEvent` / `subscribeListEvents` | `apps/api/src/lib/listEmitter.ts` | In-process EventEmitter fan-out keyed per list; SSE route subscribes on open and unsubscribes on disconnect |
|
|
| `dispatchPush` | `apps/api/src/lib/pushDispatcher.ts` | Centralised VAPID-signed push sender; handles 410/404 subscription pruning |
|
|
| `issueLocalSessionCookie` / `verifyLocalSessionCookie` | `apps/api/src/auth/localSession.ts` | Issues and verifies the `local-session` JWT cookie used by local username/password auth |
|
|
| `localAuthMiddleware` | `apps/api/src/auth/localAuthMiddleware.ts` | Reads `local-session` cookie → populates `c.get('user')`; no-op passthrough when cookie absent (OIDC guard fires for unauthenticated requests) |
|
|
| `linkOidcToUser` / `OidcLinkConflictError` | `apps/api/src/auth/linkOidc.ts` | Binds an OIDC iss+sub to an existing local user; throws `OidcLinkConflictError` on identity collision |
|
|
| `localCredentials` table | `apps/api/src/db/schema.ts` | Per-member local login credentials (scrypt PHC hash); a row exists iff the member can log in with username/password |
|
|
| `appConfig` table | `apps/api/src/db/schema.ts` | Key/value store for setup wizard output (OIDC config, VAPID public key, setup_complete flag) |
|
|
| `isSetupLocked` | `apps/api/src/lib/setupGuard.ts` | Returns true when the first-run wizard is complete; setup mutation routes call this as their first guard |
|
|
| `SessionExpiredError` | `apps/pwa/src/api/client.ts` | Typed error thrown by all fetch wrappers on 401/opaqueredirect; global `QueryCache` handler arms the session-expiry interstitial |
|
|
| Zustand stores | `apps/pwa/src/store/` | UI-only ephemeral state (open panels, selected date, active tab); server state always in TanStack Query |
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### Calendar read (typical request)
|
|
|
|
1. On mount, the PWA's `CalendarShell` computes a date window and fires `fetchEvents(start, end)` via TanStack Query.
|
|
2. `GET /api/events?start=&end=` hits the Hono `eventsRouter`.
|
|
3. The route queries `calendarEvents` + `calendars` from MariaDB, filtering by the authenticated user's accessible calendars.
|
|
4. Raw `rawVevent` blobs are passed through `expandOccurrences()` (ical.js + rrule) to produce concrete `CalendarOccurrence` objects for the window.
|
|
5. The JSON response is cached by TanStack Query; Schedule-X renders the events.
|
|
|
|
### Calendar write (create/edit/delete)
|
|
|
|
1. The PWA calls `POST /api/events/create` (or `PATCH /api/events/:uid/edit` / `DELETE /api/events/:uid`).
|
|
2. The route validates the payload with Zod, assigns a UID, and inserts a `pending` row into `calendarOutbox`. Returns `202 Accepted` immediately.
|
|
3. Every 15 seconds `runOutboxDrain` picks up `pending` rows, decrypts the member's Fastmail app password (AES-256-GCM), and dispatches the CalDAV write via `tsdav` (PUT/DELETE).
|
|
4. On success, the worker triggers a targeted `syncCalendar` re-sync for that calendar, then marks the outbox row `done`.
|
|
5. The PWA's `SyncStateToast` polls `GET /api/events/sync-status?uid=` and invalidates the `['events']` TanStack Query cache when status transitions to `done`.
|
|
|
|
### Calendar background sync (poller)
|
|
|
|
1. `startBrokerPoller` fires every 5 minutes via `setInterval`.
|
|
2. For each row in `member_credentials`, the poller decrypts the app password, builds a `tsdav` client, and runs `fetchCalendars()` (PROPFIND).
|
|
3. Each calendar's live `ctag` is compared against the stored value. If unchanged, the calendar is skipped.
|
|
4. On change, `syncCalendar` fetches all objects (REPORT), parses with ical.js, and upserts `calendarEvents` rows.
|
|
5. Detected event changes are passed to `dispatchEventChange`, which sends VAPID push notifications to non-actor members' subscriptions.
|
|
|
|
### Live list sync
|
|
|
|
1. When the PWA opens a list, `useListSSE` opens an `EventSource` to `GET /api/sse/lists` with `withCredentials: true`.
|
|
2. The SSE route resolves the caller's accessible list IDs via `getAccessibleListIds`, then calls `subscribeListEvents(listId, handler)` for each one.
|
|
3. When any list is mutated via the REST routes (`POST /api/lists`, `PATCH /api/list-items/:id`, etc.), the route calls `publishListEvent(listId, event)`.
|
|
4. The in-process `EventEmitter` fans the event out to all active `subscribeListEvents` handlers, which write it to the SSE stream.
|
|
5. The PWA's SSE handler calls `queryClient.invalidateQueries(['list', listId])`, triggering a full refetch.
|
|
6. A 30-second polling fallback (`refetchInterval: 30000`) is always active in `ListDetail` as a safety net.
|
|
|
|
### Authentication
|
|
|
|
The app supports two auth modes, selectable per-deployment and per-user. `GET /api/auth/mode` (pre-auth) tells the PWA which modes are active.
|
|
|
|
**Local auth path (Phase 19):**
|
|
|
|
1. The PWA fetches `GET /api/auth/mode`; when `localEnabled === true` it renders `/login` (`LoginPage`).
|
|
2. The user submits credentials; the PWA calls `POST /api/auth/local/login`.
|
|
3. The route verifies the scrypt hash from `local_credentials`, then calls `issueLocalSessionCookie` — a signed HS256 JWT issued as a `local-session` HttpOnly cookie.
|
|
4. On subsequent requests, `localAuthMiddleware` reads the cookie, verifies the JWT, fetches the users row, and populates `c.get('user')`. The OIDC guard is skipped when `c.get('user')` is already set.
|
|
5. A local user can optionally link an OIDC identity via `POST /api/me/link-oidc`; on completion `linkOidcToUser` binds `oidc_iss`/`oidc_sub` to the users row and deletes the `local_credentials` row, converting the account to OIDC-only.
|
|
|
|
**OIDC path (Authelia):**
|
|
|
|
1. An unauthenticated browser navigates to `/api/login`.
|
|
2. The `oidcAuthMiddleware` (`@hono/oidc-auth`) issues a `302` to Authelia's `/authorize` endpoint with PKCE (S256).
|
|
3. After login, Authelia posts the authorization code to `/callback`; `processOAuthCallback` exchanges it for tokens and issues a signed JWT session cookie.
|
|
4. `persistSessionCookie` middleware re-issues the cookie as persistent on every authenticated response so the PWA session survives browser close.
|
|
5. All `/api/*` routes require a session (local or OIDC); a missing or expired session returns a `302` which the PWA's fetch wrappers detect as `opaqueredirect` and convert to a `SessionExpiredError`.
|
|
|
|
---
|
|
|
|
## Component Interaction
|
|
|
|
### Backend modules
|
|
|
|
```text
|
|
routes/events.ts ──→ broker/expand.ts (read: RRULE expansion)
|
|
──→ calendarOutbox (DB) (write: enqueue)
|
|
──→ broker/sync.ts (write-sync after outbox drain)
|
|
|
|
broker/outboxWorker.ts ──→ broker/write.ts (CalDAV PUT/DELETE)
|
|
──→ broker/sync.ts (targeted re-sync)
|
|
──→ lib/eventChangeDispatcher.ts → lib/pushDispatcher.ts
|
|
|
|
broker/poller.ts ──→ broker/sync.ts
|
|
──→ lib/eventChangeDispatcher.ts
|
|
|
|
routes/lists.ts ──→ db (MariaDB)
|
|
──→ lib/listEmitter.ts (publishListEvent)
|
|
|
|
routes/sse.ts ──→ lib/listEmitter.ts (subscribeListEvents)
|
|
──→ lib/listAccess.ts
|
|
|
|
routes/localAuth.ts ──→ auth/localCredentials.ts (scrypt verify)
|
|
──→ auth/localSession.ts (issue cookie)
|
|
|
|
routes/admin.ts ──→ db (MariaDB: users, local_credentials, calendars)
|
|
──→ broker/credentialSync.ts (validate+encrypt+store)
|
|
──→ lib/requireAdmin.ts (role guard)
|
|
|
|
routes/setup.ts ──→ db (app_config)
|
|
──→ lib/setupGuard.ts (isSetupLocked)
|
|
```
|
|
|
|
### Frontend data ownership
|
|
|
|
| Concern | Owner |
|
|
| ----------------------------- | ---------------------------------------------- |
|
|
| Calendar events | TanStack Query `['events', start, end]` |
|
|
| List data + items | TanStack Query `['lists']`, `['list', listId]` |
|
|
| Current user | TanStack Query `['me']` |
|
|
| Writable calendars | TanStack Query `['writableCalendars']` |
|
|
| Outbox sync status | TanStack Query `['syncStatus', uid]` |
|
|
| Auth mode (local/OIDC flags) | TanStack Query `['authMode']` |
|
|
| Setup completion status | TanStack Query `['setupStatus']` |
|
|
| Selected calendar view + date | Zustand `calendarStore` |
|
|
| Event form open/mode | Zustand `calendarStore` |
|
|
| Active tab, create-list sheet | Zustand `listsStore` |
|
|
|
|
---
|
|
|
|
## Infrastructure
|
|
|
|
| Component | Technology |
|
|
| -------------------- | --------------------------------------------------------------------------------------------------------------------------- |
|
|
| Runtime | Node.js 22 LTS |
|
|
| HTTP framework | Hono 4.x (`@hono/node-server`) |
|
|
| Database | MariaDB 11 (Docker volume) |
|
|
| ORM | Drizzle ORM 0.45.x (`mysql2` dialect) |
|
|
| Auth IdP | Authelia (pre-deployed, external) — OIDC authorization code + PKCE; optional when local auth is enabled |
|
|
| Session middleware | `@hono/oidc-auth` (OIDC session — storage-less signed JWT cookies) + custom `localSession.ts` (local-auth HS256 JWT cookie) |
|
|
| Calendar source | Fastmail CalDAV (`caldav.fastmail.com`) — read via `tsdav`, write via transactional outbox |
|
|
| Calendar parsing | `ical.js` (VCALENDAR/VEVENT parse) + `rrule` (RRULE expansion) |
|
|
| App password storage | AES-256-GCM encrypted in `member_credentials.encrypted_password` |
|
|
| Local auth storage | scrypt PHC hash in `local_credentials.password_hash`; session signed with `LOCAL_SESSION_SECRET` env var |
|
|
| Push notifications | `web-push` (VAPID) → APNs (iOS) / FCM (Android) |
|
|
| Live list sync | In-process Node.js `EventEmitter` → SSE (`text/event-stream`) |
|
|
| Redis | Present in stack (image: `redis:7-alpine`); not used in current runtime (reserved for future multi-process pub/sub) |
|
|
| PWA | React 19 + Vite 8 + `vite-plugin-pwa` (Workbox `injectManifest` mode) |
|
|
| Networking | Pangolin/Newt tunnel — no open ports; split-DNS internal domain |
|
|
| Deployment | Docker Compose on Unraid; single `api` container serves both the API and the PWA static build |
|