223 lines
13 KiB
Markdown
223 lines
13 KiB
Markdown
<!-- generated-by: gsd-doc-writer -->
|
|
# FamilySync Architecture
|
|
|
|
FamilySync is a self-hosted family organization hub — a unified, color-coded calendar and shared collaborative lists — delivered as a React PWA. The system is a two-container Docker Compose stack (API + MariaDB) running on Unraid, exposed through a Pangolin/Newt tunnel with Authelia providing OIDC authentication.
|
|
|
|
---
|
|
|
|
## System Overview
|
|
|
|
FamilySync follows a layered architecture with Fastmail CalDAV as the external calendar source of truth. The React PWA talks exclusively to a single Hono API backend. The backend handles authentication (Authelia OIDC), calendar read/write via CalDAV (Fastmail), list persistence (MariaDB), and real-time push delivery.
|
|
|
|
```mermaid
|
|
graph TD
|
|
subgraph "Client (Browser / Home Screen PWA)"
|
|
PWA["React 19 PWA\n(Vite + vite-plugin-pwa)"]
|
|
SW["Service Worker\n(Workbox precache + push)"]
|
|
end
|
|
|
|
subgraph "API (apps/api — Hono on Node 22)"
|
|
AUTH["OIDC Auth\n(@hono/oidc-auth)"]
|
|
ROUTES["API Routes\n/events /lists /me /push /sse"]
|
|
BROKER["CalDAV Broker\n(tsdav + ical.js + rrule)"]
|
|
OUTBOX["Outbox Worker\n(15s drain loop)"]
|
|
POLLER["CalDAV Poller\n(5-min setInterval)"]
|
|
REMINDER["Reminder Scheduler\n(1-min setInterval)"]
|
|
SSE_LIB["List Emitter\n(in-process EventEmitter)"]
|
|
PUSH_LIB["Push Dispatcher\n(web-push VAPID)"]
|
|
end
|
|
|
|
subgraph "Persistence"
|
|
DB["MariaDB 11\n(Drizzle ORM)"]
|
|
end
|
|
|
|
subgraph "External Services"
|
|
AUTHELIA["Authelia\n(OIDC / OAuth2 IdP)"]
|
|
FASTMAIL["Fastmail CalDAV\ncaldav.fastmail.com"]
|
|
PUSH_SVC["Browser Push Services\n(APNs / FCM)"]
|
|
end
|
|
|
|
PWA -- "HTTPS (same-origin via Pangolin)" --> ROUTES
|
|
SW -- "push events" --> PWA
|
|
ROUTES --> AUTH
|
|
AUTH -- "authorization-code + PKCE" --> AUTHELIA
|
|
ROUTES --> BROKER
|
|
ROUTES --> SSE_LIB
|
|
ROUTES --> DB
|
|
BROKER -- "PROPFIND / REPORT / PUT / DELETE" --> FASTMAIL
|
|
POLLER --> BROKER
|
|
OUTBOX --> BROKER
|
|
BROKER --> DB
|
|
REMINDER --> DB
|
|
REMINDER --> PUSH_LIB
|
|
OUTBOX --> PUSH_LIB
|
|
SSE_LIB -- "text/event-stream" --> PWA
|
|
PUSH_LIB -- "VAPID push" --> PUSH_SVC
|
|
PUSH_SVC --> SW
|
|
```
|
|
|
|
---
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
familysync/
|
|
├── apps/
|
|
│ ├── api/ # Hono backend (Node 22 + TypeScript)
|
|
│ │ └── src/
|
|
│ │ ├── index.ts # App entry: mounts routes, starts background workers
|
|
│ │ ├── routes/ # HTTP route handlers
|
|
│ │ ├── auth/ # OIDC middleware + dev-bypass + session persistence
|
|
│ │ ├── broker/ # CalDAV integration layer
|
|
│ │ ├── db/ # Drizzle schema, client, migrations
|
|
│ │ └── lib/ # Shared dispatchers and utilities
|
|
│ └── pwa/ # React 19 PWA (Vite + vite-plugin-pwa)
|
|
│ └── src/
|
|
│ ├── App.tsx # BrowserRouter shell with persistent nav chrome
|
|
│ ├── routes/ # Page-level React components
|
|
│ ├── components/ # Shared UI components
|
|
│ ├── api/ # Typed fetch wrappers (client.ts, listsClient.ts)
|
|
│ ├── hooks/ # Custom hooks (useListSSE, usePushSubscription)
|
|
│ ├── store/ # Zustand UI-state stores
|
|
│ ├── lib/ # Pure utilities (event date/time, login redirect)
|
|
│ └── sw.ts # Custom Workbox service worker
|
|
├── docker-compose.yml # Production stack
|
|
└── docker-compose.dev.yml # Dev overrides (port-binds, volume mounts)
|
|
```
|
|
|
|
### Directory Rationale
|
|
|
|
| Directory | Purpose |
|
|
|-----------|---------|
|
|
| `apps/api/src/routes/` | One file per resource — `events.ts`, `lists.ts`, `me.ts`, `push.ts`, `sse.ts`, `health.ts` |
|
|
| `apps/api/src/broker/` | All CalDAV I/O: `client.ts` (tsdav factory), `sync.ts` (REPORT→DB), `poller.ts` (5-min ctag check), `outboxWorker.ts` (async write-back), `expand.ts` (RRULE expansion), `write.ts` (PUT/DELETE), `vevent.ts` (ICS builder), `crypto.ts` (AES-256-GCM for app passwords) |
|
|
| `apps/api/src/auth/` | `middleware.ts` (re-exports `@hono/oidc-auth`), `devBypass.ts` (DEV_AUTH_BYPASS inject), `persistSessionCookie.ts` (session lifetime extension), `user.ts` (upsert on first OIDC login) |
|
|
| `apps/api/src/db/` | `schema.ts` (Drizzle `mysqlTable` definitions), `client.ts` (mysql2 pool), `migrations/` (drizzle-kit output) |
|
|
| `apps/api/src/lib/` | Stateless helpers: `listEmitter.ts` (EventEmitter fan-out), `listChangeDispatcher.ts`, `eventChangeDispatcher.ts`, `pushDispatcher.ts` (VAPID send), `pushCoalescer.ts`, `listAccess.ts`, `rank.ts` (fractional indexing) |
|
|
| `apps/pwa/src/api/` | Thin typed fetch wrappers — `client.ts` (events, me, sync-status), `listsClient.ts` (lists and items) |
|
|
| `apps/pwa/src/store/` | `calendarStore.ts` and `listsStore.ts` — Zustand UI-only state (no server data) |
|
|
| `apps/pwa/src/hooks/` | `useListSSE.ts` (bounded-backoff EventSource), `usePushSubscription.ts` (VAPID subscribe) |
|
|
|
|
---
|
|
|
|
## Key Abstractions
|
|
|
|
| Abstraction | File | Description |
|
|
|-------------|------|-------------|
|
|
| `app` (Hono) | `apps/api/src/index.ts` | Root Hono app; mounts all routes and serves the PWA static build |
|
|
| Drizzle schema | `apps/api/src/db/schema.ts` | Single source of truth for all table definitions (`users`, `memberCredentials`, `calendars`, `calendarEvents`, `calendarOutbox`, `lists`, `listShares`, `listItems`, `pushSubscriptions`) |
|
|
| `syncCalendar` | `apps/api/src/broker/sync.ts` | REPORT → ical.js parse → `onDuplicateKeyUpdate` upsert into MariaDB |
|
|
| `expandOccurrences` | `apps/api/src/broker/expand.ts` | Server-side RRULE expansion using `ical.js` + `rrule`; never runs in the browser |
|
|
| `CalendarOccurrence` | `apps/api/src/broker/expand.ts` | Wire type for a single concrete event occurrence; mirrored in the PWA's `api/client.ts` |
|
|
| `calendarOutbox` table | `apps/api/src/db/schema.ts` | Transactional outbox pattern — CalDAV writes are enqueued here and drained asynchronously |
|
|
| `runOutboxDrain` | `apps/api/src/broker/outboxWorker.ts` | Drains pending outbox rows every 15s; handles retry backoff, 412 conflict, dead-lettering, and edit-as-move ordering |
|
|
| `publishListEvent` / `subscribeListEvents` | `apps/api/src/lib/listEmitter.ts` | In-process EventEmitter fan-out keyed per list; SSE route subscribes on open and unsubscribes on disconnect |
|
|
| `dispatchPush` | `apps/api/src/lib/pushDispatcher.ts` | Centralised VAPID-signed push sender; handles 410/404 subscription pruning |
|
|
| `SessionExpiredError` | `apps/pwa/src/api/client.ts` | Typed error thrown by all fetch wrappers on 401/opaqueredirect; global `QueryCache` handler arms the session-expiry interstitial |
|
|
| Zustand stores | `apps/pwa/src/store/` | UI-only ephemeral state (open panels, selected date, active tab); server state always in TanStack Query |
|
|
|
|
---
|
|
|
|
## Data Flow
|
|
|
|
### Calendar read (typical request)
|
|
|
|
1. On mount, the PWA's `CalendarShell` computes a date window and fires `fetchEvents(start, end)` via TanStack Query.
|
|
2. `GET /api/events?start=&end=` hits the Hono `eventsRouter`.
|
|
3. The route queries `calendarEvents` + `calendars` from MariaDB, filtering by the authenticated user's accessible calendars.
|
|
4. Raw `rawVevent` blobs are passed through `expandOccurrences()` (ical.js + rrule) to produce concrete `CalendarOccurrence` objects for the window.
|
|
5. The JSON response is cached by TanStack Query; Schedule-X renders the events.
|
|
|
|
### Calendar write (create/edit/delete)
|
|
|
|
1. The PWA calls `POST /api/events/create` (or `PATCH /api/events/:uid/edit` / `DELETE /api/events/:uid`).
|
|
2. The route validates the payload with Zod, assigns a UID, and inserts a `pending` row into `calendarOutbox`. Returns `202 Accepted` immediately.
|
|
3. Every 15 seconds `runOutboxDrain` picks up `pending` rows, decrypts the member's Fastmail app password (AES-256-GCM), and dispatches the CalDAV write via `tsdav` (PUT/DELETE).
|
|
4. On success, the worker triggers a targeted `syncCalendar` re-sync for that calendar, then marks the outbox row `done`.
|
|
5. The PWA's `SyncStateToast` polls `GET /api/events/sync-status?uid=` and invalidates the `['events']` TanStack Query cache when status transitions to `done`.
|
|
|
|
### Calendar background sync (poller)
|
|
|
|
1. `startBrokerPoller` fires every 5 minutes via `setInterval`.
|
|
2. For each row in `member_credentials`, the poller decrypts the app password, builds a `tsdav` client, and runs `fetchCalendars()` (PROPFIND).
|
|
3. Each calendar's live `ctag` is compared against the stored value. If unchanged, the calendar is skipped.
|
|
4. On change, `syncCalendar` fetches all objects (REPORT), parses with ical.js, and upserts `calendarEvents` rows.
|
|
5. Detected event changes are passed to `dispatchEventChange`, which sends VAPID push notifications to non-actor members' subscriptions.
|
|
|
|
### Live list sync
|
|
|
|
1. When the PWA opens a list, `useListSSE` opens an `EventSource` to `GET /api/sse/lists` with `withCredentials: true`.
|
|
2. The SSE route resolves the caller's accessible list IDs via `getAccessibleListIds`, then calls `subscribeListEvents(listId, handler)` for each one.
|
|
3. When any list is mutated via the REST routes (`POST /api/lists`, `PATCH /api/list-items/:id`, etc.), the route calls `publishListEvent(listId, event)`.
|
|
4. The in-process `EventEmitter` fans the event out to all active `subscribeListEvents` handlers, which write it to the SSE stream.
|
|
5. The PWA's SSE handler calls `queryClient.invalidateQueries(['list', listId])`, triggering a full refetch.
|
|
6. A 30-second polling fallback (`refetchInterval: 30000`) is always active in `ListDetail` as a safety net.
|
|
|
|
### Authentication
|
|
|
|
1. An unauthenticated browser navigates to `/api/login`.
|
|
2. The `oidcAuthMiddleware` (`@hono/oidc-auth`) issues a `302` to Authelia's `/authorize` endpoint with PKCE (S256).
|
|
3. After login, Authelia posts the authorization code to `/callback`; `processOAuthCallback` exchanges it for tokens and issues a signed JWT session cookie.
|
|
4. `persistSessionCookie` middleware re-issues the cookie as persistent on every authenticated response so the PWA session survives browser close.
|
|
5. All `/api/*` routes require the session cookie; a missing or expired session returns a `302` which the PWA's fetch wrappers detect as `opaqueredirect` and convert to a `SessionExpiredError`.
|
|
|
|
---
|
|
|
|
## Component Interaction
|
|
|
|
### Backend modules
|
|
|
|
```
|
|
routes/events.ts ──→ broker/expand.ts (read: RRULE expansion)
|
|
──→ calendarOutbox (DB) (write: enqueue)
|
|
──→ broker/sync.ts (write-sync after outbox drain)
|
|
|
|
broker/outboxWorker.ts ──→ broker/write.ts (CalDAV PUT/DELETE)
|
|
──→ broker/sync.ts (targeted re-sync)
|
|
──→ lib/eventChangeDispatcher.ts → lib/pushDispatcher.ts
|
|
|
|
broker/poller.ts ──→ broker/sync.ts
|
|
──→ lib/eventChangeDispatcher.ts
|
|
|
|
routes/lists.ts ──→ db (MariaDB)
|
|
──→ lib/listEmitter.ts (publishListEvent)
|
|
|
|
routes/sse.ts ──→ lib/listEmitter.ts (subscribeListEvents)
|
|
──→ lib/listAccess.ts
|
|
```
|
|
|
|
### Frontend data ownership
|
|
|
|
| Concern | Owner |
|
|
|---------|-------|
|
|
| Calendar events | TanStack Query `['events', start, end]` |
|
|
| List data + items | TanStack Query `['lists']`, `['list', listId]` |
|
|
| Current user | TanStack Query `['me']` |
|
|
| Writable calendars | TanStack Query `['writableCalendars']` |
|
|
| Outbox sync status | TanStack Query `['syncStatus', uid]` |
|
|
| Selected calendar view + date | Zustand `calendarStore` |
|
|
| Event form open/mode | Zustand `calendarStore` |
|
|
| Active tab, create-list sheet | Zustand `listsStore` |
|
|
|
|
---
|
|
|
|
## Infrastructure
|
|
|
|
| Component | Technology |
|
|
|-----------|------------|
|
|
| Runtime | Node.js 22 LTS |
|
|
| HTTP framework | Hono 4.x (`@hono/node-server`) |
|
|
| Database | MariaDB 11 (Docker volume) |
|
|
| ORM | Drizzle ORM 0.45.x (`mysql2` dialect) |
|
|
| Auth IdP | Authelia (pre-deployed, external) — OIDC authorization code + PKCE |
|
|
| Session middleware | `@hono/oidc-auth` — storage-less signed JWT cookies |
|
|
| Calendar source | Fastmail CalDAV (`caldav.fastmail.com`) — read via `tsdav`, write via transactional outbox |
|
|
| Calendar parsing | `ical.js` (VCALENDAR/VEVENT parse) + `rrule` (RRULE expansion) |
|
|
| App password storage | AES-256-GCM encrypted in `member_credentials.encrypted_password` |
|
|
| Push notifications | `web-push` (VAPID) → APNs (iOS) / FCM (Android) |
|
|
| Live list sync | In-process Node.js `EventEmitter` → SSE (`text/event-stream`) |
|
|
| Redis | Present in stack (image: `redis:7-alpine`); not used in current runtime (reserved for future multi-process pub/sub) |
|
|
| PWA | React 19 + Vite 8 + `vite-plugin-pwa` (Workbox `injectManifest` mode) |
|
|
| Networking | Pangolin/Newt tunnel — no open ports; split-DNS internal domain |
|
|
| Deployment | Docker Compose on Unraid; single `api` container serves both the API and the PWA static build |
|