familysync/.planning/research/ARCHITECTURE.md

Architecture Research

Domain: Self-hosted family calendar + list hub (Fastmail broker + MariaDB + React PWA) Researched: 2026-06-03 Confidence: MEDIUM (Fastmail personal-calendar ACL mechanics unconfirmed — see flags below)

Standard Architecture

System Overview

graph TD
    subgraph Public["Public Internet"]
        iOS["iOS PWA (Safari)"]
        Android["Android PWA (Chrome)"]
    end

    subgraph Tunnel["Pangolin/Newt Tunnel (no open ports)"]
        direction TB
        Authelia["Authelia OIDC\n(already deployed)"]
    end

    subgraph Docker["Docker Compose — Unraid"]
        direction TB
        PWA["React PWA\n(static, served by API or nginx)"]
        API["App API\n(Node/Express or Fastify)"]

        subgraph Broker["Fastmail Broker Layer"]
            CalDAVClient["CalDAV Client\n(node-caldav / tsdav)"]
            Cache["Calendar Cache\n(MariaDB: events + ctag)"]
            Poller["Background Poller\n(cron, 5-min interval)"]
        end

        subgraph Lists["List Domain"]
            ListAPI["List CRUD\n(REST endpoints)"]
            ListDB["lists / items tables\n(MariaDB)"]
        end

        subgraph Realtime["Real-time Layer"]
            WSHub["WebSocket Hub\n(ws or Socket.IO)"]
            RedisPubSub["Redis Pub/Sub\n(optional, single instance OK)"]
        end

        subgraph Push["Web Push"]
            PushSender["VAPID Push Sender\n(web-push npm)"]
            SubStore["push_subscriptions table\n(MariaDB)"]
            Scheduler["Reminder Scheduler\n(node-cron)"]
        end

        MariaDB[("MariaDB\n(users, calendars_cache,\nlists, push_subscriptions)")]
        Redis[("Redis\n(pub/sub channels)")]
    end

    subgraph Fastmail["Fastmail (external, source of truth)"]
        FM_Shared["Shared Family Calendar"]
        FM_Personal["Personal Calendar Collections\n(one per user)"]
    end

    iOS -->|HTTPS via tunnel| Authelia
    Android -->|HTTPS via tunnel| Authelia
    Authelia -->|forwards authed request| API
    PWA <-->|REST + WebSocket| API

    API --> CalDAVClient
    API --> ListAPI
    API --> WSHub

    CalDAVClient <-->|CalDAV over HTTPS\napp password auth| FM_Shared
    CalDAVClient <-->|CalDAV over HTTPS\nper-user app password OR shared ACL| FM_Personal
    CalDAVClient --> Cache

    Poller -->|every 5 min: PROPFIND ctag| CalDAVClient
    Poller -->|on ctag change: full sync| Cache

    ListAPI --> ListDB
    ListDB --> MariaDB
    Cache --> MariaDB

    WSHub <-->|subscribe/publish| RedisPubSub
    RedisPubSub --> Redis

    ListAPI -->|on write| RedisPubSub
    PushSender --> SubStore
    SubStore --> MariaDB
    Scheduler -->|check upcoming events| MariaDB
    Scheduler --> PushSender
    ListAPI -->|on important change| PushSender

Component Responsibilities

Component	Responsibility	Notes
React PWA	All UI rendering; calendar view, list co-edit, push subscription registration	Static build; served from API container or separate nginx
App API	Auth middleware (OIDC token validation), REST endpoints, WebSocket upgrade, orchestrates broker + list + push	Single Node process; Fastify recommended for performance
CalDAV Client	Issues PROPFIND/REPORT against caldav.fastmail.com using app password(s); parses iCalendar	Use tsdav (TypeScript, actively maintained) or node-ical for parsing
Calendar Cache	Stores raw VEVENT blobs + ctag per calendar in MariaDB; serves as the read path for the API	Never exposed directly; always through API
Background Poller	Cron job running inside API process; checks ctag every 5 min, triggers full REPORT sync on change	5-min polling is sufficient for family use; no Fastmail push webhook available
List CRUD	REST handlers for list/item create-read-update-delete; emits to Redis on every write	Simple; MariaDB is source of truth
WebSocket Hub	Maintains open connections per authenticated user; pushes Redis messages to correct connections	Keyed by user ID extracted from OIDC sub claim
Redis Pub/Sub	Message bus for list change events; decouples list writes from WebSocket delivery	Single-instance Redis is fine for 2-person household; no clustering needed
VAPID Push Sender	Calls browser push services (FCM, APNs Web Push) with encrypted payloads	web-push npm package; keys stored in environment, not DB
Reminder Scheduler	Cron job: queries events starting in next 15 min (configurable), fires push notifications	Runs inside API process; reads from calendar cache
MariaDB	Persistent storage: users, calendar event cache, lists, push subscriptions	Single source for everything the app owns

Recommended Project Structure

familysync/
├── apps/
│   ├── api/                    # Node backend
│   │   ├── src/
│   │   │   ├── auth/           # OIDC token validation, user upsert
│   │   │   ├── broker/         # Fastmail CalDAV client + cache sync
│   │   │   │   ├── client.ts   # tsdav wrapper
│   │   │   │   ├── poller.ts   # ctag polling cron
│   │   │   │   ├── sync.ts     # REPORT sync → DB write
│   │   │   │   └── expand.ts   # RRULE expansion + timezone normalise
│   │   │   ├── calendars/      # REST routes: GET /calendars, GET /events
│   │   │   ├── lists/          # REST routes + Redis emit
│   │   │   ├── push/           # VAPID sender, subscription routes, scheduler
│   │   │   ├── realtime/       # WebSocket server, Redis subscriber
│   │   │   ├── db/             # Knex migrations + query helpers (MariaDB)
│   │   │   └── server.ts       # Fastify app bootstrap
│   │   └── Dockerfile
│   └── pwa/                    # React PWA
│       ├── src/
│       │   ├── features/
│       │   │   ├── calendar/   # Calendar view, event form
│       │   │   └── lists/      # List view, item row, optimistic updates
│       │   ├── auth/           # OIDC redirect handling, token storage
│       │   ├── push/           # Service worker registration, push consent
│       │   └── api/            # Typed fetch client
│       ├── public/
│       │   └── sw.js           # Service worker (push + offline cache)
│       └── Dockerfile
├── db/
│   └── migrations/             # Knex migration files (versioned)
├── docker-compose.yml
└── .env.example

Structure Rationale

• apps/api/src/broker/: Isolation boundary. All Fastmail I/O lives here. Nothing else imports from broker/ except the calendars/ routes and push/scheduler. Replacing CalDAV with JMAP later touches only this module.
• apps/api/src/db/: Knex chosen over an ORM because MariaDB lacks full Prisma support for some JSON features; Knex gives SQL control without raw strings everywhere.
• apps/pwa/src/features/: Feature-sliced structure keeps calendar and list concerns separated. Both features share the same WebSocket connection managed at app root.

Architectural Patterns

Pattern 1: Broker Cache with ctag-based Invalidation

What: The API never fetches from Fastmail on a user request. All calendar reads hit the MariaDB cache. A background poller issues a lightweight CalDAV PROPFIND to check the getctag property on each calendar collection every 5 minutes. On change, it issues a REPORT calendar-query (or sync-collection if the server supports it) to fetch only changed VEVENTs, then upserts into the cache.

When to use: Always — Fastmail has no push webhook for calendar changes. Polling is the only option.

Trade-offs: 5-min staleness is acceptable for a family calendar. Reduces Fastmail API calls to near-zero during quiet periods. Cache warms on first poll after startup.

// broker/poller.ts (sketch)
async function poll(calendarUrl: string, knownCtag: string | null) {
  const ctag = await fetchCtag(calendarUrl); // PROPFIND
  if (ctag === knownCtag) return; // no change
  await syncCalendar(calendarUrl); // REPORT → upsert cache
  await db('calendars').where({ url: calendarUrl }).update({ ctag, synced_at: new Date() });
}

Pattern 2: Single-Broker-Token with Per-Calendar App Passwords

What: The broker holds credentials for the primary Fastmail account (the project owner). The shared family calendar is readable/writable with that account's app password. For personal calendars of other household members (the wife's personal calendar), Fastmail requires that the owner either:

1. Share the calendar with the broker account — the broker then sees it appear at a different URL under its own principal (confirmed: Fastmail supports CalDAV sharing with read or read-write ACL to another Fastmail user).
2. Provide a separate app password per user — the broker authenticates as each user independently to read their personal calendar.

Option 1 (sharing to the broker account) is preferred: one credential, no password management per user. The wife logs into Fastmail once, shares her personal calendar with the primary account, done. The broker discovers it via CalDAV principal discovery.

Confidence: LOW on exact URL behavior. Fastmail's sharing documentation confirms the sharing feature exists and follows CalDAV draft ACL standards, but does not explicitly document whether the shared calendar appears in the principal's list discovery. Treat this as a phase research item — test during the broker phase with a real Fastmail multi-user or sharing setup.

Trade-offs: If sharing is not discoverable by principal, fall back to individual app passwords stored encrypted in .env. Two-person household means at most 2 credentials.

Pattern 3: Redis Pub/Sub for List Co-Edit

What: On every list/item write that succeeds in MariaDB, the API handler publishes a message to a Redis channel keyed by list ID. The WebSocket hub is subscribed to all relevant channels and pushes the serialized diff to connected clients.

When to use: Whenever two users might be editing the same list concurrently. For 2 people, this is "always."

Trade-offs: Redis is optional in the stack — if Redis is unavailable, fall back to polling (client polls /lists/{id} every 10s). Design the list API to support both modes via a feature flag.

Client A writes item → POST /lists/1/items
  → DB insert
  → Redis PUBLISH list:1 { type: "item_added", item: {...} }
  → WebSocket hub receives → pushes to all sockets subscribed to list:1
Client B receives message → appends item to local state (no refetch)

Pattern 4: RRULE Expansion Client-Side from Stored Raw VEVENT

What: Store the raw VEVENT blob (including RRULE) in the cache. On read, the API returns the raw iCal data or a partially processed event object that includes the RRULE string. The PWA expands recurrences client-side using the rrule.js library for the displayed date window (current month ± 1 month).

When to use: For this scale. Server-side pre-expansion into a flat events table is over-engineering for a 2-person calendar.

Trade-offs: Timezone correctness requires the PWA to use the TZID from the VEVENT and the Intl API to resolve DST. The rrule.js library handles TZID correctly when fed the VTIMEZONE component. Store all DTSTART values in UTC in the DB alongside the raw blob for efficient range queries.

Data Flow

Calendar Read (PWA → API → Cache)

User opens calendar view (month X)
  → GET /api/events?start=...&end=...
  → API queries MariaDB: SELECT * FROM events WHERE dtstart BETWEEN ? AND ?
  → Returns event objects (RRULE included)
  → PWA rrule.js expands recurring events for display window
  → Render color-coded events (color from user_calendars.color config)

Calendar Write (PWA → API → Fastmail)

User creates/edits event in PWA
  → POST/PUT /api/events
  → API validates auth, resolves target calendar from event's calendarId
  → broker/client.ts issues CalDAV PUT (new VEVENT) to caldav.fastmail.com
  → On 201/204 success: upsert into local cache immediately (no wait for next poll)
  → Return 201 to PWA
  → PWA updates local state optimistically (already done before response)

Fastmail Sync Loop (background)

Poller tick (every 5 min)
  → PROPFIND each tracked calendar URL → fetch getctag
  → Compare ctag to DB stored value
  → If changed: REPORT calendar-query for the collection
    → Parse iCalendar response
    → For each VEVENT: upsert events table (uid as key), update dtstart UTC
    → Update calendars.ctag
  → If reminder threshold: Scheduler fires Web Push

List Co-Edit (real-time)

User A edits list item text
  → PATCH /api/lists/1/items/42 { text: "oat milk" }
  → DB UPDATE → success
  → Redis PUBLISH list:1 { op: "update", itemId: 42, text: "oat milk", userId: "A" }
  → WS Hub (subscribed to list:1) receives message
  → Push to all WebSocket connections for users in list:1 EXCEPT sender
  → User B's PWA receives → update item in list state

Web Push Delivery

Reminder Scheduler (cron, runs every minute)
  → SELECT events WHERE dtstart BETWEEN now+14min AND now+15min
    AND reminder_sent = false
  → For each event: find household users who have visibility to that calendar
  → For each user: SELECT push_subscriptions WHERE user_id = ?
  → For each subscription: web-push sendNotification(subscription, payload)
  → Mark reminder_sent = true

Identity Mapping (OIDC → App User)

User authenticates via Authelia OIDC
  → PWA redirects to /auth/login → Authelia
  → Authelia returns id_token with claims: sub, email, preferred_username, groups
  → API receives id_token in Authorization header (or httpOnly cookie)
  → Validate token signature against Authelia JWKS endpoint
  → Look up users table WHERE oidc_sub = claims.sub
  → If not found: INSERT (first login provisioning) using email as display hint
  → Attach app user_id to request context
  → user_id is used for: calendar color config, push subscriptions, WS channel auth

Key: Use oidc_sub (the sub claim) as the stable identity anchor. Never use email as a unique key — it can change per Authelia docs. Store oidc_iss + oidc_sub as the composite stable ID.

Component Build Order (Dependencies)

Build in this sequence — each layer depends on the previous:

1. Infrastructure scaffold
   docker-compose.yml (API + MariaDB + Redis), DB migrations, .env wiring

2. Auth layer
   OIDC middleware (validate Authelia tokens, user upsert), session/cookie setup
   [nothing works without identity]

3. Fastmail broker — read path only
   CalDAV client, calendar discovery, cache sync, poller, event query API
   [needed before any calendar UI]

4. Calendar PWA — read only
   React calendar view consuming /api/events, color coding per calendar
   [validates broker correctness before write complexity]

5. Fastmail broker — write path
   CalDAV PUT for create/edit/delete, optimistic cache upsert
   [built on confirmed-working read path]

6. Lists domain
   MariaDB list/item schema, REST CRUD, basic PWA list UI
   [independent of calendar; can parallel-track after step 3]

7. Real-time list sync
   Redis pub/sub, WebSocket hub, PWA WebSocket client
   [enhancement on top of working list CRUD]

8. Web Push
   VAPID key generation, push_subscriptions table, service worker, reminder scheduler
   [built last; depends on events being cached and users having identity]

Parallel tracks available: Lists (step 6) can start in parallel with calendar write (step 5) once auth and broker read are stable.

Integration Points

External Services

Service	Integration Pattern	Auth	Gotchas
Fastmail CalDAV	HTTP PROPFIND / REPORT / PUT via tsdav	App password (Basic Auth over HTTPS)	JMAP for calendars NOT available — CalDAV only. Use app password, not API token. Rate limits undocumented; 5-min polling is conservative.
Authelia OIDC	Authorization Code flow from PWA; JWT validation in API	OIDC client_id/secret in API env	Validate against Authelia JWKS URL, not a static key. Use sub not email as stable identity.
Browser Push Services (FCM/APNs Web Push)	VAPID via web-push npm	VAPID public/private key pair	iOS requires PWA installed to home screen (outside EU). Push endpoint URL comes from browser PushManager.subscribe().
Pangolin/Newt	Transparent tunnel — no changes to app	None (handled at infra layer)	API must trust X-Forwarded-For headers from Pangolin; set trustProxy: true in Fastify.

Internal Boundaries

Boundary	Communication	Notes
PWA ↔ API	REST (JSON) + WebSocket	REST for CRUD + auth; WS for real-time list push. Single domain, so no CORS needed in prod.
API ↔ Broker	Direct function call (same process)	Broker is a module, not a separate service. Keep it that way — no premature microservice split.
API ↔ MariaDB	Knex query builder	All queries go through db/ module. No raw SQL in route handlers.
API ↔ Redis	ioredis, publish on write	WS hub uses a separate ioredis subscriber connection (required by Redis subscribe mode).
Scheduler ↔ API	In-process cron (node-cron)	Poller and reminder scheduler run in same Node process. If process dies, they restart with it.

Anti-Patterns

Anti-Pattern 1: Fetching from Fastmail on Every Calendar Request

What people do: Proxy each PWA calendar load directly to CalDAV — no cache.

Why it's wrong: Fastmail CalDAV has undocumented rate limits. Cold requests are slow (multi-round-trip CalDAV discovery). Calendar loads become unavailable if Fastmail is unreachable.

Do this instead: Always read from the MariaDB cache. Write-back is synchronous (PUT + cache upsert). Background poller handles incoming changes from the Fastmail app.

Anti-Pattern 2: Using Email as the App User Key

What people do: Map Authelia email claim → users.email primary key, assume it's stable.

Why it's wrong: OpenID Connect spec does not guarantee email stability or uniqueness. Authelia docs explicitly warn against this.

Do this instead: Use oidc_iss + oidc_sub as the stable composite identity. Store email as a display/provisioning hint only.

Anti-Pattern 3: Storing Recurring Events Only as Expanded Instances

What people do: Pre-expand all RRULE events into individual rows in the DB to simplify queries.

Why it's wrong: Event edits (changing the series, adding exceptions) then require reconstructing the original RRULE plus EXDATE/RDATE logic — a nightmare. Fastmail is source of truth for RRULE; the cache should reflect it faithfully.

Do this instead: Store the raw VEVENT blob. Add a dtstart_utc column for range queries. Expand in the PWA for display.

Anti-Pattern 4: Separate Microservices for Broker, Lists, Push

What people do: Split the broker, list API, and push sender into separate containers because "separation of concerns."

Why it's wrong: For a 2-person household app on a single Unraid host, this is pure overhead. Three containers to coordinate, three deployment targets, inter-service HTTP overhead with zero benefit.

Do this instead: One API container. Three modules within it. Split to services only if one component needs independent scaling (it won't).

Anti-Pattern 5: Per-User CalDAV Credentials in the PWA

What people do: Give each household member their own Fastmail app password and have the PWA authenticate to CalDAV directly.

Why it's wrong: Exposes Fastmail credentials client-side; bypasses the broker cache; defeats the aggregation purpose.

Do this instead: All Fastmail access goes through the backend broker. The PWA authenticates only to the app API.

Scaling Considerations

This is a 2-person household app. Scaling is not a concern. The architecture should remain simple.

Concern	At 2 users (target)
CalDAV polling	Single poller, 5-min interval, negligible load
MariaDB	Single instance, no replication needed
Redis	Single instance, no clustering
WebSocket	Single Node process handles 2 connections trivially
Web Push	At most ~10 push subscriptions (phone + desktop per user)

Open Research Flags for Phase Research

These items require hands-on investigation during the broker implementation phase — they cannot be resolved by documentation alone:

1. Fastmail personal-calendar sharing visibility (MEDIUM risk): Can the broker account's CalDAV principal discovery see calendars that another Fastmail account has shared to it via the in-app sharing UI? If yes, one app password suffices. If no, the fallback is storing an encrypted app password per household member. Test this before building the broker's calendar-discovery logic.

2. Fastmail CalDAV rate limits (LOW risk): No documented rate limit for CalDAV. 5-min polling with ctag pre-check (one PROPFIND per calendar per tick) is conservative and unlikely to hit limits. Monitor HTTP 429 responses and back off exponentially if encountered.

3. iOS Push reliability (LOW risk for this use case): iOS 16.4+ supports Web Push only when the PWA is installed to the home screen. The wife will be asked to install the PWA as part of onboarding. Canada is not in the EU, so the iOS 17.4 DMA PWA restriction does not apply. Push is a "nice to have" alarm, not a safety-critical channel — missed notifications are acceptable.

Sources

• Fastmail API Documentation — CalDAV supported, JMAP calendars pending RFC finalization
• Fastmail: Calendar Sharing with Other Users — ACL tiers confirmed; cross-account token access not documented
• Fastmail: Shared Calendaring Improvements — CalDAV ACL standards alignment, per-user reminders
• Fastmail: App Passwords — CalDAV uses app passwords, not API tokens
• Using Fastmail with CalDAV libraries — Principal URL format, Cyrus IMAP internals
• Authelia OIDC Claims — iss+sub as stable identity anchor
• CalDAV ctag Extension — ctag-based efficient polling
• rrule.js — Client-side RRULE expansion
• web-push npm — VAPID push sender
• PWA iOS Push Limitations 2025 — iOS 16.4+ push support, EU exception
• Redis Pub/Sub + WebSocket pattern — Architecture reference

---

Architecture research for: FamilySync — self-hosted family calendar + list hub Researched: 2026-06-03