docs: v1.1 research (stack/features/architecture/pitfalls/summary)
This commit is contained in:
Lucas Berger
+509
-350
@@ -1,424 +1,583 @@
|
||||
# Architecture Research
|
||||
|
||||
**Domain:** Self-hosted family calendar + list hub (Fastmail broker + MariaDB + React PWA)
|
||||
**Researched:** 2026-06-03
|
||||
**Confidence:** MEDIUM (Fastmail personal-calendar ACL mechanics unconfirmed — see flags below)
|
||||
**Domain:** FamilySync v1.1 — integration analysis for Operability & Polish milestone
|
||||
**Researched:** 2026-06-10
|
||||
**Confidence:** HIGH (grounded in actual codebase)
|
||||
|
||||
## Standard Architecture
|
||||
|
||||
### System Overview
|
||||
|
||||
```mermaid
|
||||
graph TD
|
||||
subgraph Public["Public Internet"]
|
||||
iOS["iOS PWA (Safari)"]
|
||||
Android["Android PWA (Chrome)"]
|
||||
end
|
||||
|
||||
subgraph Tunnel["Pangolin/Newt Tunnel (no open ports)"]
|
||||
direction TB
|
||||
Authelia["Authelia OIDC\n(already deployed)"]
|
||||
end
|
||||
|
||||
subgraph Docker["Docker Compose — Unraid"]
|
||||
direction TB
|
||||
PWA["React PWA\n(static, served by API or nginx)"]
|
||||
API["App API\n(Node/Express or Fastify)"]
|
||||
|
||||
subgraph Broker["Fastmail Broker Layer"]
|
||||
CalDAVClient["CalDAV Client\n(node-caldav / tsdav)"]
|
||||
Cache["Calendar Cache\n(MariaDB: events + ctag)"]
|
||||
Poller["Background Poller\n(cron, 5-min interval)"]
|
||||
end
|
||||
|
||||
subgraph Lists["List Domain"]
|
||||
ListAPI["List CRUD\n(REST endpoints)"]
|
||||
ListDB["lists / items tables\n(MariaDB)"]
|
||||
end
|
||||
|
||||
subgraph Realtime["Real-time Layer"]
|
||||
WSHub["WebSocket Hub\n(ws or Socket.IO)"]
|
||||
RedisPubSub["Redis Pub/Sub\n(optional, single instance OK)"]
|
||||
end
|
||||
|
||||
subgraph Push["Web Push"]
|
||||
PushSender["VAPID Push Sender\n(web-push npm)"]
|
||||
SubStore["push_subscriptions table\n(MariaDB)"]
|
||||
Scheduler["Reminder Scheduler\n(node-cron)"]
|
||||
end
|
||||
|
||||
MariaDB[("MariaDB\n(users, calendars_cache,\nlists, push_subscriptions)")]
|
||||
Redis[("Redis\n(pub/sub channels)")]
|
||||
end
|
||||
|
||||
subgraph Fastmail["Fastmail (external, source of truth)"]
|
||||
FM_Shared["Shared Family Calendar"]
|
||||
FM_Personal["Personal Calendar Collections\n(one per user)"]
|
||||
end
|
||||
|
||||
iOS -->|HTTPS via tunnel| Authelia
|
||||
Android -->|HTTPS via tunnel| Authelia
|
||||
Authelia -->|forwards authed request| API
|
||||
PWA <-->|REST + WebSocket| API
|
||||
|
||||
API --> CalDAVClient
|
||||
API --> ListAPI
|
||||
API --> WSHub
|
||||
|
||||
CalDAVClient <-->|CalDAV over HTTPS\napp password auth| FM_Shared
|
||||
CalDAVClient <-->|CalDAV over HTTPS\nper-user app password OR shared ACL| FM_Personal
|
||||
CalDAVClient --> Cache
|
||||
|
||||
Poller -->|every 5 min: PROPFIND ctag| CalDAVClient
|
||||
Poller -->|on ctag change: full sync| Cache
|
||||
|
||||
ListAPI --> ListDB
|
||||
ListDB --> MariaDB
|
||||
Cache --> MariaDB
|
||||
|
||||
WSHub <-->|subscribe/publish| RedisPubSub
|
||||
RedisPubSub --> Redis
|
||||
|
||||
ListAPI -->|on write| RedisPubSub
|
||||
PushSender --> SubStore
|
||||
SubStore --> MariaDB
|
||||
Scheduler -->|check upcoming events| MariaDB
|
||||
Scheduler --> PushSender
|
||||
ListAPI -->|on important change| PushSender
|
||||
```
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ React PWA (apps/pwa/src/) │
|
||||
│ ┌──────────────┐ ┌──────────────┐ ┌───────────────────────────┐ │
|
||||
│ │ EventForm.tsx│ │SettingsSheet │ │ [NEW] SetupWizard / │ │
|
||||
│ │ + reminder │ │ + Admin tab │ │ AdminSettings │ │
|
||||
│ │ selector │ │ │ │ │ │
|
||||
│ └──────┬───────┘ └──────┬───────┘ └────────────┬──────────────┘ │
|
||||
│ │ api/client.ts (typed fetch wrappers) │ │
|
||||
└─────────┼──────────────────────────────────────┬─┴───────────────┘
|
||||
│ │
|
||||
▼ HTTP / SSE ▼ HTTP
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Hono API (apps/api/src/index.ts) │
|
||||
│ ┌──────────────┐ ┌────────────┐ ┌──────────────────────────────┐│
|
||||
│ │ routes/ │ │ routes/ │ │ [NEW] routes/admin.ts + ││
|
||||
│ │ events.ts │ │ push.ts │ │ routes/setup.ts ││
|
||||
│ │ (enqueue to │ │ │ │ (role-gated credential mgmt, ││
|
||||
│ │ outbox) │ │ │ │ first-run wizard endpoints) ││
|
||||
│ └──────┬───────┘ └────────────┘ └──────────────────────────────┘│
|
||||
│ │ │
|
||||
│ ┌──────▼──────────────────────────────────────────────────────┐ │
|
||||
│ │ broker/ │ │
|
||||
│ │ outboxWorker.ts (15s setInterval + NEW event-driven drain) │ │
|
||||
│ │ reminderScheduler.ts (1-min setInterval, MODIFIED: per- │ │
|
||||
│ │ event VALARM lead, variable window) │ │
|
||||
│ │ poller.ts (5-min setInterval, UNCHANGED) │ │
|
||||
│ │ vevent.ts [MODIFIED: buildVeventString adds VALARM] │ │
|
||||
│ │ sync.ts [MODIFIED: extract VALARM -> reminder_lead_minutes] │ │
|
||||
│ │ crypto.ts (AES-256-GCM, REUSED by admin credential writes) │ │
|
||||
│ └──────┬──────────────────────────────────────────────────────┘ │
|
||||
└─────────┼────────────────────────────────────────────────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────────────────────────────────────────────────────────┐
|
||||
│ Data layer (apps/api/src/db/) │
|
||||
│ schema.ts: users (+is_admin), calendars, calendarEvents │
|
||||
│ (+reminder_lead_minutes), calendarOutbox, │
|
||||
│ memberCredentials, pushSubscriptions, lists, ... │
|
||||
│ [NEW] app_config table (setup_complete flag, etc.) │
|
||||
│ │
|
||||
│ MariaDB (mariadb:11) + Redis (7-alpine, ioredis for pub/sub) │
|
||||
└──────────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
### Component Responsibilities
|
||||
|
||||
| Component | Responsibility | Notes |
|
||||
|-----------|----------------|-------|
|
||||
| React PWA | All UI rendering; calendar view, list co-edit, push subscription registration | Static build; served from API container or separate nginx |
|
||||
| App API | Auth middleware (OIDC token validation), REST endpoints, WebSocket upgrade, orchestrates broker + list + push | Single Node process; Fastify recommended for performance |
|
||||
| CalDAV Client | Issues PROPFIND/REPORT against caldav.fastmail.com using app password(s); parses iCalendar | Use `tsdav` (TypeScript, actively maintained) or `node-ical` for parsing |
|
||||
| Calendar Cache | Stores raw VEVENT blobs + ctag per calendar in MariaDB; serves as the read path for the API | Never exposed directly; always through API |
|
||||
| Background Poller | Cron job running inside API process; checks ctag every 5 min, triggers full REPORT sync on change | 5-min polling is sufficient for family use; no Fastmail push webhook available |
|
||||
| List CRUD | REST handlers for list/item create-read-update-delete; emits to Redis on every write | Simple; MariaDB is source of truth |
|
||||
| WebSocket Hub | Maintains open connections per authenticated user; pushes Redis messages to correct connections | Keyed by user ID extracted from OIDC sub claim |
|
||||
| Redis Pub/Sub | Message bus for list change events; decouples list writes from WebSocket delivery | Single-instance Redis is fine for 2-person household; no clustering needed |
|
||||
| VAPID Push Sender | Calls browser push services (FCM, APNs Web Push) with encrypted payloads | `web-push` npm package; keys stored in environment, not DB |
|
||||
| Reminder Scheduler | Cron job: queries events starting in next 15 min (configurable), fires push notifications | Runs inside API process; reads from calendar cache |
|
||||
| MariaDB | Persistent storage: users, calendar event cache, lists, push subscriptions | Single source for everything the app owns |
|
||||
| Component | File | Responsibility | v1.1 Status |
|
||||
|-----------|------|----------------|-------------|
|
||||
| Event form | `apps/pwa/src/components/EventForm.tsx` | Create/edit event UI | MODIFY: add reminder selector |
|
||||
| Settings sheet | `apps/pwa/src/components/SettingsSheet.tsx` | Notifications toggle | MODIFY: add Admin section |
|
||||
| API client | `apps/pwa/src/api/client.ts` | Typed fetch wrappers | MODIFY: admin + setup endpoints |
|
||||
| Events route | `apps/api/src/routes/events.ts` | Calendar CRUD, outbox enqueue | MODIFY: pass reminder in payload, signal drain |
|
||||
| Push route | `apps/api/src/routes/push.ts` | VAPID subscription management | UNCHANGED |
|
||||
| VEVENT builder | `apps/api/src/broker/vevent.ts` | iCalendar string construction | MODIFY: add VALARM |
|
||||
| CalDAV sync | `apps/api/src/broker/sync.ts` | Fastmail REPORT -> DB upsert | MODIFY: extract VALARM trigger |
|
||||
| Outbox worker | `apps/api/src/broker/outboxWorker.ts` | CalDAV write-back drain | MODIFY: event-driven trigger subscription |
|
||||
| Reminder scheduler | `apps/api/src/broker/reminderScheduler.ts` | Push reminders for events | MODIFY: variable VALARM-based lead |
|
||||
| Poller | `apps/api/src/broker/poller.ts` | 5-min CalDAV sync | UNCHANGED |
|
||||
| Crypto | `apps/api/src/broker/crypto.ts` | AES-256-GCM encrypt/decrypt | UNCHANGED (reused by admin) |
|
||||
| DB schema | `apps/api/src/db/schema.ts` | Drizzle table definitions | MODIFY: is_admin, reminder_lead_minutes, app_config |
|
||||
| Index / wiring | `apps/api/src/index.ts` | App bootstrap + worker startup | MODIFY: mount admin + setup routes |
|
||||
| [NEW] Admin route | `apps/api/src/routes/admin.ts` | Role-gated credential + calendar mgmt | NEW |
|
||||
| [NEW] Setup route | `apps/api/src/routes/setup.ts` | First-run wizard endpoints + validation | NEW |
|
||||
| [NEW] Admin UI | `apps/pwa/src/components/AdminSettings.tsx` | Member credential UI, shared-cal picker | NEW |
|
||||
| [NEW] Setup wizard | `apps/pwa/src/components/SetupWizard.tsx` | First-run guided bootstrap | NEW |
|
||||
| [NEW] Outbox trigger | `apps/api/src/lib/outboxTrigger.ts` | In-process EventEmitter for drain signal | NEW |
|
||||
| [NEW] CI workflow | `.gitea/workflows/ci.yml` | Lint/typecheck/test on PR | NEW |
|
||||
|
||||
## Recommended Project Structure
|
||||
---
|
||||
|
||||
```
|
||||
familysync/
|
||||
├── apps/
|
||||
│ ├── api/ # Node backend
|
||||
│ │ ├── src/
|
||||
│ │ │ ├── auth/ # OIDC token validation, user upsert
|
||||
│ │ │ ├── broker/ # Fastmail CalDAV client + cache sync
|
||||
│ │ │ │ ├── client.ts # tsdav wrapper
|
||||
│ │ │ │ ├── poller.ts # ctag polling cron
|
||||
│ │ │ │ ├── sync.ts # REPORT sync → DB write
|
||||
│ │ │ │ └── expand.ts # RRULE expansion + timezone normalise
|
||||
│ │ │ ├── calendars/ # REST routes: GET /calendars, GET /events
|
||||
│ │ │ ├── lists/ # REST routes + Redis emit
|
||||
│ │ │ ├── push/ # VAPID sender, subscription routes, scheduler
|
||||
│ │ │ ├── realtime/ # WebSocket server, Redis subscriber
|
||||
│ │ │ ├── db/ # Knex migrations + query helpers (MariaDB)
|
||||
│ │ │ └── server.ts # Fastify app bootstrap
|
||||
│ │ └── Dockerfile
|
||||
│ └── pwa/ # React PWA
|
||||
│ ├── src/
|
||||
│ │ ├── features/
|
||||
│ │ │ ├── calendar/ # Calendar view, event form
|
||||
│ │ │ └── lists/ # List view, item row, optimistic updates
|
||||
│ │ ├── auth/ # OIDC redirect handling, token storage
|
||||
│ │ ├── push/ # Service worker registration, push consent
|
||||
│ │ └── api/ # Typed fetch client
|
||||
│ ├── public/
|
||||
│ │ └── sw.js # Service worker (push + offline cache)
|
||||
│ └── Dockerfile
|
||||
├── db/
|
||||
│ └── migrations/ # Knex migration files (versioned)
|
||||
├── docker-compose.yml
|
||||
└── .env.example
|
||||
## Feature Integration Analysis
|
||||
|
||||
### (a) Per-Event Reminders: VALARM Authoring + Variable-Lead Scheduling
|
||||
|
||||
#### Write path — what changes
|
||||
|
||||
**`apps/pwa/src/components/EventForm.tsx`** — MODIFY
|
||||
|
||||
Add a "Reminder" `<select>` field. Values: `none | 5 | 10 | 15 | 30 | 60` (minutes before). Default `none`. Show in both create and edit modes. On submit, `executeSubmit()` includes `reminderMinutes?: number` in `CreateEventPayload`.
|
||||
|
||||
The existing WR-01 "omit recurrence on edit" pattern does NOT apply here — reminder lead is user-configured per save and should always be sent explicitly. Include `reminderMinutes: 0` to mean "remove VALARM".
|
||||
|
||||
The reminder selector should be disabled or hidden when `allDay = true` (RFC 5545: duration-form TRIGGER is semantically invalid for all-day events; see Anti-Pattern 4 below).
|
||||
|
||||
**`apps/pwa/src/api/client.ts`** — MODIFY
|
||||
|
||||
Extend `CreateEventPayload` type:
|
||||
```typescript
|
||||
reminderMinutes?: number // 0 = no alarm; positive = minutes before
|
||||
```
|
||||
|
||||
### Structure Rationale
|
||||
**`apps/api/src/routes/events.ts`** — MODIFY
|
||||
|
||||
- **apps/api/src/broker/:** Isolation boundary. All Fastmail I/O lives here. Nothing else imports from `broker/` except the `calendars/` routes and `push/scheduler`. Replacing CalDAV with JMAP later touches only this module.
|
||||
- **apps/api/src/db/:** Knex chosen over an ORM because MariaDB lacks full Prisma support for some JSON features; Knex gives SQL control without raw strings everywhere.
|
||||
- **apps/pwa/src/features/:** Feature-sliced structure keeps calendar and list concerns separated. Both features share the same WebSocket connection managed at app root.
|
||||
Extend `eventFieldsSchema` (line 100) and `outboxPayloadSchema` (line 71 in `outboxWorker.ts`):
|
||||
```typescript
|
||||
reminderMinutes: z.number().int().min(0).max(10080).optional() // max = 1 week
|
||||
```
|
||||
|
||||
## Architectural Patterns
|
||||
The payload is JSON-stringified into `calendar_outbox.payload` (TEXT column). No outbox schema change needed.
|
||||
|
||||
### Pattern 1: Broker Cache with ctag-based Invalidation
|
||||
**`apps/api/src/broker/vevent.ts`** — MODIFY
|
||||
|
||||
**What:** The API never fetches from Fastmail on a user request. All calendar reads hit the MariaDB cache. A background poller issues a lightweight CalDAV PROPFIND to check the `getctag` property on each calendar collection every 5 minutes. On change, it issues a `REPORT calendar-query` (or `sync-collection` if the server supports it) to fetch only changed VEVENTs, then upserts into the cache.
|
||||
|
||||
**When to use:** Always — Fastmail has no push webhook for calendar changes. Polling is the only option.
|
||||
|
||||
**Trade-offs:** 5-min staleness is acceptable for a family calendar. Reduces Fastmail API calls to near-zero during quiet periods. Cache warms on first poll after startup.
|
||||
Add `reminderMinutes?: number` to `NewEventParams` interface (line 21). In `buildVeventString`, after the RRULE block, add:
|
||||
|
||||
```typescript
|
||||
// broker/poller.ts (sketch)
|
||||
async function poll(calendarUrl: string, knownCtag: string | null) {
|
||||
const ctag = await fetchCtag(calendarUrl); // PROPFIND
|
||||
if (ctag === knownCtag) return; // no change
|
||||
await syncCalendar(calendarUrl); // REPORT → upsert cache
|
||||
await db('calendars').where({ url: calendarUrl }).update({ ctag, synced_at: new Date() });
|
||||
if (!params.allDay && params.reminderMinutes !== undefined && params.reminderMinutes > 0) {
|
||||
const valarm = new ICAL.Component('valarm')
|
||||
valarm.addPropertyWithValue('action', 'DISPLAY')
|
||||
valarm.addPropertyWithValue('description', params.summary)
|
||||
const trigger = ICAL.Duration.fromSeconds(-(params.reminderMinutes * 60))
|
||||
const triggerProp = new ICAL.Property('trigger')
|
||||
triggerProp.setValue(trigger)
|
||||
valarm.addProperty(triggerProp)
|
||||
vevent.addSubcomponent(valarm)
|
||||
}
|
||||
```
|
||||
|
||||
### Pattern 2: Single-Broker-Token with Per-Calendar App Passwords
|
||||
`ICAL.Duration.fromSeconds` is the correct ical.js API — same pattern as how `ICAL.Recur.fromString` is used for RRULE on line 144. Negative duration = before the event.
|
||||
|
||||
**What:** The broker holds credentials for the primary Fastmail account (the project owner). The shared family calendar is readable/writable with that account's app password. For personal calendars of other household members (the wife's personal calendar), Fastmail requires that the owner either:
|
||||
**`apps/api/src/broker/outboxWorker.ts`** — MODIFY
|
||||
|
||||
1. **Share the calendar** with the broker account — the broker then sees it appear at a different URL under its own principal (confirmed: Fastmail supports CalDAV sharing with read or read-write ACL to another Fastmail user).
|
||||
2. **Provide a separate app password** per user — the broker authenticates as each user independently to read their personal calendar.
|
||||
In both the `create` branch and `update` branch, pass `reminderMinutes: fields.reminderMinutes` to `buildVeventString`. One-line addition per branch. Also add `reminderMinutes` to `outboxPayloadSchema` for IN-03 re-validation.
|
||||
|
||||
Option 1 (sharing to the broker account) is preferred: one credential, no password management per user. The wife logs into Fastmail once, shares her personal calendar with the primary account, done. The broker discovers it via CalDAV principal discovery.
|
||||
#### Scheduler path — what changes
|
||||
|
||||
**Confidence: LOW on exact URL behavior.** Fastmail's sharing documentation confirms the sharing feature exists and follows CalDAV draft ACL standards, but does not explicitly document whether the shared calendar appears in the principal's list discovery. Treat this as a phase research item — test during the broker phase with a real Fastmail multi-user or sharing setup.
|
||||
**`apps/api/src/db/schema.ts`** — MODIFY
|
||||
|
||||
**Trade-offs:** If sharing is not discoverable by principal, fall back to individual app passwords stored encrypted in `.env`. Two-person household means at most 2 credentials.
|
||||
|
||||
### Pattern 3: Redis Pub/Sub for List Co-Edit
|
||||
|
||||
**What:** On every list/item write that succeeds in MariaDB, the API handler publishes a message to a Redis channel keyed by list ID. The WebSocket hub is subscribed to all relevant channels and pushes the serialized diff to connected clients.
|
||||
|
||||
**When to use:** Whenever two users might be editing the same list concurrently. For 2 people, this is "always."
|
||||
|
||||
**Trade-offs:** Redis is optional in the stack — if Redis is unavailable, fall back to polling (client polls `/lists/{id}` every 10s). Design the list API to support both modes via a feature flag.
|
||||
|
||||
```
|
||||
Client A writes item → POST /lists/1/items
|
||||
→ DB insert
|
||||
→ Redis PUBLISH list:1 { type: "item_added", item: {...} }
|
||||
→ WebSocket hub receives → pushes to all sockets subscribed to list:1
|
||||
Client B receives message → appends item to local state (no refetch)
|
||||
Add to `calendarEvents`:
|
||||
```typescript
|
||||
reminderLeadMinutes: int('reminder_lead_minutes'),
|
||||
// nullable: NULL = no VALARM; positive int = minutes before dtstart
|
||||
```
|
||||
|
||||
### Pattern 4: RRULE Expansion Client-Side from Stored Raw VEVENT
|
||||
Add `drizzle-kit generate` migration. This is a nullable column addition — safe online DDL on MariaDB 11/InnoDB.
|
||||
|
||||
**What:** Store the raw VEVENT blob (including RRULE) in the cache. On read, the API returns the raw iCal data or a partially processed event object that includes the RRULE string. The PWA expands recurrences client-side using the `rrule.js` library for the displayed date window (current month ± 1 month).
|
||||
**`apps/api/src/broker/sync.ts`** — MODIFY
|
||||
|
||||
**When to use:** For this scale. Server-side pre-expansion into a flat events table is over-engineering for a 2-person calendar.
|
||||
|
||||
**Trade-offs:** Timezone correctness requires the PWA to use the TZID from the VEVENT and the `Intl` API to resolve DST. The `rrule.js` library handles TZID correctly when fed the VTIMEZONE component. Store all DTSTART values in UTC in the DB alongside the raw blob for efficient range queries.
|
||||
|
||||
## Data Flow
|
||||
|
||||
### Calendar Read (PWA → API → Cache)
|
||||
|
||||
```
|
||||
User opens calendar view (month X)
|
||||
→ GET /api/events?start=...&end=...
|
||||
→ API queries MariaDB: SELECT * FROM events WHERE dtstart BETWEEN ? AND ?
|
||||
→ Returns event objects (RRULE included)
|
||||
→ PWA rrule.js expands recurring events for display window
|
||||
→ Render color-coded events (color from user_calendars.color config)
|
||||
When upserting a `calendarEvent`, extract VALARM from the parsed VEVENT:
|
||||
```typescript
|
||||
const valarm = vevent.getFirstSubcomponent('valarm')
|
||||
let reminderLeadMinutes: number | null = null
|
||||
if (valarm) {
|
||||
const trigger = valarm.getFirstPropertyValue('trigger')
|
||||
if (trigger instanceof ICAL.Duration) {
|
||||
reminderLeadMinutes = Math.abs(trigger.toSeconds()) / 60
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Calendar Write (PWA → API → Fastmail)
|
||||
Store `reminderLeadMinutes` in the DB upsert. This is the ground truth — the scheduler reads from DB, not from the outbox payload.
|
||||
|
||||
```
|
||||
User creates/edits event in PWA
|
||||
→ POST/PUT /api/events
|
||||
→ API validates auth, resolves target calendar from event's calendarId
|
||||
→ broker/client.ts issues CalDAV PUT (new VEVENT) to caldav.fastmail.com
|
||||
→ On 201/204 success: upsert into local cache immediately (no wait for next poll)
|
||||
→ Return 201 to PWA
|
||||
→ PWA updates local state optimistically (already done before response)
|
||||
**`apps/api/src/broker/reminderScheduler.ts`** — MODIFY (significant rewrite of `runReminderCheck`)
|
||||
|
||||
Current behavior:
|
||||
- Fixed 16-minute catch-up window: `dtstartUtc IN (now, now+16min]`
|
||||
- Fires only for `calendars.isShared = true` (D-05 constraint)
|
||||
- Dedup key: bare `uid` in `sentReminders: Map<string, number>`
|
||||
|
||||
New behavior:
|
||||
- Variable lead: query events where the alarm-fire-time falls in the current tick's window
|
||||
- Alarm-fire-time = `dtstartUtc - INTERVAL reminder_lead_minutes MINUTE`
|
||||
- Window: `(now - 1min, now + 1min]` gives a 2-minute catch-up on missed ticks (scheduler runs every 60s)
|
||||
- Events with `reminder_lead_minutes IS NULL` are excluded — no VALARM means no reminder
|
||||
|
||||
Drizzle raw SQL expression for the window predicate:
|
||||
```typescript
|
||||
sql`${calendarEvents.reminderLeadMinutes} IS NOT NULL
|
||||
AND DATE_SUB(${calendarEvents.dtstartUtc}, INTERVAL ${calendarEvents.reminderLeadMinutes} MINUTE) > ${new Date(now.getTime() - 60_000)}
|
||||
AND DATE_SUB(${calendarEvents.dtstartUtc}, INTERVAL ${calendarEvents.reminderLeadMinutes} MINUTE) <= ${new Date(now.getTime() + 60_000)}`
|
||||
```
|
||||
|
||||
### Fastmail Sync Loop (background)
|
||||
`isShared` restriction (D-05): this is a scope decision. D-05 was originally "shared calendar only" for the fixed-15-min reminder. With per-event VALARM, the user explicitly configured a reminder — it should fire regardless of which calendar the event is on. Recommend removing the `isShared` restriction in v1.1 for reminder-triggered notifications. Event-change notifications (via `eventChangeDispatcher.ts`) are a separate flow and unaffected.
|
||||
|
||||
```
|
||||
Poller tick (every 5 min)
|
||||
→ PROPFIND each tracked calendar URL → fetch getctag
|
||||
→ Compare ctag to DB stored value
|
||||
→ If changed: REPORT calendar-query for the collection
|
||||
→ Parse iCalendar response
|
||||
→ For each VEVENT: upsert events table (uid as key), update dtstart UTC
|
||||
→ Update calendars.ctag
|
||||
→ If reminder threshold: Scheduler fires Web Push
|
||||
**Catch-up guarantee is preserved.** The existing 16-min window provided catch-up by keeping missed events visible for 16 minutes. With a 2-minute window (±1min around now) and 1-min ticks, a single missed tick means the next tick has the event in window. A 2-minute outage means at most one missed reminder — acceptable for household use.
|
||||
|
||||
**Dedup stays the same.** `sentReminders: Map<uid, dtstartMs>` keyed by bare uid. This correctly prevents double-fire when the event lingers in the window across ticks. No change needed.
|
||||
|
||||
---
|
||||
|
||||
### (b) Event-Driven Outbox Drain
|
||||
|
||||
#### Current state
|
||||
|
||||
`startOutboxWorker()` in `outboxWorker.ts` (line 781) runs `runOutboxDrain()` every 15 seconds. All write routes in `routes/events.ts` insert to `calendar_outbox` and return 202 with no drain trigger.
|
||||
|
||||
#### Recommended approach: in-process EventEmitter
|
||||
|
||||
New file `apps/api/src/lib/outboxTrigger.ts` — follows the exact pattern of `listEmitter.ts` (proven, same single-process deployment):
|
||||
|
||||
```typescript
|
||||
// apps/api/src/lib/outboxTrigger.ts
|
||||
import { EventEmitter } from 'node:events'
|
||||
const emitter = new EventEmitter()
|
||||
|
||||
export function signalOutboxDrain(): void {
|
||||
emitter.emit('drain')
|
||||
}
|
||||
|
||||
export function onOutboxDrainSignal(handler: () => void): () => void {
|
||||
emitter.on('drain', handler)
|
||||
return () => emitter.off('drain', handler)
|
||||
}
|
||||
```
|
||||
|
||||
### List Co-Edit (real-time)
|
||||
**Why not Redis pub/sub:** The outbox drain is single-process by design — the `isDraining` guard in `outboxWorker.ts` (line 156) explicitly documents "SINGLE-PROCESS LIMITATION". Redis pub/sub adds a network hop and operational overhead for zero benefit. Redis is already used for SSE list fan-out — a different problem (multi-browser-tab fan-out).
|
||||
|
||||
```
|
||||
User A edits list item text
|
||||
→ PATCH /api/lists/1/items/42 { text: "oat milk" }
|
||||
→ DB UPDATE → success
|
||||
→ Redis PUBLISH list:1 { op: "update", itemId: 42, text: "oat milk", userId: "A" }
|
||||
→ WS Hub (subscribed to list:1) receives message
|
||||
→ Push to all WebSocket connections for users in list:1 EXCEPT sender
|
||||
→ User B's PWA receives → update item in list state
|
||||
**`apps/api/src/routes/events.ts`** — MODIFY
|
||||
|
||||
After each `db.insert(calendarOutbox)` in the three write handlers (create, edit/same-calendar, edit/move transaction), add:
|
||||
```typescript
|
||||
signalOutboxDrain()
|
||||
```
|
||||
|
||||
### Web Push Delivery
|
||||
Import from `'../lib/outboxTrigger.js'`. Fire-and-forget; no await.
|
||||
|
||||
```
|
||||
Reminder Scheduler (cron, runs every minute)
|
||||
→ SELECT events WHERE dtstart BETWEEN now+14min AND now+15min
|
||||
AND reminder_sent = false
|
||||
→ For each event: find household users who have visibility to that calendar
|
||||
→ For each user: SELECT push_subscriptions WHERE user_id = ?
|
||||
→ For each subscription: web-push sendNotification(subscription, payload)
|
||||
→ Mark reminder_sent = true
|
||||
**`apps/api/src/broker/outboxWorker.ts`** — MODIFY
|
||||
|
||||
In `startOutboxWorker()`:
|
||||
```typescript
|
||||
export function startOutboxWorker(): void {
|
||||
// Event-driven: drain immediately after any enqueue signal
|
||||
onOutboxDrainSignal(() => {
|
||||
runOutboxDrain().catch((err: unknown) => {
|
||||
console.error('[outboxWorker] Signal-triggered drain error:', err)
|
||||
})
|
||||
})
|
||||
// Fallback: 15s periodic drain for recovery + startup catch-up
|
||||
setInterval(() => {
|
||||
runOutboxDrain().catch((err: unknown) => {
|
||||
console.error('[outboxWorker] Interval drain error:', err)
|
||||
})
|
||||
}, 15 * 1000)
|
||||
}
|
||||
```
|
||||
|
||||
### Identity Mapping (OIDC → App User)
|
||||
**All durability guarantees are preserved.** The `isDraining` concurrency guard handles overlapping invocations from both the signal and the interval. When a signal-triggered drain is running (`isDraining = true`), any concurrent call (signal or interval) is a no-op. Create-before-delete ordering (D-04), etag re-read (WR-02), 412 conflict flow (D-08), transient backoff (D-07) — all unchanged. The trigger path only affects *when* `runOutboxDrain()` is called.
|
||||
|
||||
```
|
||||
User authenticates via Authelia OIDC
|
||||
→ PWA redirects to /auth/login → Authelia
|
||||
→ Authelia returns id_token with claims: sub, email, preferred_username, groups
|
||||
→ API receives id_token in Authorization header (or httpOnly cookie)
|
||||
→ Validate token signature against Authelia JWKS endpoint
|
||||
→ Look up users table WHERE oidc_sub = claims.sub
|
||||
→ If not found: INSERT (first login provisioning) using email as display hint
|
||||
→ Attach app user_id to request context
|
||||
→ user_id is used for: calendar color config, push subscriptions, WS channel auth
|
||||
**Rapid-successive-edits behavior:** Each `signalOutboxDrain()` call may trigger a drain that processes row N while row N+1 hasn't been inserted yet. Row N+1's `nextAttemptAt = defaultNow()`, so it appears in the next drain cycle's `pending + next_attempt_at <= NOW()` query. The 15s fallback interval ensures it is drained within 15 seconds at most.
|
||||
|
||||
---
|
||||
|
||||
### (c) Admin Role + Setup Wizard Config Storage
|
||||
|
||||
#### Admin role flag
|
||||
|
||||
**Where:** `users` table in `apps/api/src/db/schema.ts`. Add:
|
||||
```typescript
|
||||
isAdmin: boolean('is_admin').default(false).notNull(),
|
||||
```
|
||||
|
||||
**Key:** Use `oidc_sub` (the `sub` claim) as the stable identity anchor. Never use `email` as a unique key — it can change per Authelia docs. Store `oidc_iss + oidc_sub` as the composite stable ID.
|
||||
**Why DB not env:** Admin designation is per-user, mutable, and tied to identity. Env vars cannot express "user X is admin." The first user to complete the setup wizard is auto-promoted. Subsequent changes go through an admin route action.
|
||||
|
||||
## Component Build Order (Dependencies)
|
||||
**Migration:** `ALTER TABLE users ADD COLUMN is_admin BOOLEAN NOT NULL DEFAULT 0` — safe online DDL on MariaDB 11/InnoDB.
|
||||
|
||||
Build in this sequence — each layer depends on the previous:
|
||||
**Route guard:** `requireAdmin` middleware in `apps/api/src/routes/admin.ts` calls `resolveUserId` then checks `users.isAdmin`. Applied to all `/api/admin/*` routes.
|
||||
|
||||
```
|
||||
1. Infrastructure scaffold
|
||||
docker-compose.yml (API + MariaDB + Redis), DB migrations, .env wiring
|
||||
**`GET /api/me` extension:** Add `isAdmin: boolean` to the response. The PWA uses this to conditionally show the Admin section in `SettingsSheet`. The `['me']` TanStack Query is already called on app load — no new query needed.
|
||||
|
||||
2. Auth layer
|
||||
OIDC middleware (validate Authelia tokens, user upsert), session/cookie setup
|
||||
[nothing works without identity]
|
||||
#### Setup wizard config
|
||||
|
||||
3. Fastmail broker — read path only
|
||||
CalDAV client, calendar discovery, cache sync, poller, event query API
|
||||
[needed before any calendar UI]
|
||||
The wizard bootstraps four categories of state:
|
||||
|
||||
4. Calendar PWA — read only
|
||||
React calendar view consuming /api/events, color coding per calendar
|
||||
[validates broker correctness before write complexity]
|
||||
| Config | Current Home | v1.1 Approach |
|
||||
|--------|-------------|----------------|
|
||||
| VAPID keypair | `.env` (VAPID_PUBLIC_KEY, _PRIVATE_KEY, _SUBJECT) | Stays in env; wizard validates presence + offers generate-and-display |
|
||||
| AES-256 key | `.env` (APP_PASSWORD_ENCRYPTION_KEY) | Stays in env; wizard validates format (64-char hex) |
|
||||
| OIDC credentials | `.env` (OIDC_ISSUER, _CLIENT_ID, _SECRET, etc.) | Stays in env; wizard validates OIDC discovery reachability |
|
||||
| App passwords + shared-cal | DB (member_credentials, calendars.is_shared) | Same DB; wizard is the UI for what was previously a manual DB write |
|
||||
|
||||
5. Fastmail broker — write path
|
||||
CalDAV PUT for create/edit/delete, optimistic cache upsert
|
||||
[built on confirmed-working read path]
|
||||
|
||||
6. Lists domain
|
||||
MariaDB list/item schema, REST CRUD, basic PWA list UI
|
||||
[independent of calendar; can parallel-track after step 3]
|
||||
|
||||
7. Real-time list sync
|
||||
Redis pub/sub, WebSocket hub, PWA WebSocket client
|
||||
[enhancement on top of working list CRUD]
|
||||
|
||||
8. Web Push
|
||||
VAPID key generation, push_subscriptions table, service worker, reminder scheduler
|
||||
[built last; depends on events being cached and users having identity]
|
||||
**New `app_config` table** for app-level flags:
|
||||
```typescript
|
||||
export const appConfig = mysqlTable('app_config', {
|
||||
key: varchar('key', { length: 128 }).primaryKey(),
|
||||
value: text('value').notNull(),
|
||||
updatedAt: timestamp('updated_at').defaultNow().onUpdateNow(),
|
||||
})
|
||||
```
|
||||
|
||||
**Parallel tracks available:** Lists (step 6) can start in parallel with calendar write (step 5) once auth and broker read are stable.
|
||||
Initial keys: `setup_complete` (`'0'` or `'1'`), `setup_step` (resumable flow state).
|
||||
|
||||
**Why the wizard cannot write `.env`:** Env vars are set at Docker container start via Docker Compose. A running container's process cannot mutate its own env and have those mutations persist across restarts. The correct flow: wizard detects missing env vars, provides a "Generate" button that returns a keypair for the operator to copy, then re-validates after the operator restarts the container with updated env. Alternatively, the wizard writes to a `/app/.env.local` file that the Node.js process reads via dotenv — but this requires the container to have write access to its own image layer or a mounted volume. The simplest Unraid approach: wizard shows values to copy, operator adds to Docker Compose env, restarts.
|
||||
|
||||
**`apps/api/src/routes/setup.ts`** — NEW
|
||||
|
||||
Endpoints (all except `/status` are admin-only after first-run completes):
|
||||
- `GET /api/setup/status` — **unauthenticated** (mounted before OIDC guard in `index.ts`, like `/health`); returns `{ complete: boolean }`
|
||||
- `GET /api/setup/env-status` — returns which env vars are present/missing (no values)
|
||||
- `POST /api/setup/generate-vapid` — calls `webpush.generateVAPIDKeys()`, returns `{ publicKey, privateKey, subject }` for operator to copy; does NOT store in DB
|
||||
- `GET /api/setup/validate-oidc` — pings `${OIDC_ISSUER}/.well-known/openid-configuration`, returns `{ ok: boolean, error?: string }`
|
||||
- `POST /api/setup/complete` — sets `app_config.setup_complete = '1'`; sets `users.is_admin = true` for the calling user (promotes wizard-completing user to admin)
|
||||
|
||||
**First-run detection in PWA:** `App.tsx` adds a setup-status check before rendering the calendar shell. If `complete = false` and the user is authenticated (or the wizard must be completeable pre-auth), render `<SetupWizard>`. Simplest flow: require authentication first (Authelia protects everything), then the wizard runs as an authenticated user. The `GET /api/setup/status` route being pre-OIDC lets the PWA detect setup state before the OIDC redirect fires — render a "Setup required" splash instead of bouncing the user around.
|
||||
|
||||
#### Shared surface: wizard vs admin Settings
|
||||
|
||||
`POST /api/admin/credentials` is called by both the setup wizard (step: first app password) and the admin Settings page (ongoing credential management). Build this route once. The wizard in Phase D is a pure frontend consumer of the same admin API surface.
|
||||
|
||||
`PATCH /api/admin/calendars/:id/shared` is called by both the setup wizard (step: designate shared calendar) and admin Settings. Same pattern.
|
||||
|
||||
**Do not create `/api/setup/credentials` or `/api/setup/calendar-shared`** — these would duplicate `/api/admin/credentials` and `/api/admin/calendars/:id/shared`.
|
||||
|
||||
**Deferred: self-service provider onboarding (999.5).** This is a member-initiated flow where each member enters their own app password. The admin credential route is admin-only and takes a `userId` parameter. Self-service onboarding is a separate concern — do not conflate with admin settings in v1.1.
|
||||
|
||||
---
|
||||
|
||||
### (d) New vs Modified Components — Dependency-Ordered Build Sequence
|
||||
|
||||
#### Dependency graph
|
||||
|
||||
```
|
||||
DB schema migration (users.is_admin, calendarEvents.reminder_lead_minutes, app_config)
|
||||
│
|
||||
├─── Admin role + routes (admin.ts) ─────────────────────────┐
|
||||
│ │ │
|
||||
│ routes/me.ts (extend response with isAdmin) │
|
||||
│ │ │
|
||||
│ AdminSettings.tsx + SettingsSheet.tsx mod │
|
||||
│ │ │
|
||||
│ SetupWizard.tsx (reuses admin routes) ◄───┘
|
||||
│ routes/setup.ts (wizard backend)
|
||||
│ App.tsx (setup-status gate)
|
||||
│
|
||||
├─── Faster write-back ─────────────────────────────────────
|
||||
│ lib/outboxTrigger.ts (new)
|
||||
│ outboxWorker.ts (subscribe to signal)
|
||||
│ routes/events.ts (call signalOutboxDrain)
|
||||
│ [no PWA changes]
|
||||
│
|
||||
├─── VALARM write path ──────────────────────────────────────┐
|
||||
│ broker/vevent.ts (add VALARM to buildVeventString) │
|
||||
│ outboxWorker.ts (pass reminderMinutes) │
|
||||
│ routes/events.ts (extend schemas) │
|
||||
│ EventForm.tsx (reminder selector) │
|
||||
│ api/client.ts (extend CreateEventPayload) │
|
||||
│ broker/sync.ts (extract VALARM -> DB) ◄───┘
|
||||
│ reminderScheduler.ts (variable-lead query)
|
||||
│
|
||||
├─── Gitea CI (.gitea/workflows/ci.yml)
|
||||
│ [no code dependencies; parallel]
|
||||
│
|
||||
└─── Mobile test harness
|
||||
[no new backend code; playwright-cli + DEV_AUTH_BYPASS]
|
||||
```
|
||||
|
||||
#### Recommended build order
|
||||
|
||||
**Phase A — DB foundation (unblocks everything)**
|
||||
|
||||
1. `apps/api/src/db/schema.ts` — add `users.isAdmin`, `calendarEvents.reminderLeadMinutes`, `appConfig` table
|
||||
2. `drizzle-kit generate` + apply migration to dev DB
|
||||
3. `apps/api/src/lib/outboxTrigger.ts` — new file, zero dependencies
|
||||
|
||||
**Phase B — Faster write-back (low-risk, isolated)**
|
||||
|
||||
Prerequisite: `outboxTrigger.ts` from Phase A
|
||||
|
||||
4. `apps/api/src/broker/outboxWorker.ts` — subscribe to drain signal in `startOutboxWorker`
|
||||
5. `apps/api/src/routes/events.ts` — add `signalOutboxDrain()` after each outbox insert (3 call sites)
|
||||
|
||||
No PWA changes. Tests: extend `tests/broker/outboxWorker.test.ts`.
|
||||
|
||||
**Phase C — Admin role + routes**
|
||||
|
||||
6. `apps/api/src/routes/admin.ts` — new file:
|
||||
- `GET /api/admin/members` — list users with credential status (isAdmin flag determines who sees it)
|
||||
- `POST /api/admin/credentials` — upsert `memberCredentials` for a given `userId`; reuses `encryptPassword` from `broker/crypto.ts`
|
||||
- `DELETE /api/admin/credentials/:userId` — remove credential
|
||||
- `PATCH /api/admin/calendars/:id/shared` — toggle `calendars.isShared`
|
||||
- All routes: `requireAdmin` middleware
|
||||
7. `apps/api/src/routes/me.ts` — extend response with `isAdmin: boolean`
|
||||
8. `apps/api/src/index.ts` — mount `adminRouter` at `/api/admin`
|
||||
|
||||
**Phase D — Setup wizard**
|
||||
|
||||
Prerequisite: Phase C admin routes
|
||||
|
||||
9. `apps/api/src/routes/setup.ts` — new file (wizard backend endpoints)
|
||||
10. `apps/api/src/index.ts` — mount `setupRouter` at `/api/setup`; mount `GET /api/setup/status` BEFORE the OIDC guard
|
||||
11. `apps/pwa/src/components/SetupWizard.tsx` — multi-step first-run UI
|
||||
12. `apps/pwa/src/App.tsx` — setup-status check gate
|
||||
|
||||
**Phase E — Admin Settings UI**
|
||||
|
||||
Prerequisite: Phase C admin routes + Phase C `isAdmin` in `/api/me`
|
||||
|
||||
13. `apps/pwa/src/components/AdminSettings.tsx` — member list, "Set app password" flow, shared-calendar picker
|
||||
14. `apps/pwa/src/components/SettingsSheet.tsx` — add "Admin" section (shown only when `currentUser.isAdmin`)
|
||||
15. `apps/pwa/src/api/client.ts` — add admin API fetch wrappers
|
||||
|
||||
**Phase F — VALARM authoring (event form + scheduler)**
|
||||
|
||||
Prerequisite: Phase A (schema has `reminderLeadMinutes`)
|
||||
|
||||
16. `apps/api/src/broker/vevent.ts` — add VALARM to `buildVeventString`; extend `NewEventParams`
|
||||
17. `apps/api/src/broker/outboxWorker.ts` — pass `reminderMinutes` in create + update branches; add to `outboxPayloadSchema`
|
||||
18. `apps/api/src/routes/events.ts` — add `reminderMinutes` to `eventFieldsSchema`
|
||||
19. `apps/api/src/broker/sync.ts` — extract VALARM trigger; store `reminderLeadMinutes` on upsert
|
||||
20. `apps/pwa/src/components/EventForm.tsx` — add reminder selector
|
||||
21. `apps/pwa/src/api/client.ts` — extend `CreateEventPayload` type
|
||||
|
||||
Then, once sync.ts change is deployed and populating `reminder_lead_minutes`:
|
||||
|
||||
22. `apps/api/src/broker/reminderScheduler.ts` — rewrite `runReminderCheck` to use variable lead from `reminder_lead_minutes`; update `isShared` scope per feature decision
|
||||
|
||||
**Phase G — Gitea CI (independent, parallel)**
|
||||
|
||||
23. `.gitea/workflows/ci.yml` — lint/typecheck/unit/API-integration on PR; MariaDB service container; secrets for `DB_HOST=127.0.0.1`, DB creds, test env vars. Tests live in `apps/api/tests/` — confirm test runner path in CI config.
|
||||
|
||||
**Phase H — Mobile test harness (independent)**
|
||||
|
||||
24. Playwright-cli mobile viewport configuration + authenticated entry via `DEV_AUTH_BYPASS=true`; no backend changes required.
|
||||
|
||||
---
|
||||
|
||||
## Component Boundaries and Key Integration Notes
|
||||
|
||||
### Shared surface: wizard credential entry vs admin Settings
|
||||
|
||||
`POST /api/admin/credentials` is called by both the setup wizard (first app password) and the admin Settings page (ongoing management). Build once in Phase C. Do not duplicate into a `/api/setup/credentials` endpoint.
|
||||
|
||||
### VALARM round-trip: write path vs read path
|
||||
|
||||
These are independent flows that rendezvous through Fastmail:
|
||||
|
||||
- **Write:** `reminderMinutes` travels as a number in the outbox JSON payload → `buildVeventString` emits VALARM TRIGGER:-PT{n}M → tsdav PUT to Fastmail
|
||||
- **Read:** After the event syncs back from Fastmail (via targeted re-sync in `outboxWorker.ts` or 5-min poll), `sync.ts` parses the VALARM TRIGGER and stores `reminderLeadMinutes` in `calendar_events`. The scheduler reads this column — never the outbox payload.
|
||||
|
||||
This means `reminder_lead_minutes` in `calendar_events` is the ground truth. Events created before v1.1 (or by third-party clients) that have no VALARM will have `reminderLeadMinutes = NULL` and the scheduler ignores them.
|
||||
|
||||
### `outboxPayloadSchema` vs `eventFieldsSchema` — must stay in sync
|
||||
|
||||
`eventFieldsSchema` in `routes/events.ts` (line 100) and `outboxPayloadSchema` in `outboxWorker.ts` (line 71) are intentionally duplicated for defense-in-depth (IN-03). Adding `reminderMinutes` requires updating both. This is documented in the existing IN-03 comment — flag it explicitly when writing the phase plan.
|
||||
|
||||
### `isAdmin` in API responses
|
||||
|
||||
`GET /api/me` currently returns `{ id, displayName, color }`. Extend to include `isAdmin: boolean`. The PWA uses this to conditionally show the Admin section in SettingsSheet. The `['me']` TanStack Query is already called on app load — no new query, just extend the existing response and the `MeUser` type in `api/client.ts`.
|
||||
|
||||
### First-run detection: mount `/api/setup/status` before OIDC guard
|
||||
|
||||
In `apps/api/src/index.ts`, `/health` is already mounted before the OIDC guard (line 40). Mount `GET /api/setup/status` with the same pattern:
|
||||
```typescript
|
||||
app.get('/api/setup/status', setupStatusHandler) // before oidcAuthMiddleware
|
||||
```
|
||||
This lets the PWA detect first-run state before the OIDC redirect fires and render a setup splash instead of an auth bounce loop.
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Diagrams
|
||||
|
||||
### Per-Event Reminder: Create to Notification
|
||||
|
||||
```
|
||||
EventForm (reminder: 15min)
|
||||
↓ POST /api/events/create { reminderMinutes: 15, ... }
|
||||
routes/events.ts (schema validates, inserts outbox row, signalOutboxDrain())
|
||||
↓ [event-driven, ~0s]
|
||||
outboxWorker.ts runOutboxDrain()
|
||||
→ buildVeventString({ reminderMinutes: 15 })
|
||||
→ vevent.ts emits VCALENDAR with VALARM TRIGGER:-PT15M
|
||||
→ tsdav PUT to Fastmail (.ics)
|
||||
→ triggerTargetedResync() → sync.ts parses VALARM → stores reminder_lead_minutes=15
|
||||
↓ (next 1-min scheduler tick)
|
||||
reminderScheduler.ts
|
||||
→ SQL WHERE DATE_SUB(dtstart_utc, INTERVAL reminder_lead_minutes MINUTE) IN tick-window
|
||||
→ dispatchPush() to all subscribers
|
||||
↓
|
||||
Browser notification: "Starts in 15 min"
|
||||
```
|
||||
|
||||
### Admin: Set Member App Password
|
||||
|
||||
```
|
||||
AdminSettings.tsx (admin enters app password for member X)
|
||||
↓ POST /api/admin/credentials { userId: X, email: '...', password: '...' }
|
||||
routes/admin.ts (requireAdmin guard → resolveUserId → check users.isAdmin)
|
||||
→ zod validate input
|
||||
→ encryptPassword() from broker/crypto.ts (AES-256-GCM)
|
||||
→ db.insert/onDuplicateKeyUpdate on memberCredentials
|
||||
↓ 200 OK
|
||||
AdminSettings.tsx invalidates ['admin/members'] query
|
||||
poller.ts (next 5-min tick): picks up new credential, discovers calendars for that user
|
||||
```
|
||||
|
||||
### Setup Wizard: First-Run Bootstrap
|
||||
|
||||
```
|
||||
App.tsx (on load)
|
||||
↓ GET /api/setup/status [UNAUTHENTICATED, pre-OIDC]
|
||||
← { complete: false }
|
||||
→ render <SetupWizard> (after OIDC login)
|
||||
Step 1: env-var check → GET /api/setup/env-status
|
||||
Step 2: DB/OIDC validation → GET /api/setup/validate-oidc
|
||||
Step 3: VAPID keypair → POST /api/setup/generate-vapid (operator copies to .env, restarts)
|
||||
Step 4: first app password → POST /api/admin/credentials
|
||||
Step 5: shared calendar → PATCH /api/admin/calendars/:id/shared
|
||||
Step 6: POST /api/setup/complete → app_config.setup_complete='1', user.is_admin=true
|
||||
→ redirect to /
|
||||
App.tsx (on reload): GET /api/setup/status → { complete: true } → render CalendarShell
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Anti-Patterns to Avoid
|
||||
|
||||
### Anti-Pattern 1: Separate wizard endpoints that duplicate admin routes
|
||||
|
||||
The wizard is a frontend flow over the same admin API. Do not create `/api/setup/credentials` alongside `/api/admin/credentials`. One route, two callers (wizard + Settings page).
|
||||
|
||||
### Anti-Pattern 2: Storing env-var secrets in the DB
|
||||
|
||||
VAPID private keys and `APP_PASSWORD_ENCRYPTION_KEY` must stay in env vars — they are the bootstrap secrets that protect everything else. The wizard generates and displays them for the operator to copy; it never writes them to the `app_config` table or any other DB table.
|
||||
|
||||
### Anti-Pattern 3: Reintroducing node-cron
|
||||
|
||||
The codebase uses `setInterval` throughout because node-cron 4.2.1 silently skips ticks in the long-running process (documented in every worker file's header comment). Do not use `node-cron` for any new scheduled work in v1.1. The event-driven drain replaces the need for a tighter interval; the 15s `setInterval` fallback is already correct.
|
||||
|
||||
### Anti-Pattern 4: Adding VALARM to all-day events
|
||||
|
||||
RFC 5545 duration-form TRIGGER (`-PT15M`) is semantically valid only for timed events. For all-day events it requires `RELATED=END` or a DATE-form trigger. Fastmail may accept it regardless, but the semantics are wrong and would yield confusing reminder times. Guard in `buildVeventString`: `if (!params.allDay && params.reminderMinutes > 0)`. Disable the reminder selector in `EventForm.tsx` when `allDay = true`.
|
||||
|
||||
### Anti-Pattern 5: Dedup key includes lead time in reminderScheduler
|
||||
|
||||
The existing `sentReminders: Map<uid, dtstartMs>` keyed by bare uid is correct and must not be changed to `uid:leadMinutes`. A keyed `uid:lead` would cause double-fire if the event is rescheduled (new dtstart, same uid) or if the lead changes. The bare uid key + dtstart-based pruning is the correct exactly-once guarantee — leave it intact.
|
||||
|
||||
### Anti-Pattern 6: Concurrency guard changes in outboxWorker
|
||||
|
||||
The `isDraining` module-level boolean guard is the correct concurrency mechanism for the single-process deployment. Do not add a second guard, change it to an async mutex, or introduce Redis-based locking for the drain. The event-driven signal adds a new call site but does not require any guard changes — the existing guard already handles simultaneous invocations.
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### External Services
|
||||
|
||||
| Service | Integration Pattern | Auth | Gotchas |
|
||||
|---------|---------------------|------|---------|
|
||||
| Fastmail CalDAV | HTTP PROPFIND / REPORT / PUT via `tsdav` | App password (Basic Auth over HTTPS) | JMAP for calendars NOT available — CalDAV only. Use app password, not API token. Rate limits undocumented; 5-min polling is conservative. |
|
||||
| Authelia OIDC | Authorization Code flow from PWA; JWT validation in API | OIDC client_id/secret in API env | Validate against Authelia JWKS URL, not a static key. Use `sub` not `email` as stable identity. |
|
||||
| Browser Push Services (FCM/APNs Web Push) | VAPID via `web-push` npm | VAPID public/private key pair | iOS requires PWA installed to home screen (outside EU). Push endpoint URL comes from browser `PushManager.subscribe()`. |
|
||||
| Pangolin/Newt | Transparent tunnel — no changes to app | None (handled at infra layer) | API must trust `X-Forwarded-For` headers from Pangolin; set `trustProxy: true` in Fastify. |
|
||||
| Service | Integration | v1.1 Notes |
|
||||
|---------|-------------|------------|
|
||||
| Fastmail CalDAV | tsdav PUT with VCALENDAR containing VALARM | VALARM is inert to Fastmail's CalDAV server (stored as-is); sync reads it back |
|
||||
| Authelia OIDC | No change | Setup wizard is post-OIDC; admin routes use existing OIDC session |
|
||||
|
||||
### Internal Boundaries
|
||||
### Internal Module Boundaries
|
||||
|
||||
| Boundary | Communication | Notes |
|
||||
|----------|---------------|-------|
|
||||
| PWA ↔ API | REST (JSON) + WebSocket | REST for CRUD + auth; WS for real-time list push. Single domain, so no CORS needed in prod. |
|
||||
| API ↔ Broker | Direct function call (same process) | Broker is a module, not a separate service. Keep it that way — no premature microservice split. |
|
||||
| API ↔ MariaDB | Knex query builder | All queries go through `db/` module. No raw SQL in route handlers. |
|
||||
| API ↔ Redis | ioredis, publish on write | WS hub uses a separate ioredis subscriber connection (required by Redis subscribe mode). |
|
||||
| Scheduler ↔ API | In-process cron (node-cron) | Poller and reminder scheduler run in same Node process. If process dies, they restart with it. |
|
||||
| `routes/events.ts` → `outboxWorker.ts` | `signalOutboxDrain()` via EventEmitter | NEW: fire-and-forget, one-way |
|
||||
| `routes/admin.ts` → `broker/crypto.ts` | Direct import of `encryptPassword` | REUSE existing module — no changes to crypto.ts |
|
||||
| `routes/setup.ts` → `db/schema.ts` appConfig | Drizzle queries | NEW table, same db client and connection pool |
|
||||
| `broker/vevent.ts` → `ical.js` | `ICAL.Component('valarm')`, `ICAL.Duration.fromSeconds` | EXTEND existing ical.js usage |
|
||||
| `broker/sync.ts` → `db/schema.ts` | Store `reminderLeadMinutes` in events upsert | MODIFY existing upsert path in syncCalendar |
|
||||
| `reminderScheduler.ts` → `db/schema.ts` | Read `reminder_lead_minutes` in Drizzle query | MODIFY existing query; raw SQL expression |
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
### Anti-Pattern 1: Fetching from Fastmail on Every Calendar Request
|
||||
|
||||
**What people do:** Proxy each PWA calendar load directly to CalDAV — no cache.
|
||||
|
||||
**Why it's wrong:** Fastmail CalDAV has undocumented rate limits. Cold requests are slow (multi-round-trip CalDAV discovery). Calendar loads become unavailable if Fastmail is unreachable.
|
||||
|
||||
**Do this instead:** Always read from the MariaDB cache. Write-back is synchronous (PUT + cache upsert). Background poller handles incoming changes from the Fastmail app.
|
||||
|
||||
### Anti-Pattern 2: Using Email as the App User Key
|
||||
|
||||
**What people do:** Map Authelia `email` claim → `users.email` primary key, assume it's stable.
|
||||
|
||||
**Why it's wrong:** OpenID Connect spec does not guarantee `email` stability or uniqueness. Authelia docs explicitly warn against this.
|
||||
|
||||
**Do this instead:** Use `oidc_iss` + `oidc_sub` as the stable composite identity. Store email as a display/provisioning hint only.
|
||||
|
||||
### Anti-Pattern 3: Storing Recurring Events Only as Expanded Instances
|
||||
|
||||
**What people do:** Pre-expand all RRULE events into individual rows in the DB to simplify queries.
|
||||
|
||||
**Why it's wrong:** Event edits (changing the series, adding exceptions) then require reconstructing the original RRULE plus EXDATE/RDATE logic — a nightmare. Fastmail is source of truth for RRULE; the cache should reflect it faithfully.
|
||||
|
||||
**Do this instead:** Store the raw VEVENT blob. Add a `dtstart_utc` column for range queries. Expand in the PWA for display.
|
||||
|
||||
### Anti-Pattern 4: Separate Microservices for Broker, Lists, Push
|
||||
|
||||
**What people do:** Split the broker, list API, and push sender into separate containers because "separation of concerns."
|
||||
|
||||
**Why it's wrong:** For a 2-person household app on a single Unraid host, this is pure overhead. Three containers to coordinate, three deployment targets, inter-service HTTP overhead with zero benefit.
|
||||
|
||||
**Do this instead:** One API container. Three modules within it. Split to services only if one component needs independent scaling (it won't).
|
||||
|
||||
### Anti-Pattern 5: Per-User CalDAV Credentials in the PWA
|
||||
|
||||
**What people do:** Give each household member their own Fastmail app password and have the PWA authenticate to CalDAV directly.
|
||||
|
||||
**Why it's wrong:** Exposes Fastmail credentials client-side; bypasses the broker cache; defeats the aggregation purpose.
|
||||
|
||||
**Do this instead:** All Fastmail access goes through the backend broker. The PWA authenticates only to the app API.
|
||||
|
||||
## Scaling Considerations
|
||||
|
||||
This is a 2-person household app. Scaling is not a concern. The architecture should remain simple.
|
||||
|
||||
| Concern | At 2 users (target) |
|
||||
|---------|---------------------|
|
||||
| CalDAV polling | Single poller, 5-min interval, negligible load |
|
||||
| MariaDB | Single instance, no replication needed |
|
||||
| Redis | Single instance, no clustering |
|
||||
| WebSocket | Single Node process handles 2 connections trivially |
|
||||
| Web Push | At most ~10 push subscriptions (phone + desktop per user) |
|
||||
|
||||
## Open Research Flags for Phase Research
|
||||
|
||||
These items require hands-on investigation during the broker implementation phase — they cannot be resolved by documentation alone:
|
||||
|
||||
1. **Fastmail personal-calendar sharing visibility (MEDIUM risk):** Can the broker account's CalDAV principal discovery see calendars that another Fastmail account has shared to it via the in-app sharing UI? If yes, one app password suffices. If no, the fallback is storing an encrypted app password per household member. Test this before building the broker's calendar-discovery logic.
|
||||
|
||||
2. **Fastmail CalDAV rate limits (LOW risk):** No documented rate limit for CalDAV. 5-min polling with ctag pre-check (one PROPFIND per calendar per tick) is conservative and unlikely to hit limits. Monitor HTTP 429 responses and back off exponentially if encountered.
|
||||
|
||||
3. **iOS Push reliability (LOW risk for this use case):** iOS 16.4+ supports Web Push only when the PWA is installed to the home screen. The wife will be asked to install the PWA as part of onboarding. Canada is not in the EU, so the iOS 17.4 DMA PWA restriction does not apply. Push is a "nice to have" alarm, not a safety-critical channel — missed notifications are acceptable.
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- [Fastmail API Documentation](https://www.fastmail.com/dev/) — CalDAV supported, JMAP calendars pending RFC finalization
|
||||
- [Fastmail: Calendar Sharing with Other Users](https://www.fastmail.help/hc/en-us/articles/1500000279781-Sharing-calendars-with-other-users) — ACL tiers confirmed; cross-account token access not documented
|
||||
- [Fastmail: Shared Calendaring Improvements](https://www.fastmail.com/blog/shared-calendar-improvements/) — CalDAV ACL standards alignment, per-user reminders
|
||||
- [Fastmail: App Passwords](https://www.fastmail.help/hc/en-us/articles/360058752854-App-passwords) — CalDAV uses app passwords, not API tokens
|
||||
- [Using Fastmail with CalDAV libraries](https://utf9k.net/blog/fastmail-caldav/) — Principal URL format, Cyrus IMAP internals
|
||||
- [Authelia OIDC Claims](https://www.authelia.com/integration/openid-connect/openid-connect-1.0-claims/) — iss+sub as stable identity anchor
|
||||
- [CalDAV ctag Extension](https://github.com/apple/ccs-calendarserver/blob/master/doc/Extensions/caldav-ctag.txt) — ctag-based efficient polling
|
||||
- [rrule.js](https://github.com/jkbrzt/rrule) — Client-side RRULE expansion
|
||||
- [web-push npm](https://github.com/web-push-libs/web-push) — VAPID push sender
|
||||
- [PWA iOS Push Limitations 2025](https://www.magicbell.com/blog/pwa-ios-limitations-safari-support-complete-guide) — iOS 16.4+ push support, EU exception
|
||||
- [Redis Pub/Sub + WebSocket pattern](https://ably.com/blog/scaling-pub-sub-with-websockets-and-redis) — Architecture reference
|
||||
All findings are grounded in direct codebase inspection:
|
||||
|
||||
- `apps/api/src/broker/outboxWorker.ts` — drain loop, `isDraining` guard, durability decisions (D-04/D-07/D-08/CR-05)
|
||||
- `apps/api/src/broker/reminderScheduler.ts` — scheduler logic, dedup map, fixed-window rationale (D-05/D-06/D-12)
|
||||
- `apps/api/src/broker/vevent.ts` — `buildVeventString`, `NewEventParams`, ical.js RRULE pattern (VALARM follows same API)
|
||||
- `apps/api/src/broker/crypto.ts` — `encryptPassword`/`decryptPassword`; confirmed reusable by admin routes
|
||||
- `apps/api/src/db/schema.ts` — table definitions; migration impact analysis
|
||||
- `apps/api/src/routes/events.ts` — `eventFieldsSchema`, `outboxPayloadSchema` contract, IN-03 duplication requirement
|
||||
- `apps/api/src/lib/listEmitter.ts` — in-process EventEmitter precedent for `outboxTrigger.ts`
|
||||
- `apps/pwa/src/components/EventForm.tsx` — full form state machine; integration points for reminder selector
|
||||
- `apps/pwa/src/components/SettingsSheet.tsx` — existing Settings surface; Admin section attachment point
|
||||
- `apps/api/src/index.ts` — route mounting order, OIDC guard placement, `isMainModule` guard pattern
|
||||
- `apps/api/src/routes/me.ts` — `/api/me` response shape; where `isAdmin` is added
|
||||
- `apps/api/tests/` — test directory layout confirms tests in `tests/`, not `src/`; `helpers/db.ts` pattern
|
||||
|
||||
---
|
||||
*Architecture research for: FamilySync — self-hosted family calendar + list hub*
|
||||
*Researched: 2026-06-03*
|
||||
*Architecture research for: FamilySync v1.1 Operability & Polish*
|
||||
*Researched: 2026-06-10*
|
||||
|
||||
Reference in New Issue
Block a user