Lucas Berger
982438dc10
Mechanical reformat — no logic changes. 398 files changed, 19125 insertions(+), 16457 deletions(-). Prettier 3.8.4 with .prettierrc (singleQuote:true, semi:true, tabWidth:2, trailingComma:all, printWidth:100). Isolated per D-13-08 for reviewability.
106 lines
11 KiB
Markdown
106 lines
11 KiB
Markdown
# Project Research Summary
|
||
|
||
**Project:** FamilySync — v1.1 "Operability & Polish"
|
||
**Domain:** Self-hosted family calendar + lists PWA (operability/admin milestone on a shipped v1.0)
|
||
**Researched:** 2026-06-10
|
||
**Confidence:** HIGH (all findings grounded in direct v1.0 source inspection + v1.0 retrospective)
|
||
|
||
## Executive Summary
|
||
|
||
v1.1 adds six operability/polish features to the proven v1.0 stack (Node 22 + Hono + Drizzle/MariaDB + tsdav/ical.js + web-push, React 19 + Vite + Schedule-X PWA, on Unraid/Docker behind Authelia OIDC + Pangolin/Newt, self-hosted Gitea with an Actions runner). The core stack is unchanged. Every feature reuses existing capabilities; **only one new dependency is warranted — `@playwright/test` (dev, `apps/pwa` scope)** for the authenticated mobile test harness. No new runtime packages: the setup wizard's validations are all covered by `zod` + `mysql2` + native `fetch` + `Buffer`/`web-push`.
|
||
|
||
The six features: (1) **per-event reminders** — VALARM authoring on the event form + a scheduler that honors each event's lead instead of the hardcoded 15-min; (2) **event-driven outbox drain** — cut perceived write-back latency from ~15s to ~1s; (3) **admin Settings** — role-gated UI to manage encrypted app passwords and designate the shared calendar; (4) **initial setup wizard** — first-run validated bootstrap of env/VAPID/DB/app-password; (5) **Gitea CI** — regression gate on PR + Docker image publish; (6) **mobile-emulated authed Playwright harness**.
|
||
|
||
Three preservation rules are non-negotiable and drive the design: VALARM authoring must **preserve native-client alarms on edit** (never rebuild-from-scratch and silently strip); the outbox durability guarantees (optimistic-202, create-before-delete ordering, drain concurrency guard, fresh-etag-before-PUT, per-uid exactly-once dedup) must be **unchanged** when the drain goes event-driven; and wizard-collected secrets (VAPID private key, `APP_PASSWORD_ENCRYPTION_KEY`) must **stay in env — never touch the DB or any response body**.
|
||
|
||
## Key Findings
|
||
|
||
### Recommended Stack
|
||
|
||
No stack change. One new dev dependency; everything else reuses v1.0.
|
||
|
||
**Core additions:**
|
||
|
||
- **`@playwright/test`** (dev, `apps/pwa`): the global `playwright-cli` binary is interactive tooling and exposes no `storageState`/`devices` presets — `@playwright/test` is required for CI spec files doing authenticated, device-emulated runs. The two coexist. Auth via the existing `DEV_AUTH_BYPASS` avoids mocking Authelia.
|
||
- **Gitea Actions workflows** (`.gitea/workflows/*.yml`, no npm packages): GitHub-Actions-compatible syntax but `runs-on: self-hosted`; job image `catthehacker/ubuntu:act-latest`; MariaDB service container `mariadb:11` with `healthcheck.sh --connect --innodb_initialized` (NOT `mysqladmin ping` — removed in MariaDB 11); Docker push via `docker/login-action@v3` + `docker/build-push-action@v5` needs a Gitea PAT with `write:package` scope (no built-in token has registry push rights).
|
||
- **Setup wizard validation — zero new deps:** env presence via `zod.safeParse`, DB via `mysql2` connect, VAPID via `Buffer.from(key,'base64url').length === 32`, OIDC via native `fetch('/.well-known/openid-configuration')`.
|
||
|
||
### Expected Features
|
||
|
||
**Must have (table stakes):**
|
||
|
||
- Per-event reminder selector with preset offsets (None / 5m / 10m / 15m / 30m / 1h / 2h / 1d / 2d); **"None" is the default** (no VALARM, no push). All-day events fire at 9 AM on the alert day (Apple convention). Existing VALARMs round-trip — never silently stripped.
|
||
- Admin Settings scoped to exactly two tasks: rotate/re-enter a member's app password (with inline CalDAV test) and toggle `calendars.is_shared`. Single `users.is_admin` boolean gate.
|
||
- Setup wizard: validated first-run steps (DB, app URL, OIDC, session secret auto-gen, encryption key auto-gen, VAPID auto-gen + structural check, admin account, Fastmail app password CalDAV PROPFIND test). Inline per-field validation; Next disabled until step passes.
|
||
- Faster write-back: target < 2s perceived; trigger an immediate drain on enqueue, keep the interval as fallback.
|
||
|
||
**Should have (competitive / differentiator):**
|
||
|
||
- Gitea CI PR gate + on-merge Docker publish.
|
||
- Mobile Playwright harness (`devices['iPhone 15']`, stored auth via `DEV_AUTH_BYPASS`).
|
||
- Multiple alarms per event (2× VALARM) — stretch, defer to v1.2.
|
||
|
||
**Anti-features (explicitly OUT — scope creep for a 2-member household):** notification-preferences UI, reminder snooze, wizard re-run, audit log, health dashboard, user management, provider abstraction (stays backlog 999.1), self-service member onboarding mixed into the wizard (stays backlog 999.5), real-device iOS CI.
|
||
|
||
### Architecture Approach
|
||
|
||
Findings grounded in the actual v1.0 codebase. Integration points (real paths):
|
||
|
||
**Major components:**
|
||
|
||
1. **DB migration** — `users.is_admin BOOLEAN DEFAULT 0`, `calendar_events.reminder_lead_minutes INT NULL`, an `app_config`/setup-state table. Foundation; blocks the role-gated and reminder work. (generate+migrate, never `push`.)
|
||
2. **Event-driven drain** — in-process `EventEmitter` (`lib/outboxTrigger.ts`, mirroring existing `listEmitter.ts`); signal after `db.insert(calendarOutbox)` in the three write handlers in `routes/events.ts`; subscribe in `startOutboxWorker()`. The existing `isDraining` guard already covers concurrent invocations. **Redis pub/sub is wrong here** — the drain is single-process by design.
|
||
3. **VALARM write path** — `buildVeventString` in `broker/vevent.ts` gains a `reminderMinutes?` param using `ICAL.Component('valarm')` + `ICAL.Duration.fromSeconds` (same ical.js surface as RRULE). `eventFieldsSchema` (routes/events.ts) and `outboxPayloadSchema` (outboxWorker.ts) must change in sync (flagged by the IN-03 comment).
|
||
4. **Variable-lead scheduler** — `reminder_lead_minutes` populated by `sync.ts` parsing the VALARM TRIGGER; scheduler query uses `DATE_SUB(dtstart_utc, INTERVAL reminder_lead_minutes MINUTE)` over a ±1-min window; dedup key becomes compound `uid:dtstartMs`. Drop the `isShared`-only restriction for reminder pushes (a user who set an alarm wants it regardless of calendar).
|
||
5. **Admin role + setup wizard** — `routes/admin.ts` with `requireAdmin` middleware reusing `broker/crypto.ts`; setup wizard and admin Settings are two frontend consumers of the same `/api/admin/*` + `/api/setup/*` routes (do not duplicate). `GET /api/setup/status` mounts **before** the OIDC guard (like `/health`). VAPID/AES keys stay in env — wizard generates + displays for the operator to copy.
|
||
|
||
### Recommended Phase Structure (dependency-ordered)
|
||
|
||
Starting at **Phase 7** (continues v1.0 numbering). Critical path with two fully independent parallel tracks:
|
||
|
||
1. **DB foundation** (migration: is_admin, reminder_lead_minutes, setup-state) — blocks the role/reminder work.
|
||
2. **Faster write-back** (event-driven drain) — small, low-risk, immediate benefit; independent after migration.
|
||
3. **Admin role + routes** — precondition for both admin Settings and setup wizard.
|
||
4. **Setup wizard** (backend + UI) — depends on the admin/role + config plumbing.
|
||
5. **Admin Settings UI** — depends on admin role; shares routes with the wizard.
|
||
6. **Per-event reminders** (VALARM authoring + variable-lead scheduler) — independent track, largest scope; keep authoring + preserve-on-edit + scheduler in one phase.
|
||
7. **Gitea CI** — fully independent; start with a runner-probe step.
|
||
8. **Mobile Playwright harness** — fully independent.
|
||
|
||
Tracks 7 and 8 have no code dependencies and can run parallel to anything.
|
||
|
||
### Critical Pitfalls (phase-mapped)
|
||
|
||
1. **VALARM round-trip strips native alarms** (reminders phase) — update path must extract + preserve existing VALARM from `rawVevent`, not rebuild from scratch.
|
||
2. **TRIGGER serialized as VALUE=TEXT** (reminders phase) — use `ICAL.Duration.fromSeconds(-n*60)`, verify ICS has no `VALUE=TEXT`.
|
||
3. **All-day reminder semantics undefined** (reminders phase) — disable selector when `allDay` in UI; guard API.
|
||
4. **Per-event lead breaks uid-only dedup** (reminders phase) — rescheduled events go invisible; fix with `uid:dtstartMs` compound key.
|
||
5. **Event-driven drain double-execution** (write-back phase) — use a `drainRequested` flag checked by the interval callback, not direct concurrent `runOutboxDrain()` calls; preserves create-before-delete + etag handling.
|
||
6. **Setup endpoint reachable post-setup** (wizard phase) — guard checked on every invocation (member_credentials AND VAPID env present → 423), not just at startup; never log/echo the app password; secrets never persisted to DB.
|
||
7. **Gitea ghost failures** (CI phase) — MariaDB readiness race (healthy ≠ accepting connections) and runner env assumptions (Node 22 + pnpm not guaranteed) — start with a probe-only workflow.
|
||
|
||
## Research Flags (deeper investigation during planning)
|
||
|
||
- **Reminders phase:** confirm Fastmail CalDAV accepts the chosen TRIGGER value type; exact `rawVevent` round-trip on the update path.
|
||
- **Wizard phase:** OIDC discovery failure handling (retry/timeout/fallback).
|
||
- **CI phase:** probe the actual Unraid Gitea runner (Docker socket mount, Node/pnpm versions) before designing pipelines.
|
||
- **Mobile harness:** fixed user (DEV_AUTH_BYPASS user 1) vs parameterized; prod service worker must be neutralized in the base context.
|
||
|
||
**Well-documented — skip research:** event-driven drain (EventEmitter proven in `listEmitter.ts`), admin/wizard routes (standard Hono + zod + React forms), Playwright device/storageState APIs.
|
||
|
||
## Open Questions to Resolve in Requirements
|
||
|
||
1. Admin tab visibility — both members are operators; should the non-technical member see it?
|
||
2. Drain transport — confirm single-container deployment (direct in-process signal) vs any multi-replica need (would require Redis).
|
||
3. Playwright auth — DEV_AUTH_BYPASS-only vs programmatic OIDC; fixed vs parameterized user.
|
||
|
||
## Confidence
|
||
|
||
| Domain | Confidence | Notes |
|
||
| ---------------- | ---------- | ---------------------------------------------------------------------------------------- |
|
||
| Stack | HIGH | v1.0 proven; one dev dep; Gitea syntax compatible (MariaDB 11 healthcheck caveat noted) |
|
||
| Features | HIGH | All have prior art; scoped to a tiny household |
|
||
| Architecture | HIGH | Grounded in real v1.0 source; component boundaries + build order sound |
|
||
| Pitfalls | HIGH | 15 pitfalls from codebase review + RFC 5545 + v1.0 retrospective, each mapped to a phase |
|
||
| Unraid CI runner | MEDIUM | Runner Docker-socket/Node/pnpm state unknown until probed |
|