Lucas BergerandClaude Opus 4.8
7fbb3cca9d
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
131 lines
30 KiB
Markdown
131 lines
30 KiB
Markdown
# FamilySync
|
||
|
||
## What This Is
|
||
|
||
FamilySync is a self-hosted, Dockerized family organization hub for a two-person household, modeled loosely on Skylight. It gives the family one unified, color-coded calendar and shared collaborative lists (groceries, gift ideas), reachable from a single low-friction React PWA — across a mixed Android/Fastmail and Apple ecosystem — built as one custom app on top of the family's existing paid Fastmail account. Email stays with members' existing mail clients and is out of scope.
|
||
|
||
## Core Value
|
||
|
||
The household can see and co-edit one color-coded family calendar (shared + each member's personal) and shared lists from a single low-friction PWA — cross-ecosystem, no app store, no per-member calendar credential juggling.
|
||
|
||
## Current State
|
||
|
||
**Shipped: v1.1 Operability & Polish (2026-06-18)** — 14 phases (7–20), 57 plans. Full detail in [`MILESTONES.md`](MILESTONES.md) and [`milestones/v1.1-ROADMAP.md`](milestones/v1.1-ROADMAP.md).
|
||
|
||
v1.1 turned the v1.0 MVP into a configurable, administrable, maintainable app: guided first-run setup wizard, role-gated in-app admin (credential rotation, shared-calendar designation, member editor), per-event reminders with a variable-lead scheduler, near-instant (~1–2s) event write-back, local-auth (no-OIDC) mode, and auto timezone detection — all backed by a full self-hosted Gitea CI/CD pipeline (mobile + desktop Playwright regression, a real ESLint gate, dependency/secret/security scanning, dev↔prod image hygiene, and Docker publish). No more hand-editing env files or the database.
|
||
|
||
Deferred to backlog: self-service provider onboarding (999.5), provider abstraction (999.1), dark mode / theming (999.20), and a broader modern-styling refresh (999.21 — future milestone).
|
||
|
||
## Next Milestone
|
||
|
||
Not yet defined. Start with `/gsd-new-milestone` (questioning → research → requirements → roadmap). Candidate seeds in the backlog: dark mode / theming (999.20), modern visual refresh (999.21), self-service onboarding (999.5), provider abstraction (999.1), dev-user full-app exercise without a real calendar (999.19), and acting on the CI dependency report (999.18).
|
||
|
||
## Requirements
|
||
|
||
### Validated
|
||
|
||
<!-- Shipped and confirmed valuable. -->
|
||
|
||
- [x] Unified, color-coded calendar view aggregating all Fastmail-hosted calendars (shared family + each member's personal) — **Validated in Phase 2 (calendar-display)**: read-only day/week/month/agenda views, server-side recurrence expansion (DST-correct), all-day no-shift, color routing by member/shared. Operator UAT approved. (Shared/rose lane activates once a shared calendar is marked — deferred per D-16.)
|
||
- [x] Create / edit / delete events written back to the correct Fastmail calendar via the app's single broker token — **Validated in Phase 3 (Gate 2, live 2026-06-07)**: create (timed/all-day/weekly-recurring), edit, delete, and recurring-series delete all round-trip to caldav.fastmail.com; 412-conflict handled. Recurring repeat-bound + per-occurrence-duration UX are "create+display only in v1" gaps (backlog 999.7/999.8).
|
||
- [x] Authelia OIDC login for every member (true SSO) — **Validated in Phase 3 (Gate 2, live 2026-06-07)**: both members log in via real Authelia OIDC over Pangolin; distinct stable colors; session carried transparently by Authelia SSO. (Full-name legend needs an Authelia ID-token `claims_policy` — operator step.)
|
||
- [x] React PWA installable on iPhone via "Add to Home Screen" (no App Store) — **Validated in Phase 3 (Gate 2, live 2026-06-07)**: iOS install + full-screen standalone launch + standalone OIDC login (load-bearing) confirmed on the wife's iPhone. Android install walkthrough deferred (B5, not yet device-tested).
|
||
- [x] Shared collaborative lists (groceries, gift ideas) co-edited by both members, stored in MariaDB — **Validated in Phase 4 (shared-lists-live-sync)**: list + item CRUD, fractional-rank drag-reorder, member-scoped access (no cross-tenant leak proven at route layer).
|
||
- [x] Live list sync so co-edits appear without manual refresh — **Validated in Phase 4**: scoped SSE fan-out over Pangolin (transport smoke-tested), bounded-backoff reconnect, co-edits land within seconds.
|
||
- [x] Web Push notifications for event reminders and list changes — **Validated in Phase 5 (web-push-notifications)**: VAPID push for reminders, event-change, and coalesced list alerts; on-device UAT 1/2/5 PASS (iOS reminder delivery, iOS push, coalescing). Android event-change on-device confirmation + iOS standalone spinner remain device-only spot-checks at go-live.
|
||
- [x] Faster write-back so edits reach Fastmail in ~1–2s instead of ~15s (CAL-15) — **Validated in Phase 9 (faster-write-back)**: event-driven outbox drain via a zero-dependency in-process EventEmitter (`outboxTrigger.ts`); a committed enqueue publishes a fire-and-forget `signalOutboxDrain()` that funnels through the existing `isDraining`-guarded drain with a `drainRequested` trailing-re-drain, preserving optimistic-202, create-before-delete on moves, exactly-once per uid, and the 15s `setInterval` fallback. 5/5 success criteria verified; trigger-wiring tests assert SC-1/D-05/D-07.
|
||
- [x] Per-event reminders — choose a reminder lead per event (None / 5m / 10m / 15m / 30m / 1h / 2h / 1d / 2d, all-day → day-granularity + 9 AM fire), serialized as a VALARM, with a variable-lead scheduler that honors each event's lead (CAL-13/CAL-14, NOTIF-04/05/06) — **Validated in Phase 11 (per-event-reminders)**: pure VALARM serialization/classification layer (`buildTimedValarm`/`buildAllDayValarm`/`classifyValarms`/`extractValarms`/`computeAlertInstantUtc`); variable-lead scheduler with `uid:dtstartMs` dedup, dropped fixed-15-min/shared-only restriction, all-day 9 AM-local branch; `reminderLeadMinutes` threaded end-to-end with preserve-on-no-change (D-08); allDay-aware reminder picker with edit pre-population. Gap-closure (Plan 11-05) fixed two code-review blockers — custom/other-client VALARMs are now preserved on edit via a surfaced `reminderIsCustom` signal (CAL-14 / Pitfall 1), and the all-day push body no longer reads "Starts in 0 min" — plus post-event-trigger classification, a server-side max bound, and helper-text gating. 5/5 must-haves verified; 347 API + 206 PWA tests green. **Deferred:** live Fastmail VALARM round-trip + on-device push fire (untestable in dev — no provider connected; backlog 999.19).
|
||
- [x] Admin role + role-gated settings surface to rotate member Fastmail app passwords and designate the shared calendar (ADMIN-01/02/03) — **Validated in Phase 10 (admin-role-settings)**: v1.1 DB foundation (`users.is_admin`, `member_credentials.provider_type`+`unique(user_id)`, `calendar_events.reminder_lead_minutes`, `app_config`) via an additive generate+migrate migration; DB-backed `requireAdmin` gating all `/api/admin/*` (client `isAdmin` UX-only, server 403 the real boundary, D-03); one shared `validateEncryptAndStoreCredential` helper for admin rotation + member self-service `/api/me/credential` (400-no-echo, session-userId only); exclusive shared-calendar designation made transactional + 404-guarded (CR-01 fix); gated `/admin` PWA route + conditional nav + `SetupBanner`. 12/12 must-haves verified; admin route-guard/nav-gating green in real Chromium (e2e 5/5). Deferred follow-ups: WR-01 bootstrap-race (Phase 12 reworks the bootstrap), broker `credentialSync.ts`/`CredentialSheet.tsx` crypto re-audit under full read access.
|
||
- [x] Initial setup wizard — first-run validated bootstrap of env/secrets/DB/OIDC/VAPID/app-password instead of hand-editing files; locks once complete (SETUP-01/02/03/04) — **Validated in Phase 12 (initial-setup-wizard)**: pre-auth `/api/setup/*` router mounted before the OIDC guard; each input validated (DB connects, VAPID decodes to 32 bytes + pairs with the public key, OIDC discovery resolves, app password reaches CalDAV); generated secrets shown for env copy and never persisted to the DB; completing user promoted to admin and a 423 guard enforced on every invocation. First-login-claims rework in `upsertUser` (no email coupling).
|
||
- [x] Self-hosted Gitea CI/CD + automated browser test coverage (TEST-01/02, CI-01/02) — **Validated in Phases 7/8/13/14/15/16**: a mobile (iPhone/WebKit + Pixel/Chromium) **and** desktop Playwright harness reached via `DEV_AUTH_BYPASS`; a PR pipeline gating lint (real ESLint flat config) / typecheck / unit / MariaDB-backed API integration / the headless harness; doc-only PRs skip the slow jobs via an always-running `gate` aggregate; dependency audit + gitleaks + eslint-plugin-security + dev↔prod image-hygiene assertions; and a publish job pushing the API production image on merge to `main`.
|
||
- [x] Local-auth (no-OIDC) mode (AUTH-LOCAL-*) — **Validated in Phase 19 (local-auth-no-oidc-mode)**: full local username/password account model — scrypt hashing, stateless `local-session` JWT cookie, `local_credentials` table, rate-limit/lockout login + logout, admin create/reset member, self-change password, OIDC-link to claim a local user, and a break-glass reset-admin CLI — coexisting with the Authelia OIDC path, removing the hard dependency on a deployed Authelia for solo/small self-hosters.
|
||
- [x] Household timezone as an explicit, stored, auto-detected, admin-changeable setting (Phase 18 D-01..D-07) — **Validated in Phase 18 (auto-timezone-detection)**: `getHouseholdTimezone(db)` with IANA validation is the source of truth for the all-day "9 AM local" reminder computation (replacing the implicit `process.env.TZ`), seeded from the browser at first run and changeable from `/admin`; browser-local display/timed-write path untouched.
|
||
- [x] PWA visual identity + phone-layout polish + admin member editor (Phase 17 D-01..D-10, Phase 20 D-01..D-07) — **Validated in Phases 17 & 20**: fixed the phone BottomTabBar/FAB overlap (with a CI regression guard), shipped the real FamilySync logo + full favicon/PWA-icon set + brand accent, restructured `tokens.css` into a themeable semantic-token layer (light-only groundwork), added a logout control + desktop-centered sheets; and replaced the per-row Rotate/Reset buttons with a single tappable member-editor sheet over `PATCH /api/admin/members/:id` (last-admin guard), retiring the confusing "Rotate" copy.
|
||
|
||
### Active
|
||
|
||
<!-- Carried into v1.x — partially validated or device-pending. -->
|
||
|
||
- [ ] Android PWA install walkthrough verified on a real Android device (iOS validated Phase 3; Android = carried Gate 2 row B5)
|
||
- [ ] Low-friction onboarding for the non-technical Apple member — visit one URL, sign in. **Partially validated Phase 3** (wife logged in + installed unaided); the per-member Fastmail app-password provider-setup step is still missing (backlog 999.5)
|
||
|
||
### Out of Scope
|
||
|
||
<!-- Explicit boundaries with reasoning to prevent re-adding. -->
|
||
|
||
- Wall-display / kiosk dashboard — deferred to v2; calendar + lists are the v1 core, display is the deferrable piece
|
||
- Email features — members keep existing mail clients; never the product's job
|
||
- Self-hosted calendar server (Baikal) — Fastmail hosts all calendars; one fewer service. Reversible if self-hosted calendar is ever wanted
|
||
- Vikunja / external task backend — lists live in the custom MariaDB; cross-ecosystem native task sync is impossible anyway
|
||
- DAVx5 — not needed
|
||
- React Native / App Store publishing — PWA delivers app-like UX without publishing overhead
|
||
- PostgreSQL — not in the existing stack; MariaDB is the database
|
||
- Forking existing apps — the value is the aggregation/display layer, not the backends
|
||
|
||
## Context
|
||
|
||
- **Household:** Primary user (project owner) on Fastmail + Android, uses the Fastmail app natively. Other member (wife) on Apple (iPhone/Mac). Wife adoption is a hard UX constraint.
|
||
- **One app to build:** backend + MariaDB + React PWA behind Authelia, sitting on top of Fastmail. This is far lighter than the prior Baikal + Vikunja + PWA design that was evaluated and dropped.
|
||
- **Calendars are all on Fastmail.** Personal calendars are also Fastmail-hosted calendar collections, so the app reads and writes everything (shared + personal) through a single Fastmail JMAP/CalDAV broker token — no external ICS feeds, no per-member credential management. The primary user gets the shared calendar natively in the Fastmail app; Apple members use the PWA (optionally subscribe in native Apple Calendar via CalDAV).
|
||
- **Prior exploration:** Architecture was revised across two sessions (explore → Opus verification). See `.planning/notes/familysync-architecture.md` for full reasoning behind the dropped options.
|
||
- **Shipped state (v1.0, 2026-06-10):** pnpm monorepo — `apps/api` (Hono + Drizzle/MariaDB + tsdav/ical.js CalDAV broker) and `apps/pwa` (React 19 + Vite + vite-plugin-pwa + Schedule-X). Live over real Authelia OIDC + Pangolin/Newt. ~338 files changed across the milestone.
|
||
- **Open questions — resolved this milestone:**
|
||
- Fastmail API → **CalDAV via tsdav** (JMAP calendars unavailable on Fastmail); locked.
|
||
- PWA Web Push → **sufficient with caveats**: VAPID direct push works on iOS 16.4+ installed PWAs and Android; iOS revokes subscriptions after 3 silent pushes (every push must be visible) and standalone install is mandatory. No FCM broker needed.
|
||
- Per-member personal calendar → **CAL-08 GO**: a per-member Fastmail app password reaches all of that account's calendars; no cross-account ACL. Onboarding flow to collect each member's app password is backlog 999.5.
|
||
- **Known issues / tech debt carried to v1.x:** event write-back latency (15s outbox drain, 999.13); per-event reminder config / VALARM authoring (999.4); first-login provider setup (999.5); admin Settings + setup wizard (999.10/999.11); Gitea CI (999.14); mobile-emulated authed browser testing (999.12).
|
||
|
||
## Constraints
|
||
|
||
- **Tech stack**: MariaDB for the database — PostgreSQL is not available in the stack
|
||
- **Tech stack**: Redis available (optional, for live list sync / push)
|
||
- **Infrastructure**: Unraid host running Docker + Docker Compose
|
||
- **Auth**: Authelia (already deployed) — OIDC/OAuth2 for the custom app; all members authenticate through it
|
||
- **Calendar backend**: Fastmail (paid, existing) is the single source for all calendars via JMAP/CalDAV
|
||
- **Frontend**: React PWA only — no React Native, no App Store
|
||
- **Networking**: Split-DNS internal domain, private IPs internally; public exposure via Pangolin/Newt tunnel, no open ports
|
||
- **UX**: Must be slick and low-friction for a non-technical Apple member — hard constraint, drives design tradeoffs
|
||
- **Scope**: No email features; no forking existing apps
|
||
|
||
## Key Decisions
|
||
|
||
| Decision | Rationale | Outcome |
|
||
| ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||
| Calendar hosted on Fastmail, not self-hosted (Baikal dropped) | Fastmail reaches the whole household native-or-PWA with one fewer self-hosted service; Fastmail Android app can't show self-hosted CalDAV anyway | ✓ Validated (v1.0) |
|
||
| Personal calendars also Fastmail-hosted collections, aggregated via one broker token | Avoids external ICS feeds and per-member credential management; uniform read/write path | ✓ Validated (v1.0, CAL-08 GO) |
|
||
| Shared lists in custom MariaDB, not Vikunja | Cross-ecosystem native task sync is impossible; a list table in the backend being built anyway is trivial vs another container + SSO integration | ✓ Validated (v1.0, Phase 4) |
|
||
| React PWA only, no React Native / App Store | App-like UX with one codebase for all surfaces; lowest onboarding friction (one URL) | ✓ Validated (v1.0, Phase 3) |
|
||
| Authelia OIDC for all members | True SSO consistent with existing infra; no separate calendar-credential problem | ✓ Validated (v1.0, Phase 3) |
|
||
| Web Push notifications in v1 (not deferred) | Family alerts judged essential day one; build push into the foundation | ✓ Validated (v1.0, Phase 5) |
|
||
| Personal-calendar overlay in v1 (not shared-only) | Unified view of everyone's schedules is the Skylight magic worth shipping early | — Validated (CAL-08 GO, Phase 1): per-member app password reaches all of an account's calendars; no cross-account ACL needed |
|
||
| **D-14:** Defer Phase 1 Gate 2 (live Authelia/Pangolin verification). SSE-over-Pangolin smoke = hard gate before Phase 4; live AUTH smoke incl. iOS standalone-PWA folded into Phase 3; full 2-member prod login verified there. Phases 2–3 develop behind a documented dev-auth bypass. | Gate 2 needs operator infra (Authelia config + tunnel) + docs that didn't exist; deferring unblocks Phase 2/3 code without rework risk, since the broker data path (CAL-01/CAL-08) is already proven live. SSE must still be verified before Phase 4 to avoid building live-sync on an unverified transport (#1034). | Tracked: `01-HUMAN-UAT.md`, `docs/deployment.md` |
|
||
| **D-15:** Validate the real external topology via a **local Newt connector + test subdomain** through existing Pangolin (Mode A), not an Unraid deploy. Unraid (Mode B) reserved for go-live. | Authelia OIDC + SSE pass-through behaviour live in Authelia + Pangolin/Newt, not in where the origin runs — so a local Newt rig faithfully tests both, decoupling "does the topology work" from "is it in production." Newt dials outbound (no open ports). Only shared touch is an additive, reversible Authelia client. | ✓ Validated (Gate 2 executed live in Phase 3 / D-17) |
|
||
| **D-16 (2026-06-05, Phase 2):** No dedicated Fastmail "broker" account. The **shared-family calendar is a calendar collection created on the operator's primary Fastmail account** (`me@lucasberger.ca`) and shared out to the wife + others via Fastmail's own calendar sharing. The app's single app password enumerates it like any other collection; the `calendars.is_shared` flag (operator-set) marks which row is the shared one. | Clarified during the Wave 2 checkpoint: "broker account" was only ever the role the primary account's app password plays. id=1 ("Calendar") is the operator's **personal** calendar, not the shared one — so it must NOT be marked `is_shared`. Aggregating each _other_ member's **personal** calendar still follows the D-09 per-member app-password model (open for Phase 3 onboarding: a member may get a personal color lane, or only the shared calendar). | ✓ Resolved (2026-06-10): "FamilySync" shared calendar created on the primary account, synced as `calendars.id=10`, marked `is_shared=1`; shared lane + reminders now active |
|
||
| **D-17 (2026-06-07, Phase 3):** Phase 1 Gate 2 (deferred per D-14) was executed live during Phase 3 against real Authelia OIDC over Pangolin/Newt (Mode A), clearing the load-bearing iOS-standalone-login risk. The full event write path (create/all-day/recurring/edit/delete/conflict) is verified end-to-end to Fastmail. | Live bring-up surfaced bugs the dev-bypass build could not (newt MTU blackhole, OIDC state-cookie race, write-path timezone/identity/join/cache bugs, all-day off-by-one, color collisions). All fixed; UX gaps captured as backlog 999.3–999.9. | — Validated (Gate 2, `03-GATE2-RESULTS.md`). Carried: Android install (B5), SSE smoke (Phase 4 entry gate, D-14). |
|
||
| **D-18 (2026-06-12, Phase 9):** Faster write-back uses a **zero-dependency in-process EventEmitter** drain signal, not Redis — the drain is single-process by design; Redis stays only for list SSE. | The optimistic-202 outbox is single-process; an in-process signal funnelled through the existing `isDraining` guard preserves all durability guarantees without a new external dependency. (Redis was later removed entirely — quick 260618-smr — as it was unused at runtime.) | ✓ Validated (v1.1, Phase 9, CAL-15) |
|
||
| **D-19 (2026-06-17, Phase 19):** FamilySync ships **local username/password auth as a first-class mode coexisting with Authelia OIDC**, not OIDC-only. | The operator runs it this way; a hard dependency on a deployed Authelia is too heavy for solo/small self-hosters. A local user can be linked to an OIDC identity later (claim flow, never email-matched per D-10). | ✓ Validated (v1.1, Phase 19) |
|
||
| **D-20 (2026-06-11, Phase 8):** CI runs on the self-hosted Gitea runner with `runs-on: ubuntu-latest` (no self-hosted label) in Docker-executor mode; MariaDB readiness uses `healthcheck.sh --connect`, never `mysqladmin ping` (removed in MariaDB 11); the secret is `REGISTRY_PAT` (the `GITEA_` prefix is silently dropped). | Established by the runner-probe-first approach (PITFALLS 11/12); these constraints are load-bearing for every CI workflow in the repo. | ✓ Validated (v1.1, Phases 8/16, CI-01/02) |
|
||
|
||
## Evolution
|
||
|
||
This document evolves at phase transitions and milestone boundaries.
|
||
|
||
**After each phase transition** (via `/gsd-transition`):
|
||
|
||
1. Requirements invalidated? → Move to Out of Scope with reason
|
||
2. Requirements validated? → Move to Validated with phase reference
|
||
3. New requirements emerged? → Add to Active
|
||
4. Decisions to log? → Add to Key Decisions
|
||
5. "What This Is" still accurate? → Update if drifted
|
||
|
||
**After each milestone** (via `/gsd-complete-milestone`):
|
||
|
||
1. Full review of all sections
|
||
2. Core Value check — still the right priority?
|
||
3. Audit Out of Scope — reasons still valid?
|
||
4. Update Context with current state
|
||
|
||
---
|
||
|
||
_Last updated: 2026-06-18 after v1.1 milestone — Operability & Polish shipped (Phases 7–20, 57 plans): guided setup, in-app admin + member editor, per-event reminders, faster write-back, local-auth mode, auto timezone, and full Gitea CI/CD. Next milestone undefined — start with `/gsd-new-milestone`._
|