diff --git a/.planning/phases/04-shared-lists-live-sync/04-CONTEXT.md b/.planning/phases/04-shared-lists-live-sync/04-CONTEXT.md new file mode 100644 index 0000000..4d63c9c --- /dev/null +++ b/.planning/phases/04-shared-lists-live-sync/04-CONTEXT.md @@ -0,0 +1,131 @@ +# Phase 4: Shared Lists + Live Sync - Context + +**Gathered:** 2026-06-07 +**Status:** Ready for planning + + +## Phase Boundary + +Deliver **app-native shared lists** (stored in MariaDB, NOT CalDAV/Fastmail) with real-time co-edit sync: + +- Create and delete named lists (LIST-01) +- Add, check off, and delete items (LIST-02) +- Reorder items by drag-and-drop (LIST-03) +- Live co-edit sync over SSE — one member's change appears for the other within seconds, surviving a brief reconnect (LIST-04, success criterion 3) + +Lists are entirely app-owned data — no CalDAV write-back, no Fastmail involvement. This is the one track independent of the calendar write path. + +**⚠️ ENTRY GATE (D-14, issue #1034 — STILL UNVERIFIED as of 2026-06-07):** The 5-minute SSE-over-Pangolin smoke test must PASS before the live-sync layer is built (`/api/sse/heartbeat` held open 5+ min through the tunnel without being cut — see `docs/deployment.md` Gate 2 row 5). This is an operator/infra task requiring the Authelia+Pangolin/Newt rig. If it FAILS: fix Pangolin idle-timeout/buffering, OR the polling fallback (decided below) becomes mandatory rather than optional. Do not build live sync on an unverified transport. + + + + +## Implementation Decisions + +### Sharing model (List & item behavior) +- **D-01:** Lists support **shared and private** visibility. New lists **default to Shared** (visible+editable by both members); creator can toggle a single list to Private. Default-shared chosen deliberately — the grocery/family-hub use case is collaborative and default-private would add friction to the primary action. +- **D-02:** Data model is a **`list_shares` join table** (list has an `owner`; join table records who each list is shared with) — NOT a simple boolean. v1 UI is only shared/private, but the schema must be **member-count-agnostic** so granular N-recipient sharing is a future UI addition, not a migration. +- **D-03:** A private list still **live-syncs across its owner's own devices** (phone + tablet); it is never pushed to other members. +- **D-04:** **SSE fan-out MUST be scoped to who can see a list.** A list's change events broadcast only to members with access (owner + shares), never to all connected clients. This is the load-bearing consequence of the sharing model — get it right or private lists leak. + +### Item behavior +- **D-05:** Checked-off items **sink to a "completed" section** at the bottom (active items stay on top). Not strikethrough-in-place, not immediate-disappear — keeps the active list clean for groceries while preserving "what was done." +- **D-06:** **Confirm-on-delete for whole lists only.** Individual items delete instantly (live sync makes mistakes visible; easy to re-add). Reuse Phase 3's `DeleteConfirmationDialog` component for the list-delete dialog. + +### Live feel & conflict resolution +- **D-07:** **Optimistic UI** — the editing member's change shows instantly, then reconciles against the server (rollback on rejection). Fits the low-friction constraint. Use React Query optimistic updates. +- **D-08:** **Per-field writes + per-field last-write-wins** ("field-level merge", BOUNDED — no CRDT). The API PATCHes only the changed field (`checked`, `text`, or `position`), not the whole row; the server applies last-write-wins per field on a server timestamp. Result: "one toggles checked while the other edits text" → both stick. Same-field collisions fall back to last-write-wins. Do NOT build CRDTs or per-field vector clocks. +- **D-09:** **Delete-wins** — if one member deletes an item while the other edits it, deletion is final; the in-flight edit is dropped (editor sees it vanish via live sync). Edits never resurrect deleted items. + +### Reconnect & transport (success criterion 3) +- **D-10:** **Full refetch on reconnect** — on SSE reconnect, React Query invalidates and refetches the affected list(s) fresh. No server-side event log / Last-Event-ID replay. Lists are tiny so refetch is cheap and guaranteed-correct. +- **D-11:** **Silent auto-recover with capped backoff, then a visible indicator.** Reconnect silently with bounded (capped exponential) backoff; after backoff is exhausted, surface a visible "disconnected / updates paused" indicator and stop hammering. NOTE for planner: raw `EventSource` auto-reconnects forever with no backoff control — implementing bounded backoff + a give-up indicator requires wrapping `EventSource` in a manual reconnect loop or using a small SSE client lib. +- **D-12:** **Polling fallback via React Query `refetchInterval`** if SSE is unavailable/flaky through Pangolin. Already have React Query; trivial to add. Guarantees criterion 3 even if the tunnel misbehaves. (Mandatory if the entry-gate smoke test fails.) + +### Reordering (LIST-03) +- **D-13:** **String-based fractional rank** for item positions (e.g., the `fractional-indexing` approach) — NOT raw floats (precision exhausts fast on repeated mid-point inserts) and NOT integer-renumber (a single move rewrites many rows, noisy over SSE). A move rewrites only the moved item's rank — one-row write, plays well with live sync and concurrent reorders. +- **D-14:** **Animate to new order** when a remote reorder arrives (smooth transition, matches the live-sync promise). +- **D-15:** **Last-write-wins with brief settle** on concurrent reorder of the same item — both see their local drag instantly (optimistic), server resolves to the last write, both converge within ~1s. No drag-locking / drag-state broadcasting. + +### Navigation / app shell +- **D-16:** **Bottom tab bar** (Calendar | Lists) — thumb-reachable, matches native iOS/Android, low-friction for the non-technical member. Currently `App.tsx` renders `CalendarShell` directly with no nav. +- **D-17:** **Add react-router** for real URLs (e.g. `/lists/:id`). No router is installed today. Real URLs enable Phase 5 push deep-linking ("tap to open Groceries"), browser back button, and PWA shortcuts. Small dependency that pays off next phase. + +### Project-level principle (applies beyond this phase) +- **D-18:** **Design for N family members, not hard-coded two.** Schema, auth/access checks, and SSE fan-out must be member-count-agnostic. Same philosophy as treating Fastmail as a generic provider — set the framework now for future expansion to more family members. The `list_shares` table (D-02) and scoped fan-out (D-04) are the first applications. + +### Claude's Discretion (deferred to research/planner) +- **Fan-out mechanism:** in-memory EventEmitter vs Redis pub/sub. API runs as a **single Node process** today (no replicas), so in-memory is the YAGNI default; Redis is in docker-compose but `ioredis` is NOT installed. Planner must address this explicitly and justify the choice against D-18 (multi-process future). +- Exact position-rank datatype/column, SSE auth/middleware wiring, and React Query cache-key structure. + +### Reviewed Todos +- **Adopt drizzle generate+migrate workflow (retire `db:push` on MariaDB)** — directly relevant: Phase 4 adds new tables (`lists`, `list_items`, `list_shares`). `drizzle-kit push` is unsafe on populated MariaDB (emits false destructive diff — see memory). New tables MUST use `drizzle-kit generate` + `migrate`, not `push`. Folded as a hard constraint on this phase's schema work. + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Entry gate & transport +- `docs/deployment.md` §"SSE idle timeout (Phase 4 dependency, issue #1034)" and §"Gate 2 — Live verification checklist" row 5 — the SSE-over-Pangolin smoke-test procedure that is this phase's entry gate +- `apps/api/src/routes/sse.ts` — existing `/api/sse/heartbeat` SSE pattern (Hono `streamSSE`, `stream.aborted` loop); the live-list SSE endpoint(s) build on this + +### Prior decisions & requirements +- `.planning/ROADMAP.md` §"Phase 4: Shared Lists + Live Sync" — goal, success criteria, entry gate +- `.planning/REQUIREMENTS.md` — LIST-01 through LIST-04 +- `.planning/STATE.md` §Decisions — D-14 (SSE-over-WebSocket choice, entry gate), real-time transport notes +- `.planning/phases/01-foundation-broker-spike/01-CONTEXT.md` §D-08 — why the SSE smoke test was folded into Phase 1 to de-risk Phase 4 transport + +### Schema & code patterns +- `apps/api/src/db/schema.ts` — Drizzle table conventions (mysqlTable, indexes, unique keys, `references`/`onDelete`); model new list tables on these +- `apps/pwa/src/components/DeleteConfirmationDialog.tsx` — reuse for list-delete confirmation (D-06) +- `apps/pwa/src/App.tsx` / `apps/pwa/src/components/CalendarShell.tsx` — current shell with no router; tab-bar + react-router (D-16/D-17) wrap this + + + + +## Existing Code Insights + +### Reusable Assets +- `apps/api/src/routes/sse.ts` — working Hono `streamSSE` heartbeat; the live-list event stream extends this pattern (auth via existing `/api/*` middleware). +- `apps/pwa/src/components/DeleteConfirmationDialog.tsx` — Phase 3 confirmation dialog, reuse for list delete. +- React Query + Zustand already established (CLAUDE.md split: React Query = server state, Zustand = UI-only state). Optimistic updates (D-07) and polling fallback (D-12) use React Query; tab/route UI state is Zustand-adjacent. +- CSS token layer + colorUtils from Phase 2 available for list theming. + +### Established Patterns +- `/api/*` routes sit behind OIDC middleware (or dev-auth bypass) — list routes inherit this; identity resolved to `users.id` via oidc iss+sub (D-10 from prior phases). +- Drizzle schema conventions in `schema.ts`: int autoincrement PKs, `references(() => x.id, { onDelete: 'cascade' })`, composite unique keys, named indexes. +- Schema migrations: **generate+migrate, never `push`** on MariaDB (see Reviewed Todos). + +### Integration Points +- New `/api/lists` (+ items + SSE) routes mount in `apps/api/src/index.ts` alongside `eventsRouter`, `sseRouter`. +- New `lists` / `list_items` / `list_shares` tables in `apps/api/src/db/schema.ts`. +- PWA gains a router + bottom tab bar in `App.tsx`; Lists surface is a sibling of `CalendarShell`. +- SSE fan-out must integrate with the (TBD) in-memory-vs-Redis pub/sub decision; `redis` service exists in docker-compose, `ioredis` not yet a dependency. + + + + +## Specific Ideas + +- "Sink to bottom" for checked items modeled on a clean active-list / completed-section split (grocery-list mental model). +- Sharing UI vision (future): pick specific recipients from the user DB; v1 collapses this to shared/private but the `list_shares` model preserves the path. +- Backoff-then-pause reconnect UX: "set backoff and then display an indicator to pause more updates" — i.e., don't retry forever silently; tell the user when data may be stale. + + + + +## Deferred Ideas + +- **Anonymous list sharing via a unique public URL** (share a list with a non-member through a link) — NEW CAPABILITY, its own phase. Introduces unauthenticated access that bypasses the Authelia OIDC model (every `/api/*` route is currently authenticated), plus link-token generation, revocation, and abuse handling. Explicitly out of scope for Phase 4; revisit as a dedicated "external/guest sharing" phase. +- **Granular per-recipient sharing UI** (a member picker) — the `list_shares` data model (D-02) supports it, but no picker UI in v1 (only two members; "shared" == shared with the other person). Becomes relevant once the household has 3+ members (D-18). +- **List metadata** (icons, per-list colors, max items) — not raised as required; standard approaches fine unless a future UI phase wants them. + + + +--- + +*Phase: 4-Shared Lists + Live Sync* +*Context gathered: 2026-06-07* diff --git a/.planning/phases/04-shared-lists-live-sync/04-DISCUSSION-LOG.md b/.planning/phases/04-shared-lists-live-sync/04-DISCUSSION-LOG.md new file mode 100644 index 0000000..ef5e5b2 --- /dev/null +++ b/.planning/phases/04-shared-lists-live-sync/04-DISCUSSION-LOG.md @@ -0,0 +1,122 @@ +# Phase 4: Shared Lists + Live Sync - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-06-07 +**Phase:** 4-Shared Lists + Live Sync +**Areas discussed:** List & item behavior, Live feel & conflicts, Reconnect catch-up, Reordering behavior, Lists navigation + +--- + +## List & item behavior — Sharing + +| Option | Description | Selected | +|--------|-------------|----------| +| All lists shared | Every list visible+editable by both; no permission model | | +| Shared + private | Lists can be private; owner/visibility model + permission checks | ✓ (refined below) | + +**User's choice:** Shared + private — initially "default private, share to specific userdb members, optionally anonymous via unique URL." +**Notes:** Claude challenged three points: (1) anonymous URL = scope creep + unauthenticated security surface → deferred; (2) recipient picker = YAGNI for two members → boolean shared/private UI but `list_shares` join table underneath; (3) default-private fights the collaborative grocery use case → recommend default-shared. User accepted re-framing. + +## List & item behavior — Privacy (refined) + +| Option | Description | Selected | +|--------|-------------|----------| +| Default Shared, toggle to Private | Family-hub default; flip individual list private; boolean model | ✓ | +| Default Private, toggle to Shared | Owner-only default; explicit share step | | +| All shared, no private | Drop private entirely | | + +**User's choice:** Default Shared, toggle to Private. +**Notes:** Anonymous URL → Defer it. Data model → `list_shares` join table, with the explicit instruction: "remember this project is intended to expand to other family members in the future... similar to treating fastmail like a generic provider helps set that framework." Captured as project-level principle D-18. + +## List & item behavior — Checked items & delete guard + +| Option | Description | Selected | +|--------|-------------|----------| +| Strikethrough in place | Item stays, struck-through | | +| Sink to bottom | Checked move to completed section | ✓ | +| Disappear immediately | Removed from view | | +| Confirm list delete only | Dialog for lists; items delete instantly | ✓ | +| Confirm both | Dialog for lists and items | | +| No confirmation | Everything instant | | + +**User's choice:** Sink to bottom; confirm list-delete only. +**Notes:** Private lists still live-sync across the owner's own devices (owner-only visibility, multi-device sync). + +--- + +## Live feel & conflicts + +| Option | Description | Selected | +|--------|-------------|----------| +| Optimistic (instant local, reconcile) | Snappy; rollback on failure | ✓ | +| Server-confirmed | Wait for round-trip | | +| Last-write-wins | Later write wins per row | | +| Field-level merge | Merge non-conflicting fields | ✓ (bounded) | +| Delete wins | Deletion final, edit dropped | ✓ | +| Edit resurrects | Edit re-creates deleted item | | + +**User's choice:** Optimistic UI; field-level merge; delete wins. +**Notes:** Claude bounded "field-level merge" to per-field PATCH + per-field last-write-wins (no CRDT) to prevent over-engineering. User context implied agreement (momentum). + +--- + +## Reconnect catch-up + +| Option | Description | Selected | +|--------|-------------|----------| +| Full refetch on reconnect | React Query invalidate+refetch | ✓ | +| Last-Event-ID replay | Server replays missed events | | +| Hybrid | Replay, refetch fallback | | +| Silent + auto-recover | EventSource native reconnect, no UI | ✓ (refined) | +| Subtle indicator when offline | Show reconnecting hint | | +| Polling fallback (refetchInterval) | Periodic refetch if SSE drops | ✓ | +| SSE only, fix the proxy | Commit to SSE, no fallback | | + +**User's choice:** Full refetch; silent auto-recover with capped backoff then a "pause updates" indicator; polling fallback. +**Notes:** User specified "set back off and then display an indicator to pause more updates." Claude flagged that raw EventSource has no backoff control → needs a manual reconnect wrapper or SSE client lib. + +--- + +## Reordering behavior + +| Option | Description | Selected | +|--------|-------------|----------| +| Fractional rank | One-row write per move | ✓ (string-based) | +| Integer position + renumber | Many-row writes per move | | +| Animate to new order | Smooth remote reorder | ✓ | +| Update on next interaction | No animation | | +| Last-write-wins, brief settle | Optimistic, converge ~1s | ✓ | +| Lock during drag | Broadcast drag state | | + +**User's choice:** Fractional rank; animate to new order; last-write-wins settle. +**Notes:** Claude steered fractional rank to a string-based fractional index (e.g. `fractional-indexing`) rather than raw floats to avoid precision exhaustion on repeated mid-point inserts. + +--- + +## Lists navigation + +| Option | Description | Selected | +|--------|-------------|----------| +| Bottom tab bar | Persistent Calendar \| Lists tabs | ✓ | +| Hamburger drawer | Slide-out menu | | +| Top segmented control | Calendar/Lists toggle at top | | +| Add a router (real URLs) | react-router; deep-linkable lists | ✓ | +| Zustand view toggle (no router) | UI-state flag, no URLs | | + +**User's choice:** Bottom tab bar + react-router (real URLs). +**Notes:** No router installed today (`App.tsx` renders `CalendarShell` directly). Real URLs justified by Phase 5 push deep-linking to specific lists. + +--- + +## Claude's Discretion + +- Fan-out mechanism (in-memory EventEmitter vs Redis pub/sub) — single Node process today; in-memory is YAGNI default; planner must address explicitly vs the N-member future. +- Position-rank column datatype, SSE auth/middleware wiring, React Query cache-key structure. + +## Deferred Ideas + +- Anonymous list sharing via unique public URL (unauthenticated, bypasses Authelia) — own phase. +- Granular per-recipient sharing UI (member picker) — `list_shares` model supports it; relevant at 3+ members. +- List metadata (icons, per-list colors, max items) — not required; standard approaches fine.