2026-06-10 17:39:19 -04:00
2 changed files with 253 additions and 0 deletions
@@ -0,0 +1,131 @@
+# Phase 4: Shared Lists + Live Sync - Context
+
+**Gathered:** 2026-06-07
+**Status:** Ready for planning
+
+<domain>
+## Phase Boundary
+
+Deliver **app-native shared lists** (stored in MariaDB, NOT CalDAV/Fastmail) with real-time co-edit sync:
+
+- Create and delete named lists (LIST-01)
+- Add, check off, and delete items (LIST-02)
+- Reorder items by drag-and-drop (LIST-03)
+- Live co-edit sync over SSE — one member's change appears for the other within seconds, surviving a brief reconnect (LIST-04, success criterion 3)
+
+Lists are entirely app-owned data — no CalDAV write-back, no Fastmail involvement. This is the one track independent of the calendar write path.
+
+**⚠️ ENTRY GATE (D-14, issue #1034 — STILL UNVERIFIED as of 2026-06-07):** The 5-minute SSE-over-Pangolin smoke test must PASS before the live-sync layer is built (`/api/sse/heartbeat` held open 5+ min through the tunnel without being cut — see `docs/deployment.md` Gate 2 row 5). This is an operator/infra task requiring the Authelia+Pangolin/Newt rig. If it FAILS: fix Pangolin idle-timeout/buffering, OR the polling fallback (decided below) becomes mandatory rather than optional. Do not build live sync on an unverified transport.
+
+</domain>
+
+<decisions>
+## Implementation Decisions
+
+### Sharing model (List & item behavior)
+- **D-01:** Lists support **shared and private** visibility. New lists **default to Shared** (visible+editable by both members); creator can toggle a single list to Private. Default-shared chosen deliberately — the grocery/family-hub use case is collaborative and default-private would add friction to the primary action.
+- **D-02:** Data model is a **`list_shares` join table** (list has an `owner`; join table records who each list is shared with) — NOT a simple boolean. v1 UI is only shared/private, but the schema must be **member-count-agnostic** so granular N-recipient sharing is a future UI addition, not a migration.
+- **D-03:** A private list still **live-syncs across its owner's own devices** (phone + tablet); it is never pushed to other members.
+- **D-04:** **SSE fan-out MUST be scoped to who can see a list.** A list's change events broadcast only to members with access (owner + shares), never to all connected clients. This is the load-bearing consequence of the sharing model — get it right or private lists leak.
+
+### Item behavior
+- **D-05:** Checked-off items **sink to a "completed" section** at the bottom (active items stay on top). Not strikethrough-in-place, not immediate-disappear — keeps the active list clean for groceries while preserving "what was done."
+- **D-06:** **Confirm-on-delete for whole lists only.** Individual items delete instantly (live sync makes mistakes visible; easy to re-add). Reuse Phase 3's `DeleteConfirmationDialog` component for the list-delete dialog.
+
+### Live feel & conflict resolution
+- **D-07:** **Optimistic UI** — the editing member's change shows instantly, then reconciles against the server (rollback on rejection). Fits the low-friction constraint. Use React Query optimistic updates.
+- **D-08:** **Per-field writes + per-field last-write-wins** ("field-level merge", BOUNDED — no CRDT). The API PATCHes only the changed field (`checked`, `text`, or `position`), not the whole row; the server applies last-write-wins per field on a server timestamp. Result: "one toggles checked while the other edits text" → both stick. Same-field collisions fall back to last-write-wins. Do NOT build CRDTs or per-field vector clocks.
+- **D-09:** **Delete-wins** — if one member deletes an item while the other edits it, deletion is final; the in-flight edit is dropped (editor sees it vanish via live sync). Edits never resurrect deleted items.
+
+### Reconnect & transport (success criterion 3)
+- **D-10:** **Full refetch on reconnect** — on SSE reconnect, React Query invalidates and refetches the affected list(s) fresh. No server-side event log / Last-Event-ID replay. Lists are tiny so refetch is cheap and guaranteed-correct.
+- **D-11:** **Silent auto-recover with capped backoff, then a visible indicator.** Reconnect silently with bounded (capped exponential) backoff; after backoff is exhausted, surface a visible "disconnected / updates paused" indicator and stop hammering. NOTE for planner: raw `EventSource` auto-reconnects forever with no backoff control — implementing bounded backoff + a give-up indicator requires wrapping `EventSource` in a manual reconnect loop or using a small SSE client lib.
+- **D-12:** **Polling fallback via React Query `refetchInterval`** if SSE is unavailable/flaky through Pangolin. Already have React Query; trivial to add. Guarantees criterion 3 even if the tunnel misbehaves. (Mandatory if the entry-gate smoke test fails.)
+
+### Reordering (LIST-03)
+- **D-13:** **String-based fractional rank** for item positions (e.g., the `fractional-indexing` approach) — NOT raw floats (precision exhausts fast on repeated mid-point inserts) and NOT integer-renumber (a single move rewrites many rows, noisy over SSE). A move rewrites only the moved item's rank — one-row write, plays well with live sync and concurrent reorders.
+- **D-14:** **Animate to new order** when a remote reorder arrives (smooth transition, matches the live-sync promise).
+- **D-15:** **Last-write-wins with brief settle** on concurrent reorder of the same item — both see their local drag instantly (optimistic), server resolves to the last write, both converge within ~1s. No drag-locking / drag-state broadcasting.
+
+### Navigation / app shell
+- **D-16:** **Bottom tab bar** (Calendar | Lists) — thumb-reachable, matches native iOS/Android, low-friction for the non-technical member. Currently `App.tsx` renders `CalendarShell` directly with no nav.
+- **D-17:** **Add react-router** for real URLs (e.g. `/lists/:id`). No router is installed today. Real URLs enable Phase 5 push deep-linking ("tap to open Groceries"), browser back button, and PWA shortcuts. Small dependency that pays off next phase.
+
+### Project-level principle (applies beyond this phase)
+- **D-18:** **Design for N family members, not hard-coded two.** Schema, auth/access checks, and SSE fan-out must be member-count-agnostic. Same philosophy as treating Fastmail as a generic provider — set the framework now for future expansion to more family members. The `list_shares` table (D-02) and scoped fan-out (D-04) are the first applications.
+
+### Claude's Discretion (deferred to research/planner)
+- **Fan-out mechanism:** in-memory EventEmitter vs Redis pub/sub. API runs as a **single Node process** today (no replicas), so in-memory is the YAGNI default; Redis is in docker-compose but `ioredis` is NOT installed. Planner must address this explicitly and justify the choice against D-18 (multi-process future).
+- Exact position-rank datatype/column, SSE auth/middleware wiring, and React Query cache-key structure.
+
+### Reviewed Todos
+- **Adopt drizzle generate+migrate workflow (retire `db:push` on MariaDB)** — directly relevant: Phase 4 adds new tables (`lists`, `list_items`, `list_shares`). `drizzle-kit push` is unsafe on populated MariaDB (emits false destructive diff — see memory). New tables MUST use `drizzle-kit generate` + `migrate`, not `push`. Folded as a hard constraint on this phase's schema work.
+
+</decisions>
+
+<canonical_refs>
+## Canonical References
+
+**Downstream agents MUST read these before planning or implementing.**
+
+### Entry gate & transport
+- `docs/deployment.md` §"SSE idle timeout (Phase 4 dependency, issue #1034)" and §"Gate 2 — Live verification checklist" row 5 — the SSE-over-Pangolin smoke-test procedure that is this phase's entry gate
+- `apps/api/src/routes/sse.ts` — existing `/api/sse/heartbeat` SSE pattern (Hono `streamSSE`, `stream.aborted` loop); the live-list SSE endpoint(s) build on this
+
+### Prior decisions & requirements
+- `.planning/ROADMAP.md` §"Phase 4: Shared Lists + Live Sync" — goal, success criteria, entry gate
+- `.planning/REQUIREMENTS.md` — LIST-01 through LIST-04
+- `.planning/STATE.md` §Decisions — D-14 (SSE-over-WebSocket choice, entry gate), real-time transport notes
+- `.planning/phases/01-foundation-broker-spike/01-CONTEXT.md` §D-08 — why the SSE smoke test was folded into Phase 1 to de-risk Phase 4 transport
+
+### Schema & code patterns
+- `apps/api/src/db/schema.ts` — Drizzle table conventions (mysqlTable, indexes, unique keys, `references`/`onDelete`); model new list tables on these
+- `apps/pwa/src/components/DeleteConfirmationDialog.tsx` — reuse for list-delete confirmation (D-06)
+- `apps/pwa/src/App.tsx` / `apps/pwa/src/components/CalendarShell.tsx` — current shell with no router; tab-bar + react-router (D-16/D-17) wrap this
+
+</canonical_refs>
+
+<code_context>
+## Existing Code Insights
+
+### Reusable Assets
+- `apps/api/src/routes/sse.ts` — working Hono `streamSSE` heartbeat; the live-list event stream extends this pattern (auth via existing `/api/*` middleware).
+- `apps/pwa/src/components/DeleteConfirmationDialog.tsx` — Phase 3 confirmation dialog, reuse for list delete.
+- React Query + Zustand already established (CLAUDE.md split: React Query = server state, Zustand = UI-only state). Optimistic updates (D-07) and polling fallback (D-12) use React Query; tab/route UI state is Zustand-adjacent.
+- CSS token layer + colorUtils from Phase 2 available for list theming.
+
+### Established Patterns
+- `/api/*` routes sit behind OIDC middleware (or dev-auth bypass) — list routes inherit this; identity resolved to `users.id` via oidc iss+sub (D-10 from prior phases).
+- Drizzle schema conventions in `schema.ts`: int autoincrement PKs, `references(() => x.id, { onDelete: 'cascade' })`, composite unique keys, named indexes.
+- Schema migrations: **generate+migrate, never `push`** on MariaDB (see Reviewed Todos).
+
+### Integration Points
+- New `/api/lists` (+ items + SSE) routes mount in `apps/api/src/index.ts` alongside `eventsRouter`, `sseRouter`.
+- New `lists` / `list_items` / `list_shares` tables in `apps/api/src/db/schema.ts`.
+- PWA gains a router + bottom tab bar in `App.tsx`; Lists surface is a sibling of `CalendarShell`.
+- SSE fan-out must integrate with the (TBD) in-memory-vs-Redis pub/sub decision; `redis` service exists in docker-compose, `ioredis` not yet a dependency.
+
+</code_context>
+
+<specifics>
+## Specific Ideas
+
+- "Sink to bottom" for checked items modeled on a clean active-list / completed-section split (grocery-list mental model).
+- Sharing UI vision (future): pick specific recipients from the user DB; v1 collapses this to shared/private but the `list_shares` model preserves the path.
+- Backoff-then-pause reconnect UX: "set backoff and then display an indicator to pause more updates" — i.e., don't retry forever silently; tell the user when data may be stale.
+
+</specifics>
+
+<deferred>
+## Deferred Ideas
+
+- **Anonymous list sharing via a unique public URL** (share a list with a non-member through a link) — NEW CAPABILITY, its own phase. Introduces unauthenticated access that bypasses the Authelia OIDC model (every `/api/*` route is currently authenticated), plus link-token generation, revocation, and abuse handling. Explicitly out of scope for Phase 4; revisit as a dedicated "external/guest sharing" phase.
+- **Granular per-recipient sharing UI** (a member picker) — the `list_shares` data model (D-02) supports it, but no picker UI in v1 (only two members; "shared" == shared with the other person). Becomes relevant once the household has 3+ members (D-18).
+- **List metadata** (icons, per-list colors, max items) — not raised as required; standard approaches fine unless a future UI phase wants them.
+
+</deferred>
+
+---
+
+*Phase: 4-Shared Lists + Live Sync*
+*Context gathered: 2026-06-07*
@@ -0,0 +1,122 @@
+# Phase 4: Shared Lists + Live Sync - Discussion Log
+
+> **Audit trail only.** Do not use as input to planning, research, or execution agents.
+> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered.
+
+**Date:** 2026-06-07
+**Phase:** 4-Shared Lists + Live Sync
+**Areas discussed:** List & item behavior, Live feel & conflicts, Reconnect catch-up, Reordering behavior, Lists navigation
+
+---
+
+## List & item behavior — Sharing
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| All lists shared | Every list visible+editable by both; no permission model | |
+| Shared + private | Lists can be private; owner/visibility model + permission checks | ✓ (refined below) |
+
+**User's choice:** Shared + private — initially "default private, share to specific userdb members, optionally anonymous via unique URL."
+**Notes:** Claude challenged three points: (1) anonymous URL = scope creep + unauthenticated security surface → deferred; (2) recipient picker = YAGNI for two members → boolean shared/private UI but `list_shares` join table underneath; (3) default-private fights the collaborative grocery use case → recommend default-shared. User accepted re-framing.
+
+## List & item behavior — Privacy (refined)
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Default Shared, toggle to Private | Family-hub default; flip individual list private; boolean model | ✓ |
+| Default Private, toggle to Shared | Owner-only default; explicit share step | |
+| All shared, no private | Drop private entirely | |
+
+**User's choice:** Default Shared, toggle to Private.
+**Notes:** Anonymous URL → Defer it. Data model → `list_shares` join table, with the explicit instruction: "remember this project is intended to expand to other family members in the future... similar to treating fastmail like a generic provider helps set that framework." Captured as project-level principle D-18.
+
+## List & item behavior — Checked items & delete guard
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Strikethrough in place | Item stays, struck-through | |
+| Sink to bottom | Checked move to completed section | ✓ |
+| Disappear immediately | Removed from view | |
+| Confirm list delete only | Dialog for lists; items delete instantly | ✓ |
+| Confirm both | Dialog for lists and items | |
+| No confirmation | Everything instant | |
+
+**User's choice:** Sink to bottom; confirm list-delete only.
+**Notes:** Private lists still live-sync across the owner's own devices (owner-only visibility, multi-device sync).
+
+---
+
+## Live feel & conflicts
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Optimistic (instant local, reconcile) | Snappy; rollback on failure | ✓ |
+| Server-confirmed | Wait for round-trip | |
+| Last-write-wins | Later write wins per row | |
+| Field-level merge | Merge non-conflicting fields | ✓ (bounded) |
+| Delete wins | Deletion final, edit dropped | ✓ |
+| Edit resurrects | Edit re-creates deleted item | |
+
+**User's choice:** Optimistic UI; field-level merge; delete wins.
+**Notes:** Claude bounded "field-level merge" to per-field PATCH + per-field last-write-wins (no CRDT) to prevent over-engineering. User context implied agreement (momentum).
+
+---
+
+## Reconnect catch-up
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Full refetch on reconnect | React Query invalidate+refetch | ✓ |
+| Last-Event-ID replay | Server replays missed events | |
+| Hybrid | Replay, refetch fallback | |
+| Silent + auto-recover | EventSource native reconnect, no UI | ✓ (refined) |
+| Subtle indicator when offline | Show reconnecting hint | |
+| Polling fallback (refetchInterval) | Periodic refetch if SSE drops | ✓ |
+| SSE only, fix the proxy | Commit to SSE, no fallback | |
+
+**User's choice:** Full refetch; silent auto-recover with capped backoff then a "pause updates" indicator; polling fallback.
+**Notes:** User specified "set back off and then display an indicator to pause more updates." Claude flagged that raw EventSource has no backoff control → needs a manual reconnect wrapper or SSE client lib.
+
+---
+
+## Reordering behavior
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Fractional rank | One-row write per move | ✓ (string-based) |
+| Integer position + renumber | Many-row writes per move | |
+| Animate to new order | Smooth remote reorder | ✓ |
+| Update on next interaction | No animation | |
+| Last-write-wins, brief settle | Optimistic, converge ~1s | ✓ |
+| Lock during drag | Broadcast drag state | |
+
+**User's choice:** Fractional rank; animate to new order; last-write-wins settle.
+**Notes:** Claude steered fractional rank to a string-based fractional index (e.g. `fractional-indexing`) rather than raw floats to avoid precision exhaustion on repeated mid-point inserts.
+
+---
+
+## Lists navigation
+
+| Option | Description | Selected |
+|--------|-------------|----------|
+| Bottom tab bar | Persistent Calendar \| Lists tabs | ✓ |
+| Hamburger drawer | Slide-out menu | |
+| Top segmented control | Calendar/Lists toggle at top | |
+| Add a router (real URLs) | react-router; deep-linkable lists | ✓ |
+| Zustand view toggle (no router) | UI-state flag, no URLs | |
+
+**User's choice:** Bottom tab bar + react-router (real URLs).
+**Notes:** No router installed today (`App.tsx` renders `CalendarShell` directly). Real URLs justified by Phase 5 push deep-linking to specific lists.
+
+---
+
+## Claude's Discretion
+
+- Fan-out mechanism (in-memory EventEmitter vs Redis pub/sub) — single Node process today; in-memory is YAGNI default; planner must address explicitly vs the N-member future.
+- Position-rank column datatype, SSE auth/middleware wiring, React Query cache-key structure.
+
+## Deferred Ideas
+
+- Anonymous list sharing via unique public URL (unauthenticated, bypasses Authelia) — own phase.
+- Granular per-recipient sharing UI (member picker) — `list_shares` model supports it; relevant at 3+ members.
+- List metadata (icons, per-list colors, max items) — not required; standard approaches fine.