Lucas BergerandClaude Opus 4.8
c7955a46b9
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
209 lines
20 KiB
Markdown
209 lines
20 KiB
Markdown
---
|
||
phase: 09-faster-write-back
|
||
plan: 01
|
||
type: tdd
|
||
wave: 1
|
||
depends_on: []
|
||
files_modified:
|
||
- apps/api/src/lib/outboxTrigger.ts
|
||
- apps/api/src/broker/outboxWorker.ts
|
||
- apps/api/tests/broker/outboxWorker.test.ts
|
||
autonomous: true
|
||
requirements: [CAL-15]
|
||
|
||
must_haves:
|
||
truths:
|
||
- "Calling signalOutboxDrain() while no drain is in flight invokes runOutboxDrain promptly (no 15s wait)"
|
||
- "A signal arriving during an in-flight drain triggers exactly one trailing re-drain (D-05)"
|
||
- "Concurrent scheduleOutboxDrain() calls dispatch each pending row exactly once — no duplicate CalDAV PUT (D-07)"
|
||
- "runOutboxDrain's body and the isDraining guard are behaviorally unchanged — its existing 15 tests still pass"
|
||
- "scheduleOutboxDrain catches drain errors so a listener error never escapes the wrapper (D-02)"
|
||
- "No debounce/coalesce window and no rejection of mid-drain signals: drainRequested is a plain boolean that collapses every mid-drain signal into exactly one trailing drain — never rate-limited or dropped (D-06)"
|
||
- "Verification is fully automated by the trigger-wiring Vitest tests; no operator-stopwatch human checkpoint is required and the CI-guarded test is the durable latency evidence (D-10)"
|
||
artifacts:
|
||
- path: "apps/api/src/lib/outboxTrigger.ts"
|
||
provides: "Zero-dependency in-process EventEmitter drain signal (signalOutboxDrain, onOutboxDrain)"
|
||
exports: ["signalOutboxDrain", "onOutboxDrain"]
|
||
contains: "node:events"
|
||
- path: "apps/api/src/broker/outboxWorker.ts"
|
||
provides: "scheduleOutboxDrain wrapper + drainRequested flag + initOutboxTrigger subscription"
|
||
exports: ["scheduleOutboxDrain", "initOutboxTrigger"]
|
||
contains: "drainRequested"
|
||
- path: "apps/api/tests/broker/outboxWorker.test.ts"
|
||
provides: "Trigger-wiring test block (SC-1, SC-4, D-05)"
|
||
contains: "scheduleOutboxDrain"
|
||
key_links:
|
||
- from: "apps/api/src/broker/outboxWorker.ts (initOutboxTrigger)"
|
||
to: "apps/api/src/lib/outboxTrigger.ts (onOutboxDrain)"
|
||
via: "onOutboxDrain(() => scheduleOutboxDrain())"
|
||
pattern: "onOutboxDrain\\("
|
||
- from: "apps/api/src/broker/outboxWorker.ts (scheduleOutboxDrain)"
|
||
to: "apps/api/src/broker/outboxWorker.ts (runOutboxDrain)"
|
||
via: "guarded call through isDraining + drainRequested"
|
||
pattern: "scheduleOutboxDrain"
|
||
---
|
||
|
||
<objective>
|
||
Build the event-driven drain signal mechanism: a zero-dependency in-process `EventEmitter` (`outboxTrigger.ts`) and the `scheduleOutboxDrain` scheduler wrapper in `outboxWorker.ts` that funnels signals through the existing `isDraining`-guarded `runOutboxDrain()` with a `drainRequested` trailing-re-drain loop (D-05). Prove SC-1, SC-4, and D-05 with automated trigger-wiring tests (D-09).
|
||
|
||
Purpose: Removes the up-to-15s queueing delay before the CalDAV round-trip by signalling the drain on enqueue, while preserving every outbox durability guarantee — `runOutboxDrain`'s internals and the `isDraining` guard stay behaviorally unchanged (CAL-15 / D-02 / D-07).
|
||
Output: `outboxTrigger.ts` (new), the `scheduleOutboxDrain` + `drainRequested` + `initOutboxTrigger` additions to `outboxWorker.ts`, and a new trigger-wiring describe block in `outboxWorker.test.ts`.
|
||
</objective>
|
||
|
||
<execution_context>
|
||
@$HOME/.claude/gsd-core/workflows/execute-plan.md
|
||
@$HOME/.claude/gsd-core/templates/summary.md
|
||
</execution_context>
|
||
|
||
<context>
|
||
@.planning/PROJECT.md
|
||
@.planning/ROADMAP.md
|
||
@.planning/STATE.md
|
||
@.planning/phases/09-faster-write-back/09-CONTEXT.md
|
||
@.planning/phases/09-faster-write-back/09-RESEARCH.md
|
||
@.planning/phases/09-faster-write-back/09-PATTERNS.md
|
||
@.planning/phases/09-faster-write-back/09-VALIDATION.md
|
||
</context>
|
||
|
||
<artifacts_this_phase_produces>
|
||
New symbols this phase creates (none exist yet — do not expect them in API-SURFACE.md):
|
||
|
||
| Symbol | File | Created By | Kind |
|
||
|--------|------|-----------|------|
|
||
| `apps/api/src/lib/outboxTrigger.ts` | (new file) | Plan 01 Task 1 | module |
|
||
| `signalOutboxDrain` | `apps/api/src/lib/outboxTrigger.ts` | Plan 01 Task 1 | exported function |
|
||
| `onOutboxDrain` | `apps/api/src/lib/outboxTrigger.ts` | Plan 01 Task 1 | exported function |
|
||
| `drainRequested` | `apps/api/src/broker/outboxWorker.ts` | Plan 01 Task 3 | module-level flag |
|
||
| `scheduleOutboxDrain` | `apps/api/src/broker/outboxWorker.ts` | Plan 01 Task 3 | exported function |
|
||
| `initOutboxTrigger` | `apps/api/src/broker/outboxWorker.ts` | Plan 01 Task 3 | exported function |
|
||
|
||
Plan 02 consumes `signalOutboxDrain` (in `events.ts`) and `initOutboxTrigger` (in `index.ts`).
|
||
</artifacts_this_phase_produces>
|
||
|
||
<tasks>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 1: Create outboxTrigger.ts zero-dependency EventEmitter signal module</name>
|
||
<files>apps/api/src/lib/outboxTrigger.ts</files>
|
||
<read_first>
|
||
- apps/api/src/lib/outboxTrigger.ts (the file being created — confirm it does not yet exist)
|
||
- apps/api/src/lib/listEmitter.ts (canonical analog: module-level `const emitter = new EventEmitter()`, named publish/subscribe wrappers, zero internal imports)
|
||
</read_first>
|
||
<behavior>
|
||
- signalOutboxDrain() emits the 'drain' event on the module-level emitter (synchronous; fire-and-forget; returns void)
|
||
- onOutboxDrain(handler) registers handler on 'drain' and returns an unsubscribe function that calls emitter.off('drain', handler)
|
||
- A handler registered via onOutboxDrain is invoked exactly once per signalOutboxDrain() call
|
||
- After the returned unsubscribe is called, a subsequent signalOutboxDrain() does NOT invoke the handler
|
||
</behavior>
|
||
<action>
|
||
Create `apps/api/src/lib/outboxTrigger.ts` following the `listEmitter.ts` module-level singleton pattern exactly. Import `EventEmitter` from `node:events`. Declare `const emitter = new EventEmitter();` at module scope. Do NOT call `setMaxListeners` — there is exactly one subscriber (the `initOutboxTrigger` listener), so the default limit of 10 is correct (unlike `listEmitter.ts` which sets 200 for SSE fan-out per its T-04-04 comment). Export `signalOutboxDrain(): void` that calls `emitter.emit('drain')` (fire-and-forget per D-04). Export `onOutboxDrain(handler: () => void): () => void` that calls `emitter.on('drain', handler)` and returns `() => emitter.off('drain', handler)`. This module MUST import nothing from `broker/`, `routes/`, `db/`, or `index.ts` — only `node:events` — to guarantee no circular import (same zero-internal-dependency rule as `listEmitter.ts`). Add a header comment noting: single subscriber, default listener limit fine, signal is fire-and-forget, published only after enqueue commit (D-01/D-03/D-04).
|
||
</action>
|
||
<verify>
|
||
<automated>cd apps/api && npx tsc --noEmit 2>&1 | grep -v '^$' | grep -i 'outboxTrigger' ; test ${PIPESTATUS[1]} -ne 0 || echo "no tsc errors in outboxTrigger"; grep -q "export function signalOutboxDrain" src/lib/outboxTrigger.ts && grep -q "export function onOutboxDrain" src/lib/outboxTrigger.ts && grep -q "node:events" src/lib/outboxTrigger.ts && echo OK</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- `apps/api/src/lib/outboxTrigger.ts` exists and exports `signalOutboxDrain` and `onOutboxDrain`
|
||
- File imports `EventEmitter` from `node:events` and nothing from `broker/`, `routes/`, `db/`, or `index.ts` (`grep -v 'node:events' src/lib/outboxTrigger.ts | grep -E "from '\\.\\./(broker|routes|db)" ` returns nothing)
|
||
- No `setMaxListeners` call present
|
||
- `cd apps/api && npx tsc --noEmit` reports no errors referencing `outboxTrigger.ts`
|
||
</acceptance_criteria>
|
||
<done>outboxTrigger.ts exists, exports signalOutboxDrain + onOutboxDrain, depends only on node:events, typechecks clean.</done>
|
||
</task>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 2: RED — add failing trigger-wiring tests for SC-1, D-05, D-07</name>
|
||
<files>apps/api/tests/broker/outboxWorker.test.ts</files>
|
||
<read_first>
|
||
- apps/api/tests/broker/outboxWorker.test.ts (full file: the `vi.hoisted` + `vi.mock` scaffold, `mockPendingRows`, `makeRow`, `makeResponse`, `wireMockChain`, the `beforeEach` reset block, and the existing describe blocks — extend, do not rewrite)
|
||
- apps/api/src/lib/outboxTrigger.ts (created in Task 1 — import `signalOutboxDrain` from here)
|
||
- apps/api/src/broker/outboxWorker.ts (the module under test — `runOutboxDrain` already exported; `scheduleOutboxDrain` will be added in Task 3 and must be imported here in RED so the test fails on the missing export)
|
||
- apps/api/tests/lib/listEmitter.test.ts (analog for `vi.fn()` handler + unsubscribe test shape)
|
||
</read_first>
|
||
<behavior>
|
||
- Test A (SC-1): with one pending row and `createCalendarEvent` mocked to resolve 201, calling `signalOutboxDrain()` then flushing microtasks via `await new Promise(r => setImmediate(r))` results in `createCalendarEvent` called exactly once — with NO `vi.useFakeTimers()` / no 15s advance
|
||
- Test B (SC-4 / D-05): with `createCalendarEvent` first call held on a manually-resolved promise and two pending rows, calling `signalOutboxDrain()` (drain 1 starts), then `signalOutboxDrain()` twice more while drain 1 is in flight, then releasing drain 1 and flushing, results in exactly one trailing re-drain — `createCalendarEvent` total calls equal (rows in drain 1) + (rows in trailing drain), never a third drain pass
|
||
- Test C (SC-4 / D-07): with `createCalendarEvent` mocked to a `setTimeout(20ms)`-delayed resolve and one pending row, calling `scheduleOutboxDrain()` twice synchronously then `await vi.runAllTimersAsync()` results in `createCalendarEvent` called exactly once (second concurrent call no-ops via `isDraining`)
|
||
</behavior>
|
||
<action>
|
||
Extend `apps/api/tests/broker/outboxWorker.test.ts`. Update the top import to add `scheduleOutboxDrain` to the existing `import { runOutboxDrain, assembleRruleString } from '../../src/broker/outboxWorker.js';` line, and add `import { signalOutboxDrain } from '../../src/lib/outboxTrigger.js';`. Reuse the existing module-scope helpers (`mockPendingRows`, `makeRow`, `makeResponse`, `wireMockChain`) and the `createCalendarEvent` mock from `../../src/broker/write.js` — do NOT redeclare them. Add ONE new describe block after the last existing describe block: `describe('scheduleOutboxDrain — trigger wiring (D-09)', ...)`. Inside, replicate the existing `beforeEach` (`vi.resetAllMocks(); mockPendingRows = []; wireMockChain();`). Because `isDraining`/`drainRequested` are module-level flags that persist across tests in the same module instance, end each test that leaves a drain potentially in flight by awaiting two `setImmediate` flushes so the module flags settle before the next test. Write Test A, Test B, Test C per the `<behavior>` block. For Test A and Test B use `await new Promise<void>(resolve => setImmediate(resolve))` for microtask flushing (the signal→drain path is microtask-driven, no timers — per RESEARCH Q5); for Test B use `vi.mocked(createCalendarEvent).mockImplementationOnce(async () => { await firstDone; return makeResponse(201); }).mockResolvedValue(makeResponse(201))` with a manually-captured `resolveFirst`. For Test C use `vi.useFakeTimers()` + `vi.runAllTimersAsync()` and restore real timers in that test's cleanup. Assert exact call counts per `<behavior>`. This task is RED: it imports `scheduleOutboxDrain` which does not exist yet, so the file fails to resolve / the new tests fail. Do NOT implement `scheduleOutboxDrain` in this task. Do NOT touch the existing 15 tests.
|
||
</action>
|
||
<verify>
|
||
<automated>cd apps/api && npx vitest run tests/broker/outboxWorker.test.ts 2>&1 | tail -20; echo "EXPECT: RED — new 'trigger wiring' tests fail or module fails to resolve scheduleOutboxDrain"</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- The new `describe('scheduleOutboxDrain — trigger wiring (D-09)', ...)` block exists with three tests (Test A SC-1, Test B D-05, Test C D-07)
|
||
- The test file imports `scheduleOutboxDrain` from `../../src/broker/outboxWorker.js` and `signalOutboxDrain` from `../../src/lib/outboxTrigger.js`
|
||
- Running `npx vitest run tests/broker/outboxWorker.test.ts` shows the new trigger-wiring tests RED (fail) — confirming the test exercises behavior not yet implemented
|
||
- The 15 pre-existing tests are unmodified (no edits inside their describe blocks)
|
||
</acceptance_criteria>
|
||
<done>Three new trigger-wiring tests written and failing (RED); existing 15 tests untouched.</done>
|
||
</task>
|
||
|
||
<task type="auto" tdd="true">
|
||
<name>Task 3: GREEN — add scheduleOutboxDrain wrapper, drainRequested flag, initOutboxTrigger; switch setInterval to the wrapper</name>
|
||
<files>apps/api/src/broker/outboxWorker.ts</files>
|
||
<read_first>
|
||
- apps/api/src/broker/outboxWorker.ts (full file — confirm: `let isDraining = false;` at L159; `runOutboxDrain` guard `if (isDraining) return; isDraining = true;` at L602–605; `finally { isDraining = false; }` at L783–785; `startOutboxWorker` setInterval calling `runOutboxDrain().catch(...)` at L796–802; the file-header export-rationale comment at L19–22)
|
||
- apps/api/src/lib/outboxTrigger.ts (import `onOutboxDrain` from here for `initOutboxTrigger`)
|
||
- apps/api/tests/broker/outboxWorker.test.ts (the RED tests from Task 2 that this task turns GREEN)
|
||
</read_first>
|
||
<action>
|
||
Modify `apps/api/src/broker/outboxWorker.ts`. (1) Add `import { onOutboxDrain } from '../lib/outboxTrigger.js';` with the other imports. (2) Immediately after `let isDraining = false;` (L159) add `let drainRequested = false;`. (3) Add and EXPORT a new `scheduleOutboxDrain(): void` function (place it between the `isDraining`/`drainRequested` declarations and `runOutboxDrain`): if `isDraining` is true, set `drainRequested = true` and return; otherwise call `runOutboxDrain()` and chain `.catch((err: unknown) => console.error('[outboxWorker] Unhandled runOutboxDrain error:', err))` then `.finally(() => { if (drainRequested) { drainRequested = false; scheduleOutboxDrain(); } })`. The `drainRequested = false` reset MUST come BEFORE the recursive `scheduleOutboxDrain()` call (Pitfall 3 — resetting after would allow an infinite loop). (4) Do NOT modify `runOutboxDrain`'s body or its `isDraining` guard/`finally` in any way (preserves the existing 15 tests and D-02/D-07). (5) Add and EXPORT `initOutboxTrigger(): void` that calls `onOutboxDrain(() => scheduleOutboxDrain())` — placed alongside `startOutboxWorker` in the scheduler section. (6) Update `startOutboxWorker`'s `setInterval` body to call `scheduleOutboxDrain()` instead of `runOutboxDrain().catch(...)` — the per-call `.catch` is now absorbed into `scheduleOutboxDrain`, so the interval body becomes a bare `scheduleOutboxDrain();` call (keep the 15 * 1000 interval exactly per D-08). (7) Extend the file-header export-rationale comment (L19–22) with a line: `scheduleOutboxDrain is exported for unit testing; it wraps runOutboxDrain with the isDraining guard + drainRequested trailing-re-drain loop (D-05).` This task is GREEN: the Task 2 tests must now pass.
|
||
</action>
|
||
<verify>
|
||
<automated>cd apps/api && npx vitest run tests/broker/outboxWorker.test.ts 2>&1 | tail -8; npx tsc --noEmit 2>&1 | grep -E 'outboxWorker|outboxTrigger' || echo "tsc clean for outbox files"; grep -q "export function scheduleOutboxDrain" src/broker/outboxWorker.ts && grep -q "export function initOutboxTrigger" src/broker/outboxWorker.ts && grep -q "let drainRequested = false" src/broker/outboxWorker.ts && echo OK</automated>
|
||
</verify>
|
||
<acceptance_criteria>
|
||
- `outboxWorker.ts` exports `scheduleOutboxDrain` and `initOutboxTrigger`, and declares `let drainRequested = false`
|
||
- `runOutboxDrain`'s guard line `if (isDraining) return;`, `isDraining = true;`, and `finally { isDraining = false; }` are byte-for-byte unchanged from before this plan (the trailing re-drain lives only in `scheduleOutboxDrain`, never inside `runOutboxDrain`)
|
||
- In `scheduleOutboxDrain`'s `.finally`, `drainRequested = false` precedes the recursive `scheduleOutboxDrain()` call (Pitfall 3)
|
||
- `startOutboxWorker`'s `setInterval` body calls `scheduleOutboxDrain()` and keeps the `15 * 1000` interval (D-08)
|
||
- `initOutboxTrigger` calls `onOutboxDrain(() => scheduleOutboxDrain())`
|
||
- `npx vitest run tests/broker/outboxWorker.test.ts` is GREEN (all trigger-wiring tests pass AND the 15 pre-existing tests still pass)
|
||
- `cd apps/api && npx tsc --noEmit` reports no errors in `outboxWorker.ts` / `outboxTrigger.ts`
|
||
</acceptance_criteria>
|
||
<done>scheduleOutboxDrain + drainRequested + initOutboxTrigger added; setInterval routes through the wrapper; runOutboxDrain internals unchanged; full outboxWorker.test.ts suite GREEN; tsc clean.</done>
|
||
</task>
|
||
|
||
</tasks>
|
||
|
||
<threat_model>
|
||
## Trust Boundaries
|
||
|
||
| Boundary | Description |
|
||
|----------|-------------|
|
||
| route handler → in-process EventEmitter | A committed enqueue publishes a `'drain'` signal in-process. No network surface, no new auth boundary, no new user input crosses here — the signal carries no payload. |
|
||
| EventEmitter listener → runOutboxDrain | The single subscriber funnels the signal into the existing `isDraining`-guarded drain. Existing trust boundaries (CalDAV credential decryption, Fastmail I/O) are unchanged downstream. |
|
||
|
||
## STRIDE Threat Register
|
||
|
||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||
|-----------|----------|-----------|-------------|-----------------|
|
||
| T-09-01 | Denial of Service | `scheduleOutboxDrain` trailing-re-drain loop | mitigate | `drainRequested` is a boolean that collapses all mid-drain signals into exactly one trailing drain (D-05/D-06); it is reset BEFORE the recursive call (Pitfall 3), so a burst can never produce an unbounded re-drain chain against Fastmail rate limits. Test B (D-05) asserts exactly one trailing drain. |
|
||
| T-09-02 | Denial of Service | listener registration in test process | mitigate | `onOutboxDrain` is invoked only from `initOutboxTrigger`, which (in Plan 02) is called only under `isMainModule()` in `index.ts`. Tests import `runOutboxDrain`/`scheduleOutboxDrain` directly and never register a listener — no open-handle leak preventing process exit. Single subscriber → default 10-listener limit suffices, no `setMaxListeners`. |
|
||
| T-09-03 | Denial of Service | error in drain escaping the wrapper | mitigate | `scheduleOutboxDrain` wraps `runOutboxDrain()` in `.catch(...)` that logs and swallows, matching the existing setInterval error-swallowing pattern (D-02). A thrown drain error cannot crash the process or escape into the EventEmitter `emit` call frame. |
|
||
| T-09-04 | Tampering | signal-induced reentrancy bypassing isDraining | accept→mitigate | The signal NEVER calls `runOutboxDrain()` directly; it always goes through `scheduleOutboxDrain`, which checks `isDraining` before dispatch (D-02). Test C (D-07) asserts concurrent `scheduleOutboxDrain` calls dispatch each row exactly once. No new tampering vector — the guard is reused verbatim. |
|
||
|
||
No `npm`/`pip`/`cargo` install in this plan (zero-dependency, `node:events` only) — no T-09-SC supply-chain threat applies. RESEARCH Package Legitimacy Audit confirms no new external packages.
|
||
</threat_model>
|
||
|
||
<verification>
|
||
- `cd apps/api && npx vitest run tests/broker/outboxWorker.test.ts` — full broker suite GREEN (15 existing + 3 new trigger-wiring tests)
|
||
- `cd apps/api && npx tsc --noEmit` — no type errors (Vitest passes while tsc fails; typecheck is mandatory per [[vitest-passes-tsc-fails]])
|
||
- `grep -q "export function scheduleOutboxDrain" apps/api/src/broker/outboxWorker.ts` — wrapper exported
|
||
- `runOutboxDrain` guard/finally unchanged: diff the `runOutboxDrain` body region against the pre-plan version — no edits inside it
|
||
</verification>
|
||
|
||
<success_criteria>
|
||
- outboxTrigger.ts created (zero-dependency, node:events only, single subscriber, no setMaxListeners)
|
||
- scheduleOutboxDrain implements the isDraining-guarded drain with a drainRequested trailing-re-drain loop (D-05), errors caught (D-02), exactly-once preserved (D-07)
|
||
- runOutboxDrain internals + isDraining guard behaviorally unchanged (existing 15 tests still pass)
|
||
- 15s setInterval routes through scheduleOutboxDrain, interval length unchanged (D-08)
|
||
- Trigger-wiring tests for SC-1, D-05, D-07 pass (D-09)
|
||
- tsc --noEmit clean
|
||
</success_criteria>
|
||
|
||
<output>
|
||
Create `.planning/phases/09-faster-write-back/09-01-SUMMARY.md` when done.
|
||
</output>
|