docs: capture D-14/D-15 (Gate 2 deferral + local-Newt test rig) + deployment runbook
- docs/deployment.md: operator runbook — Mode A (local Newt test rig) vs Mode B (Unraid prod), Authelia client block, env reference, Pangolin SSE idle-timeout note, Gate 2 checklist. - ROADMAP: Phase 1 verification-status note (Gate 1 done, Gate 2 deferred); Phase 3 gains live AUTH + iOS standalone-PWA criterion; Phase 4 gains hard SSE-smoke entry gate (#1034). - PROJECT.md: D-14 (Gate 2 deferral split) + D-15 (local Newt rig); CAL-08 marked validated. - STATE: decisions/blockers updated to reflect deferral and dev-auth-bypass approach.
This commit is contained in:
Lucas Berger
@@ -0,0 +1,237 @@
|
||||
# FamilySync — Deployment & Live-Verification Runbook
|
||||
|
||||
This document is the operator runbook for getting FamilySync running behind Authelia (OIDC)
|
||||
and Pangolin/Newt (public tunnel), and for executing the **Phase 1 Gate 2** live-verification
|
||||
items (`01-HUMAN-UAT.md`). It covers two deployment modes:
|
||||
|
||||
- **Mode A — Local test rig** (recommended for Gate 2): familysync + a Newt connector run on
|
||||
your dev box, routed through your existing Pangolin under a *test* subdomain. Validates the
|
||||
real external topology (HTTPS, Pangolin SSE pass-through, Authelia OIDC) **without** deploying
|
||||
to Unraid and without touching the production stack.
|
||||
- **Mode B — Unraid production**: the real household deployment. Identical app + config; only
|
||||
the host and the Newt site differ.
|
||||
|
||||
> The behaviours Gate 2 is checking — Authelia OIDC redirect/session, and SSE survival through
|
||||
> the tunnel — live in **Authelia** and **Pangolin/Newt**, not in *where* the origin container
|
||||
> runs. So Mode A is a faithful test of both. Reserve Unraid (Mode B) for go-live.
|
||||
|
||||
---
|
||||
|
||||
## Topology
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph Public
|
||||
User[Browser / iOS PWA]
|
||||
Pangolin[Pangolin edge<br/>public HTTPS + WAF]
|
||||
end
|
||||
subgraph Private[Private network - no inbound ports]
|
||||
Newt[Newt connector<br/>outbound tunnel]
|
||||
API[familysync api<br/>Hono :3000]
|
||||
DB[(MariaDB)]
|
||||
Redis[(Redis - Phase 4)]
|
||||
end
|
||||
Authelia[Authelia OIDC<br/>auth.DOMAIN]
|
||||
|
||||
User -->|https://familysync.DOMAIN| Pangolin
|
||||
Pangolin -->|tunnel| Newt
|
||||
Newt --> API
|
||||
API --> DB
|
||||
API -.Phase 4.-> Redis
|
||||
User -->|OIDC redirect| Authelia
|
||||
API -->|token exchange / userinfo| Authelia
|
||||
```
|
||||
|
||||
Key property: **Newt dials outbound to Pangolin** — there are no open inbound ports on the
|
||||
private network (honours the project networking constraint). This is true for both modes.
|
||||
|
||||
---
|
||||
|
||||
## Prerequisites (both modes)
|
||||
|
||||
- A Pangolin instance you control, with a wildcard or per-host cert for `*.DOMAIN`.
|
||||
- Authelia already deployed and reachable at `https://auth.DOMAIN` (project constraint).
|
||||
- The familysync image builds: `docker compose build` (see repo `docker-compose.yml`).
|
||||
- A Fastmail app password per member (scope "Mail, Contacts & Calendars") — see
|
||||
`CAL-08-DECISION.md`. **Never commit it; it lives in a gitignored `.env`/`.env.spike`.**
|
||||
|
||||
### ⚠️ Same-parent-domain requirement (Pitfall 1)
|
||||
|
||||
FamilySync **must** be served under the same parent domain as Authelia so the session cookie is
|
||||
same-site. e.g. Authelia at `auth.DOMAIN` and the app at `familysync.DOMAIN` (Mode B) or
|
||||
`familysync-dev.DOMAIN` (Mode A). A different apex domain will break the OIDC session cookie.
|
||||
|
||||
---
|
||||
|
||||
## Step 1 — Register the OIDC client in Authelia
|
||||
|
||||
Authelia client registration is **additive** — adding a new `client_id` does not affect existing
|
||||
clients, and is trivially reversible. For Mode A use a distinct id + redirect so it never collides
|
||||
with the eventual production client.
|
||||
|
||||
Generate a hashed client secret:
|
||||
|
||||
```bash
|
||||
authelia crypto hash generate pbkdf2 --variant sha512 --random --random.length 72
|
||||
# Record BOTH the plaintext (for the app's OIDC_CLIENT_SECRET) and the hash (for Authelia).
|
||||
```
|
||||
|
||||
Add to Authelia `configuration.yml` under `identity_providers.oidc.clients`:
|
||||
|
||||
```yaml
|
||||
identity_providers:
|
||||
oidc:
|
||||
clients:
|
||||
- client_id: 'familysync' # Mode A: 'familysync-dev'
|
||||
client_name: 'FamilySync'
|
||||
client_secret: '$pbkdf2-sha512$...' # the HASH from the command above
|
||||
public: false
|
||||
authorization_policy: 'one_factor'
|
||||
redirect_uris:
|
||||
- 'https://familysync.DOMAIN/callback' # Mode A: https://familysync-dev.DOMAIN/callback
|
||||
scopes: [openid, profile, email]
|
||||
response_types: [code]
|
||||
grant_types: [authorization_code, refresh_token]
|
||||
token_endpoint_auth_method: client_secret_basic
|
||||
require_pkce: true
|
||||
pkce_challenge_method: S256
|
||||
```
|
||||
|
||||
Reload Authelia (`docker restart authelia` or its reload mechanism). These match the locked
|
||||
auth params in `CLAUDE.md` (code flow + PKCE S256 + client_secret_basic).
|
||||
|
||||
---
|
||||
|
||||
## Step 2 — App environment (`.env`)
|
||||
|
||||
Copy `.env.example` → `.env` and fill in. Generate secrets as noted:
|
||||
|
||||
```bash
|
||||
# Session cookie signing secret for @hono/oidc-auth
|
||||
OIDC_AUTH_SECRET=$(openssl rand -base64 32)
|
||||
# Broker app-password encryption key (32 bytes hex)
|
||||
APP_PASSWORD_ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")
|
||||
```
|
||||
|
||||
```dotenv
|
||||
# Database
|
||||
DB_HOST=mariadb
|
||||
DB_PORT=3306
|
||||
DB_USER=familysync
|
||||
DB_PASSWORD=<strong>
|
||||
DB_NAME=familysync
|
||||
DB_ROOT_PASSWORD=<strong>
|
||||
|
||||
# OIDC (Authelia)
|
||||
OIDC_AUTH_SECRET=<openssl rand -base64 32>
|
||||
OIDC_ISSUER=https://auth.DOMAIN
|
||||
OIDC_CLIENT_ID=familysync # or familysync-dev (Mode A)
|
||||
OIDC_CLIENT_SECRET=<plaintext secret matching the Authelia hash>
|
||||
OIDC_REDIRECT_URI=https://familysync.DOMAIN/callback
|
||||
# MANDATORY behind a tunnel — without it @hono/oidc-auth builds redirect_uri from the
|
||||
# internal container hostname, which will not match the registered URI.
|
||||
OIDC_AUTH_EXTERNAL_URL=https://familysync.DOMAIN
|
||||
|
||||
# Broker
|
||||
APP_PASSWORD_ENCRYPTION_KEY=<64-hex>
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 3 — Apply the database schema
|
||||
|
||||
The image does not auto-migrate. Bring up MariaDB and push the Drizzle schema once:
|
||||
|
||||
```bash
|
||||
docker compose up -d mariadb
|
||||
# from the repo root, host-side (dev override exposes 3306):
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d mariadb
|
||||
DB_HOST=127.0.0.1 DB_PORT=3306 DB_USER=familysync DB_NAME=familysync DB_PASSWORD=<value> \
|
||||
pnpm --filter @familysync/api exec drizzle-kit push
|
||||
# verify: SHOW TABLES; -> users, member_credentials, calendars, calendar_events
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 4 — Pangolin route + Newt connector
|
||||
|
||||
In Pangolin, create a **resource/route** for the hostname:
|
||||
|
||||
- Host: `familysync.DOMAIN` (Mode A: `familysync-dev.DOMAIN`)
|
||||
- Upstream: the Newt connector → `http://<api-host>:3000`
|
||||
- Auth: leave Pangolin's own auth **off** for this route — FamilySync does its own Authelia OIDC
|
||||
at the app layer. (Do not double-gate.)
|
||||
|
||||
### ⚠️ SSE idle timeout (Phase 4 dependency, issue #1034)
|
||||
|
||||
FamilySync uses Server-Sent Events for live list sync (Phase 4). Long-lived SSE streams can be
|
||||
killed by a proxy idle timeout. In the Pangolin route config, ensure response buffering is
|
||||
**off** and the idle/read timeout is **>= 120s** (ideally higher). The Gate 2 SSE smoke test
|
||||
below is what confirms this end to end — **it must pass before Phase 4 is built.**
|
||||
|
||||
### Newt connector
|
||||
|
||||
- **Mode A (local):** run Newt on your dev box pointing at your Pangolin site token. It dials out;
|
||||
no local ports are exposed. `familysync` (api) listens on `:3000` reachable by Newt.
|
||||
- **Mode B (Unraid):** run the Newt container in the same Unraid stack (see Step 6).
|
||||
|
||||
```bash
|
||||
# Newt connector (example — use the site token Pangolin issues for this site)
|
||||
docker run -d --name newt --restart unless-stopped \
|
||||
-e PANGOLIN_ENDPOINT=https://pangolin.DOMAIN \
|
||||
-e NEWT_ID=<site-id> -e NEWT_SECRET=<site-secret> \
|
||||
fosrl/newt:latest
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Step 5 — Bring up the app
|
||||
|
||||
```bash
|
||||
docker compose up -d --build
|
||||
curl -s http://localhost:3000/health # local sanity: {"ok":true,"db":"up"}
|
||||
```
|
||||
|
||||
Then proceed to Gate 2 verification through the public URL.
|
||||
|
||||
---
|
||||
|
||||
## Step 6 — Unraid production (Mode B only)
|
||||
|
||||
1. Copy the repo (or just `docker-compose.yml`, `apps/api/Dockerfile`, built image) to Unraid.
|
||||
2. Create the `.env` on the Unraid host (do **not** commit it; store via Unraid's secrets/template).
|
||||
3. Add the `newt` service to the production compose (or run as a separate Unraid container) bound
|
||||
to the production Pangolin site.
|
||||
4. Use a named Docker volume for `mariadb_data` on the array (not a throwaway volume).
|
||||
5. `docker compose up -d --build`, then `drizzle-kit push` once (Step 3) against the prod DB.
|
||||
6. Register the **production** Authelia client (`client_id: familysync`, prod redirect URI) if you
|
||||
used `familysync-dev` for Mode A.
|
||||
|
||||
Differences from Mode A are limited to: host, Newt site token, volume location, and the OIDC
|
||||
client id/redirect. The app code and `docker-compose.yml` are identical.
|
||||
|
||||
---
|
||||
|
||||
## Gate 2 — Live verification checklist (`01-HUMAN-UAT.md`)
|
||||
|
||||
Run from an **external** network (phone on cellular is ideal for a true external path).
|
||||
|
||||
| # | Item | Pass condition |
|
||||
|---|------|----------------|
|
||||
| 1 | **AUTH-01** login | `https://familysync.DOMAIN` → redirects to Authelia → after login, shell shows name + color + one cached event |
|
||||
| 2 | **AUTH-02** session | Fully close + reopen browser → no re-login |
|
||||
| 3 | **AUTH-03** colors | Wife logs in on iPhone → distinct, stable color |
|
||||
| 4 | **iOS PWA** (pairs with Phase 3) | Add-to-Home-Screen, launch standalone, login completes in standalone mode (watch for redirect breaking out of standalone) |
|
||||
| 5 | **SSE smoke** (gate before Phase 4) | Hold the stream open 5+ min without it being cut: |
|
||||
|
||||
```bash
|
||||
# get the session cookie from the browser after logging in (DevTools → Application → Cookies)
|
||||
curl -N -H "Cookie: oidc-auth=<value>" https://familysync.DOMAIN/api/sse/heartbeat
|
||||
# expect a `heartbeat` event ~every 10s for 5+ minutes
|
||||
```
|
||||
|
||||
- **SSE PASS** → SSE transport confirmed for Phase 4.
|
||||
- **SSE FAIL** (stream cut early) → adjust Pangolin idle-timeout/buffering; if still failing, record
|
||||
as a Phase 4 constraint and plan a reconnect/fallback strategy.
|
||||
|
||||
Record results in `.planning/phases/01-foundation-broker-spike/01-HUMAN-UAT.md`.
|
||||
Reference in New Issue
Block a user