familysync/docs/deployment.md

FamilySync — Deployment & Live-Verification Runbook

This document is the operator runbook for getting FamilySync running behind Authelia (OIDC) and Pangolin/Newt (public tunnel), and for executing the Phase 1 Gate 2 live-verification items (01-HUMAN-UAT.md). It covers two deployment modes:

• Mode A — Local test rig (recommended for Gate 2): familysync + a Newt connector run on your dev box, routed through your existing Pangolin under a test subdomain. Validates the real external topology (HTTPS, Pangolin SSE pass-through, Authelia OIDC) without deploying to Unraid and without touching the production stack.
• Mode B — Unraid production: the real household deployment. Identical app + config; only the host and the Newt site differ.

The behaviours Gate 2 is checking — Authelia OIDC redirect/session, and SSE survival through the tunnel — live in Authelia and Pangolin/Newt, not in where the origin container runs. So Mode A is a faithful test of both. Reserve Unraid (Mode B) for go-live.

---

Topology

flowchart LR
  subgraph Public
    User[Browser / iOS PWA]
    Pangolin[Pangolin edge<br/>public HTTPS + WAF]
  end
  subgraph Private[Private network - no inbound ports]
    Newt[Newt connector<br/>outbound tunnel]
    API[familysync api<br/>Hono :3000]
    DB[(MariaDB)]
    Redis[(Redis - Phase 4)]
  end
  Authelia[Authelia OIDC<br/>auth.DOMAIN]

  User -->|https://familysync.DOMAIN| Pangolin
  Pangolin -->|tunnel| Newt
  Newt --> API
  API --> DB
  API -.Phase 4.-> Redis
  User -->|OIDC redirect| Authelia
  API -->|token exchange / userinfo| Authelia

Key property: Newt dials outbound to Pangolin — there are no open inbound ports on the private network (honours the project networking constraint). This is true for both modes.

---

Prerequisites (both modes)

• A Pangolin instance you control, with a wildcard or per-host cert for *.DOMAIN.
• Authelia already deployed and reachable at https://auth.DOMAIN (project constraint).
• The familysync image builds: docker compose build (see repo docker-compose.yml).
• A Fastmail app password per member (scope "Mail, Contacts & Calendars") — see CAL-08-DECISION.md. Never commit it; it lives in a gitignored .env/.env.spike.

⚠️ Same-parent-domain requirement (Pitfall 1)

FamilySync must be served under the same parent domain as Authelia so the session cookie is same-site. e.g. Authelia at auth.DOMAIN and the app at familysync.DOMAIN (Mode B) or familysync-dev.DOMAIN (Mode A). A different apex domain will break the OIDC session cookie.

---

Step 1 — Register the OIDC client in Authelia

Authelia client registration is additive — adding a new client_id does not affect existing clients, and is trivially reversible. For Mode A use a distinct id + redirect so it never collides with the eventual production client.

Generate a hashed client secret:

authelia crypto hash generate pbkdf2 --variant sha512 --random --random.length 72
# Record BOTH the plaintext (for the app's OIDC_CLIENT_SECRET) and the hash (for Authelia).

Add to Authelia configuration.yml under identity_providers.oidc.clients:

identity_providers:
  oidc:
    clients:
      - client_id: 'familysync'            # Mode A: 'familysync-dev'
        client_name: 'FamilySync'
        client_secret: '$pbkdf2-sha512$...' # the HASH from the command above
        public: false
        authorization_policy: 'one_factor'
        redirect_uris:
          - 'https://familysync.DOMAIN/callback'      # Mode A: https://familysync-dev.DOMAIN/callback
        scopes: [openid, profile, email]
        response_types: [code]
        grant_types: [authorization_code, refresh_token]
        token_endpoint_auth_method: client_secret_basic
        require_pkce: true
        pkce_challenge_method: S256

Reload Authelia (docker restart authelia or its reload mechanism). These match the locked auth params in CLAUDE.md (code flow + PKCE S256 + client_secret_basic).

---

Step 2 — App environment (.env)

Copy .env.example → .env and fill in. Generate secrets as noted:

# Session cookie signing secret for @hono/oidc-auth
OIDC_AUTH_SECRET=$(openssl rand -base64 32)
# Broker app-password encryption key (32 bytes hex)
APP_PASSWORD_ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")

# Database
DB_HOST=mariadb
DB_PORT=3306
DB_USER=familysync
DB_PASSWORD=<strong>
DB_NAME=familysync
DB_ROOT_PASSWORD=<strong>

# OIDC (Authelia)
OIDC_AUTH_SECRET=<openssl rand -base64 32>
OIDC_ISSUER=https://auth.DOMAIN
OIDC_CLIENT_ID=familysync            # or familysync-dev (Mode A)
OIDC_CLIENT_SECRET=<plaintext secret matching the Authelia hash>
OIDC_REDIRECT_URI=https://familysync.DOMAIN/callback
# MANDATORY behind a tunnel — without it @hono/oidc-auth builds redirect_uri from the
# internal container hostname, which will not match the registered URI.
OIDC_AUTH_EXTERNAL_URL=https://familysync.DOMAIN

# Broker
APP_PASSWORD_ENCRYPTION_KEY=<64-hex>

---

Step 3 — Apply the database schema

The image does not auto-migrate. Bring up MariaDB and apply the committed migrations once:

WARNING — do NOT use the push subcommand of drizzle-kit on this MariaDB. The mysql dialect misreads MariaDB 11.x metadata and schedules a false truncate/recreate that wipes data. The push workflow has been removed from the project scripts for this reason. Always use the committed-migration path: db:generate to author a new migration (diffs schema.ts against committed JSON snapshots, never the live DB), db:migrate to apply it.

docker compose up -d mariadb
# from the repo root, host-side (dev override exposes 3306):
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d mariadb
DB_HOST=127.0.0.1 DB_PORT=3306 DB_USER=familysync DB_NAME=familysync DB_PASSWORD=<value> \
  pnpm --filter @familysync/api exec drizzle-kit migrate
# verify: SHOW TABLES; -> users, member_credentials, calendars, calendar_events

To author a future schema change: edit apps/api/src/db/schema.ts, run pnpm --filter @familysync/api run db:generate (diffs schema against committed snapshots — no DB connection needed), commit the generated SQL, then apply with db:migrate.

---

Step 4 — Pangolin route + Newt connector

In Pangolin, create a resource/route for the hostname:

• Host: familysync.DOMAIN (Mode A: familysync-dev.DOMAIN)
• Upstream: the Newt connector → http://<api-host>:3000
• Auth: leave Pangolin's own auth off for this route — FamilySync does its own Authelia OIDC at the app layer. (Do not double-gate.)

⚠️ SSE idle timeout (Phase 4 dependency, issue #1034)

FamilySync uses Server-Sent Events for live list sync (Phase 4). Long-lived SSE streams can be killed by a proxy idle timeout. In the Pangolin route config, ensure response buffering is off and the idle/read timeout is >= 120s (ideally higher). The Gate 2 SSE smoke test below is what confirms this end to end — it must pass before Phase 4 is built.

Newt connector

• Mode A (local): run Newt on your dev box pointing at your Pangolin site token. It dials out; no local ports are exposed. familysync (api) listens on :3000 reachable by Newt.
• Mode B (Unraid): run the Newt container in the same Unraid stack (see Step 6).

# Newt connector (example — use the site token Pangolin issues for this site)
docker run -d --name newt --restart unless-stopped \
  -e PANGOLIN_ENDPOINT=https://pangolin.DOMAIN \
  -e NEWT_ID=<site-id> -e NEWT_SECRET=<site-secret> \
  fosrl/newt:latest

---

Step 5 — Bring up the app

docker compose up -d --build
curl -s http://localhost:3000/health   # local sanity: {"ok":true,"db":"up"}

Then proceed to Gate 2 verification through the public URL.

---

Step 6 — Unraid production (Mode B only)

1. Copy the repo (or just docker-compose.yml, apps/api/Dockerfile, built image) to Unraid.
2. Create the .env on the Unraid host (do not commit it; store via Unraid's secrets/template).
3. Add the newt service to the production compose (or run as a separate Unraid container) bound to the production Pangolin site.
4. Use a named Docker volume for mariadb_data on the array (not a throwaway volume).
5. docker compose up -d --build, then drizzle-kit migrate once (Step 3) against the prod DB.
6. Register the production Authelia client (client_id: familysync, prod redirect URI) if you used familysync-dev for Mode A.

Differences from Mode A are limited to: host, Newt site token, volume location, and the OIDC client id/redirect. The app code and docker-compose.yml are identical.

---

Gate 2 — Live verification checklist (01-HUMAN-UAT.md)

Run from an external network (phone on cellular is ideal for a true external path).

#	Item	Pass condition
1	AUTH-01 login	https://familysync.DOMAIN → redirects to Authelia → after login, shell shows name + color + one cached event
2	AUTH-02 session	Fully close + reopen browser → no re-login
3	AUTH-03 colors	Wife logs in on iPhone → distinct, stable color
4	iOS PWA (pairs with Phase 3)	Add-to-Home-Screen, launch standalone, login completes in standalone mode (watch for redirect breaking out of standalone)
5	SSE smoke (gate before Phase 4)	Hold the stream open 5+ min without it being cut:

# get the session cookie from the browser after logging in (DevTools → Application → Cookies)
curl -N -H "Cookie: oidc-auth=<value>" https://familysync.DOMAIN/api/sse/heartbeat
# expect a `heartbeat` event ~every 10s for 5+ minutes

• SSE PASS → SSE transport confirmed for Phase 4.
• SSE FAIL (stream cut early) → adjust Pangolin idle-timeout/buffering; if still failing, record as a Phase 4 constraint and plan a reconnect/fallback strategy.

Record results in .planning/phases/01-foundation-broker-spike/01-HUMAN-UAT.md.

---

Dev-auth bypass (Phase 2+ local development)

FamilySync builds Phase 2 and Phase 3 features behind a dev-auth bypass so live Authelia is not required during development (D-14). The bypass injects a fixed dev user into the request context and short-circuits the OIDC guard.

Activation (local dev only):

# In your local .env:
DEV_AUTH_BYPASS=true
NODE_ENV=development   # or test, or any value other than 'production'

Hard production guard:

The bypass middleware's FIRST conditional is process.env.NODE_ENV === 'production'. If this is true, the middleware returns a no-op passthrough regardless of any other env var. This means:

• Even if DEV_AUTH_BYPASS=true is accidentally present in a production container, it has zero effect. The OIDC guard fires normally.
• The hard guard is checked before DEV_AUTH_BYPASS is read — there is no code path where production + bypass = unauthenticated access.

Production Docker Compose prohibition:

The production docker-compose.yml MUST NOT include DEV_AUTH_BYPASS in the environment block. The .env.example entry for DEV_AUTH_BYPASS is commented out by default as a reminder.

What the bypass does:

Sets c.set('user', DEV_USER) in the Hono context before oidcAuthMiddleware runs. Routes that read c.get('user') receive a fixed dev user { id: 1, displayName: 'Dev User', color: '#4A90D9' }. Routes that call getAuth(c) from @hono/oidc-auth will still return null (no OIDC cookie is present) — those routes must be updated to prefer c.get('user') when building Phase 2+.