familysync/docs/deployment.md

FamilySync — Deployment & Live-Verification Runbook

This document is the operator runbook for getting FamilySync running behind Authelia (OIDC) and Pangolin/Newt (public tunnel), and for executing the Phase 1 Gate 2 live-verification items (01-HUMAN-UAT.md). It covers two deployment modes:

• Mode A — Local test rig (recommended for Gate 2): familysync + a Newt connector run on your dev box, routed through your existing Pangolin under a test subdomain. Validates the real external topology (HTTPS, Pangolin SSE pass-through, Authelia OIDC) without deploying to Unraid and without touching the production stack.
• Mode B — Unraid production: the real household deployment. Identical app + config; only the host and the Newt site differ.

The behaviours Gate 2 is checking — Authelia OIDC redirect/session, and SSE survival through the tunnel — live in Authelia and Pangolin/Newt, not in where the origin container runs. So Mode A is a faithful test of both. Reserve Unraid (Mode B) for go-live.

---

Topology

flowchart LR
  subgraph Public
    User[Browser / iOS PWA]
    Pangolin[Pangolin edge<br/>public HTTPS + WAF]
  end
  subgraph Private[Private network - no inbound ports]
    Newt[Newt connector<br/>outbound tunnel]
    API[familysync api<br/>Hono :3000]
    DB[(MariaDB)]
    Redis[(Redis - Phase 4)]
  end
  Authelia[Authelia OIDC<br/>auth.DOMAIN]

  User -->|https://familysync.DOMAIN| Pangolin
  Pangolin -->|tunnel| Newt
  Newt --> API
  API --> DB
  API -.Phase 4.-> Redis
  User -->|OIDC redirect| Authelia
  API -->|token exchange / userinfo| Authelia

Key property: Newt dials outbound to Pangolin — there are no open inbound ports on the private network (honours the project networking constraint). This is true for both modes.

---

Prerequisites (both modes)

• A Pangolin instance you control, with a wildcard or per-host cert for *.DOMAIN.
• Authelia already deployed and reachable at https://auth.DOMAIN (project constraint).
• The familysync image builds: docker compose build (see repo docker-compose.yml).
• A Fastmail app password per member (scope "Mail, Contacts & Calendars") — see CAL-08-DECISION.md. Never commit it; it lives in a gitignored .env/.env.spike.

⚠️ Same-parent-domain requirement (Pitfall 1)

FamilySync must be served under the same parent domain as Authelia so the session cookie is same-site. e.g. Authelia at auth.DOMAIN and the app at familysync.DOMAIN (Mode B) or familysync-dev.DOMAIN (Mode A). A different apex domain will break the OIDC session cookie.

---

Step 1 — Register the OIDC client in Authelia

Authelia client registration is additive — adding a new client_id does not affect existing clients, and is trivially reversible. For Mode A use a distinct id + redirect so it never collides with the eventual production client.

Generate a hashed client secret:

authelia crypto hash generate pbkdf2 --variant sha512 --random --random.length 72
# Record BOTH the plaintext (for the app's OIDC_CLIENT_SECRET) and the hash (for Authelia).

Add to Authelia configuration.yml under identity_providers.oidc.clients:

identity_providers:
  oidc:
    clients:
      - client_id: 'familysync'            # Mode A: 'familysync-dev'
        client_name: 'FamilySync'
        client_secret: '$pbkdf2-sha512$...' # the HASH from the command above
        public: false
        authorization_policy: 'one_factor'
        redirect_uris:
          - 'https://familysync.DOMAIN/callback'      # Mode A: https://familysync-dev.DOMAIN/callback
        scopes: [openid, profile, email]
        response_types: [code]
        grant_types: [authorization_code, refresh_token]
        token_endpoint_auth_method: client_secret_basic
        require_pkce: true
        pkce_challenge_method: S256

Reload Authelia (docker restart authelia or its reload mechanism). These match the locked auth params in CLAUDE.md (code flow + PKCE S256 + client_secret_basic).

---

Step 2 — App environment (.env)

Copy .env.example → .env and fill in. Generate secrets as noted:

# Session cookie signing secret for @hono/oidc-auth
OIDC_AUTH_SECRET=$(openssl rand -base64 32)
# Broker app-password encryption key (32 bytes hex)
APP_PASSWORD_ENCRYPTION_KEY=$(node -e "console.log(require('crypto').randomBytes(32).toString('hex'))")

# Database
DB_HOST=mariadb
DB_PORT=3306
DB_USER=familysync
DB_PASSWORD=<strong>
DB_NAME=familysync
DB_ROOT_PASSWORD=<strong>

# OIDC (Authelia)
OIDC_AUTH_SECRET=<openssl rand -base64 32>
OIDC_ISSUER=https://auth.DOMAIN
OIDC_CLIENT_ID=familysync            # or familysync-dev (Mode A)
OIDC_CLIENT_SECRET=<plaintext secret matching the Authelia hash>
OIDC_REDIRECT_URI=https://familysync.DOMAIN/callback
# MANDATORY behind a tunnel — without it @hono/oidc-auth builds redirect_uri from the
# internal container hostname, which will not match the registered URI.
OIDC_AUTH_EXTERNAL_URL=https://familysync.DOMAIN

# Broker
APP_PASSWORD_ENCRYPTION_KEY=<64-hex>

---

Step 3 — Apply the database schema

The image does not auto-migrate. Bring up MariaDB and apply the committed migrations once:

WARNING — do NOT use the push subcommand of drizzle-kit on this MariaDB. The mysql dialect misreads MariaDB 11.x metadata and schedules a false truncate/recreate that wipes data. The push workflow has been removed from the project scripts for this reason. Always use the committed-migration path: db:generate to author a new migration (diffs schema.ts against committed JSON snapshots, never the live DB), db:migrate to apply it.

docker compose up -d mariadb
# from the repo root, host-side (dev override exposes 3306):
docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d mariadb
DB_HOST=127.0.0.1 DB_PORT=3306 DB_USER=familysync DB_NAME=familysync DB_PASSWORD=<value> \
  pnpm --filter @familysync/api exec drizzle-kit migrate
# verify: SHOW TABLES; -> users, member_credentials, calendars, calendar_events

To author a future schema change: edit apps/api/src/db/schema.ts, run pnpm --filter @familysync/api run db:generate (diffs schema against committed snapshots — no DB connection needed), commit the generated SQL, then apply with db:migrate.

---

Step 4 — Pangolin route + Newt connector

In Pangolin, create a resource/route for the hostname:

• Host: familysync.DOMAIN (Mode A: familysync-dev.DOMAIN)
• Upstream: the Newt connector → http://<api-host>:3000
• Auth: leave Pangolin's own auth off for this route — FamilySync does its own Authelia OIDC at the app layer. (Do not double-gate.)

⚠️ SSE idle timeout (Phase 4 dependency, issue #1034)

FamilySync uses Server-Sent Events for live list sync (Phase 4). Long-lived SSE streams can be killed by a proxy idle timeout. In the Pangolin route config, ensure response buffering is off and the idle/read timeout is >= 120s (ideally higher). The Gate 2 SSE smoke test below is what confirms this end to end — it must pass before Phase 4 is built.

Newt connector

• Mode A (local): run Newt on your dev box pointing at your Pangolin site token. It dials out; no local ports are exposed. familysync (api) listens on :3000 reachable by Newt.
• Mode B (Unraid): run the Newt container in the same Unraid stack (see Step 6).

# Newt connector (example — use the site token Pangolin issues for this site)
docker run -d --name newt --restart unless-stopped \
  -e PANGOLIN_ENDPOINT=https://pangolin.DOMAIN \
  -e NEWT_ID=<site-id> -e NEWT_SECRET=<site-secret> \
  fosrl/newt:latest

---

Step 5 — Bring up the app

docker compose up -d --build
curl -s http://localhost:3000/health   # local sanity: {"ok":true,"db":"up"}

Then proceed to Gate 2 verification through the public URL.

---

Step 6 — Unraid production (Mode B only)

1. Copy the repo (or just docker-compose.yml, apps/api/Dockerfile, built image) to Unraid.
2. Create the .env on the Unraid host (do not commit it; store via Unraid's secrets/template).
3. Add the newt service to the production compose (or run as a separate Unraid container) bound to the production Pangolin site.
4. Use a named Docker volume for mariadb_data on the array (not a throwaway volume).
5. docker compose up -d --build, then drizzle-kit migrate once (Step 3) against the prod DB.
6. Register the production Authelia client (client_id: familysync, prod redirect URI) if you used familysync-dev for Mode A.

Differences from Mode A are limited to: host, Newt site token, volume location, and the OIDC client id/redirect. The app code and docker-compose.yml are identical.

---

Gate 2 — Live verification checklist (01-HUMAN-UAT.md)

Run from an external network (phone on cellular is ideal for a true external path).

#	Item	Pass condition
1	AUTH-01 login	https://familysync.DOMAIN → redirects to Authelia → after login, shell shows name + color + one cached event
2	AUTH-02 session	Fully close + reopen browser → no re-login
3	AUTH-03 colors	Wife logs in on iPhone → distinct, stable color
4	iOS PWA (pairs with Phase 3)	Add-to-Home-Screen, launch standalone, login completes in standalone mode (watch for redirect breaking out of standalone)
5	SSE smoke (gate before Phase 4)	Hold the stream open 5+ min without it being cut:

# get the session cookie from the browser after logging in (DevTools → Application → Cookies)
curl -N -H "Cookie: oidc-auth=<value>" https://familysync.DOMAIN/api/sse/heartbeat
# expect a `heartbeat` event ~every 10s for 5+ minutes

• SSE PASS → SSE transport confirmed for Phase 4.
• SSE FAIL (stream cut early) → adjust Pangolin idle-timeout/buffering; if still failing, record as a Phase 4 constraint and plan a reconnect/fallback strategy.

Record results in .planning/phases/01-foundation-broker-spike/01-HUMAN-UAT.md.

---

Dev-auth bypass (Phase 2+ local development)

FamilySync builds Phase 2 and Phase 3 features behind a dev-auth bypass so live Authelia is not required during development (D-14). The bypass injects a fixed dev user into the request context and short-circuits the OIDC guard.

Activation (local dev only):

# In your local .env:
DEV_AUTH_BYPASS=true
NODE_ENV=development   # or test, or any value other than 'production'

Hard production guard:

The bypass middleware's FIRST conditional is process.env.NODE_ENV === 'production'. If this is true, the middleware returns a no-op passthrough regardless of any other env var. This means:

• Even if DEV_AUTH_BYPASS=true is accidentally present in a production container, it has zero effect. The OIDC guard fires normally.
• The hard guard is checked before DEV_AUTH_BYPASS is read — there is no code path where production + bypass = unauthenticated access.

Production Docker Compose prohibition:

The production docker-compose.yml MUST NOT include DEV_AUTH_BYPASS in the environment block. The .env.example entry for DEV_AUTH_BYPASS is commented out by default as a reminder.

What the bypass does:

Sets c.set('user', DEV_USER) in the Hono context before oidcAuthMiddleware runs. Routes that read c.get('user') receive a fixed dev user { id: 1, displayName: 'Dev User', color: '#4A90D9' }. Routes that call getAuth(c) from @hono/oidc-auth will still return null (no OIDC cookie is present) — those routes must be updated to prefer c.get('user') when building Phase 2+.

Running locally (host-side, no Docker)

Use this when you want to run the API and PWA directly on the host (no docker compose up for the app containers), e.g. during Phase 2+ feature development with the dev-auth bypass active.

Prerequisite: dev MariaDB must be running with the host port exposed.

Use the dev compose override from Step 3 — it binds MariaDB to localhost:3306:

docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d mariadb

Why a plain pnpm --filter @familysync/api dev is not enough:

The API dev script is node --watch dist/index.js. It does not auto-load .env — there is no dotenv call and no --env-file flag in the script. Without any env vars, db/client.ts defaults DB_HOST to 'localhost', which works for a host-side run. However, once you source the root .env to pick up OIDC secrets and other variables, the problem surfaces: root .env sets DB_HOST=mariadb (the Docker service name, only resolvable inside the Docker network). On the host, mariadb does not resolve, so the DB connection fails.

The fix is to source .env for all the other variables and then immediately override DB_HOST back to localhost.

Run the API and PWA:

Open two terminals from the repo root.

Terminal 1 — API:

# Build first (the dev script runs the compiled output, not ts-node)
pnpm --filter @familysync/api build

# Source root .env, then override DB_HOST and activate the bypass
set -a; source .env; set +a && DEV_AUTH_BYPASS=true DB_HOST=localhost pnpm --filter @familysync/api dev

Terminal 2 — PWA:

pnpm --filter @familysync/pwa dev

The set -a; source .env; set +a idiom exports every variable from the root .env into the shell environment. The DEV_AUTH_BYPASS=true DB_HOST=localhost prefix on the same command line then overrides those two specific vars for the pnpm child process — DB_HOST=localhost wins over the DB_HOST=mariadb that was exported from .env.

Why --env-file is not baked into the dev script:

If --env-file .env were added to the API dev script, it would load DB_HOST=mariadb automatically on every pnpm dev invocation. That value only works inside the Docker network; on the host it resolves to nothing and the DB connection fails. Keeping .env loading out of the script is intentional — the developer sources it manually and overrides DB_HOST as shown above.