Lucas Berger
f39bd308b2
- Update apps/api/test/setup.ts header: clarify tests run against familysync_test
(provisioned by global-setup.ts), document users-cleanup decision (intact across
tests), and note CI-vs-local env difference
- Add apps/api/README.md "Running API tests locally" section: documents the test
DB isolation, run command, DB_ROOT_PASSWORD requirement, and CI no-op behaviour
- Fix apps/api/test/global-setup.ts: switch from drizzle({ client, mode }) to
drizzle(pool, { mode }) — drizzle-orm@0.45.2 isConfig() has a tautological OR
in the `mode` branch that always returns false, causing the combined-config form
to pass the config object as the client (client.query is not a function); two-arg
form routes correctly; 244/244 tests pass against familysync_test
173 lines
8.6 KiB
Markdown
173 lines
8.6 KiB
Markdown
<!-- generated-by: gsd-doc-writer -->
|
|
|
|
# @familysync/api
|
|
|
|
The Hono backend for FamilySync. Acts as a calendar broker over Fastmail CalDAV, stores collaborative lists in MariaDB, enforces OIDC auth via Authelia, and delivers live list updates via SSE and push notifications via VAPID.
|
|
|
|
Part of the [FamilySync monorepo](../../README.md).
|
|
|
|
## What it does
|
|
|
|
- **Calendar broker** — polls Fastmail CalDAV every 5 minutes via `tsdav`; parses iCalendar payloads with `ical.js` and expands recurrence rules with `ical.js`'s `ICAL.RecurExpansion`; writes changes back to Fastmail through an outbox worker
|
|
- **Collaborative lists** — creates, reorders (fractional indexing), and syncs grocery/gift lists in MariaDB via Drizzle ORM
|
|
- **OIDC auth** — all `/api/*` routes protected by `@hono/oidc-auth` with authorization-code + PKCE flow against Authelia; `DEV_AUTH_BYPASS=true` skips OIDC for local development
|
|
- **Live sync** — Server-Sent Events stream list mutations to connected PWA clients in real time
|
|
- **Push notifications** — web-push (VAPID) delivers reminders for shared timed events to subscribed browsers
|
|
|
|
## Source layout
|
|
|
|
```text
|
|
src/
|
|
index.ts Hono app entrypoint; server startup; background worker initialization
|
|
routes/
|
|
events.ts CalDAV event CRUD endpoints
|
|
lists.ts List and list-item CRUD endpoints
|
|
me.ts Authenticated user profile endpoint
|
|
push.ts Push subscription registration
|
|
sse.ts SSE stream for live list updates
|
|
health.ts Unauthenticated health check
|
|
db/
|
|
schema.ts Drizzle table definitions (MariaDB/mysql2)
|
|
client.ts Drizzle client singleton
|
|
migrations/ SQL migrations generated by drizzle-kit
|
|
auth/
|
|
middleware.ts oidcAuthMiddleware + processOAuthCallback
|
|
devBypass.ts DEV_AUTH_BYPASS passthrough (non-production only)
|
|
persistSessionCookie.ts Re-issues session cookie as persistent for PWA
|
|
user.ts User upsert on first login
|
|
broker/
|
|
poller.ts 5-minute setInterval CalDAV ctag change-detection
|
|
outboxWorker.ts 15-second drain of pending CalDAV writes to Fastmail
|
|
reminderScheduler.ts 1-minute scan for upcoming shared events → push
|
|
client.ts tsdav client factory
|
|
sync.ts REPORT → ical.js → DB upsert logic
|
|
write.ts CalDAV PUT/DELETE helpers
|
|
expand.ts recurrence expansion via ICAL.RecurExpansion
|
|
vevent.ts VEVENT ↔ DB row mapping
|
|
crypto.ts AES-256-GCM encrypt/decrypt for stored app passwords
|
|
lib/
|
|
listEmitter.ts In-process EventEmitter for SSE fan-out
|
|
listChangeDispatcher.ts Publishes list mutations to listEmitter
|
|
eventChangeDispatcher.ts Publishes calendar mutations
|
|
pushDispatcher.ts Dispatches VAPID push payloads
|
|
pushCoalescer.ts Debounces push for rapid successive edits
|
|
listAccess.ts List permission helpers
|
|
rank.ts Fractional indexing helpers
|
|
```
|
|
|
|
## Running in the workspace
|
|
|
|
All commands below run from the monorepo root via the `--filter` flag, or from `apps/api/` directly.
|
|
|
|
### Prerequisites
|
|
|
|
- Node.js 22 LTS
|
|
- `pnpm` (see root `package.json` for version)
|
|
- MariaDB reachable at the coordinates in your `.env`
|
|
- Authelia OIDC provider (or use `DEV_AUTH_BYPASS=true` for local development)
|
|
|
|
### Development
|
|
|
|
`dev` runs the compiled `dist/` with `node --watch`. You must build first — `tsc` output in `dist/` is the source of truth at runtime.
|
|
|
|
```bash
|
|
# From monorepo root:
|
|
pnpm --filter @familysync/api build # compile TypeScript → dist/
|
|
pnpm --filter @familysync/api dev # node --watch dist/index.js
|
|
|
|
# Or from apps/api/:
|
|
pnpm build
|
|
pnpm dev
|
|
```
|
|
|
|
Rebuild after any source change; `node --watch` reloads on `dist/` file changes but does not invoke `tsc` itself.
|
|
|
|
### Production
|
|
|
|
```bash
|
|
pnpm --filter @familysync/api build
|
|
pnpm --filter @familysync/api start # node dist/index.js
|
|
```
|
|
|
|
The server listens on port `3000`.
|
|
|
|
## Database migrations
|
|
|
|
Never use `drizzle-kit push` against a populated MariaDB instance — it emits false destructive diffs and will truncate data.
|
|
|
|
```bash
|
|
# 1. Generate SQL migration files from schema changes:
|
|
pnpm --filter @familysync/api db:generate
|
|
|
|
# 2. Apply pending migrations:
|
|
pnpm --filter @familysync/api db:migrate
|
|
```
|
|
|
|
Migration files are written to `src/db/migrations/` and checked into source control.
|
|
|
|
## Environment variables
|
|
|
|
| Variable | Required | Description |
|
|
| --------------------------- | ------------------- | ---------------------------------------------------------------------- |
|
|
| `DB_HOST` | Yes | MariaDB host |
|
|
| `DB_USER` | Yes | MariaDB user |
|
|
| `DB_PASSWORD` | Yes | MariaDB password |
|
|
| `DB_NAME` | Yes | MariaDB database name |
|
|
| `DB_PORT` | No (default `3306`) | MariaDB port |
|
|
| `OIDC_ISSUER` | Yes (production) | Authelia issuer URL |
|
|
| `OIDC_CLIENT_ID` | Yes (production) | OIDC client ID |
|
|
| `OIDC_CLIENT_SECRET` | Yes (production) | OIDC client secret |
|
|
| `OIDC_AUTH_EXTERNAL_URL` | Yes (production) | External-facing URL for redirect_uri behind Pangolin tunnel |
|
|
| `VAPID_SUBJECT` | Yes (push) | `mailto:` or `https:` operator identifier |
|
|
| `VAPID_PUBLIC_KEY` | Yes (push) | VAPID public key |
|
|
| `VAPID_PRIVATE_KEY` | Yes (push) | VAPID private key |
|
|
| `CREDENTIAL_ENCRYPTION_KEY` | Yes | AES-256-GCM key for stored Fastmail app passwords |
|
|
| `DEV_AUTH_BYPASS` | No | Set to `true` (non-production only) to skip OIDC and inject a dev user |
|
|
| `NODE_ENV` | No | Set to `production` to enforce OIDC unconditionally |
|
|
|
|
See [../../docs/CONFIGURATION.md](../../docs/CONFIGURATION.md) for the full reference.
|
|
|
|
## Tests
|
|
|
|
Tests live in `tests/` (integration, route, broker unit) and `test/setup.ts` (global setup/teardown).
|
|
|
|
```bash
|
|
# Run full suite (sequential — shared MariaDB requires serial file execution):
|
|
pnpm --filter @familysync/api test
|
|
|
|
# Watch mode:
|
|
pnpm --filter @familysync/api test:watch
|
|
|
|
# Type-check without emitting:
|
|
pnpm --filter @familysync/api typecheck
|
|
```
|
|
|
|
Integration tests that hit MariaDB require a running dev DB with `DB_HOST=127.0.0.1` and credentials from your `.env`. See [../../docs/TESTING.md](../../docs/TESTING.md) for the full setup.
|
|
|
|
## Running API tests locally
|
|
|
|
Local test runs use a dedicated `familysync_test` database so the dev `familysync` database is never mutated. `test/global-setup.ts` creates and migrates `familysync_test` automatically on the first run.
|
|
|
|
**Prerequisites:**
|
|
|
|
- Dev MariaDB running and port-bound (`127.0.0.1:3306`) — start with `docker compose -f docker-compose.yml -f docker-compose.dev.yml up -d mariadb`
|
|
- `.env` sourced in your shell (provides `DB_PASSWORD`, `DB_ROOT_PASSWORD`, and other credentials)
|
|
|
|
**Run command:**
|
|
|
|
```bash
|
|
set -a; source .env; set +a
|
|
DB_HOST=127.0.0.1 pnpm --filter @familysync/api test
|
|
```
|
|
|
|
`DB_ROOT_PASSWORD` must be set in `.env` for the one-time `CREATE DATABASE` / `GRANT` that provisions `familysync_test`. Subsequent runs skip the provisioning step if the database already exists (`CREATE DATABASE IF NOT EXISTS`).
|
|
|
|
**CI is unaffected.** `test/global-setup.ts` returns immediately when `CI` is set (the CI `api` job provisions its own `familysync` service DB and runs `db:migrate` before the test step). The `test.env` DB override in `vitest.config.ts` is also a no-op under CI.
|
|
|
|
## Further reading
|
|
|
|
- [Architecture](../../docs/ARCHITECTURE.md) — system overview and component diagram
|
|
- [API reference](../../docs/API.md) — endpoint table, request/response shapes, auth flow
|
|
- [Configuration](../../docs/CONFIGURATION.md) — all environment variables
|
|
- [Deployment](../../docs/deployment.md) — Docker Compose, Unraid setup, VAPID key generation
|