diff --git a/.planning/phases/16-ci-dependency-audit-and-security-checks/16-CONTEXT.md b/.planning/phases/16-ci-dependency-audit-and-security-checks/16-CONTEXT.md new file mode 100644 index 0000000..4e303d2 --- /dev/null +++ b/.planning/phases/16-ci-dependency-audit-and-security-checks/16-CONTEXT.md @@ -0,0 +1,124 @@ +# Phase 16: CI Dependency Audit, Security Checks & Image Hygiene - Context + +**Gathered:** 2026-06-12 +**Status:** Ready for planning + + +## Phase Boundary + +Extend the **existing** Gitea CI with three families of gates — implemented as additions to the current `.gitea/workflows/ci.yml` (PR workflow: `changes → fast-checks / api / harness → gate`) and `.gitea/workflows/publish.yml` (push-to-main publish, builds `--target production`). **Not** a new pipeline, **not** a new runtime dependency, **not** a new external service. + +1. **Dependency audit** — `pnpm audit` against the lockfile (fail on High+Critical) + a `pnpm outdated` advisory report. +2. **Security checks baseline** — secret scanning (gitleaks) + a static security lint (eslint-plugin-security folded into the existing ESLint gate). +3. **Dev↔prod image hygiene (absorbs backlog 999.17)** — provably confine `DEV_AUTH_BYPASS` and any dev affordance/secret/seed data to local dev; the published production image must never carry them and must refuse to run with dev-bypass enabled. + +**Explicitly out of scope:** automated dependency *upgrade* bots (Renovate/Dependabot), Trivy/image CVE scanning, removing dev-bypass (still needed for local + Phase 7/8 harness). + + + +## Implementation Decisions + +### Security-Check Baseline +- **D-01:** Baseline = **secret scanning + static security lint**. Trivy/image CVE scanning is **dropped** — not even backlogged; revisit only if a future need arises. +- **D-02:** **Secret scanning via gitleaks** (tool choice researcher may confirm vs trufflehog). Scope = **per-PR diff (blocking) + a one-time full-history/full-tree baseline scan** to catch anything already committed. The app handles real family credentials (encryption key, OIDC secret, Fastmail app passwords), so secret scanning is core. +- **D-03:** **eslint-plugin-security folded into the existing Phase 13 ESLint gate, as blocking errors** (not warnings). Accepted consequence: the plugin is heuristic/noisy (e.g. `detect-object-injection`); the executor must triage existing code — add targeted `eslint-disable` with justification or rule-tune — to get the gate green. This is expected work, not a blocker. + +### Dependency Audit & Outdated +- **D-04:** `pnpm audit` **fails the build on High + Critical**; moderate/low are advisory only. +- **D-05:** Unfixable/transitive advisories are waived via a **committed allowlist file in the repo** — advisory IDs (CVE/GHSA) each with a reason + reviewer, reviewed through PR. A wrapper filters `pnpm audit` output against it (or pnpm's native `auditConfig.ignore*` if the researcher finds it cleaner — but keep it auditable and self-documenting, not silent). +- **D-06:** Outdated reporting **runs and never gates** (respects the deliberate exact-version pins in CLAUDE.md). Exact balance is an **open research question** — see OQ-01. + +### Dev/Prod Image Hygiene (999.17) +- **D-07:** **Bake `ENV NODE_ENV=production` into the production Dockerfile stage.** Today the `production` stage sets no `NODE_ENV` and `CMD` runs `node dist/index.js` with no env baked in, so `devBypass.ts`'s "hard guard" (`NODE_ENV==='production'` checked first) is **not actually engaged** in the shipped image — it's only safe because the second check (`DEV_AUTH_BYPASS !== 'true'`) passes through when unset. Baking `NODE_ENV=production` engages the hard guard. +- **D-08:** Add a **boot-time refuse-to-boot guard**: on startup, if `NODE_ENV==='production'` AND `DEV_AUTH_BYPASS==='true'`, **throw and exit non-zero** instead of silently no-op'ing. Unit-tested. (This is defense-in-depth on top of D-07 — turns a silent misconfig into a loud failure.) +- **D-09:** Create a **full `.dockerignore`** (none exists today — the whole repo root is currently sent to the Docker daemon as build context). Scope = **secrets + dev + bulk**: `.env` / `.env.*` (incl. `apps/api/.env.spike`), `apps/api/scripts/seed-credential.mjs`, `.git`, `node_modules`, `dist`, `test`/`tests`, `e2e`, `.planning`, `*.sql`/dumps, Playwright artifacts (researcher enumerates the exact list against the current tree). +- **D-10:** **CI assertion = static + boot-smoke.** Static: assert `.dockerignore` exists & covers the forbidden patterns, and `publish.yml` still pins `--target production`. Boot-smoke: start the built production image with `NODE_ENV=production` + `DEV_AUTH_BYPASS=true` and assert it **refuses to boot** (exits non-zero, proving D-08 in the actual image). Full filesystem forensics deemed unnecessary — the `production` stage already copies only `apps/api/dist` + `apps/pwa/dist`. + +### Gating & Noise Posture +- **D-11:** Blocking vs advisory split — **block:** gitleaks (secret found), eslint-plugin-security, `pnpm audit` High+Critical, image-hygiene boot-smoke + static checks. **Advisory (never gates):** `pnpm outdated`. +- **D-12:** **Doc-only PR behavior:** gitleaks **runs on every PR** including doc-only (a secret can land in a doc/config). `pnpm audit` + `pnpm outdated` are **gated behind the code/lockfile `changes` filter** like `api`/`harness` (mirrors Phase 15's doc-only-skip model). +- **D-13:** **Advisory results surface in the job log only** — no PR comment / Gitea API wiring. Gitea doesn't render GitHub-style annotations (see `ci.yml` Pitfall 5 / D-06: reporter `github` is overridden). Blocking checks surface via failed status + the `gate` aggregate. +- **D-14:** Any new **blocking** job must be wired into the `gate` aggregator (`if: always()`, individual `needs.X.result` checks per the Gitea 1.26.2 wildcard bug #31007) and, if it becomes a required context, into branch protection on `main`. + +### Claude's Discretion / Researcher Decides +- **Job decomposition** (D-15): how the new PR-time checks (secret scan, audit, outdated) are laid out in `ci.yml` — a dedicated parallel `security` job vs folding into `fast-checks` — is **left to the researcher/planner** against runner constraints (no `actions/cache` — times out; ~30s install per job). Recommendation surfaced in discussion: a new `security` job parallel to `fast-checks` keeps the critical path fast and isolates advisory churn; eslint-plugin-security folds into the existing lint step regardless. +- Exact secret-scan tool (gitleaks vs trufflehog) and exact `.dockerignore` line list — researcher confirms. + +### Folded Todos +None folded. (See Deferred — the one matched todo was already delivered in Phase 8.) + + + +## Open Research Questions + +- **OQ-01 (outdated-vs-pins balance):** Design a pragmatic `pnpm outdated` reporting policy that **respects the intentional exact-version pins** in CLAUDE.md but still **surfaces when a pin is a liability** — e.g. the pinned version is multiple major versions behind latest, or the pinned version itself carries a known advisory. The user's words: "Version pins are fine but if there's an issue with them or if they are too far behind there should be a balance here." Output should be a concrete, advisory-only mechanism (what's reported, how a "dangerously behind" pin is flagged distinctly from routine drift). Never gates the build (D-06). + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Existing CI to extend +- `.gitea/workflows/ci.yml` — PR workflow being extended. Jobs: `changes` (dorny/paths-filter), `fast-checks` (lint/format/md-lint/typecheck/pwa-unit), `api` (DB-backed, MariaDB service container), `harness` (Playwright iphone+pixel+desktop), `gate` (`if: always()` aggregator). Documents runner pitfalls: no `actions/cache`, no `mysql` CLI, `localhost`→`::1` vs IPv4, Gitea annotation non-rendering, wildcard-needs bug #31007. +- `.gitea/workflows/publish.yml` — push-to-main publish. Builds `docker build --target production -f apps/api/Dockerfile .`; tags `:latest` + `:-`; `REGISTRY_PAT` secret (Gitea forbids `GITEA_` prefix). This is where the image-hygiene boot-smoke + static assertion attach. + +### Image hygiene (999.17) +- `apps/api/Dockerfile` — multi-stage: `base`/`builder`/`dev`/`pwa-builder`/`production`. `production` copies only `apps/api/dist` + `apps/pwa/dist`; **no `ENV NODE_ENV`** (the gap D-07 fixes). `dev` target shares the file. +- `apps/api/src/auth/devBypass.ts` — the dev-bypass middleware + `DEV_USER` (id=1). Hard guard checks `NODE_ENV==='production'` first; boot-time refuse-to-boot (D-08) extends this. +- (no `.dockerignore` exists yet — D-09 creates it) + +### Constraints / precedent +- `CLAUDE.md` — pins exact dependency versions intentionally (drives D-06 / OQ-01); Technology Stack + Version Compatibility tables. +- `.planning/phases/15-doc-only-ci-skip-and-md-lint/15-CONTEXT.md` — gate-aggregation + doc-only-skip noise-control precedent (D-12, D-14 mirror it). +- `.planning/phases/08-gitea-ci/08-CONTEXT.md` — original CI design decisions + runner-constraint probes (D-PROBE-*). + +### Memory (operator-confirmed gotchas) +- Gitea CI runner gotchas: `ubuntu-latest` label, no `actions/cache`, no `mysql` CLI, `GITEA_` secret prefix forbidden (use `REGISTRY_PAT`), `act` reaps backgrounded procs at step boundary. + + + +## Existing Code Insights + +### Reusable Assets +- **`gate` job pattern** (`ci.yml`) — new blocking jobs plug into its per-`needs.X.result` aggregation; copy the success/skipped tolerance logic. +- **`changes`/paths-filter** (`ci.yml`) — reuse `needs.changes.outputs.code` to gate audit/outdated on code/lockfile changes (D-12). +- **Existing ESLint gate** (Phase 13, run in `fast-checks` `pnpm lint`) — eslint-plugin-security plugs into the same config/step (D-03). +- **Phase 8 inline-Node-via-mysql2 pattern** — precedent for runner steps without extra CLIs (no `mysql`, no extra binaries assumed available). + +### Established Patterns +- **Image build only at publish** (`publish.yml`, push-to-main). The boot-smoke (D-10) builds/runs the image where it's already built — at publish — rather than adding a full image build to every PR. +- **Throwaway CI creds** scoped to ephemeral service containers — never reuse for any new secret-handling step. +- **Job-log-only result surfacing** — Gitea annotation non-rendering already forced `--reporter=list,html` over `github`; advisory output follows the same constraint (D-13). + +### Integration Points +- New `security` (or folded) checks → `ci.yml` jobs + `gate` aggregator + possibly branch-protection required contexts. +- Boot-time guard → `apps/api` startup path (alongside/within `devBypass.ts` usage in `index.ts`) + a unit test. +- `.dockerignore` → repo root; `ENV NODE_ENV=production` → `production` stage of `apps/api/Dockerfile`. +- Boot-smoke + static assertion → `publish.yml` (post-build, pre/around push). + + + +## Specific Ideas + +- gitleaks preferred for secret scanning (single binary, easy on a self-hosted runner); per-PR diff + one-time full-history baseline. +- eslint-plugin-security must be **blocking** even though it's noisy — the user explicitly chose `error` over `warn`. +- The `NODE_ENV` gap in the production image was the concrete "aha" of this discussion — fixing it (D-07) is the highest-leverage, lowest-cost hardening. +- Keep the whole phase additive to existing CI — no rewrite of `ci.yml`/`publish.yml` structure. + + + +## Deferred Ideas + +- **Renovate / Dependabot automated dependency upgrades** — out of scope; detection/enforcement only this phase. Self-hosted Renovate on Gitea is its own setup + interplay with the pin strategy. Candidate for a future phase/backlog. +- **Trivy / image CVE scanning** — dropped, not backlogged per the user; reconsider only if a concrete need arises (base-image `node:22-alpine` CVE exposure). +- **PR-comment surfacing of advisory results** (Gitea API) — deferred in favor of job-log-only (D-13); revisit if visibility proves insufficient. + +### Reviewed Todos (not folded) +- `2026-06-10-gitea-ci-regression-and-docker-publish.md` ("Gitea CI — full regression on PR to main + build/publish Docker image") — matched on keywords but **already delivered in Phase 8** (CI-01/CI-02). Not in Phase 16 scope; this is a stale pending-todo that should be archived. + + +--- + +*Phase: 16-ci-dependency-audit-and-security-checks* +*Context gathered: 2026-06-12* diff --git a/.planning/phases/16-ci-dependency-audit-and-security-checks/16-DISCUSSION-LOG.md b/.planning/phases/16-ci-dependency-audit-and-security-checks/16-DISCUSSION-LOG.md new file mode 100644 index 0000000..1c7b42c --- /dev/null +++ b/.planning/phases/16-ci-dependency-audit-and-security-checks/16-DISCUSSION-LOG.md @@ -0,0 +1,160 @@ +# Phase 16: CI Dependency Audit, Security Checks & Image Hygiene - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-06-12 +**Phase:** 16-ci-dependency-audit-and-security-checks +**Areas discussed:** Security-check baseline, Audit + outdated policy, Image-hygiene enforcement (999.17), Gating & noise posture + +--- + +## Security-Check Baseline + +### Which checks form the baseline (multiSelect) + +| Option | Description | Selected | +|--------|-------------|----------| +| Secret scan on diff | gitleaks/trufflehog scans the PR diff for committed secrets | ✓ | +| Static security lint | eslint-plugin-security or CodeQL | ✓ | +| Trivy image scan | scan built production image for OS/package CVEs | ✗ (later dropped) | +| Dependency-review action | GitHub's PR action — depends on GitHub API, not on Gitea | | + +**User's choice:** Secret scan + static security lint + (initially) trivy — then dropped trivy in the follow-up. + +### Secret-scan scope + +| Option | Description | Selected | +|--------|-------------|----------| +| Diff + one full-history scan | per-PR diff + a one-time full-history baseline | ✓ | +| PR diff only | only changed commits | | +| Full tree every run | whole repo every PR | | + +### Trivy gate + +| Option | Description | Selected | +|--------|-------------|----------| +| Publish-time, advisory | scan image, never block | | +| Publish-time, block on CRITICAL | fail publish on CRITICAL | | +| Per-PR (build image in PR) | earliest feedback | | + +**User's choice:** "Drop Trivy for now. I don't want it in the backlog, but it can be some future thing if we need it." → Trivy removed from scope entirely. + +### Static security lint severity + +| Option | Description | Selected | +|--------|-------------|----------| +| Warn (advisory) | surfaced but non-blocking | | +| Error (blocking) | fails the lint gate | ✓ | + +**Notes:** User accepts that blocking eslint-plugin-security will require triaging/disabling existing heuristic findings to reach green. + +--- + +## Audit + Outdated Policy + +### pnpm audit severity threshold + +| Option | Description | Selected | +|--------|-------------|----------| +| High + Critical | fail on high/critical | ✓ | +| Critical only | fail only on critical | | +| Moderate+ | fail on moderate and above | | + +### Waiver mechanism + +| Option | Description | Selected | +|--------|-------------|----------| +| Allowlist file in repo | committed advisory-ID list + reason + reviewer | ✓ | +| pnpm overrides / config | auditConfig.ignore* in package.json | | +| No waiver mechanism yet | deal with it if/when it blocks | | + +### Outdated reporting vs intentional pins + +| Option | Description | Selected | +|--------|-------------|----------| +| Advisory PR comment, never gates | pnpm outdated -r as PR comment | | +| Advisory, job-log only | print to job log | | +| Skip outdated entirely | rely on audit only | | + +**User's choice:** Deferred to researcher (OQ-01). "Version pins are fine but if there's an issue with them or if they are too far behind there should be a balance here." Outcome locked: advisory, never gates; researcher designs the "dangerously behind / pinned-version-has-advisory" flagging. + +--- + +## Image-Hygiene Enforcement (999.17) + +### Enforcement mechanism (multiSelect) + +| Option | Description | Selected | +|--------|-------------|----------| +| Bake NODE_ENV=production into image | engages devBypass hard guard in shipped image | ✓ | +| Boot-time refuse-to-boot | throw + non-zero exit on prod + dev-bypass | ✓ | +| Build-time abort | fail build/publish on dev target/arg | | + +**Notes:** publish.yml already pins `--target production`; the static CI assertion covers "stays that way." + +### CI assertion depth + +| Option | Description | Selected | +|--------|-------------|----------| +| Static + boot smoke | .dockerignore + --target assertion + run image with dangerous combo, assert refuses to boot | ✓ | +| Full filesystem forensics | export image fs, grep for secrets/seed/.git | | +| Static checks only | no container built/run | | + +### .dockerignore scope + +| Option | Description | Selected | +|--------|-------------|----------| +| Secrets + dev + bulk | .env*, seed-credential.mjs, .git, node_modules, dist, tests, e2e, .planning, *.sql, playwright artifacts | ✓ | +| Secrets-only minimal | only secret/seed/data files | | +| Researcher proposes the list | capture intent, enumerate later | | + +--- + +## Gating & Noise Posture + +### Job layout + +| Option | Description | Selected | +|--------|-------------|----------| +| New 'security' job, parallel | gitleaks+audit+outdated parallel to fast-checks | | +| Fold into fast-checks | steps in existing job | | +| Researcher decides layout | pick against runner constraints | ✓ | + +**Notes:** Recommendation surfaced (dedicated parallel `security` job) but final decomposition left to researcher/planner. + +### Doc-only PR behavior + +| Option | Description | Selected | +|--------|-------------|----------| +| Secret scan always; audit/outdated code-only | gitleaks universal, audit/outdated behind changes filter | ✓ | +| All new checks code-only | whole security job skips doc-only | | +| All new checks always run | run on every PR | | + +### Result surfacing + +| Option | Description | Selected | +|--------|-------------|----------| +| Job-log summary only | advisory output to job log | ✓ | +| PR comment via Gitea API | step posts/updates a PR comment | | + +### Renovate / Dependabot + +| Option | Description | Selected | +|--------|-------------|----------| +| Defer | out of scope; capture as deferred | ✓ | +| In scope | add upgrade-bot config this phase | | + +--- + +## Claude's Discretion + +- Job decomposition for the new PR-time checks (D-15) — researcher/planner. +- Exact secret-scan tool (gitleaks vs trufflehog) and exact `.dockerignore` line list — researcher confirms. + +## Deferred Ideas + +- Renovate / Dependabot automated dependency upgrades — future phase/backlog. +- Trivy / image CVE scanning — dropped, not backlogged (revisit only if needed). +- PR-comment surfacing of advisory results — deferred in favor of job-log-only. +- Stale pending todo `2026-06-10-gitea-ci-regression-and-docker-publish.md` — already delivered in Phase 8; should be archived.