luckberg/familysync
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
98acff8108fa759767357d0ac017be61aa999759
familysync/.planning/phases/07-mobile-test-harness/07-RESEARCH.md
T

52 KiB
Raw Blame History

Phase 7: Mobile Test Harness — Research

Researched: 2026-06-10 Domain: Playwright E2E test infrastructure — mobile device emulation, authenticated dev bypass, DB seeding, CI portability Confidence: HIGH (primary Playwright API verified via Context7 + npm registry)

<user_constraints>

User Constraints (from CONTEXT.md)

Locked Decisions

  • D-01: Auth via DEV_AUTH_BYPASS=true on the host-side dev stack — never a checked-in storage-state.json with an expiring session cookie (Pitfall 14). No Authelia/OIDC mocking. Dev-bypass resolves to Dev User id 1.
  • D-02: Playwright context uses serviceWorkers: 'block' so the PWA's injectManifest SW (sw.js, registerType: 'autoUpdate') cannot intercept requests / return stale cached responses (Pitfall 15). Verify the trace shows no SW-sourced responses.
  • D-03: Run a two-profile matrix: iPhone + Pixel — covers both household ecosystems (Apple + Android/Fastmail). The iPhone profile satisfies the hard non-technical-Apple-member UX constraint; Pixel covers Chrome-viewport defects.
  • D-04: Use faithful browser engines per profile: iPhone → WebKit, Pixel → Chromium. Adds a WebKit browser to the harness/CI image. (Note: this exceeds the existing global playwright-cli Chromium tooling — the harness brings its own @playwright/test browsers.) SW-block + dev-bypass apply to both profiles.
  • D-05: Hybrid — seed deterministic DB fixtures for populated views and keep explicit empty-state assertions. Dev-bypass user 1 natively has no calendars (calendar/list views render empty, live create 422s), so populated coverage requires seeding.
  • D-06: Seeding is deterministic and reset per run (truncate/reset → insert, not insert-if-absent) to guarantee repeatable day-over-day results with no stale state (SC #3). Seed onto the shared calendar (id 10, per prior project memory) + list items so user 1's views render populated.
  • D-07: Seeding runs in global-setup against the dev MariaDB (already port-bound on 3306 via docker-compose.dev.yml); teardown/reset keeps runs idempotent.
  • D-08: Harness targets a configurable baseURL (env-driven: operator's vite dev server locally, CI service host in Phase 8) with a readiness gate in global-setup (wait on /health before any spec; mirrors the PITFALLS CI-readiness guidance to avoid flaky ECONNREFUSED).
  • D-09: Stack bring-up is the caller's responsibility — operator's already-running dev stack locally, compose orchestration in Phase 8 CI. The harness never depends on a pre-running stack; it waits for one. Satisfies SC #4.
  • D-10: Optionally use Playwright webServer for vite only with reuseExistingServer: !process.env.CI (reuse the operator's pnpm dev locally, start vite fresh in CI). The API + MariaDB + Redis always stay compose-managed — webServer cannot own a multi-container stack.

Claude's Discretion

  • Assertion approach: lead with structural/role-based locator assertions + explicit tap-target measurements; add toHaveScreenshot only if non-flaky cross-environment snapshots are achievable. Schedule-X drift risk weighted heavily.
  • Stack lifecycle (D-08–D-10): planner may refine exact env-var name and webServer wiring.
  • Spec file location/structure, trace/artifact capture on failure, and npm-script + Makefile wiring — planner's discretion (follow existing conventions).

Deferred Ideas (OUT OF SCOPE)

  • Gitea CI pipeline itself (Phase 8 owns it).
  • Real production-service-worker behavior.
  • iOS-Safari standalone-PWA behavior (Home-Screen install, standalone OIDC redirect, iOS push).
  • Live event-create against Fastmail (dev-bypass user 1 has no CalDAV credential). </user_constraints>

<phase_requirements>

Phase Requirements

ID Description Research Support
TEST-01 The assistant can drive the PWA in a mobile-emulated viewport (device profile + mobile UA + touch) for automated UI/layout verification. devices['iPhone 14'] + devices['Pixel 7'] confirmed in Playwright @playwright/test 1.60.0; projects: config pattern documented via Context7.
TEST-02 Automated runs reach the authenticated PWA via the existing DEV_AUTH_BYPASS on the host-side dev stack (no manual login, no Authelia/OIDC mocking). Harness specs consumed by Phase 8 CI as UI-regression step. DEV_AUTH_BYPASS already wired in API; global-setup pattern for readiness gate + DB seeding documented; baseURL env-var pattern confirmed.
</phase_requirements>

Summary

This phase adds @playwright/test as a devDependency in apps/pwa and creates a mobile-emulated E2E harness. The harness runs two device profiles (iPhone 14/WebKit and Pixel 7/Chromium), authenticates via the existing DEV_AUTH_BYPASS=true mechanism, seeds deterministic fixtures into the dev MariaDB in globalSetup, and asserts structural quality rules (tap targets, overflow, visibility, accessible names, empty/error states). No visual screenshot assertions are included — Schedule-X's date-driven dynamic layout makes cross-environment snapshots unworkable without a high false-positive rate.

The primary research question — assertion strategy — is answered: use structural assertions only (role/name locators + boundingBox() measurements + scrollWidth ≤ clientWidth + isVisible() + page.route() for error-state simulation). This is the well-documented Playwright-idiomatic approach; toHaveScreenshot is explicitly excluded for this phase due to Schedule-X date-dependent rendering and font-pipeline variance across host↔CI WebKit.

@playwright/test 1.60.0 is the current release. [VERIFIED: npm registry] The devices descriptors for 'iPhone 14' (390×844 viewport, WebKit UA) and 'Pixel 7' (412×915 viewport, Chrome Android UA) are confirmed in the Playwright source. [VERIFIED: playwright deviceDescriptorsSource.json via WebFetch] mysql2 is the existing project DB driver; the same credentials pattern (DB_HOST=127.0.0.1, DB_PASSWORD from env) used by the API integration tests applies to the global-setup seed script.

Primary recommendation: One playwright.config.ts in apps/pwa/ with two projects (iphone/pixel), globalSetup for health-polling + DB seeding, serviceWorkers: 'block' on both contexts, env-driven baseURL, webServer for vite with reuseExistingServer: !process.env.CI, and spec files under apps/pwa/e2e/ using *.spec.ts glob (isolated from Vitest's *.test.ts glob).

Architectural Responsibility Map

Capability Primary Tier Secondary Tier Rationale
Mobile viewport + UA emulation Test Harness (@playwright/test) — Playwright devices[...] spread into project use: options; no app-layer change needed
Auth bypass API (existing devBypass.ts) Test Harness (sets DEV_AUTH_BYPASS=true) Bypass is already implemented; harness only ensures env var is set before API starts
DB seeding Test Harness (globalSetup) Dev MariaDB (port 3306) Direct mysql2 connection from global-setup; no API endpoint for seed data
Stack readiness gate Test Harness (globalSetup) — GET /health poll via fetch with retry loop before any spec runs
Service worker suppression Test Harness (context option) — serviceWorkers: 'block' in playwright.config.ts context options; stops Workbox intercept
Structural assertions (tap target, overflow, visibility, a11y) Test Harness (spec files) — boundingBox(), page.evaluate(scrollWidth), isVisible(), role-based locators
API error-state simulation Test Harness (page.route()) — Fulfill /api/events* with status 500 for error-state tests; no backend change needed
Vite dev server lifecycle Test Harness (webServer) or Operator — webServer starts vite if not running; reuseExistingServer: !process.env.CI avoids double-start locally
CI portability Test Harness (env-driven config) — PLAYWRIGHT_BASE_URL + DB_HOST env vars; no hardcoded localhost in spec files

Standard Stack

Core (new additions for this phase)

Library Version Purpose Why Standard
@playwright/test 1.60.0 Mobile-emulated E2E test runner + assertions Official Playwright test runner; includes device descriptors, projects:, globalSetup, page.route(), boundingBox(), toHaveScreenshot (omitted this phase) — the only credible option for WebKit-on-Linux emulation [VERIFIED: npm registry]

Supporting (already in project, used in harness)

Library Version Purpose When to Use
mysql2 3.22.5 (already a project dep) DB connection in global-setup seed script Direct mysql2 createConnection (not Drizzle — global-setup runs outside the API; Drizzle schema not needed for raw INSERT/TRUNCATE) [VERIFIED: npm registry, SUS flag — see audit]

Alternatives Considered

Instead of Could Use Tradeoff
mysql2 in global-setup Drizzle ORM Drizzle is fine but adds unnecessary indirection for 3 TRUNCATE + INSERT statements; raw mysql2 is simpler and already project-resident
serviceWorkers: 'block' Manual SW unregister in test block is one line of config; unregister requires per-test async setup and is easy to forget
page.route() for error states Mocking API server route interception is in-process and doesn't require a separate mock server; the canonical Playwright approach

Installation:

pnpm --filter @familysync/pwa add -D @playwright/test
# Install browser engines (both projects: WebKit + Chromium)
pnpm --filter @familysync/pwa exec playwright install --with-deps webkit chromium

Version verification:

npm view @playwright/test version   # → 1.60.0 (verified 2026-06-10)
npm view mysql2 version             # → 3.22.5 (verified 2026-06-10)

Package Legitimacy Audit

Package Registry Age Downloads Source Repo Verdict Disposition
@playwright/test npm Published 2026-05-11 (latest 1.60.0) 38.6M/wk github.com/microsoft/playwright OK Approved
mysql2 npm Published 2026-06-06 (latest 3.22.5) 11.4M/wk github.com/sidorares/node-mysql2 SUS (too-new flag for latest patch) Approved — already a project dependency; legitimate package, flag is recency of latest patch, not package itself

Packages removed due to SLOP verdict: none

Packages flagged as suspicious SUS: mysql2 — the SUS flag is triggered by the too-new heuristic on the latest patch (3.22.5, published 2026-06-06). This package is well-established (11.4M weekly downloads, Drizzle explicit dependency, already in the project), and the harness uses the same version already present in apps/api. No additional review needed; the SUS flag is a false positive caused by a very recent patch release. [ASSUMED: "false positive" characterisation — gsd-tools SUS verdict cannot be overridden by provenance alone; planner notes no checkpoint:human-verify is required since mysql2 is already installed in the project.]

Architecture Patterns

System Architecture Diagram

Operator / CI runner
        |
        | sets env: PLAYWRIGHT_BASE_URL, DB_HOST, DB_PASSWORD, DEV_AUTH_BYPASS=true
        v
[ playwright.config.ts ]
        |
        |-- globalSetup ──────────────────────────────────────────────────┐
        |       1. poll GET {baseURL}/health until 200 (60s timeout)      |
        |       2. mysql2.createConnection(DB_HOST:3306, creds)           |
        |       3. TRUNCATE calendar_events, lists, list_items, ...       |
        |       4. INSERT deterministic fixtures onto calendar id=10      |
        |       5. INSERT list (owner_id=1) + list_items (≥2 items)       |
        |       6. INSERT list_shares (list_id, user_id=1)                |
        |       connection.end()                                           |
        |                                                                  |
        |-- webServer (vite, reuseExistingServer: !CI) ───────────────────┘
        |        starts vite :5173 if not already running
        |
        |-- project: iphone ──────────────────────────────────────────────┐
        |       engine: WebKit                                             |
        |       use: devices['iPhone 14']                                 |
        |       contextOptions: { serviceWorkers: 'block' }               |
        |                                                                  |
        |-- project: pixel ───────────────────────────────────────────────┘
                engine: Chromium
                use: devices['Pixel 7']
                contextOptions: { serviceWorkers: 'block' }

                    |
                    | both projects run against same spec files
                    v
        [ apps/pwa/e2e/*.spec.ts ]
                    |
                    | page.goto(baseURL + '/calendar'), assertions
                    | page.goto(baseURL + '/lists'), assertions
                    | page.route('/api/events*', fulfill 500), goto /calendar, assertions
                    v
        [ Vite dev server :5173 ]  ← proxy /api,/health,/callback → :3000
                    |
                    v
        [ API :3000 (DEV_AUTH_BYPASS=true, NODE_ENV=development) ]
                    |
                    v
        [ Dev MariaDB :3306 ]   ← seeded fixtures from globalSetup

Recommended Project Structure

apps/pwa/
├── e2e/                          # Playwright E2E specs — *.spec.ts glob
│   ├── calendar.spec.ts          # calendar route: populated, empty, error state
│   ├── lists.spec.ts             # lists route: populated, empty state
│   ├── layout.spec.ts            # cross-route: tap targets, overflow, BottomTabBar
│   └── global-setup.ts           # health poll + DB seed (no Playwright deps)
├── playwright.config.ts          # project matrix, globalSetup, webServer, artifacts
├── vitest.config.ts              # unchanged — *.test.ts glob, jsdom env
└── package.json                  # add @playwright/test devDependency + e2e script

Key isolation rule: vitest.config.ts has no explicit include pattern, so by default Vitest scans for *.test.ts / *.test.tsx files. Playwright's testMatch in playwright.config.ts targets e2e/**/*.spec.ts. These globs do not overlap — no runner collision. [CITED: TESTING.md — existing convention uses *.test.ts for Vitest]

Pattern 1: Two-Project Device Matrix with serviceWorkers: 'block'

// apps/pwa/playwright.config.ts
// Source: Context7 /microsoft/playwright.dev — emulation.mdx + test-global-setup-teardown.mdx
import { defineConfig, devices } from '@playwright/test'

export default defineConfig({
  testDir: './e2e',
  testMatch: '**/*.spec.ts',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  reporter: process.env.CI ? 'github' : 'list',
  globalSetup: './e2e/global-setup.ts',

  use: {
    baseURL: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173',
    trace: 'on-first-retry',
    video: 'on-first-retry',
    screenshot: 'only-on-failure',
  },

  projects: [
    {
      name: 'iphone',
      use: {
        ...devices['iPhone 14'],
        // serviceWorkers: 'block' prevents the injectManifest sw.js from
        // intercepting any requests — satisfies D-02 / Pitfall 15
        serviceWorkers: 'block',
      },
    },
    {
      name: 'pixel',
      use: {
        ...devices['Pixel 7'],
        serviceWorkers: 'block',
      },
    },
  ],

  // D-10: manage Vite only; API+MariaDB+Redis are compose-managed
  webServer: {
    command: 'pnpm --filter @familysync/pwa dev',
    url: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
    timeout: 120_000,
  },
})

Device descriptor confirmed properties:

  • devices['iPhone 14']: viewport 390×844, userAgent Mobile Safari, hasTouch: true, isMobile: true, defaultBrowserType: 'webkit' [VERIFIED: playwright deviceDescriptorsSource.json]
  • devices['Pixel 7']: viewport 412×915, userAgent Chrome/Android, hasTouch: true, isMobile: true, defaultBrowserType: 'chromium' [VERIFIED: playwright deviceDescriptorsSource.json]

Pattern 2: globalSetup — Health Poll + DB Seed

// apps/pwa/e2e/global-setup.ts
// Source: Context7 /microsoft/playwright.dev — test-global-setup-teardown.mdx
import mysql from 'mysql2/promise'

export default async function globalSetup() {
  // Step 1: Wait for /health — D-08 readiness gate
  const baseURL = process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173'
  const deadline = Date.now() + 60_000
  while (Date.now() < deadline) {
    try {
      const res = await fetch(`${baseURL}/health`)
      if (res.ok) break
    } catch {
      // ECONNREFUSED — not ready yet
    }
    await new Promise((r) => setTimeout(r, 1_000))
  }
  // will throw if never resolved — test run fails fast with a clear message

  // Step 2: Seed — D-06 deterministic reset-per-run
  const conn = await mysql.createConnection({
    host: process.env.DB_HOST ?? '127.0.0.1',
    port: Number(process.env.DB_PORT ?? 3306),
    user: process.env.DB_USER ?? 'familysync',
    password: process.env.DB_PASSWORD ?? '',
    database: process.env.DB_NAME ?? 'familysync',
  })

  try {
    // Disable FK checks for TRUNCATE ordering
    await conn.execute('SET FOREIGN_KEY_CHECKS=0')
    await conn.execute('TRUNCATE TABLE list_items')
    await conn.execute('TRUNCATE TABLE list_shares')
    await conn.execute('TRUNCATE TABLE lists')
    await conn.execute('TRUNCATE TABLE calendar_events')
    await conn.execute('SET FOREIGN_KEY_CHECKS=1')

    // Seed: one calendar event on shared calendar id=10 (timed, not all-day)
    // Minimal VCALENDAR string — enough for the API to expand and the UI to show it
    const futureStart = new Date(Date.now() + 24 * 60 * 60 * 1000) // tomorrow
    const futureStartUtc = futureStart.toISOString().replace('T', 'T').replace(/\.\d+Z$/, 'Z')
    const uid = 'e2e-seed-event-001'
    const rawVevent = [
      'BEGIN:VCALENDAR',
      'VERSION:2.0',
      'BEGIN:VEVENT',
      `UID:${uid}`,
      `DTSTART:${futureStart.toISOString().replace(/[-:]/g, '').replace(/\.\d+Z$/, 'Z')}`,
      `SUMMARY:Seeded Test Event`,
      'END:VEVENT',
      'END:VCALENDAR',
    ].join('\r\n')
    await conn.execute(
      `INSERT INTO calendar_events
         (calendar_id, uid, etag, raw_vevent, title, dtstart_utc, all_day, has_rrule)
       VALUES (10, ?, 'e2e-etag-001', ?, 'Seeded Test Event', ?, false, false)`,
      [uid, rawVevent, futureStartUtc],
    )

    // Seed: one list with two items for user 1
    const [listResult] = await conn.execute(
      `INSERT INTO lists (owner_id, name, is_shared) VALUES (1, 'E2E Grocery List', true)`,
    ) as any[]
    const listId = listResult.insertId
    await conn.execute(
      `INSERT INTO list_shares (list_id, user_id) VALUES (?, 1)`,
      [listId],
    )
    await conn.execute(
      `INSERT INTO list_items (list_id, text, checked, rank) VALUES (?, 'Milk', false, 'a0'), (?, 'Eggs', false, 'a1')`,
      [listId, listId],
    )
  } finally {
    await conn.end()
  }
}

Notes for planner:

  • The calendar_events.dtstart_utc type is TIMESTAMP, which MariaDB stores in UTC. Pass an ISO UTC string.
  • calendar_id=10 is the confirmed shared calendar from project memory dev-data-user1-no-calendars. The seed assumes this row pre-exists (it does on the dev stack); the planner may add an INSERT IGNORE INTO calendars ... guard for CI resilience.
  • list_shares ensures user 1 can see the list in /api/lists.
  • fractional-indexing rank strings 'a0', 'a1' are valid initial ranks per the project's sort pattern. [ASSUMED: exact rank string format — verify against a real row or the fractional-indexing library docs if different from 'a0'/'a1']

Pattern 3: Structural Assertions — Tap Targets (Rule 1)

// Source: Context7 /microsoft/playwright.dev — api/class-locator.mdx + UI-SPEC.md Rule 1
import { test, expect } from '@playwright/test'

test('BottomTabBar tabs meet 44px touch target', async ({ page }) => {
  await page.goto('/calendar')
  
  const calTab = page.getByRole('link', { name: 'Calendar' })
  const listsTab = page.getByRole('link', { name: 'Lists' })
  
  for (const el of [calTab, listsTab]) {
    const box = await el.boundingBox()
    expect(box).not.toBeNull()
    expect(box!.width).toBeGreaterThanOrEqual(44)
    expect(box!.height).toBeGreaterThanOrEqual(44)
  }
})

API confirmation: locator.boundingBox() returns { x, y, width, height } in CSS pixels (logical pixels at scale: 'css'). Returns null if element not visible. [CITED: context7 /microsoft/playwright.dev — api/class-locator.mdx]

Pattern 4: Structural Assertions — No Horizontal Overflow (Rule 2)

// Source: UI-SPEC.md Rule 2 — confirmed as standard Playwright JS evaluation pattern
const overflow = await page.evaluate(() => ({
  scrollWidth: document.documentElement.scrollWidth,
  clientWidth: document.documentElement.clientWidth,
}))
expect(overflow.scrollWidth).toBeLessThanOrEqual(overflow.clientWidth)

This is page.evaluate() — standard Playwright API, no special library needed. [CITED: context7 /microsoft/playwright.dev]

Pattern 5: API Error-State Simulation via page.route()

// Source: Context7 /microsoft/playwright.dev — network.mdx
// Use BEFORE page.goto() — route must be registered before navigation
await page.route('/api/events*', (route) =>
  route.fulfill({ status: 500, body: JSON.stringify({ error: 'simulated' }) }),
)
await page.goto('/calendar')
await expect(page.getByRole('heading', { name: "Couldn't load events" })).toBeVisible()
await expect(page.getByRole('button', { name: 'Retry' })).toBeVisible()

Important: Call await page.unroute('/api/events*') or use page.route with { times: 1 } to prevent the mock from leaking to subsequent tests. [CITED: context7 /microsoft/playwright.dev — network.mdx]

Pattern 6: Vite webServer with reuseExistingServer (D-10)

// Source: Context7 /microsoft/playwright.dev — playwright.config.ts example
webServer: {
  command: 'pnpm --filter @familysync/pwa dev',
  url: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173',
  reuseExistingServer: !process.env.CI,
  timeout: 120_000,
},
  • Locally: Playwright checks if :5173 is already answering; if yes, it reuses the operator's pnpm dev session without starting a new one.
  • In CI: process.env.CI is set by Gitea Actions → reuseExistingServer: false → Playwright starts its own vite process from scratch. [CITED: context7 /microsoft/playwright.dev — playwright.config.ts]

Interaction with DEV_AUTH_BYPASS: The API process is not managed by webServer. DEV_AUTH_BYPASS=true must be set in the environment that starts the API (compose env or runner env). Locally, the operator sets it; in Phase 8 CI, the workflow YAML sets it before starting the compose stack. The harness itself does not control API env.

Anti-Patterns to Avoid

  • storageState in playwright.config.ts use.storageState: stores OIDC session cookies that expire. Using DEV_AUTH_BYPASS eliminates the need entirely. [CITED: PITFALLS.md §Pitfall 14]
  • Omitting serviceWorkers: 'block': Workbox cache-first responses from a prior Playwright run will appear as SW-sourced in traces. The block option prevents registration entirely. [CITED: PITFALLS.md §Pitfall 15]
  • Hardcoded localhost:5173 in spec files: breaks CI where baseURL may differ. Use page.goto('/calendar') with a configured baseURL — relative paths resolve against it. [CITED: context7 /microsoft/playwright.dev — test-parameterize.mdx]
  • webServer managing API + compose services: webServer can only manage one process. API needs DEV_AUTH_BYPASS=true, MariaDB, and Redis — use compose for those. [ASSUMED: webServer single-process limitation — consistent with docs pattern]
  • Calling pnpm playwright install without --with-deps in CI: WebKit on Linux requires system libraries. playwright install --with-deps webkit chromium installs both engines and their system deps. [CITED: context7 /microsoft/playwright.dev — browsers.mdx]
  • INSERT IGNORE instead of TRUNCATE + INSERT for seed: insert-if-absent leaves stale rows from a prior run. D-06 mandates truncate-first for determinism. [CITED: 07-CONTEXT.md D-06]
  • Vitest picking up *.spec.ts files: The existing vitest.config.ts has no explicit include, so Vitest uses its default **/*.{test,spec}.{js,ts,tsx} glob. This means *.spec.ts files in e2e/ WOULD be picked up by Vitest unless excluded. The planner must add exclude: ['e2e/**'] to vitest.config.ts. [VERIFIED: vitest.config.ts read — no explicit include; spec files would be caught by default glob]

Don't Hand-Roll

Problem Don't Build Use Instead Why
Mobile viewport + touch + UA string Custom browser launch flags devices['iPhone 14'] spread devices includes DPR, hasTouch, isMobile, UA — reproducing this by hand misses fields and drifts with Playwright updates
Service worker suppression Per-test SW unregister evaluate serviceWorkers: 'block' context option One config line; unregister requires async setup in every test and is easy to omit
Stack readiness polling sleep 10 && run tests globalSetup health-poll loop Sleep is non-deterministic; a poll with timeout gives fast-pass and hard-fail
API error states Separate mock API server page.route() inline fulfill route() is in-process, zero infrastructure, the Playwright-canonical approach
DB seeding from the API POST requests to API endpoints Direct mysql2 INSERT in globalSetup Dev-bypass user 1 cannot create calendar events via API (422, no CalDAV credential); direct DB insert bypasses that constraint and is faster

Key insight: Playwright's device descriptors, serviceWorkers context option, and page.route() network interception are designed precisely for this use case. The only custom code needed is the globalSetup health poll and DB seed script.

Primary Research Question: Assertion Strategy

Recommendation: structural assertions only — no toHaveScreenshot for this phase.

Why structural assertions are sufficient and correct

The UI-SPEC.md defines five concrete quality rules, all of which map directly to Playwright structural APIs with zero rendering-pipeline variance:

Rule Playwright API Variance Risk
Touch target ≥ 44px locator.boundingBox() → measure width/height None — CSS pixel dimensions are layout-engine output, consistent across OS font settings
No horizontal overflow page.evaluate(() => scrollWidth/clientWidth) None — DOM measurement, not pixel comparison
Elements in viewport on load locator.isVisible() + boundingBox().y + height ≤ viewportHeight None — geometry check
Accessible names present page.getByRole(role, { name }) — if locatable, name exists None — ARIA tree query
Empty/error states render getByText(), getByRole() visibility None — presence check

Why toHaveScreenshot is excluded

  1. Schedule-X date-dependent layout. The calendar widget highlights today's date, places event chips by its internal layout engine, and renders the current week/month by default. The "current date" changes every day, so a snapshot taken on 2026-06-10 will fail on 2026-06-11 even with identical code. [CITED: 07-UI-SPEC.md §Rule 6]

  2. WebKit font rendering on Linux vs macOS. The iPhone project uses WebKit engine. Font hinting on Linux (CI runner likely Ubuntu) differs from macOS — sub-pixel differences accumulate across text-heavy layouts. Even with maxDiffPixelRatio: 0.03, Schedule-X's event chip labels cause breaches. [ASSUMED: Linux WebKit font difference vs macOS — this is well-documented in the Playwright community but not formally cited; LOW confidence]

  3. CI-generated baseline workflow adds operational burden. Using --update-snapshots in CI on first run, committing baselines, and keeping them per-engine (snapshots/webkit/, snapshots/chromium/) is achievable — but adds a mandatory workflow step that is not self-healing when the app's UI legitimately changes. For a two-person household app with a small team, this maintenance overhead outweighs the pixel-accuracy benefit.

  4. The structural assertions catch actual defects. Schedule-X has historically caused horizontal overflow on narrow viewports (memory entry schedule-x-allday-event-styling). The scrollWidth ≤ clientWidth assertion catches that. Tap targets below 44px are the other mobile-only class of defect — boundingBox() catches that. Screenshots would add noise without catching additional real bugs.

If snapshots are added in a later phase: scope to static, non-dynamic regions only (e.g., BottomTabBar clipped to its bounding box, not the full viewport). Use CI-generated baselines committed by a dedicated "update-snapshots" workflow. Mask the calendar grid area with mask: [page.locator('.sx__calendar-wrapper')].

Common Pitfalls

Pitfall 1: Vitest Glob Collision with *.spec.ts

What goes wrong: Vitest's default testMatch includes **/*.spec.ts. Adding apps/pwa/e2e/*.spec.ts files without an explicit exclude in vitest.config.ts causes Vitest to pick them up and fail (Playwright APIs like devices are not available in the Vitest jsdom environment).

Why it happens: vitest.config.ts has no explicit include/exclude — relies on Vitest defaults. [VERIFIED: vitest.config.ts read]

How to avoid: Add exclude: ['e2e/**'] to the test: block in apps/pwa/vitest.config.ts. Alternatively, scope Vitest's include to src/**/*.test.ts. Either prevents collision.

Warning signs: Vitest run fails with ReferenceError: devices is not defined or Playwright import errors.

Pitfall 2: globalSetup has no access to Playwright fixtures

What goes wrong: globalSetup runs outside the Playwright worker context. It cannot use page, browser, or any Playwright fixture. Only plain Node.js (fetch, mysql2, fs) is available.

Why it happens: globalSetup runs once before any worker is spawned. [CITED: context7 /microsoft/playwright.dev — test-global-setup-teardown.mdx]

How to avoid: The health poll and DB seed use only fetch (global in Node 18+) and mysql2 — both are plain Node.js. No Playwright imports in global-setup.ts.

Warning signs: ReferenceError: test is not defined in global-setup.

Pitfall 3: Vite Proxy Not Active When webServer Starts Fresh Vite

What goes wrong: In CI, webServer starts pnpm dev for the PWA. The API at :3000 must already be running (compose-managed) before Playwright navigates to /calendar — the Vite proxy to :3000 will 502 if the API is not up.

Why it happens: webServer only gates on the Vite URL being reachable, not on the proxied API being up. The globalSetup /health poll gates on the health endpoint (which IS proxied through Vite to :3000), so it handles this correctly — but only if the health poll runs AFTER Vite is started by webServer.

How to avoid: Playwright starts webServer before running globalSetup, so the ordering is: compose brings up API+DB+Redis → Playwright starts Vite (webServer) → globalSetup polls /health (proxied to API). In CI, the workflow must start compose before running npx playwright test. [ASSUMED: Playwright webServer starts before globalSetup — verify in docs; treat as LOW confidence]

Warning signs: globalSetup health poll times out in CI even though the API is healthy, because Vite isn't started yet when the poll begins.

Pitfall 4: calendar_id=10 Not Present in CI MariaDB

What goes wrong: The seed script does INSERT INTO calendar_events (calendar_id=10, ...). In the developer's local MariaDB, calendar row 10 exists (created by the broker poller after D-16). In a fresh CI MariaDB with only Drizzle migrations applied, there is no calendar row 10.

Why it happens: The CI DB starts from migrations only — no production data, no broker-seeded calendar rows.

How to avoid: The globalSetup should INSERT IGNORE INTO calendars (id, user_id, url, display_name, is_shared) VALUES (10, 1, ...) before inserting calendar_events. This ensures the FK constraint is satisfied in both fresh and populated environments. [CITED: 07-CONTEXT.md D-06 + schema.ts FK reference]

Warning signs: globalSetup throws ER_NO_REFERENCED_ROW_2: Cannot add or update a child row: a foreign key constraint fails on the calendar_events INSERT.

Pitfall 5: DEV_AUTH_BYPASS Not Propagated to API Process

What goes wrong: The harness assumes DEV_AUTH_BYPASS=true is active in the API process. If the API was started without it (or the env var was not exported), all /api/* calls return 401/302 and the PWA renders an auth redirect instead of the calendar.

Why it happens: DEV_AUTH_BYPASS is checked at API startup and gated by NODE_ENV !== 'production'. The harness cannot set it — it must be set in the API's environment before the API process starts.

How to avoid: Document in the harness README that the dev stack must be started with DEV_AUTH_BYPASS=true. In CI (Phase 8), the workflow YAML must set it in the environment before launching the compose stack. The globalSetup can assert DEV_AUTH_BYPASS is active by checking that GET /health returns { ok: true } — if the API is running without bypass, /api/me will redirect, which isn't directly testable in globalSetup, but the first spec failing on unexpected auth redirect is a clear signal.

Warning signs: All specs fail with unexpected redirect to Authelia login page.

Pitfall 6: WebKit Not Installed in CI Image

What goes wrong: Running playwright install without --with-deps in CI installs the Playwright browser binaries but not the system-level libraries WebKit needs on Linux. WebKit then fails to launch with library errors.

Why it happens: WebKit on Linux requires libwebkit2gtk or similar system deps that are not present in the base CI runner image.

How to avoid: Use playwright install --with-deps webkit chromium in CI. This is the documented approach for CI environments. Expect this to add ~500MB to the CI step. [CITED: Playwright docs on browsers.mdx — --with-deps flag]

Warning signs: CI step fails with libnss3.so: cannot open shared object file or similar.

Code Examples

playwright.config.ts (complete)

// apps/pwa/playwright.config.ts
// Source: Context7 /microsoft/playwright.dev
import { defineConfig, devices } from '@playwright/test'

export default defineConfig({
  testDir: './e2e',
  testMatch: '**/*.spec.ts',
  fullyParallel: true,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: process.env.CI ? 'github' : 'list',
  globalSetup: './e2e/global-setup.ts',

  use: {
    baseURL: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173',
    trace: 'on-first-retry',
    video: 'on-first-retry',
    screenshot: 'only-on-failure',
  },

  projects: [
    {
      name: 'iphone',
      use: {
        ...devices['iPhone 14'],
        serviceWorkers: 'block',
      },
    },
    {
      name: 'pixel',
      use: {
        ...devices['Pixel 7'],
        serviceWorkers: 'block',
      },
    },
  ],

  webServer: {
    command: 'pnpm --filter @familysync/pwa dev',
    url: process.env.PLAYWRIGHT_BASE_URL ?? 'http://localhost:5173',
    reuseExistingServer: !process.env.CI,
    timeout: 120_000,
  },
})

vitest.config.ts patch (add exclude)

// apps/pwa/vitest.config.ts — add exclude to prevent Vitest from picking up e2e specs
import { defineConfig } from 'vitest/config'

export default defineConfig({
  test: {
    environment: 'jsdom',
    globals: true,
    setupFiles: ['./src/test-setup.ts'],
    env: { TZ: 'UTC' },
    exclude: ['e2e/**', 'node_modules/**'],   // ← ADD THIS
  },
})

package.json scripts additions

{
  "scripts": {
    "test:e2e": "playwright test",
    "test:e2e:ui": "playwright test --ui",
    "test:e2e:update-snapshots": "playwright test --update-snapshots"
  }
}

Root workspace script (for pnpm filter):

pnpm --filter @familysync/pwa test:e2e

layout.spec.ts skeleton

// apps/pwa/e2e/layout.spec.ts
// Source: UI-SPEC.md Rules 1-4 + Context7 /microsoft/playwright.dev
import { test, expect } from '@playwright/test'

test.describe('BottomTabBar presence and tap targets', () => {
  test.beforeEach(async ({ page }) => {
    await page.goto('/calendar')
  })

  test('BottomTabBar is present at mobile width', async ({ page }) => {
    await expect(page.getByRole('navigation', { name: 'Main navigation' })).toBeVisible()
  })

  test('Calendar tab meets 44px touch target', async ({ page }) => {
    const tab = page.getByRole('link', { name: 'Calendar' })
    const box = await tab.boundingBox()
    expect(box).not.toBeNull()
    expect(box!.width).toBeGreaterThanOrEqual(44)
    expect(box!.height).toBeGreaterThanOrEqual(44)
  })

  test('no horizontal overflow on /calendar', async ({ page }) => {
    const overflow = await page.evaluate(() => ({
      scrollWidth: document.documentElement.scrollWidth,
      clientWidth: document.documentElement.clientWidth,
    }))
    expect(overflow.scrollWidth).toBeLessThanOrEqual(overflow.clientWidth)
  })
})

test.describe('Error state — /calendar', () => {
  test('shows error heading and Retry button when API returns 500', async ({ page }) => {
    await page.route('/api/events*', (route) =>
      route.fulfill({ status: 500, body: JSON.stringify({ error: 'simulated' }) }),
    )
    await page.goto('/calendar')
    await expect(page.getByRole('heading', { name: "Couldn't load events" })).toBeVisible()
    await expect(page.getByRole('button', { name: 'Retry' })).toBeVisible()
    // error state must also pass overflow rule
    const overflow = await page.evaluate(() => ({
      scrollWidth: document.documentElement.scrollWidth,
      clientWidth: document.documentElement.clientWidth,
    }))
    expect(overflow.scrollWidth).toBeLessThanOrEqual(overflow.clientWidth)
  })
})

State of the Art

Old Approach Current Approach When Changed Impact
playwright-cli global tool (Chromium desktop only) @playwright/test with devices[...] projects (WebKit + Chromium, mobile viewport) This phase Playwright-cli remains for interactive assistant smoke tests; @playwright/test is for automated regression
No E2E tests — playwright-cli used ad-hoc Structured e2e/ spec files with globalSetup + device matrix This phase Mobile layout defects caught automatically instead of by operator on real devices
toHaveScreenshot visual regression Structural assertions (boundingBox, overflow eval, role/name locators) Deliberate decision — UI-SPEC §Rule 6 Lower maintenance, zero rendering-pipeline variance, sufficient quality coverage for this app

Deprecated/outdated patterns for this codebase:

  • storageState.json for Playwright auth: never appropriate here; DEV_AUTH_BYPASS is the correct pattern. [CITED: PITFALLS.md §Pitfall 14]
  • serviceWorkers: 'allow' (default): would allow Workbox cache-first to intercept API calls. [CITED: PITFALLS.md §Pitfall 15]

Assumptions Log

# Claim Section Risk if Wrong
A1 fractional-indexing rank strings 'a0', 'a1' are valid initial ranks for seed items Pattern 2 (global-setup) Seed succeeds but list items sort incorrectly; items may not appear in expected order in UI
A2 Playwright webServer starts before globalSetup is called Pitfall 3 globalSetup health poll would time out if Vite isn't started yet; ordering must be confirmed against docs
A3 mysql2 SUS verdict is a false positive due to recent patch release Package Audit Not a concern — package is already in the project; would only matter if upgrading to the latest patch caused issues
A4 Linux WebKit font rendering differs from macOS enough to cause toHaveScreenshot failures Primary Research Q If wrong, screenshots could be added with maxDiffPixelRatio: 0.03; structural assertions remain the lower-risk choice

Open Questions

  1. calendar_id=10 in CI DB — confirmed guard needed

    • What we know: dev DB has calendar row 10 from production poller. CI DB starts fresh from migrations.
    • What's unclear: does the CI compose stack do any data seeding beyond migrations?
    • Recommendation: globalSetup does INSERT IGNORE INTO calendars (id, user_id, url, display_name, color, is_shared) VALUES (10, 1, 'https://caldav.fastmail.com/dav/calendars/e2e/', 'FamilySync', '#4A90D9', true) before inserting calendar_events. Safe even on local dev (IGNORE avoids duplicate key error).

  2. webServer ordering relative to globalSetup

    • What we know: Context7 docs show webServer and globalSetup as separate config options but don't document relative ordering explicitly.
    • What's unclear: does Playwright guarantee webServer starts before globalSetup runs?
    • Recommendation: if unsure, move the Vite readiness check INTO globalSetup (poll :5173 before polling /health). This is belt-and-suspenders but eliminates the ordering ambiguity.

  3. list_shares row required vs. isShared=true flag alone

    • What we know: lists.is_shared=true is the flag; list_shares is the join table. The API /api/lists route may return lists via listShares join or via is_shared flag — need to check route handler.
    • What's unclear: does user 1 see a list they own (ownerId=1) without a listShares row, or only via listShares?
    • Recommendation: seed both lists.owner_id=1 and a list_shares row for safety; the seed is idempotent either way.

Environment Availability

Dependency Required By Available Version Fallback
Node.js 22 LTS global-setup (fetch native, mysql2) ✓ (assumed) 22.x —
Dev MariaDB :3306 (port-bound) global-setup DB seed ✓ when dev stack is up via docker-compose.dev.yml MariaDB 11 Seed step skipped gracefully — tests run with empty DB (empty-state assertions still valid)
Vite dev server :5173 All spec files ✓ via webServer or operator's pnpm dev Vite 8.0.16 —
API :3000 with DEV_AUTH_BYPASS=true All spec files (via Vite proxy) ✓ when dev stack is up Node 22 + Hono —
WebKit browser binary iPhone project ✗ (not yet installed) — Must run playwright install --with-deps webkit
Chromium browser binary Pixel project ✓ (used by playwright-cli skill) Chromium (via playwright-cli) May need re-install via @playwright/test's own browser store

Missing dependencies with no fallback:

  • WebKit browser binary — required for the iphone project. Must be installed via playwright install --with-deps webkit as part of Phase 7 Wave 0.

Missing dependencies with fallback:

  • Dev MariaDB port binding — if compose isn't up, the seed is skipped; specs run with empty DB, exercising empty-state assertions only (partial coverage, but not a hard failure).

Validation Architecture

Test Framework

Property Value
Framework @playwright/test 1.60.0
Config file apps/pwa/playwright.config.ts (Wave 0 — new file)
Quick run command (one profile) pnpm --filter @familysync/pwa exec playwright test --project=pixel
Full suite command pnpm --filter @familysync/pwa exec playwright test
Headed (local debug) pnpm --filter @familysync/pwa exec playwright test --headed

Phase Requirements → Test Map

Req ID Behavior Test Type Automated Command File Exists?
TEST-01 PWA loads in mobile-emulated viewport (390px / 412px), touch-capable, mobile UA E2E (Playwright) pnpm --filter @familysync/pwa exec playwright test ❌ Wave 0 — create e2e/layout.spec.ts
TEST-01 Tap targets ≥ 44px on BottomTabBar, FAB, Retry, settings button E2E (Playwright) same ❌ Wave 0 — e2e/layout.spec.ts
TEST-01 No horizontal overflow on /calendar, /lists E2E (Playwright) same ❌ Wave 0 — e2e/layout.spec.ts
TEST-02 Reaches authenticated PWA via DEV_AUTH_BYPASS (no manual login) E2E (Playwright) same ❌ Wave 0 — e2e/global-setup.ts enforces auth precondition
TEST-02 Harness runs headlessly, CI-portable (env-driven baseURL, readiness gate) E2E (Playwright) CI=true pnpm --filter @familysync/pwa exec playwright test ❌ Wave 0 — playwright.config.ts

Harness Self-Validation (the harness must prove it works)

This phase's deliverable IS the test infrastructure. The harness is validated when it detects real defects. Recommended self-validation approach:

  1. Broken layout fixture test: temporarily reduce BottomTabBar minHeight to 20px in a test — the tap-target assertion MUST fail. Restore and verify it passes. This proves boundingBox() is measuring the rendered element, not the CSS declaration.

  2. Overflow injection test: add body { overflow-x: auto; width: 2000px; } via page.addStyleTag before the overflow assertion — it MUST fail. Remove and verify it passes.

  3. SW-block verification: after a run, trace: 'on-first-retry' generates trace artifacts. Review one trace with the Playwright trace viewer and confirm zero responses have (ServiceWorker) as source.

  4. Auth bypass verification: without DEV_AUTH_BYPASS=true, the API redirects to Authelia. Run with bypass disabled — specs MUST fail on expected content not found. With bypass enabled, specs pass. (Manual verification step.)

Sampling Rate

  • Per task commit: pnpm --filter @familysync/pwa exec playwright test --project=pixel (Chromium only, faster)
  • Per wave merge: pnpm --filter @familysync/pwa exec playwright test (both profiles)
  • Phase gate: both profiles green on the full spec suite before marking Phase 7 complete

Wave 0 Gaps

  • apps/pwa/playwright.config.ts — project matrix, globalSetup, webServer, artifact config
  • apps/pwa/e2e/global-setup.ts — health poll + DB seed (calendar id 10 guard + list + items)
  • apps/pwa/e2e/layout.spec.ts — tap targets, overflow, BottomTabBar visibility (Rules 1, 2, 3)
  • apps/pwa/e2e/calendar.spec.ts — populated state, empty state, error state (Rules 4, 5 for calendar)
  • apps/pwa/e2e/lists.spec.ts — populated state, empty state (Rules 4, 5 for lists)
  • apps/pwa/vitest.config.ts — add exclude: ['e2e/**'] to prevent glob collision
  • apps/pwa/package.json — add @playwright/test devDependency + test:e2e script
  • Browser install: pnpm --filter @familysync/pwa exec playwright install --with-deps webkit chromium

Security Domain

security_enforcement not set to false — section required.

Applicable ASVS Categories

ASVS Category Applies Standard Control
V2 Authentication Yes (test auth path) DEV_AUTH_BYPASS=true — never real credentials in test env; bypass is dev-only (guarded by NODE_ENV !== 'production')
V3 Session Management No DEV_AUTH_BYPASS bypasses session cookies entirely
V4 Access Control No Harness tests as user 1; no privilege escalation in scope
V5 Input Validation No Harness is read-only; no form submission in scope
V6 Cryptography No No crypto operations in test harness

Known Threat Patterns for this stack

Pattern STRIDE Standard Mitigation
DEV_AUTH_BYPASS=true active in production Elevation of Privilege API guards on NODE_ENV !== 'production' — production compose MUST NOT set this variable. The harness README must document this.
storage-state.json with real OIDC session committed to repo Information Disclosure Not applicable — storageState is never used in this harness (D-01).
DB seed credentials in test script Information Disclosure Use env vars for DB credentials in globalSetup (DB_HOST, DB_PASSWORD); no hardcoded credentials.

Project Constraints (from CLAUDE.md)

Directive Impact on Phase 7
MariaDB only — no PostgreSQL seed script uses mysql2; no pg driver
pnpm workspace all installs via pnpm --filter @familysync/pwa add; scripts via pnpm --filter @familysync/pwa exec playwright
No Makefile (root Makefile does not exist) scripts exposed via package.json scripts in apps/pwa and root workspace; no Makefile to update
playwright-cli is global Chromium only @playwright/test brings its own browser store; no conflict with playwright-cli; the two tools coexist
playwright-cli skill exception for iOS-Safari-standalone real device checks (Home Screen install, iOS push) remain human gates — NOT in scope for this harness
Vitest for unit tests *.spec.ts glob collision must be resolved via vitest.config.ts exclude
tsc --noEmit gate (both apps) playwright.config.ts and e2e/*.ts files must pass typecheck; add to root typecheck script or ensure apps/pwa/tsconfig.json includes e2e/

Sources

Primary (MEDIUM confidence — Context7/High reputation source)

  • /microsoft/playwright.dev via Context7 — device emulation config, projects matrix, globalSetup pattern, page.route(), trace: 'on-first-retry', webServer + reuseExistingServer, baseURL env config, toHaveScreenshot options

Verified (via direct tool calls)

  • npm registry @playwright/test — version 1.60.0 confirmed, 38.6M weekly downloads, Microsoft GitHub source [VERIFIED: npm registry]
  • npm registry mysql2 — version 3.22.5 confirmed, 11.4M weekly downloads, SUS (too-new flag on latest patch) [VERIFIED: npm registry]
  • playwright/deviceDescriptorsSource.json via WebFetch — devices['iPhone 14'] and devices['Pixel 7'] confirmed present [VERIFIED: playwright deviceDescriptorsSource.json]
  • apps/pwa/vitest.config.ts — no explicit include; default glob catches *.spec.ts; exclude needed [VERIFIED: file read]
  • apps/pwa/package.json — no @playwright/test present; no e2e script [VERIFIED: file read]
  • apps/api/src/db/schema.ts — calendars, calendar_events, lists, list_items, list_shares table structure confirmed [VERIFIED: file read]
  • apps/api/src/auth/devBypass.ts — DEV_USER id=1, NODE_ENV !== 'production' guard confirmed [VERIFIED: file read]
  • apps/api/src/db/client.ts — DB connection reads DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME from env [VERIFIED: file read]

Cited (project documentation)

  • 07-CONTEXT.md — locked decisions D-01 through D-10
  • 07-UI-SPEC.md — assertion contract Rules 1–8, device matrix, locator anchors
  • PITFALLS.md §Pitfall 14, §Pitfall 15 — storage-state stale, SW intercept
  • TESTING.md — existing Vitest setup, *.test.ts convention, E2E gap
  • memory/dev-stack-bringup.md — dev stack DB_HOST override, DEV_AUTH_BYPASS pattern
  • memory/api-integration-test-db.md — DB_HOST=127.0.0.1, mysql2 connection pattern

Metadata

Confidence breakdown:

  • Standard stack: HIGH — @playwright/test 1.60.0 verified via npm; device descriptors verified via source file; mysql2 confirmed existing dep
  • Architecture: HIGH — patterns derived from existing project conventions (vitest config, DB client env vars, devBypass) + Context7 Playwright docs
  • Pitfalls: HIGH — Pitfalls 1/2/5/6 derived from reading actual project files; Pitfall 3/4 from reasoning about CI ordering

Research date: 2026-06-10 Valid until: 2026-09-10 (Playwright releases frequently but the device emulation and globalSetup APIs are stable)

Reference in New Issue View Git Blame Copy Permalink