unraid-docker-manager/.planning/research/SUMMARY.md

# Project Research Summary

**Project:** Unraid Docker Manager v1.3 — Update Status Sync
**Domain:** Docker container management integration with Unraid server
**Researched:** 2026-02-08
**Confidence:** HIGH

## Executive Summary

Unraid tracks container update status through an internal state file (`/var/lib/docker/unraid-update-status.json`) that is only updated when updates are initiated through Unraid's WebGUI or GraphQL API. When containers are updated externally via Docker API (as the bot currently does), Unraid's state becomes stale, showing false "update available" badges and sending duplicate notifications. This is the core pain point for v1.3.

The recommended approach is to use Unraid's official GraphQL API with the `updateContainer` mutation. This leverages the same mechanism Unraid's WebGUI uses and automatically handles status file synchronization, image digest comparison, and UI refresh. The GraphQL API is available natively in Unraid 7.2+ or via the Connect plugin for earlier versions. This approach is significantly more robust than direct file manipulation, is version-safe, and has proper error handling.

Key risks include race conditions between bot updates and Unraid's periodic update checker, version compatibility across Unraid 6.x-7.x, and ensuring n8n container has proper network access to the Unraid host's GraphQL endpoint. These are mitigated through atomic operations via the GraphQL API, version detection, and using `host.docker.internal` networking with proper container configuration.

## Key Findings

### Recommended Stack

**Use Unraid's GraphQL API via n8n's built-in HTTP Request node** — no new dependencies required. The GraphQL API is the official Unraid interface and handles all status synchronization internally through the DockerService and Dynamix Docker Manager integration.

**Core technologies:**
- **Unraid GraphQL API (7.2+ or Connect plugin)**: Container update status sync — Official API, same mechanism as WebGUI, handles all internal state updates automatically
- **HTTP Request (n8n built-in)**: GraphQL client — Already available, no new dependencies, simple POST request pattern
- **Unraid API Key**: Authentication — Same credential pattern as existing n8n API keys, stored in `.env.unraid-api`

**Network requirements:**
- n8n container → Unraid host via `http://host.docker.internal/graphql`
- Requires `--add-host=host.docker.internal:host-gateway` in n8n container config
- Alternative: Use Unraid IP directly (`http://tower.local/graphql` or `http://192.168.x.x/graphql`)

**Critical finding from stack research:** Direct file manipulation of `/var/lib/docker/unraid-update-status.json` was investigated but rejected. The file format is undocumented, doesn't trigger UI refresh, and is prone to race conditions. The GraphQL API is the proper abstraction layer.

### Expected Features

Based on feature research, the v1.3 scope is tightly focused on solving the core pain point.

**Must have (table stakes):**
- Clear "update available" badge after bot updates container — Users expect Unraid UI to reflect reality after external updates
- Prevent duplicate update notifications — After bot updates, Unraid shouldn't send false-positive Telegram notifications
- Automatic sync after every update — Zero user intervention, bot handles sync transparently

**Should have (v1.4+):**
- Manual sync command (`/sync`) — Trigger when user updates via other tools (Portainer, CLI, Watchtower)
- Read Unraid update status for better detection — Parse Unraid's view to enhance bot's container selection UI
- Batch status sync — Optimize multi-container update operations

**Defer (v2+):**
- Bidirectional status awareness — Use Unraid's update detection as source of truth instead of Docker digest comparison
- Persistent monitoring daemon — Real-time sync via Docker events (conflicts with n8n's workflow execution model)
- Full Unraid API integration — Authentication, template parsing, web session management (overly complex for cosmetic badge)

**Anti-features identified:**
- Automatic template XML regeneration (breaks container configuration)
- Sync status for ALL containers on every operation (performance impact, unnecessary)
- Full Unraid API integration via authentication/sessions (complexity overkill for status sync)

### Architecture Approach

**Extend n8n-update.json sub-workflow with a single HTTP Request node** that calls Unraid's GraphQL `updateContainer` mutation after successful Docker API update. This keeps sync tightly coupled to update operations, requires minimal architectural changes, and maintains single responsibility (update sub-workflow owns all update-related actions).

**Major components:**
1. **Clear Unraid Status (NEW node)** — HTTP Request to GraphQL API after "Remove Old Image (Success)", calls `mutation { docker { updateContainer(id: "docker:containerName") } }`
2. **n8n container configuration (modified)** — Add `--add-host=host.docker.internal:host-gateway` for network access to Unraid host
3. **Unraid API credential (NEW)** — API key stored in `.env.unraid-api`, loaded via n8n credential system, requires `DOCKER:UPDATE_ANY` permission

**Data flow:**
```
Update sub-workflow success → Extract container name → HTTP Request to GraphQL
→ Unraid's DockerService executes syncVersions() → Update status file written
→ Return to sub-workflow → Success response to user
```

**Rejected alternatives:**
- **Direct file write** — Brittle, undocumented format, no UI refresh, race conditions
- **New sub-workflow (n8n-unraid-sync.json)** — Overkill for single operation
- **Main workflow integration** — Adds latency, harder to test
- **Helper script on host** — Unnecessary complexity when GraphQL API exists

### Critical Pitfalls

Research identified 7 critical pitfalls with phase-specific prevention strategies:

1. **State Desync Between Docker API and Unraid's Internal Tracking** — GraphQL API solves this by using Unraid's official update mechanism instead of direct file manipulation. Validation: Update via bot, verify Unraid shows "up-to-date" after manual "Check for Updates"

2. **Race Condition Between Unraid's Periodic Update Check and Bot Sync** — GraphQL `updateContainer` mutation is idempotent and atomic. Even if Unraid's update checker runs concurrently, no file corruption or lost updates occur. The API handles synchronization internally.

3. **Unraid Version Compatibility** — GraphQL API is stable across Unraid 6.9+ (via Connect plugin) and 7.2+ (native). Version detection should check `/etc/unraid-version` and verify API availability before sync. If GraphQL unavailable, log error and skip sync (preserve bot functionality).

4. **Docker Socket Proxy Blocks Filesystem Access** — Resolved by using GraphQL API instead of direct file access. n8n only needs HTTP access to Unraid host, not filesystem mounts. This preserves security boundaries.

5. **Notification Spam During Batch Updates** — Batch updates should collect all container names and call `updateContainers` (plural) mutation once after batch completes, not per-container. This triggers single status recalculation instead of N notifications.

6. **n8n Workflow State Doesn't Persist** — Sync happens within same execution (no cross-execution state needed). Batch updates already collect results in single execution, sync after loop completes. No reliance on static data.

7. **Unraid's br0 Network Recreate** — Not directly related to status sync, but update flow must preserve network config. Current n8n-update.json uses Docker API recreate — verify network settings are copied from old container inspect to new create body.

## Implications for Roadmap

Based on combined research, v1.3 should be structured as 3 focused phases:

### Phase 1: API Setup & Network Access
**Rationale:** Validate the GraphQL approach before n8n integration. Infrastructure changes first, workflow changes second.

**Delivers:**
- Unraid API key created with `DOCKER:UPDATE_ANY` permission
- Network access verified from n8n container to Unraid host
- Container ID format documented (via GraphQL query test)

**Tasks:**
- Create API key via `unraid-api apikey --create` CLI command
- Store in `.env.unraid-api` (gitignored)
- Add `--add-host=host.docker.internal:host-gateway` to n8n container config
- Test GraphQL query from command line: `curl -X POST http://host.docker.internal/graphql -H "x-api-key: ..." -d '{"query": "{ docker { containers { id name } } }"}'`
- Verify container ID format returned (likely `docker:<name>`)

**Avoids:**
- Pitfall 4 (filesystem access issues) — uses API not file mounts
- Pitfall 3 (version compatibility) — validates API availability before implementation

**Research flag:** NEEDS DEEPER RESEARCH — Network access testing, container ID format verification, API permission validation

### Phase 2: n8n Workflow Integration
**Rationale:** Once API access is proven, integrate into update sub-workflow. Single node addition is minimal risk.

**Delivers:**
- n8n-update.json calls GraphQL `updateContainer` after successful Docker API update
- Error handling for API failures (continue on error, log for debugging)
- Single-container updates sync automatically

**Tasks:**
- Add HTTP Request node to n8n-update.json after "Remove Old Image (Success)"
- Configure GraphQL mutation: `mutation { docker { updateContainer(id: "docker:{{$json.containerName}}") { id name state } } }`
- Set authentication: Header Auth with `x-api-key: {{$env.UNRAID_API_KEY}}`
- Error handling: `continueRegularOutput` (don't fail update if sync fails)
- Connect to "Return Success" node
- Test with single container update

**Uses:**
- Unraid GraphQL API from Phase 1
- n8n HTTP Request node (built-in)

**Implements:**
- Clear Unraid Status component from architecture research
- Post-Action Sync pattern (sync after primary operation completes)

**Avoids:**
- Pitfall 1 (state desync) — uses official sync mechanism
- Pitfall 2 (race conditions) — GraphQL API handles atomicity

**Research flag:** STANDARD PATTERNS — HTTP Request node usage well-documented, unlikely to need additional research

### Phase 3: Batch Optimization & UAT
**Rationale:** After core functionality works, optimize for batch operations and validate across scenarios.

**Delivers:**
- Batch updates use `updateContainers` (plural) mutation for efficiency
- Validation across Unraid versions (6.12, 7.0, 7.2)
- Confirmation that network config preservation works

**Tasks:**
- Modify batch update flow to collect all updated container IDs
- Call `updateContainers(ids: ["docker:container1", "docker:container2"])` once after batch loop completes
- Test on Unraid 6.12 (with Connect plugin) and 7.2 (native API)
- Verify no notification spam during batch updates (5+ containers)
- Test container on custom network (`br0`) — verify DNS resolution after update
- Document manual sync option in README (if GraphQL sync fails, user clicks "Check for Updates")

**Avoids:**
- Pitfall 5 (notification spam) — batch mutation prevents per-container notifications
- Pitfall 3 (version compatibility) — multi-version UAT catches breaking changes
- Pitfall 7 (network issues) — UAT includes network-dependent scenarios

**Research flag:** STANDARD PATTERNS — Batch optimization is iteration of Phase 2 pattern

### Phase Ordering Rationale

- **Infrastructure before integration** — Phase 1 validates GraphQL API access before modifying workflows. If network access fails, can pivot to alternative (e.g., helper script) without workflow rework.

- **Single-container before batch** — Phase 2 proves core sync mechanism with simplest case. Batch optimization (Phase 3) is safe iteration once foundation works.

- **Validation last** — Phase 3 UAT happens after implementation complete. Testing earlier wastes time if implementation changes.

**Dependencies discovered:**
- Phase 2 depends on Phase 1 (API access must work)
- Phase 3 depends on Phase 2 (batch builds on single-container pattern)
- No parallelization opportunities (sequential phases)

### Research Flags

Phases likely needing deeper research during planning:

- **Phase 1:** Network access testing, API permission verification — Some unknowns around container ID format and `host.docker.internal` behavior in Unraid's Docker implementation. Low risk (fallback to IP-based access), but needs validation.

Phases with standard patterns (skip research-phase):

- **Phase 2:** HTTP Request node integration — Well-documented n8n pattern, GraphQL mutation structure is simple
- **Phase 3:** Batch optimization and UAT — Iteration of Phase 2, no new concepts

## Confidence Assessment

| Area | Confidence | Notes |
|------|------------|-------|
| Stack | HIGH | GraphQL API is official Unraid interface, documented in schema and DeepWiki. HTTP Request node is n8n built-in. |
| Features | MEDIUM | Core pain point well-understood from community forums. Feature prioritization based on user impact analysis, but batch optimization impact is estimated. |
| Architecture | HIGH | Extension of existing sub-workflow pattern. GraphQL integration is standard HTTP Request usage. Rejected alternatives are well-reasoned. |
| Pitfalls | MEDIUM | Critical pitfalls identified from community reports and source code analysis. Race condition and version compatibility need UAT validation. |

**Overall confidence:** HIGH

The GraphQL API approach is well-documented and official. Core functionality (sync after single-container update) is low-risk. Batch optimization and multi-version support add complexity but are defer-able if needed.

### Gaps to Address

Areas where research was inconclusive or needs validation during implementation:

- **Container ID format** — GraphQL schema shows `id: ID!` but exact format (`docker:<name>` vs just `<name>`) needs testing. Resolution: Phase 1 queries containers to get actual ID format.

- **API rate limiting** — Not documented. Impact: LOW (bot updates are infrequent, <10/min even in batch). Resolution: Monitor during UAT, no preemptive handling.

- **GraphQL subscription for real-time status** — Schema includes subscriptions but unclear if Docker status changes are subscribable. Impact: None for v1.3 (defer to future enhancement). Resolution: Document as v2+ exploration.

- **Unraid 6.9-7.1 Connect plugin availability** — Research confirms Connect plugin exists but installation process not validated. Impact: MEDIUM (users on pre-7.2 need this). Resolution: Phase 1 should document plugin installation steps and test.

- **Exact permission name for API key** — Research shows `DOCKER:UPDATE_ANY` but this wasn't explicitly stated in official docs (inferred from schema). Impact: LOW (easily testable). Resolution: Phase 1 tests API key creation and documents exact permission syntax.

## Sources

### Primary (HIGH confidence)
- [Unraid API Schema](https://raw.githubusercontent.com/unraid/api/main/api/generated-schema.graphql) — GraphQL mutations, container ID types, query structure
- [DeepWiki — Docker Integration](https://deepwiki.com/unraid/api/2.4-docker-integration) — DockerService architecture, updateContainer mutation behavior
- [limetech/dynamix source](https://github.com/limetech/dynamix/blob/master/plugins/dynamix.docker.manager/include/DockerClient.php) — syncVersions() function, update status file handling
- [Unraid API Documentation](https://docs.unraid.net/API/) — API key management, version availability
- [Unraid API Key Management](https://docs.unraid.net/API/programmatic-api-key-management/) — API key creation, permission scopes

### Secondary (MEDIUM confidence)
- [Unraid Forum: Incorrect Update Notification](https://forums.unraid.net/bug-reports/stable-releases/regression-incorrect-docker-update-notification-r2807/) — Community-identified pain point, file deletion workaround
- [Watchtower + Unraid Discussion](https://github.com/containrrr/watchtower/discussions/1389) — External update detection issues, Unraid doesn't auto-sync
- [Unraid Forum: Watchtower Status Not Reflected](https://forums.unraid.net/topic/149953-docker-update-via-watchtower-status-not-reflected-in-unraid/) — Confirmation of stale status after external updates
- [Unraid MCP Server](https://github.com/jmagar/unraid-mcp) — Reference GraphQL client implementation
- [Home Assistant Unraid Integration](https://github.com/domalab/unraid-api-client) — Additional GraphQL usage examples
- [Docker host.docker.internal guide](https://eastondev.com/blog/en/posts/dev/20251217-docker-host-access/) — Network access pattern

### Tertiary (LOW confidence)
- [Unraid Forums — Docker Update Status](https://forums.unraid.net/topic/114415-plugin-docker-compose-manager/page/9/) — Status file structure (observed, not documented)
- Community reports of notification spam — Anecdotal, but consistent pattern across forums

---
*Research completed: 2026-02-08*
*Ready for roadmap: yes*