diff --git a/.planning/MILESTONES.md b/.planning/MILESTONES.md index af92c9a..3e8a910 100644 --- a/.planning/MILESTONES.md +++ b/.planning/MILESTONES.md @@ -1,5 +1,33 @@ # Project Milestones: Unraid Docker Manager +## v1.2 Modularization & Polish (Shipped: 2026-02-08) + +**Delivered:** Modular sub-workflow architecture with "Update All" functionality — 7 domain sub-workflows, bitmap-encoded batch selection, and comprehensive documentation. + +**Phases completed:** 10-13 + 10.1, 10.2 (6 phases, 25 plans) + +**Key accomplishments:** +- Decomposed monolithic workflow into 7 domain sub-workflows (Update, Actions, Logs, Batch UI, Status, Confirmation, Matching) — 287 total nodes +- Bitmap-encoded batch selection eliminating Telegram's 64-byte callback limit (supports 50+ containers) +- "Update All :latest" via text command and inline keyboard with infrastructure container exclusion +- Correlation ID tracking for request tracing across sub-workflow boundaries +- Comprehensive documentation overhaul (README with architecture, configuration, troubleshooting) +- 9 UAT bugs fixed during Update All verification including infrastructure self-destruction protection + +**Stats:** +- 96 files modified (+51,319/-4,862 lines) +- 10,987 lines across 8 workflow JSON files +- 6 phases, 25 plans, 153 commits +- 4 days (2026-02-04 → 2026-02-08) + +**Git range:** v1.1 → `0471565` + +**Tech debt accepted:** 4 non-blocking items (descoped logging features, 3 orphan nodes, legacy batch parsers, missing Phase 12 verification) + +**What's next:** v2.0 with resource monitoring or proactive notifications. + +--- + ## v1.1 n8n Integration & Polish (Shipped: 2026-02-04) **Delivered:** Inline keyboard UX and Docker security hardening — button-driven container control with filtered socket proxy. @@ -47,6 +75,4 @@ **Git range:** Initial commit → `e5c02f9` -**What's next:** Consider v1.1 with n8n API integration for faster development iteration, or additional features like resource monitoring. - --- diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md index bd6710b..dc3cd56 100644 --- a/.planning/PROJECT.md +++ b/.planning/PROJECT.md @@ -2,7 +2,7 @@ ## What This Is -A Telegram bot that lets you manage Docker containers on your Unraid server via inline keyboard buttons and text commands. Control containers from your phone without needing to open a laptop — check status, view logs, start/stop/restart/update containers, batch operations on multiple containers at once. +A Telegram bot that lets you manage Docker containers on your Unraid server via inline keyboard buttons and text commands. Built on a modular n8n sub-workflow architecture with 7 domain-specific sub-workflows. Control containers from your phone — check status, view logs, start/stop/restart/update containers, batch operations, and update all :latest containers at once. ## Core Value @@ -30,16 +30,23 @@ When you get a container update notification or notice a service is down, you ca - ✓ Confirmation dialogs for dangerous actions (stop, update) — v1.1 - ✓ Progress feedback during operations (message edits) — v1.1 +**v1.2:** +- ✓ Workflow modularization into 7 domain sub-workflows — v1.2 +- ✓ Sub-workflows callable from main without code duplication — v1.2 +- ✓ Update all :latest containers via text command ("update all") — v1.2 +- ✓ Update all :latest containers via inline keyboard button — v1.2 +- ✓ Bitmap-encoded batch selection (supports 50+ containers, eliminates 64-byte limit) — v1.2 +- ✓ Batch selection supports containers with long names — v1.2 +- ✓ Unraid update badge documented as known limitation — v1.2 +- ✓ Environment variable documentation (TELEGRAM_USERID, BOT_TOKEN) — v1.2 +- ✓ README documents proxy architecture and all v1.2 features — v1.2 +- ✓ Duplicate --max-time flags fixed — v1.2 +- ✓ Update flow consolidated (no duplicate logic) — v1.2 +- ✓ Correlation ID tracking across sub-workflow boundaries — v1.2 + ### Active -**Milestone v1.2: Modularization & Polish** - -- [ ] Workflow modularization (break main workflow into sub-workflows) -- [ ] Update all containers via text command and inline keyboard -- [ ] Fix callback data limits (batch selection >2 containers, long names) -- [ ] Unraid update sync (clear "update available" when bot updates container) -- [ ] Environment variable audit (verify TELEGRAM_USERID/BOT_TOKEN necessity) -- [ ] Documentation update (proxy architecture, new features) +(No active requirements — define in next milestone) ### Out of Scope @@ -47,14 +54,16 @@ When you get a container update notification or notice a service is down, you ca - Deploying new containers — manage existing only, not create new ones - Natural language understanding — simple keyword matching sufficient, Claude API adds complexity - Proactive monitoring/notifications — bot is reactive (you ask, it answers) -- Resource queries — "what's using the most memory?" deferred to future version +- Offline mode — real-time Docker API access is core to functionality +- Ring buffer / persistent debug logging — n8n static data is execution-scoped, not workflow-scoped ## Current State -**Shipped:** v1.1 (2026-02-04) +**Shipped:** v1.2 (2026-02-08) **Tech stack:** n8n workflow + Telegram Bot API + Docker socket proxy -**Files:** n8n-workflow.json (~8,500 lines), README.md -**Architecture:** tecnativa/docker-socket-proxy filters Docker API access +**Architecture:** 1 main workflow (166 nodes) + 7 sub-workflows (121 nodes) = 287 total nodes +**Files:** 8 workflow JSON files (10,987 LOC), README.md (264 lines), DEPLOY-SUBWORKFLOWS.md (725 lines) +**Sub-workflows:** Update, Actions, Logs, Batch UI, Status, Confirmation, Matching ## Context @@ -70,7 +79,13 @@ When you get a container update notification or notice a service is down, you ca - Matching: Keyword/substring with exact-match priority - Auth: Single user via Telegram ID - Logs: Configurable line count, default 50, max 1000 -- Callback data: 64-byte Telegram limit (affects batch/selection features) +- Callback data: Bitmap encoding overcomes 64-byte Telegram limit +- n8n static data: Execution-scoped only (no persistent cross-execution state) + +**Known tech debt:** +- 3 orphan nodes in main workflow (legacy dead code, unreachable) +- Legacy batch parsers retained alongside bitmap parsers (graceful migration) +- Phase 10.2 logging features descoped (n8n platform limitation) ## Key Decisions @@ -92,18 +107,12 @@ When you get a container update notification or notice a service is down, you ca | Batch stop requires confirmation | Fuzzy matching risk for destructive operations | ✓ Good | | Two-phase batch execution | Callbacks have names but no IDs - need lookup | ✓ Good | | Update all filters to :latest | Performance optimization - full check would be slow | ✓ Good | - -## Current Milestone: v1.2 Modularization & Polish - -**Goal:** Modularize workflow for maintainability, add "update all" functionality, fix callback limits, and polish remaining issues. - -**Target features:** -- Workflow modularization into sub-workflows -- "Update all" command (text and inline keyboard) -- Callback data limit fix (batch selection, long names) -- Unraid badge sync (UNR-01) -- Environment audit (ENV-01, ENV-02) -- Documentation overhaul (proxy architecture) +| 7 domain sub-workflows | Clean boundaries: Update, Actions, Logs, Batch UI, Status, Confirmation, Matching | ✓ Good | +| Bitmap-encoded batch callbacks | Base36 BigInt supports 50+ containers in 64-byte limit | ✓ Good | +| Action-based sub-workflow routing | Sub-workflow returns action field, main routes to Telegram handlers | ✓ Good | +| Correlation IDs without persistent logging | Timestamp+random string traces requests; ring buffer non-viable on n8n | ✓ Good | +| Infrastructure container exclusion | Exclude n8n and socket-proxy from "update all" to prevent self-destruction | ✓ Good | +| Document Unraid badge as limitation | Unraid API integration adds complexity for cosmetic issue | ✓ Good | --- -*Last updated: 2026-02-04 after v1.1 milestone* +*Last updated: 2026-02-08 after v1.2 milestone* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 769da7a..09cdeff 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -2,187 +2,59 @@ ## Milestones -- **v1.0 Docker Control via Telegram** — Phases 1-5 (shipped 2026-02-02) -> [Archive](milestones/v1.0-ROADMAP.md) -- **v1.1 n8n Integration & Polish** — Phases 6-9 (shipped 2026-02-04) -> [Archive](milestones/v1.1-ROADMAP.md) -- **v1.2 Modularization & Polish** — Phases 10-13 + 10.1, 10.2 (planned) +- ✅ **v1.0 Docker Control via Telegram** — Phases 1-5 (shipped 2026-02-02) -> [Archive](milestones/v1.0-ROADMAP.md) +- ✅ **v1.1 n8n Integration & Polish** — Phases 6-9 (shipped 2026-02-04) -> [Archive](milestones/v1.1-ROADMAP.md) +- ✅ **v1.2 Modularization & Polish** — Phases 10-13 + 10.1, 10.2 (shipped 2026-02-08) -> [Archive](milestones/v1.2-ROADMAP.md) ---- +## Phases -## v1.2: Modularization & Polish +
+✅ v1.0 Docker Control via Telegram (Phases 1-5) — SHIPPED 2026-02-02 -Modularize the workflow for maintainability, add "update all" functionality, fix callback data limits, and polish remaining issues. +- [x] Phase 1: Foundation (2/2 plans) +- [x] Phase 2: Container Actions (2/2 plans) +- [x] Phase 3: Core Operations (4/4 plans) +- [x] Phase 4: Container Logs (1/1 plan) +- [x] Phase 5: Polish & Ship (3/3 plans) -### Phase 10: Workflow Modularization +
-**Goal:** Break main workflow into modular sub-workflows for maintainability +
+✅ v1.1 n8n Integration & Polish (Phases 6-9) — SHIPPED 2026-02-04 -**Dependencies:** None +- [x] Phase 6: n8n API Access (1/1 plan) +- [x] Phase 7: Socket Security (3/3 plans) +- [x] Phase 8: Inline Keyboard Infrastructure (3/3 plans) +- [x] Phase 9: Batch Operations (4/4 plans) -**Requirements:** MOD-01, MOD-02, DEBT-03 +
-**Plans:** 7 plans +
+✅ v1.2 Modularization & Polish (Phases 10-13 + 10.1, 10.2) — SHIPPED 2026-02-08 -Plans: -- [x] 10-01-PLAN.md — Orphan node cleanup (removed 2 orphan nodes) -- [x] 10-02-PLAN.md — Extract container update sub-workflow (consolidates DEBT-03) -- [x] 10-03-PLAN.md — Extract container actions sub-workflow (start/stop/restart) -- [x] 10-04-PLAN.md — Integration verification and user checkpoint -- [x] 10-05-PLAN.md — Complete modularization (batch operations, logs sub-workflow) -- [x] 10-06-PLAN.md — Remediation: fix routing gaps, wire logs, cleanup Python scripts -- [x] 10-07-PLAN.md — UAT gap closure: race conditions, data chains, fuzzy matching, error handling +- [x] Phase 10: Workflow Modularization (7/7 plans) +- [x] Phase 10.1: Aggressive Workflow Modularization (9/9 plans) (INSERTED) +- [x] Phase 10.2: Better Logging & Log Management (4/4 plans) (INSERTED) +- [x] Phase 11: Update All & Callback Limits (2/2 plans) +- [x] Phase 12: Polish & Audit (2/2 plans) +- [x] Phase 13: Documentation Overhaul (1/1 plan) -**Success Criteria:** -1. ✓ Workflow split into logical sub-workflows (update, actions, logs) -2. ✓ Sub-workflows callable from main without code duplication -3. ✓ Update flow consolidated between single and batch paths -4. ✓ Actions flow consolidated between single and batch paths -5. ✓ Main workflow reduced from 209 to 192 nodes (-8%) -6. ✓ All existing functionality still works after modularization - -**Note:** Deeper modularization (target 120-140 nodes) deferred to Phase 10.1 - ---- - -### Phase 10.1: Aggressive Workflow Modularization (INSERTED) - -**Goal:** Decompose main workflow to minimal trigger/auth/routing (~50-80 nodes) with domain sub-workflows - -**Dependencies:** Phase 10 (initial modularization complete) - -**Requirements:** MOD-03 (new) - -**Plans:** 9 plans - -Plans: -- [x] 10.1-01-PLAN.md — Rename sub-workflows, analyze domain boundaries, get user approval -- [x] 10.1-02-PLAN.md — Extract Batch UI sub-workflow (~50 nodes) -- [x] 10.1-03-PLAN.md — Extract Container Status sub-workflow (~10-15 nodes) -- [x] 10.1-04-PLAN.md — Extract Confirmation sub-workflow (~15-20 nodes) -- [x] 10.1-05-PLAN.md — Integration verification and UAT -- [x] 10.1-06-PLAN.md — Gap closure: Extract Matching/Disambiguation sub-workflow -- [x] 10.1-07-PLAN.md — Gap closure: Code node classification + contract documentation -- [x] 10.1-08-PLAN.md — UAT gap closure: Fix action result statusCode handling (n8n-actions.json) -- [x] 10.1-09-PLAN.md — UAT gap closure: Fix batch keyboard, cancel routing, /list command (n8n-workflow.json) - -**Success Criteria:** -1. Main workflow contains only: trigger, auth, keyword routing, sub-workflow dispatch -2. UX/Keyboard sub-workflow handles all batch selection UI and pagination -3. Container Status sub-workflow handles list and status display -4. Confirmation sub-workflow handles all confirmation dialogs -5. Main workflow reduced to ~50-80 nodes (from 192) -6. All sub-workflows have clean input/output contracts - ---- - -### Phase 10.2: Better Logging and Log Management (INSERTED) - -**Goal:** Add correlation ID tracking for request tracing across sub-workflow boundaries - -**Dependencies:** Phase 10.1 (aggressive modularization complete) - -**Requirements:** LOG-01 (error ring buffer), LOG-02 (sub-workflow error propagation), LOG-03 (debug commands), LOG-04 (debug mode tracing) - -**Plans:** 4 plans - -Plans: -- [x] 10.2-01-PLAN.md -- Error ring buffer foundation + hidden Telegram debug commands -- [x] 10.2-02-PLAN.md -- Sub-workflow error propagation + correlation ID tracking -- [x] 10.2-03-PLAN.md -- Debug mode tracing + deployment verification -- [x] 10.2-04-PLAN.md -- UAT gap closure: wire correlation ID generators and remove orphan nodes - -**Success Criteria:** (descoped — n8n static data does not persist between executions) -1. ~~Errors from sub-workflow failures automatically captured in ring buffer~~ (removed — platform limitation) -2. ~~/errors, /clear-errors, /debug, /trace hidden commands~~ (removed — platform limitation) -3. ✓ Correlation IDs trace single user requests across main + sub-workflow boundaries -4. ~~Debug mode captures sub-workflow I/O boundary data~~ (removed — platform limitation) -5. ✓ No regression to existing bot functionality after deployment -6. ✓ All 7 sub-workflows return structured error objects (success/false + error details) - -**Note:** n8n workflow static data is execution-scoped, not workflow-scoped. Ring buffer architecture not viable. Retained: correlation IDs, structured error returns, correlationId pass-through. - ---- - -### Phase 11: Update All & Callback Limits - -**Goal:** Add "update all" functionality and fix callback data limits for batch selection - -**Dependencies:** Phase 10 (modularization provides cleaner base for new features) - -**Requirements:** BATCH-04, BATCH-05, BATCH-06, BATCH-07 - -**Plans:** 2 plans - -Plans: -- [x] 11-01-PLAN.md — Bitmap-encoded batch selection (replaces CSV-in-callback to eliminate 64-byte limit) -- [x] 11-02-PLAN.md — Update All inline keyboard button, deployment, and UAT verification - -**Success Criteria:** -1. ✓ User can type "update all" to update all :latest containers with confirmation -2. ✓ User can tap "Update All" in inline keyboard to update all :latest containers -3. ✓ Batch selection keyboard allows selecting 5+ containers without hitting callback limit -4. ✓ Containers with long names (20+ chars) can be selected in batch keyboard - ---- - -### Phase 12: Polish & Audit - -**Goal:** Clear Unraid update badges, verify environment configuration, and fix remaining tech debt - -**Dependencies:** Phase 11 (features complete before polish) - -**Requirements:** UNR-01, ENV-01, ENV-02, DEBT-02 - -**Plans:** 2 plans - -Plans: -- [x] 12-01-PLAN.md — Documentation update (README, env vars, DEBT-02 verification) + Unraid badge investigation -- [x] 12-02-PLAN.md — Deferred UAT: Update All text command (BATCH-04) and inline keyboard (BATCH-05) - -**Success Criteria:** -1. ✓ Unraid update badge behavior documented as known limitation with workaround (bot bypasses Unraid template system) -2. ✓ Documentation clarifies whether TELEGRAM_USERID env var is required or can be hardcoded -3. ✓ Documentation clarifies whether TELEGRAM_BOT_TOKEN env var is required or if n8n credential suffices -4. ✓ Image pull command has single --max-time flag (600s) - ---- - -### Phase 13: Documentation Overhaul - -**Goal:** Update README and documentation to reflect current architecture and features - -**Dependencies:** Phase 12 (core features complete before documentation) - -**Requirements:** DEBT-01 - -**Plans:** 1 plan - -Plans: -- [x] 13-01-PLAN.md — README overhaul (architecture, configuration, troubleshooting, v1.2 features) + remove outdated DEPLOYMENT_GUIDE.md - -**Success Criteria:** -1. ✓ README documents docker-socket-proxy architecture (not direct socket mount) -2. ✓ README documents all v1.2 features (update all, batch selection improvements) -3. ✓ Setup instructions verified accurate for clean install - ---- +
## Progress -| Phase | Name | Milestone | Status | -|-------|------|-----------|--------| -| 1-5 | Foundation through Polish | v1.0 | Complete | -| 6 | n8n API Access | v1.1 | Complete | -| 7 | Socket Security | v1.1 | Complete | -| 8 | Inline Keyboard Infrastructure | v1.1 | Complete | -| 9 | Batch Operations | v1.1 | Complete | -| 10 | Workflow Modularization | v1.2 | Complete | -| 10.1 | Aggressive Workflow Modularization | v1.2 | Complete | -| 10.2 | Better Logging & Log Management | v1.2 | Complete (descoped) | -| 11 | Update All & Callback Limits | v1.2 | Complete | -| 12 | Polish & Audit | v1.2 | Complete | -| 13 | Documentation Overhaul | v1.2 | Complete | +| Phase | Name | Milestone | Plans Complete | Status | Completed | +|-------|------|-----------|----------------|--------|-----------| +| 1-5 | Foundation through Polish | v1.0 | 12/12 | Complete | 2026-02-02 | +| 6-9 | API, Security, Keyboard, Batch | v1.1 | 11/11 | Complete | 2026-02-04 | +| 10 | Workflow Modularization | v1.2 | 7/7 | Complete | 2026-02-05 | +| 10.1 | Aggressive Modularization | v1.2 | 9/9 | Complete | 2026-02-06 | +| 10.2 | Better Logging & Log Management | v1.2 | 4/4 | Complete | 2026-02-07 | +| 11 | Update All & Callback Limits | v1.2 | 2/2 | Complete | 2026-02-08 | +| 12 | Polish & Audit | v1.2 | 2/2 | Complete | 2026-02-08 | +| 13 | Documentation Overhaul | v1.2 | 1/1 | Complete | 2026-02-08 | -**v1.2 Coverage:** 12+ requirements mapped across 7 phases +**Total: 3 milestones shipped, 13 phases, 48 plans** --- -*Updated: 2026-02-08 — Phase 13 complete (1/1 plans, v1.2 milestone COMPLETE)* +*Updated: 2026-02-08 — v1.2 milestone SHIPPED* diff --git a/.planning/STATE.md b/.planning/STATE.md index c1fe744..a65f6be 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,11 +2,16 @@ ## Current Position -- **Milestone:** v1.2 -- Modularization & Polish -- **Phase:** 13 of 13 (Documentation Overhaul) -- **Plan:** 1 of 1 complete -- **Status:** Phase 13 COMPLETE (README overhaul, documentation consolidation) -- **Last activity:** 2026-02-08 -- Completed 13-01 (README architecture/config/troubleshooting, removed outdated DEPLOYMENT_GUIDE.md) +- **Milestone:** v1.2 SHIPPED +- **Status:** Milestone archived, tagged v1.2 +- **Last activity:** 2026-02-08 -- v1.2 milestone completed and archived + +## Project Reference + +See: .planning/PROJECT.md (updated 2026-02-08) + +**Core value:** When you get a container update notification or notice a service is down, you can immediately investigate and act from your phone. +**Current focus:** Planning next milestone ## Progress @@ -14,7 +19,7 @@ v1.0: [**********] 100% SHIPPED v1.1: [**********] 100% SHIPPED -v1.2: [**********] 100% COMPLETE +v1.2: [**********] 100% SHIPPED Phase 10: Workflow Modularization [**********] 100% COMPLETE (+ 10-07 UAT fixes) Phase 10.1: Aggressive Modularization [**********] 100% COMPLETE (9/9 plans + UAT closure) diff --git a/.planning/milestones/v1.2-MILESTONE-AUDIT.md b/.planning/milestones/v1.2-MILESTONE-AUDIT.md new file mode 100644 index 0000000..fbff03b --- /dev/null +++ b/.planning/milestones/v1.2-MILESTONE-AUDIT.md @@ -0,0 +1,146 @@ +--- +milestone: v1.2 +audited: 2026-02-08 +status: tech_debt +scores: + requirements: 12/12 + phases: 6/6 + integration: 7/7 + flows: 6/6 +gaps: + requirements: [] + integration: [] + flows: [] +tech_debt: + - phase: 10.2-better-logging-and-log-management + items: + - "Descoped: ring buffer, debug commands, trace logging (n8n static data platform limitation)" + - "Only correlation IDs + structured errors retained from original plan" + - phase: 10-workflow-modularization + items: + - "3 orphan nodes remain: Build Action Command, Build Immediate Action Command, Prepare Cancel Return (legacy dead code, unreachable)" + - phase: 12-polish-audit + items: + - "Missing VERIFICATION.md (phase has SUMMARY.md files, all requirements closed, but no formal verification report)" + - phase: 11-update-all-callback-limits + items: + - "Old batch parsers retained for graceful migration (batch:toggle:, batch:nav:, batch:exec: formats)" +--- + +# Milestone v1.2 Audit Report: Modularization & Polish + +**Audited:** 2026-02-08 +**Status:** tech_debt (all requirements met, no critical blockers, accumulated non-critical debt) + +## Requirements Coverage + +| Requirement | Description | Phase | Status | +|-------------|-------------|-------|--------| +| MOD-01 | Main workflow broken into logical sub-workflows | 10/10.1 | ✓ SATISFIED | +| MOD-02 | Sub-workflows callable without duplication | 10/10.1 | ✓ SATISFIED | +| BATCH-04 | Update all via text command | 11/12 | ✓ SATISFIED | +| BATCH-05 | Update all via inline keyboard | 11/12 | ✓ SATISFIED | +| BATCH-06 | Batch selection 5+ containers | 11 | ✓ SATISFIED | +| BATCH-07 | Long container names in batch | 11 | ✓ SATISFIED | +| UNR-01 | Unraid update badge | 12 | ✓ SATISFIED (documented limitation) | +| ENV-01 | TELEGRAM_USERID documentation | 12 | ✓ SATISFIED | +| ENV-02 | TELEGRAM_BOT_TOKEN documentation | 12 | ✓ SATISFIED | +| DEBT-01 | README documents proxy architecture | 12 | ✓ SATISFIED | +| DEBT-02 | Fix duplicate --max-time flags | 12 | ✓ SATISFIED (verified fixed) | +| DEBT-03 | Consolidate duplicate update flow | 10 | ✓ SATISFIED | + +**Score: 12/12 requirements satisfied** + +## Phase Verification + +| Phase | Name | Plans | Verification | Status | +|-------|------|-------|-------------|--------| +| 10 | Workflow Modularization | 7/7 | 10-VERIFICATION.md (passed, 6/6) | ✓ Complete | +| 10.1 | Aggressive Modularization | 9/9 | 10.1-VERIFICATION.md (passed, 16/16) | ✓ Complete | +| 10.2 | Better Logging & Log Management | 4/4 | 10.2-VERIFICATION.md (passed, 4/4, descoped) | ✓ Complete | +| 11 | Update All & Callback Limits | 2/2 | 11-VERIFICATION.md (human_needed 7/9, completed in 12-02) | ✓ Complete | +| 12 | Polish & Audit | 2/2 | No VERIFICATION.md (all requirements closed via SUMMARY.md) | ✓ Complete | +| 13 | Documentation Overhaul | 1/1 | 13-VERIFICATION.md (passed, 7/7) | ✓ Complete | + +**Score: 6/6 phases complete (25/25 plans executed)** + +## Cross-Phase Integration + +| Connection | From | To | Status | +|------------|------|-----|--------| +| Sub-workflow wiring | Phase 10/10.1 | All phases | ✓ 17 Execute Workflow nodes properly connected | +| Correlation IDs | Phase 10.2 | All sub-workflows | ✓ 19 Prepare Input nodes pass correlationId | +| Bitmap encoding | Phase 11 | Batch UI sub-workflow | ✓ Base36 BigInt encoding integrated | +| Update All button | Phase 11 | Status sub-workflow | ✓ uall:start callback wired | +| UAT completion | Phase 11 → 12 | BATCH-04/BATCH-05 | ✓ Deferred UAT completed, 9 bugs fixed | +| Documentation chain | Phase 12 → 13 | README/DEPLOY-SUBWORKFLOWS | ✓ Architecture, config, troubleshooting sections | +| Sub-workflow cross-call | Confirmation | Actions | ✓ Confirmed stop actions execute via n8n-actions.json | + +**Score: 7/7 integrations verified** + +## E2E User Flows + +| Flow | Path | Status | +|------|------|--------| +| Text status | User → auth → correlation ID → Keyword Router → Status sub-workflow → Telegram | ✓ Complete | +| Callback routing | User → auth → correlation ID → Parse Callback → Route → sub-workflow → Telegram | ✓ Complete | +| Update All (text) | "update all" → Get Containers → Filter Infra → Confirmation → Batch Loop → Summary | ✓ Complete | +| Update All (keyboard) | uall:start → Answer → Get Containers → Confirmation → Batch Loop → Summary | ✓ Complete | +| Batch selection | Batch mode → Batch UI (bitmap) → Toggle → Execute → batch loop → Summary | ✓ Complete | +| Logs command | "logs X" → Matching sub-workflow → Logs sub-workflow → Send Logs Response | ✓ Complete | + +**Score: 6/6 flows verified** + +## Tech Debt + +### Phase 10.2: Descoped Features +- **Ring buffer, debug commands, trace logging** — removed due to n8n static data platform limitation (execution-scoped, not workflow-scoped) +- Only correlation IDs + structured error returns retained +- Impact: Manual debugging via n8n UI only (no Telegram debug commands) + +### Phase 10: Orphan Nodes (3) +- **Build Action Command, Build Immediate Action Command, Prepare Cancel Return** — legacy dead code, unreachable from any user interaction +- Pre-modularization inline action execution paths +- Impact: None (dead code, 3 nodes in 166-node workflow) + +### Phase 12: Missing VERIFICATION.md +- Phase completed successfully (12-01, 12-02 SUMMARY.md files exist, all requirements closed) +- No formal verification report was generated +- Impact: Minor documentation gap, no functional impact + +### Phase 11: Legacy Batch Parsers +- Old `batch:toggle:`, `batch:nav:`, `batch:exec:` parsers retained alongside new bitmap `b:`, `bn:`, `be:` parsers +- Intended for graceful migration of in-flight messages (30-second window) +- Impact: Minor code bloat, no functional impact + +**Total: 4 tech debt items across 4 phases (none blocking)** + +## Architecture Summary + +``` +Telegram Bot → Main Workflow (166 nodes) + ├── n8n-update.json (34 nodes) — Container Update + ├── n8n-actions.json (11 nodes) — Start/Stop/Restart + ├── n8n-logs.json (9 nodes) — Container Logs + ├── n8n-batch-ui.json (17 nodes) — Batch Selection UI + ├── n8n-status.json (11 nodes) — Container Status/List + ├── n8n-confirmation.json (16 nodes) — Confirmation Dialogs + └── n8n-matching.json (23 nodes) — Container Matching + ↓ + docker-socket-proxy + ↓ + Docker Engine +``` + +**Total system nodes:** 287 (166 main + 121 sub-workflows) +**Documentation:** README.md (264 lines), DEPLOY-SUBWORKFLOWS.md (725 lines) + +## Conclusion + +Milestone v1.2 has met all 12 requirements with no critical gaps. Cross-phase integration is solid across all 7 sub-workflows. All 6 E2E user flows verified. The 4 tech debt items are non-blocking and can be tracked in backlog. + +**Recommendation:** Proceed to milestone completion (`/gsd:complete-milestone`). + +--- +*Audited: 2026-02-08* +*Auditor: Claude (milestone audit workflow)* diff --git a/.planning/REQUIREMENTS.md b/.planning/milestones/v1.2-REQUIREMENTS.md similarity index 96% rename from .planning/REQUIREMENTS.md rename to .planning/milestones/v1.2-REQUIREMENTS.md index 6b7cb0d..4369c6d 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/milestones/v1.2-REQUIREMENTS.md @@ -1,3 +1,12 @@ +# Requirements Archive: v1.2 Modularization & Polish + +**Archived:** 2026-02-08 +**Status:** SHIPPED + +For current requirements, see `.planning/REQUIREMENTS.md`. + +--- + # Requirements: Unraid Docker Manager **Defined:** 2026-02-04 diff --git a/.planning/milestones/v1.2-ROADMAP.md b/.planning/milestones/v1.2-ROADMAP.md new file mode 100644 index 0000000..769da7a --- /dev/null +++ b/.planning/milestones/v1.2-ROADMAP.md @@ -0,0 +1,188 @@ +# Roadmap — Unraid Docker Manager + +## Milestones + +- **v1.0 Docker Control via Telegram** — Phases 1-5 (shipped 2026-02-02) -> [Archive](milestones/v1.0-ROADMAP.md) +- **v1.1 n8n Integration & Polish** — Phases 6-9 (shipped 2026-02-04) -> [Archive](milestones/v1.1-ROADMAP.md) +- **v1.2 Modularization & Polish** — Phases 10-13 + 10.1, 10.2 (planned) + +--- + +## v1.2: Modularization & Polish + +Modularize the workflow for maintainability, add "update all" functionality, fix callback data limits, and polish remaining issues. + +### Phase 10: Workflow Modularization + +**Goal:** Break main workflow into modular sub-workflows for maintainability + +**Dependencies:** None + +**Requirements:** MOD-01, MOD-02, DEBT-03 + +**Plans:** 7 plans + +Plans: +- [x] 10-01-PLAN.md — Orphan node cleanup (removed 2 orphan nodes) +- [x] 10-02-PLAN.md — Extract container update sub-workflow (consolidates DEBT-03) +- [x] 10-03-PLAN.md — Extract container actions sub-workflow (start/stop/restart) +- [x] 10-04-PLAN.md — Integration verification and user checkpoint +- [x] 10-05-PLAN.md — Complete modularization (batch operations, logs sub-workflow) +- [x] 10-06-PLAN.md — Remediation: fix routing gaps, wire logs, cleanup Python scripts +- [x] 10-07-PLAN.md — UAT gap closure: race conditions, data chains, fuzzy matching, error handling + +**Success Criteria:** +1. ✓ Workflow split into logical sub-workflows (update, actions, logs) +2. ✓ Sub-workflows callable from main without code duplication +3. ✓ Update flow consolidated between single and batch paths +4. ✓ Actions flow consolidated between single and batch paths +5. ✓ Main workflow reduced from 209 to 192 nodes (-8%) +6. ✓ All existing functionality still works after modularization + +**Note:** Deeper modularization (target 120-140 nodes) deferred to Phase 10.1 + +--- + +### Phase 10.1: Aggressive Workflow Modularization (INSERTED) + +**Goal:** Decompose main workflow to minimal trigger/auth/routing (~50-80 nodes) with domain sub-workflows + +**Dependencies:** Phase 10 (initial modularization complete) + +**Requirements:** MOD-03 (new) + +**Plans:** 9 plans + +Plans: +- [x] 10.1-01-PLAN.md — Rename sub-workflows, analyze domain boundaries, get user approval +- [x] 10.1-02-PLAN.md — Extract Batch UI sub-workflow (~50 nodes) +- [x] 10.1-03-PLAN.md — Extract Container Status sub-workflow (~10-15 nodes) +- [x] 10.1-04-PLAN.md — Extract Confirmation sub-workflow (~15-20 nodes) +- [x] 10.1-05-PLAN.md — Integration verification and UAT +- [x] 10.1-06-PLAN.md — Gap closure: Extract Matching/Disambiguation sub-workflow +- [x] 10.1-07-PLAN.md — Gap closure: Code node classification + contract documentation +- [x] 10.1-08-PLAN.md — UAT gap closure: Fix action result statusCode handling (n8n-actions.json) +- [x] 10.1-09-PLAN.md — UAT gap closure: Fix batch keyboard, cancel routing, /list command (n8n-workflow.json) + +**Success Criteria:** +1. Main workflow contains only: trigger, auth, keyword routing, sub-workflow dispatch +2. UX/Keyboard sub-workflow handles all batch selection UI and pagination +3. Container Status sub-workflow handles list and status display +4. Confirmation sub-workflow handles all confirmation dialogs +5. Main workflow reduced to ~50-80 nodes (from 192) +6. All sub-workflows have clean input/output contracts + +--- + +### Phase 10.2: Better Logging and Log Management (INSERTED) + +**Goal:** Add correlation ID tracking for request tracing across sub-workflow boundaries + +**Dependencies:** Phase 10.1 (aggressive modularization complete) + +**Requirements:** LOG-01 (error ring buffer), LOG-02 (sub-workflow error propagation), LOG-03 (debug commands), LOG-04 (debug mode tracing) + +**Plans:** 4 plans + +Plans: +- [x] 10.2-01-PLAN.md -- Error ring buffer foundation + hidden Telegram debug commands +- [x] 10.2-02-PLAN.md -- Sub-workflow error propagation + correlation ID tracking +- [x] 10.2-03-PLAN.md -- Debug mode tracing + deployment verification +- [x] 10.2-04-PLAN.md -- UAT gap closure: wire correlation ID generators and remove orphan nodes + +**Success Criteria:** (descoped — n8n static data does not persist between executions) +1. ~~Errors from sub-workflow failures automatically captured in ring buffer~~ (removed — platform limitation) +2. ~~/errors, /clear-errors, /debug, /trace hidden commands~~ (removed — platform limitation) +3. ✓ Correlation IDs trace single user requests across main + sub-workflow boundaries +4. ~~Debug mode captures sub-workflow I/O boundary data~~ (removed — platform limitation) +5. ✓ No regression to existing bot functionality after deployment +6. ✓ All 7 sub-workflows return structured error objects (success/false + error details) + +**Note:** n8n workflow static data is execution-scoped, not workflow-scoped. Ring buffer architecture not viable. Retained: correlation IDs, structured error returns, correlationId pass-through. + +--- + +### Phase 11: Update All & Callback Limits + +**Goal:** Add "update all" functionality and fix callback data limits for batch selection + +**Dependencies:** Phase 10 (modularization provides cleaner base for new features) + +**Requirements:** BATCH-04, BATCH-05, BATCH-06, BATCH-07 + +**Plans:** 2 plans + +Plans: +- [x] 11-01-PLAN.md — Bitmap-encoded batch selection (replaces CSV-in-callback to eliminate 64-byte limit) +- [x] 11-02-PLAN.md — Update All inline keyboard button, deployment, and UAT verification + +**Success Criteria:** +1. ✓ User can type "update all" to update all :latest containers with confirmation +2. ✓ User can tap "Update All" in inline keyboard to update all :latest containers +3. ✓ Batch selection keyboard allows selecting 5+ containers without hitting callback limit +4. ✓ Containers with long names (20+ chars) can be selected in batch keyboard + +--- + +### Phase 12: Polish & Audit + +**Goal:** Clear Unraid update badges, verify environment configuration, and fix remaining tech debt + +**Dependencies:** Phase 11 (features complete before polish) + +**Requirements:** UNR-01, ENV-01, ENV-02, DEBT-02 + +**Plans:** 2 plans + +Plans: +- [x] 12-01-PLAN.md — Documentation update (README, env vars, DEBT-02 verification) + Unraid badge investigation +- [x] 12-02-PLAN.md — Deferred UAT: Update All text command (BATCH-04) and inline keyboard (BATCH-05) + +**Success Criteria:** +1. ✓ Unraid update badge behavior documented as known limitation with workaround (bot bypasses Unraid template system) +2. ✓ Documentation clarifies whether TELEGRAM_USERID env var is required or can be hardcoded +3. ✓ Documentation clarifies whether TELEGRAM_BOT_TOKEN env var is required or if n8n credential suffices +4. ✓ Image pull command has single --max-time flag (600s) + +--- + +### Phase 13: Documentation Overhaul + +**Goal:** Update README and documentation to reflect current architecture and features + +**Dependencies:** Phase 12 (core features complete before documentation) + +**Requirements:** DEBT-01 + +**Plans:** 1 plan + +Plans: +- [x] 13-01-PLAN.md — README overhaul (architecture, configuration, troubleshooting, v1.2 features) + remove outdated DEPLOYMENT_GUIDE.md + +**Success Criteria:** +1. ✓ README documents docker-socket-proxy architecture (not direct socket mount) +2. ✓ README documents all v1.2 features (update all, batch selection improvements) +3. ✓ Setup instructions verified accurate for clean install + +--- + +## Progress + +| Phase | Name | Milestone | Status | +|-------|------|-----------|--------| +| 1-5 | Foundation through Polish | v1.0 | Complete | +| 6 | n8n API Access | v1.1 | Complete | +| 7 | Socket Security | v1.1 | Complete | +| 8 | Inline Keyboard Infrastructure | v1.1 | Complete | +| 9 | Batch Operations | v1.1 | Complete | +| 10 | Workflow Modularization | v1.2 | Complete | +| 10.1 | Aggressive Workflow Modularization | v1.2 | Complete | +| 10.2 | Better Logging & Log Management | v1.2 | Complete (descoped) | +| 11 | Update All & Callback Limits | v1.2 | Complete | +| 12 | Polish & Audit | v1.2 | Complete | +| 13 | Documentation Overhaul | v1.2 | Complete | + +**v1.2 Coverage:** 12+ requirements mapped across 7 phases + +--- +*Updated: 2026-02-08 — Phase 13 complete (1/1 plans, v1.2 milestone COMPLETE)*