docs(10.2-01): complete error ring buffer foundation plan

- Create 10.2-01-SUMMARY.md with full plan execution details
- Update STATE.md: Phase 10.2 plan 1 of 3 complete (33% progress)
- Document all technical decisions and architecture patterns
- Self-check verification passed: all files and commits present
- Duration: 156 seconds (2.6 minutes)
- Node count: 168 -> 172 (+4 nodes: 2 command, 2 utility)
- Ring buffer infrastructure ready for Plan 02 wiring
This commit is contained in:
Lucas Berger
2026-02-08 12:49:54 -05:00
parent 030118efb3
commit 6833641ad1
2 changed files with 236 additions and 10 deletions
+27 -10
View File
@@ -3,10 +3,10 @@
## Current Position ## Current Position
- **Milestone:** v1.2 -- Modularization & Polish - **Milestone:** v1.2 -- Modularization & Polish
- **Phase:** 10.1 of 13 (Aggressive Workflow Modularization) - **Phase:** 10.2 of 13 (Better Logging & Log Management)
- **Plan:** 9 of 9 complete - **Plan:** 1 of 3 complete
- **Status:** Phase 10.1 COMPLETE (all UAT gaps closed) - **Status:** Phase 10.2 IN PROGRESS (error ring buffer foundation complete)
- **Last activity:** 2026-02-08 -- Completed 10.1-09 (Data flow fixes: actionType, cancel return, /list command) - **Last activity:** 2026-02-08 -- Completed 10.2-01 (Error ring buffer foundation and hidden debug commands)
## Progress ## Progress
@@ -18,7 +18,7 @@ v1.2: [*******___] 70%
Phase 10: Workflow Modularization [**********] 100% COMPLETE (+ 10-07 UAT fixes) Phase 10: Workflow Modularization [**********] 100% COMPLETE (+ 10-07 UAT fixes)
Phase 10.1: Aggressive Modularization [**********] 100% COMPLETE (9/9 plans + UAT closure) Phase 10.1: Aggressive Modularization [**********] 100% COMPLETE (9/9 plans + UAT closure)
Phase 10.2: Better Logging & Log Management [ ] Pending (INSERTED) Phase 10.2: Better Logging & Log Management [***_______] 33% (1/3 plans complete)
Phase 11: Update All & Callback Limits [ ] Pending Phase 11: Update All & Callback Limits [ ] Pending
Phase 12: Polish & Audit [ ] Pending Phase 12: Polish & Audit [ ] Pending
Phase 13: Documentation Overhaul [ ] Pending Phase 13: Documentation Overhaul [ ] Pending
@@ -47,7 +47,7 @@ Phase 13: Documentation Overhaul [ ] Pending
## Key Artifacts ## Key Artifacts
- `n8n-workflow.json` -- Main workflow (168 nodes, down from 192) - `n8n-workflow.json` -- Main workflow (172 nodes after 10.2-01 logging infrastructure)
- `n8n-batch-ui.json` -- Batch UI sub-workflow (16 nodes) -- ID: `ZJhnGzJT26UUmW45` - `n8n-batch-ui.json` -- Batch UI sub-workflow (16 nodes) -- ID: `ZJhnGzJT26UUmW45`
- `n8n-status.json` -- Container Status sub-workflow (11 nodes) -- ID: `lqpg2CqesnKE2RJQ` - `n8n-status.json` -- Container Status sub-workflow (11 nodes) -- ID: `lqpg2CqesnKE2RJQ`
- `n8n-confirmation.json` -- Confirmation Dialogs sub-workflow (16 nodes) -- ID: `fZ1hu8eiovkCk08G` - `n8n-confirmation.json` -- Confirmation Dialogs sub-workflow (16 nodes) -- ID: `fZ1hu8eiovkCk08G`
@@ -119,6 +119,9 @@ Phase 13: Documentation Overhaul [ ] Pending
| 10.1-08 | HTTP 304 treated as success | Docker API returns 304 for already-in-state, better UX than error | | 10.1-08 | HTTP 304 treated as success | Docker API returns 304 for already-in-state, better UX than error |
| 10.1-09 | /list command as alias for status | Status command already provides list functionality; alias simpler than duplication | | 10.1-09 | /list command as alias for status | Status command already provides list functionality; alias simpler than duplication |
| 10.1-09 | Dynamic predecessor reference pattern | Use $input.item.json for nodes with multiple incoming paths | | 10.1-09 | Dynamic predecessor reference pattern | Use $input.item.json for nodes with multiple incoming paths |
- [Phase 10.2-01]: Ring buffer size set to 50 entries for both errors and traces
- [Phase 10.2-01]: Debug mode auto-disables after 100 executions to prevent performance impact
- [Phase 10.2-01]: All 4 debug commands use single unified code node for maintainability
## Phase 10.1 Progress ## Phase 10.1 Progress
@@ -160,16 +163,30 @@ All 7 sub-workflows deployed and operational:
- n8n-confirmation.json -- `fZ1hu8eiovkCk08G` - n8n-confirmation.json -- `fZ1hu8eiovkCk08G`
- n8n-matching.json -- `kL4BoI8ITSP9Oxek` - n8n-matching.json -- `kL4BoI8ITSP9Oxek`
## Phase 10.2 Progress
| Plan | Description | Status |
|------|-------------|--------|
| 10.2-01 | Error Ring Buffer Foundation and Hidden Debug Commands | Complete |
| 10.2-02 | Wire Error Logging to Main Workflow | Pending |
| 10.2-03 | Add Debug Tracing to Sub-workflow Boundaries | Pending |
**Achievements (10.2-01):**
- Ring buffer infrastructure in workflow static data (max 50 errors, 50 traces)
- 4 hidden debug commands: /errors, /clear-errors, /debug, /trace
- Process Debug Command unified handler node with HTML formatting
- Log Error utility node with field truncation and pass-through
- Log Trace utility node with debug mode toggle and auto-disable
- Main workflow: 168 -> 172 nodes (+4 nodes)
## Next Step ## Next Step
Phase 10.1 complete. All 9 plans executed, all UAT gaps closed. Ready for UAT re-test to verify fixes. Phase 10.2 in progress. Plan 01 complete (ring buffer foundation). Next: Plan 02 (wire error logging to main workflow error paths).
Proceed to Phase 10.2 (Better Logging & Log Management) or Phase 11 (Update All & Callback Limits).
## Session Continuity ## Session Continuity
Last session: 2026-02-08 Last session: 2026-02-08
Stopped at: Completed Phase 10.1 (all 9 plans complete) Stopped at: Completed 10.2-01-PLAN.md (Error ring buffer foundation and hidden debug commands)
Resume file: None Resume file: None
--- ---
@@ -0,0 +1,209 @@
---
phase: 10.2-better-logging-and-log-management
plan: 01
subsystem: logging-infrastructure
tags: [error-logging, debug-commands, ring-buffer, telegram-commands]
dependency_graph:
requires: []
provides: [error-ring-buffer, debug-toggle, trace-infrastructure, hidden-commands]
affects: [main-workflow]
tech_stack:
added: [workflow-static-data, ring-buffer-pattern]
patterns: [ring-buffer, structured-logging, command-routing]
key_files:
created: []
modified: [n8n-workflow.json]
decisions:
- "Ring buffer size set to 50 entries for both errors and traces"
- "Debug mode auto-disables after 100 executions to prevent performance impact"
- "Field truncation: error stack 500 chars, raw response 1000 chars"
- "All 4 debug commands use single unified code node for maintainability"
- "Commands use startsWith operator to prevent false matches with regular text"
metrics:
duration: 156
completed: 2026-02-08T17:47:57Z
---
# Phase 10.2 Plan 01: Error Ring Buffer Foundation and Hidden Debug Commands Summary
**Built error ring buffer infrastructure and hidden Telegram debug commands in the main workflow, establishing centralized error/trace storage using workflow static data and providing command-line interface for quick diagnostics.**
## Completed Tasks
### Task 1: Add hidden command routing and debug command processor
**Status:** Complete
**Commit:** daff5bc
Added 4 new keyword routing rules to the Keyword Router switch node for /errors, /clear-errors, /debug, and /trace commands. All use `startsWith` operator (not `contains`) to prevent false matches with regular text. Created unified Process Debug Command code node that handles all 4 commands in a single code block, implementing:
- Static data initialization (errorLog structure with debug, errors, traces)
- `/errors [N]` command: displays last N errors (default 5, max 50) with formatted output
- `/clear-errors` command: resets error ring buffer
- `/debug on|off|status` command: toggles debug mode, shows status
- `/trace <correlationId>` command: queries all entries matching a correlation ID
Added Send Debug Response Telegram node with HTML parse mode for formatted output. Positioned nodes at [1120, -200] and [1340, -200] to keep them visually grouped above existing menu path. Commands remain hidden (not listed in /start help menu).
**Key changes:**
- Keyword Router: 9 rules -> 13 rules (+4 hidden commands)
- New node: Process Debug Command (id: `code-process-debug-command`)
- New node: Send Debug Response (id: `telegram-send-debug-response`)
- Connections: Keyword Router -> Process Debug Command -> Send Debug Response
- Node count: 168 -> 170 (+2 nodes)
### Task 2: Add error logging utility function and ring buffer write helper
**Status:** Complete
**Commit:** 4c2194c
Created two standalone utility Code nodes that will be wired by Plan 02 and Plan 03:
**Log Error node** (id: `code-log-error`) - Centralized error logging entry point:
- Accepts structured error input (correlationId, workflow, node, operation, userMessage, errorMessage, httpCode, rawResponse, contextData)
- Initializes static data structure if missing
- Creates error entry with auto-incrementing ID (err_001, err_002, etc.)
- Implements ring buffer: max 50 entries, auto-rotates (shift oldest when full)
- Truncates large fields: error stack to 500 chars, raw response to 1000 chars
- Passes through all input data with _errorLogged flag for downstream nodes
- Positioned at [2600, -200] in utility area
**Log Trace node** (id: `code-log-trace`) - Debug mode trace logging:
- Checks debug.enabled flag before logging (passes through unchanged if off)
- Increments execution count and auto-disables at 100 executions
- Stores trace entries in separate ring buffer (max 50)
- Trace entry fields: id, correlationId, timestamp, executionId, event, workflow, node, data
- Supports two event types: "sub-workflow-call" and "callback-routing"
- Passes through all input data unchanged with _traceLogged flag
- Positioned at [2600, -400] in utility area
Both nodes are standalone (no incoming/outgoing connections) and ready for wiring in Plan 02.
**Key changes:**
- New node: Log Error (id: `code-log-error`)
- New node: Log Trace (id: `code-log-trace`)
- Node count: 170 -> 172 (+2 utility nodes)
## Technical Implementation
### Ring Buffer Pattern
Both error and trace buffers use simple array-based ring buffer with auto-rotation:
```javascript
buffer.push(entry);
if (buffer.length > MAX_SIZE) {
buffer.shift(); // Remove oldest
}
```
### Static Data Structure
```javascript
staticData.errorLog = {
debug: { enabled: false, executionCount: 0 },
errors: { buffer: [], nextId: 1, count: 0, lastCleared: ISO_DATE },
traces: { buffer: [], nextId: 1 }
}
```
### Command Routing
All 4 debug commands route through single Process Debug Command node:
- `/errors [N]` - Display recent errors (HTML formatted)
- `/clear-errors` - Reset error buffer
- `/debug on|off|status` - Toggle/query debug mode
- `/trace <correlationId>` - Query by correlation ID
### Auto-Disable Mechanism
Debug mode automatically disables after 100 executions to prevent:
- Performance impact from continuous tracing
- Ring buffer fill-up with trace noise
- Forgotten debug mode affecting production
## Deviations from Plan
None - plan executed exactly as written.
## Architecture Decisions
**1. Unified command handler**
Used single Process Debug Command code node for all 4 commands rather than separate nodes per command. This improves maintainability (single code block to update) and reduces node count.
**2. Ring buffer size: 50 entries**
Conservative size based on research recommendations. Prevents workflow static data size limits while providing sufficient history for debugging. Can be adjusted if needed.
**3. Field truncation**
Error stack: 500 chars, raw response: 1000 chars. Balances diagnostic value with static data size constraints.
**4. startsWith operator for keywords**
Used `startsWith` instead of `contains` to prevent false matches (e.g., user typing "debug the container" wouldn't trigger /debug command).
**5. HTML formatting for Telegram output**
Used `parse_mode: HTML` with `<b>`, `<pre>`, `<code>` tags for better readability in Telegram. Escapes `>` as `&gt;` to prevent HTML parsing errors.
## Success Criteria Met
- [x] Main workflow has 172 nodes (168 + 4 new)
- [x] Ring buffer infrastructure initialized in workflow static data
- [x] /errors, /clear-errors, /debug, /trace commands routed and handled
- [x] Log Error and Log Trace utility nodes ready for wiring
- [x] No regression to existing functionality
- [x] All new nodes use correct n8n typeVersion (2 for code, 1.2 for Telegram)
- [x] Commands remain hidden (not in /start menu)
## Self-Check
Running verification of created files and commits:
**Files created:**
```bash
# No new files - only modified n8n-workflow.json
```
**Files modified:**
```bash
$ [ -f "/home/luc/Projects/unraid-docker-manager/n8n-workflow.json" ] && echo "FOUND: n8n-workflow.json"
```
FOUND: n8n-workflow.json
**Commits created:**
```bash
$ git log --oneline --all | grep -q "daff5bc" && echo "FOUND: daff5bc"
$ git log --oneline --all | grep -q "4c2194c" && echo "FOUND: 4c2194c"
```
FOUND: daff5bc
FOUND: 4c2194c
**Node count verification:**
```bash
$ python3 -c "import json; wf=json.load(open('n8n-workflow.json')); print(f'Node count: {len(wf[\"nodes\"])}')"
```
Node count: 172
## Self-Check: PASSED
All files, commits, and node counts verified. Plan executed successfully.
## Next Steps
**Plan 02:** Wire error logging to main workflow error paths
- Connect Log Error node to sub-workflow error returns
- Add error context capture (sub-workflow input/output)
- Wire error propagation from sub-workflows to centralized storage
**Plan 03:** Add debug tracing to sub-workflow boundaries and callback routing
- Wire Log Trace node to sub-workflow call points (capture I/O)
- Add trace logging to callback routing decisions
- Test debug mode toggle and auto-disable behavior
## Metrics
- **Duration:** 156 seconds (2.6 minutes)
- **Tasks completed:** 2/2
- **Commits:** 2 (1 per task)
- **Files modified:** 1 (n8n-workflow.json)
- **Nodes added:** 4 (2 command handler nodes, 2 utility nodes)
- **Node count:** 168 -> 172 (+2.4%)
- **Rules added:** 4 (hidden command keywords)
- **Connections added:** 5 (4 keyword outputs + 1 command-to-response)
---
*Plan completed: 2026-02-08*
*Phase: 10.2-better-logging-and-log-management*
*Execution agent: Claude Sonnet 4.5*