diff --git a/.planning/phases/10.1-aggressive-workflow-modularization/10.1-RESEARCH.md b/.planning/phases/10.1-aggressive-workflow-modularization/10.1-RESEARCH.md new file mode 100644 index 0000000..7cf4949 --- /dev/null +++ b/.planning/phases/10.1-aggressive-workflow-modularization/10.1-RESEARCH.md @@ -0,0 +1,700 @@ +# Phase 10.1: Aggressive Workflow Modularization - Research + +**Researched:** 2026-02-04 +**Domain:** n8n workflow decomposition and sub-workflow architecture +**Confidence:** HIGH + +## Summary + +This research investigates best practices for decomposing large n8n workflows into modular sub-workflows. The primary goal is to reduce the main workflow from 192 nodes (~260KB) to 50-80 nodes by extracting domain-specific logic into dedicated sub-workflows while maintaining a thin orchestration layer. + +n8n's sub-workflow system uses the Execute Workflow node (in parent) paired with the Execute Workflow Trigger node (in sub-workflow). The platform's modularization best practices recommend 5-10 nodes per workflow for optimal maintainability, with sub-workflow conversion features built into the UI for refactoring existing workflows. Key architectural decisions include input/output contracts (field-based schemas vs passthrough), execution modes (once vs per-item), and error propagation strategies. + +Current state analysis reveals the main workflow has 192 nodes with heavy batch orchestration (48+ batch-related nodes), pagination/list management (10+ nodes), confirmation dialogs, and Telegram response handling (26 Telegram nodes). Existing sub-workflows successfully demonstrate the pattern: Container Update (34 nodes), Container Actions (11 nodes), and Container Logs (9 nodes) all use typed field schemas with common fields (chatId, messageId, responseMode) plus domain-specific fields. + +**Primary recommendation:** Extract cohesive domain workflows (8-15+ nodes) with typed input schemas following existing pattern (common fields + domain fields), keep main workflow as thin orchestrator handling only trigger/auth/routing/Telegram responses, and use grouped extraction with automated verification for safe migration. + +## Standard Stack + +n8n workflow modularization uses built-in nodes without external dependencies. + +### Core +| Component | Version | Purpose | Why Standard | +|-----------|---------|---------|--------------| +| Execute Workflow node | n8n-nodes-base.executeWorkflow | Calls sub-workflows from parent workflow | Built-in n8n node for sub-workflow invocation | +| Execute Workflow Trigger | n8n-nodes-base.executeWorkflowTrigger | Receives calls in sub-workflow | Built-in trigger for sub-workflow entry point | +| Switch node | n8n-nodes-base.switch v3.2 | Routes logic to different paths | Standard routing mechanism in n8n | +| Code node | n8n-nodes-base.code | Data transformation and validation | Standard for complex logic in n8n | + +### Supporting +| Component | Version | Purpose | When to Use | +|-----------|---------|---------|-------------| +| If node | n8n-nodes-base.if | Simple conditional routing | Binary decisions (2 paths) | +| Sub-workflow conversion | n8n UI feature | Converts selected nodes to sub-workflow | Initial extraction from large workflows | +| Error output paths | Built-in n8n feature | Node-level error handling | Graceful degradation within workflows | + +### Alternatives Considered +| Instead of | Could Use | Tradeoff | +|------------|-----------|----------| +| Execute Workflow (wait) | Execute Workflow (no wait) | Async execution faster but loses error propagation | +| Field-based schema | Accept all data | Less type safety but more flexible | +| Sub-workflows | Monolithic workflow | Simpler data flow but poor maintainability at scale | + +**Installation:** +No external dependencies. All features are built into n8n core nodes. + +## Architecture Patterns + +### Current State Analysis + +**Main Workflow:** 192 nodes, ~260KB JSON file +- Trigger + auth + routing: ~10-15 nodes +- Batch orchestration: ~48 nodes (Batch Keyboard, Confirmation, Loop, State, Summary, Clear, Nav, etc.) +- Pagination/List UI: ~10 nodes (Container List Keyboard, Paginated List, Edit List, etc.) +- Confirmation dialogs: ~10 nodes (Stop Confirmation, Update Confirmation, Expired handling) +- Telegram responses: 26 Telegram nodes (sendMessage, editMessage, answerCallbackQuery) +- Container operations routing: ~15 nodes (Parse commands, Route actions) +- Update operations: ~15 nodes (Parse Update Command, Match Count, Multiple, etc.) +- Logs operations: ~5 nodes (Parse Logs Command, Send Response) +- Docker API calls: ~20 nodes (List Containers, various HTTP requests) +- Error handling: 2 explicit error nodes (Send Docker Error, Send Update Error) +- Data transformation: 75+ Code nodes + +**Existing Sub-Workflows:** +- Container Update: 34 nodes, ~31KB (INPUT: containerId, containerName, chatId, messageId, responseMode) +- Container Actions: 11 nodes, ~13KB (INPUT: containerId, containerName, action, chatId, messageId, responseMode) +- Container Logs: 9 nodes, ~9KB (INPUT: containerId/containerName, lineCount, chatId, messageId, responseMode) + +**Locked Decisions from CONTEXT.md:** +- Main workflow keeps: trigger, auth, keyword routing, sub-workflow dispatch +- Main workflow sends ALL Telegram responses (sub-workflows return data only) +- Centralized error handling: sub-workflows return/throw errors, main catches and responds +- Grouped extraction: extract related domains together, verify as group +- File naming: `n8n-{domain}.json` pattern +- Rename existing: n8n-update.json, n8n-actions.json, n8n-logs.json + +### Recommended Project Structure + +``` +/ +├── n8n-workflow.json # Main orchestrator (~50-80 nodes) +├── n8n-update.json # Container update operations (renamed from n8n-container-update.json) +├── n8n-actions.json # Container actions (renamed from n8n-container-actions.json) +├── n8n-logs.json # Container logs (renamed from n8n-container-logs.json) +├── n8n-batch-ui.json # Batch selection UI and pagination (NEW - candidate) +├── n8n-container-status.json # Container list and status display (NEW - candidate) +├── n8n-confirmation.json # Confirmation dialogs (NEW - candidate) +└── n8n-telegram.json # Telegram response abstraction (NEW - candidate, user discretion) +``` + +### Pattern 1: Sub-workflow Input Contract (Field-Based Schema) + +**What:** Define typed input fields in Execute Workflow Trigger node for type safety and documentation. + +**When to use:** All sub-workflows (locked decision: common fields + domain fields pattern) + +**Example:** +```javascript +// Execute Workflow Trigger node configuration +{ + "parameters": { + "inputSource": "passthrough", + "schema": { + "schemaType": "fromFields", + "fields": [ + // Common fields (all sub-workflows) + { + "fieldName": "chatId", + "fieldType": "number" + }, + { + "fieldName": "messageId", + "fieldType": "number" + }, + { + "fieldName": "responseMode", + "fieldType": "string" + }, + // Domain-specific fields + { + "fieldName": "containerId", + "fieldType": "string" + }, + { + "fieldName": "action", + "fieldType": "string" + } + ] + } + } +} +``` + +**Source:** Existing pattern from n8n-container-update.json, n8n-container-actions.json + +### Pattern 2: Sub-workflow Invocation with Wait + +**What:** Execute sub-workflow synchronously, waiting for completion to handle response/errors. + +**When to use:** When parent needs sub-workflow result or error handling (locked decision for all sub-workflows) + +**Example:** +```javascript +// Execute Workflow node configuration +{ + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "mode": "list", + "value": "7AvTzLtKXM2hZTio92_mC" + }, + "mode": "once", + "options": { + "waitForSubWorkflow": true + } + } +} +``` + +**Source:** Existing pattern from n8n-workflow.json Execute Workflow nodes + +### Pattern 3: Data Return from Sub-workflow + +**What:** Sub-workflows return structured data, parent handles Telegram responses. + +**When to use:** All sub-workflows (locked decision: sub-workflows return data, main sends responses) + +**Example:** +```javascript +// Last node in sub-workflow (Code node) +return { + json: { + success: true, + message: "Container updated successfully", + containerId: containerId, + containerName: containerName, + // Additional result data + image: imageTag, + status: newStatus + } +}; + +// Parent workflow handles response +// Code node after Execute Workflow +const result = $json; +if (result.success) { + return { + json: { + chatId: result.chatId, + text: `✅ ${result.message}\nContainer: ${result.containerName}` + } + }; +} +``` + +**Source:** Design pattern from locked decisions, inferred from current responseMode pattern + +### Pattern 4: Sub-workflow Conversion (UI Feature) + +**What:** n8n's built-in feature to extract selected nodes into a new sub-workflow. + +**When to use:** Initial extraction of cohesive node groups from main workflow. + +**How it works:** +1. Select desired nodes on canvas (8-15+ nodes recommended) +2. Right-click canvas background → "Convert to sub-workflow" +3. n8n automatically: + - Creates new workflow with Execute Workflow Trigger + - Adds Execute Workflow node in parent + - Updates expressions referencing other nodes + - Adds parameters to trigger node +4. Manual work needed: + - Define type constraints for inputs (default: accept all types) + - Configure output fields in Edit Fields + - Test data flow and error handling + +**Source:** [n8n Sub-workflow conversion docs](https://docs.n8n.io/workflows/subworkflow-conversion/), [n8n Sub-workflows docs](https://docs.n8n.io/flow-logic/subworkflows/) + +### Pattern 5: Error Handling Centralization + +**What:** Sub-workflows return error data, parent workflow catches and sends Telegram error responses. + +**When to use:** All sub-workflows (locked decision) + +**Example:** +```javascript +// Sub-workflow error return +try { + // ... operation ... +} catch (error) { + return { + json: { + success: false, + error: true, + message: error.message, + containerId: containerId, + chatId: chatId, + messageId: messageId + } + }; +} + +// Parent workflow error handling +const result = $json; +if (result.error) { + return { + json: { + chatId: result.chatId, + text: `❌ Error: ${result.message}` + } + }; +} +``` + +**Source:** Design pattern from locked decisions (centralized error handling) + +### Pattern 6: Workflow ID Reference (TypeVersion 1.2 Requirement) + +**What:** Sub-workflow references use resource locator format with database source. + +**When to use:** All Execute Workflow nodes. + +**Example:** +```javascript +{ + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "mode": "list", + "value": "" // Assigned by n8n on import + } + } +} +``` + +**Source:** Existing pattern from STATE.md phase 10-05 decision, technical notes + +### Pattern 7: Batch Reuse Pattern + +**What:** Batch operations call single-container sub-workflows in a loop. + +**When to use:** Batch operations (locked decision: already working this way) + +**Example:** +```javascript +// Batch loop in main workflow or batch sub-workflow +// For each container in batch selection: +for (const container of selectedContainers) { + // Call single-container sub-workflow + executeWorkflow('n8n-actions.json', { + containerId: container.id, + containerName: container.name, + action: 'stop', + chatId: chatId, + messageId: 0, // No intermediate responses + responseMode: 'silent' + }); +} +// Aggregate results, send batch summary +``` + +**Source:** User-specified pattern from CONTEXT.md specific ideas + +### Anti-Patterns to Avoid + +- **Extracting 2-3 node groups:** Overhead not worth it, keep small utilities in parent (locked decision: 8-15+ node threshold) +- **Sub-workflow sends Telegram responses:** Violates centralized response pattern, breaks error handling (locked decision) +- **Accept all data without validation:** Type safety lost, harder to debug, no documentation of contract +- **Deeply nested sub-workflows:** Execution becomes harder to trace, increases memory overhead +- **Large data passing between workflows:** Memory consumption increases, risk of memory errors (use chunking) +- **Forgetting waitForSubWorkflow:** Parent continues without result, race conditions, error handling breaks + +## Don't Hand-Roll + +Problems that look simple but have existing solutions: + +| Problem | Don't Build | Use Instead | Why | +|---------|-------------|-------------|-----| +| Extracting nodes to sub-workflow | Manual JSON editing | n8n UI "Convert to sub-workflow" | Automatically updates expressions and creates proper structure | +| Sub-workflow type definitions | Untyped passthrough | Execute Workflow Trigger field-based schema | Provides type safety, documentation, and validation | +| Complex routing logic | Multiple nested If nodes | Switch node with named outputs | More maintainable, clearer routing paths | +| Error propagation | Custom error data structure | Built-in error output paths | Native n8n feature, works with error workflows | +| Memory optimization for large data | Custom batching | Split workflows into sub-workflows | n8n frees sub-workflow memory after completion | +| Workflow versioning | Manual file copies | Git integration with commit messages | Proper version history, team collaboration | + +**Key insight:** n8n provides powerful built-in features for modularization. Use the platform's native capabilities (sub-workflow conversion UI, error output paths, Switch routing) rather than building custom abstractions. + +## Common Pitfalls + +### Pitfall 1: Memory Issues from Large Workflows + +**What goes wrong:** Large workflows (150+ nodes) consume significant memory during execution, especially with many Code nodes and manual executions. Can lead to "out of memory" errors or slow performance. + +**Why it happens:** n8n keeps all node data in memory during execution. Manual executions double memory usage (copy for frontend). Code nodes increase consumption. Large workflows with many HTTP nodes compound the problem. + +**How to avoid:** +- Split into sub-workflows (target: no workflow exceeds 80-100 nodes) - locked decision +- Sub-workflows only hold data for current execution, memory freed after completion +- Minimize data passed between parent and sub-workflow +- Use "once" execution mode when processing multiple items +- Avoid manual executions on large workflows in production + +**Warning signs:** +- Workflow execution times increasing +- n8n worker memory usage climbing +- "Existing execution data is too large" errors +- Canvas becoming slow to render + +**Sources:** [n8n Memory-related errors](https://docs.n8n.io/hosting/scaling/memory-errors/), [N8N Performance Optimization](https://www.wednesday.is/writing-articles/n8n-performance-optimization-maximizing-workflow-efficiency), [n8n Performance for High-Volume Workflows](https://www.wednesday.is/writing-articles/n8n-performance-optimization-for-high-volume-workflows) + +### Pitfall 2: Breaking Data References During Extraction + +**What goes wrong:** When manually extracting nodes to sub-workflow, expressions like `$('Node Name').json.field` break because referenced nodes are in different workflows. + +**Why it happens:** n8n expressions reference nodes by name within the same workflow context. Sub-workflows can't access parent workflow node data. + +**How to avoid:** +- Use n8n's built-in "Convert to sub-workflow" feature - it automatically updates expressions +- If manual extraction needed: identify all data dependencies first, pass as inputs to sub-workflow +- Test data flow thoroughly after extraction +- Use Execute Workflow Trigger field schema to document required inputs + +**Warning signs:** +- Sub-workflow execution fails with "node not found" errors +- Undefined values in sub-workflow despite parent having data +- Expressions showing red in n8n editor + +**Sources:** [n8n Sub-workflow conversion docs](https://docs.n8n.io/workflows/subworkflow-conversion/), community forum discussions + +### Pitfall 3: Error Handling Gaps + +**What goes wrong:** Errors in sub-workflows don't propagate to parent, or error messages lost, or Telegram user sees no feedback. + +**Why it happens:** Sub-workflows are separate execution contexts. If parent doesn't check sub-workflow result or if sub-workflow crashes without returning data, errors are silent. + +**How to avoid:** +- Always use `waitForSubWorkflow: true` (locked decision for this project) +- Sub-workflows return structured error data: `{ success: false, error: true, message: "..." }` +- Parent checks result and handles errors with Telegram responses (locked decision: centralized error handling) +- Use error output paths for critical nodes within sub-workflows +- Consider error workflow for unhandled failures + +**Warning signs:** +- User receives no response when operations fail +- Workflow shows success but expected outcome didn't happen +- Errors visible in n8n logs but not communicated to user + +**Sources:** [n8n Error handling docs](https://docs.n8n.io/flow-logic/error-handling/), [Creating error workflows in n8n](https://blog.n8n.io/creating-error-workflows-in-n8n/) + +### Pitfall 4: Circular Dependencies + +**What goes wrong:** Workflow A calls Workflow B, which calls Workflow A, creating infinite loop or stack overflow. + +**Why it happens:** Poor domain boundary definition leads to bi-directional dependencies between workflows. + +**How to avoid:** +- Define clear hierarchy: orchestrator (main) → domain sub-workflows (update, actions, logs, etc.) +- Domain sub-workflows never call other domain sub-workflows +- If shared logic needed, extract to separate utility sub-workflow called by both +- Document dependencies explicitly + +**Warning signs:** +- Workflow execution never completes +- Stack overflow errors +- Confusing data flow diagrams + +**Sources:** General software architecture principles, n8n community best practices + +### Pitfall 5: Telegram Sub-workflow Handoff Complexity + +**What goes wrong:** If Telegram responses are abstracted to a sub-workflow, the handoff becomes complex: domain sub-workflow → returns data → parent formats → calls Telegram sub-workflow → Telegram sub-workflow sends. Multiple execution contexts increase latency and error surface area. + +**Why it happens:** Over-abstraction of Telegram responses creates unnecessary indirection. Telegram API calls are simple (sendMessage, editMessage) and don't have complex business logic worth isolating. + +**How to avoid:** +- Keep Telegram responses in main workflow (locked decision already made) +- Sub-workflows return data only +- Only create Telegram sub-workflow if analysis validates value (user discretion in CONTEXT.md) +- Evaluate: does Telegram sub-workflow centralize meaningful quirks (keyboards, error formatting, callback answers) or just add overhead? + +**Warning signs:** +- Telegram responses becoming slower +- Error handling getting complicated +- Debugging requires tracing through multiple workflows +- Simple message sends require multiple sub-workflow calls + +**Sources:** User-specified concern from CONTEXT.md, general microservices architecture pitfalls + +### Pitfall 6: Inconsistent Naming Conventions + +**What goes wrong:** Workflows named inconsistently make them hard to find, sort, and understand purpose. Example: mixing patterns like "n8n-container-update.json" and "n8n-update.json". + +**Why it happens:** Incremental development without established naming convention. Different developers or phases use different patterns. + +**How to avoid:** +- Adopt consistent pattern: `n8n-{domain}.json` (locked decision) +- Rename existing workflows: n8n-update.json, n8n-actions.json, n8n-logs.json (locked decision) +- Use descriptive domain names: "batch-ui" not "batch", "container-status" not "status" +- Document naming convention for future additions + +**Warning signs:** +- Hard to locate workflow files +- Unclear which workflow handles which domain +- Inconsistent patterns across files + +**Sources:** [Best Practices for Naming Your Workflows](https://spacetag.co.uk/blogs/workflow-automation/best-practices-for-naming-your-workflows-in-n8n-zapier-and-make/), [n8n workflow naming conventions](https://community.n8n.io/t/best-practices-for-structuring-n8n-workflows-for-scale-and-long-term-maintainability/248671) + +## Code Examples + +Verified patterns from existing project workflows: + +### Execute Workflow Node Configuration + +```json +{ + "parameters": { + "source": "database", + "workflowId": { + "__rl": true, + "mode": "list", + "value": "7AvTzLtKXM2hZTio92_mC" + }, + "mode": "once", + "options": { + "waitForSubWorkflow": true + } + }, + "id": "exec-text-update-subworkflow", + "name": "Execute Text Update", + "type": "n8n-nodes-base.executeWorkflow", + "typeVersion": 1.2 +} +``` + +**Source:** /home/luc/Projects/unraid-docker-manager/n8n-workflow.json + +### Execute Workflow Trigger with Field Schema + +```json +{ + "parameters": { + "inputSource": "passthrough", + "schema": { + "schemaType": "fromFields", + "fields": [ + { + "fieldName": "containerId", + "fieldType": "string" + }, + { + "fieldName": "containerName", + "fieldType": "string" + }, + { + "fieldName": "chatId", + "fieldType": "number" + }, + { + "fieldName": "messageId", + "fieldType": "number" + }, + { + "fieldName": "responseMode", + "fieldType": "string" + } + ] + } + }, + "id": "sub-trigger", + "name": "When executed by another workflow", + "type": "n8n-nodes-base.executeWorkflowTrigger", + "typeVersion": 1.1 +} +``` + +**Source:** /home/luc/Projects/unraid-docker-manager/n8n-container-update.json + +### Input Validation in Sub-workflow + +```javascript +// Parse and validate input (Code node immediately after trigger) +const input = $json; + +// Get required fields +const containerId = input.containerId || ''; +const containerName = input.containerName || ''; +const chatId = input.chatId; +const messageId = input.messageId || 0; +const responseMode = input.responseMode || 'text'; + +// Validation +if (!containerId && !containerName) { + throw new Error('Either containerId or containerName required'); +} + +if (!chatId) { + throw new Error('chatId required'); +} + +return { + json: { + containerId: containerId, + containerName: containerName, + chatId: chatId, + messageId: messageId, + responseMode: responseMode + } +}; +``` + +**Source:** /home/luc/Projects/unraid-docker-manager/n8n-container-logs.json (Parse Input node) + +### Switch Node for Domain Routing + +```json +{ + "parameters": { + "rules": { + "values": [ + { + "id": "route-message", + "conditions": { + "options": { + "caseSensitive": true, + "typeValidation": "loose" + }, + "conditions": [ + { + "id": "has-message", + "leftValue": "={{ $json.message?.text }}", + "rightValue": "", + "operator": { + "type": "string", + "operation": "notEmpty" + } + } + ], + "combinator": "and" + }, + "renameOutput": true, + "outputKey": "message" + }, + { + "id": "route-callback", + "conditions": { + "options": { + "caseSensitive": true, + "typeValidation": "loose" + }, + "conditions": [ + { + "id": "has-callback", + "leftValue": "={{ $json.callback_query?.id }}", + "rightValue": "", + "operator": { + "type": "string", + "operation": "notEmpty" + } + } + ], + "combinator": "and" + }, + "renameOutput": true, + "outputKey": "callback_query" + } + ] + }, + "options": { + "fallbackOutput": "none" + } + }, + "type": "n8n-nodes-base.switch", + "typeVersion": 3.2 +} +``` + +**Source:** /home/luc/Projects/unraid-docker-manager/n8n-workflow.json (Route Update Type node) + +## State of the Art + +| Old Approach | Current Approach | When Changed | Impact | +|--------------|------------------|--------------|--------| +| Manual JSON editing for sub-workflows | UI-based "Convert to sub-workflow" | n8n recent versions | Automatic expression updates, safer refactoring | +| Monolithic workflows | Recommended 5-10 nodes per workflow | 2025-2026 best practices | Better maintainability, debugging, memory usage | +| Untyped sub-workflow inputs | Field-based schemas with type definitions | n8n typeVersion 1.1+ | Type safety, documentation, validation | +| Simple workflow ID reference | Resource locator format (`__rl`, mode, value) | n8n typeVersion 1.2 | Required for newer n8n versions | +| Accept all data mode default | Field-based schema recommended | 2025-2026 | Better contracts, easier testing | + +**Deprecated/outdated:** +- **Simple string workflow ID references:** n8n typeVersion 1.2+ requires resource locator format with `__rl: true` +- **Monolithic workflow pattern:** Community consensus shifted to 5-10 node workflows for optimal maintainability +- **No error handling in sub-workflows:** Modern practice requires structured error returns and parent-side handling + +## Open Questions + +Things that couldn't be fully resolved: + +1. **Optimal domain boundaries for batch operations** + - What we know: 48+ batch-related nodes exist in main workflow, batch operations reuse single-container sub-workflows + - What's unclear: Should batch orchestration be in main workflow, single batch sub-workflow, or multiple batch sub-workflows (batch-ui, batch-exec)? User wants Claude to analyze and recommend. + - Recommendation: Plan phase should analyze batch node cohesion and propose options (single vs multiple) with tradeoffs + +2. **Telegram sub-workflow value proposition** + - What we know: 26 Telegram nodes in main workflow (sendMessage, editMessage, answerCallbackQuery), user concerned about handoff complexity + - What's unclear: Do Telegram quirks (keyboards, error formatting, callback answers) merit centralization, or is it over-abstraction? + - Recommendation: Plan phase should evaluate specific Telegram patterns in main workflow and assess if abstraction reduces complexity or increases it. This is explicitly marked as "Claude's discretion" in CONTEXT.md. + +3. **Exact extraction threshold** + - What we know: User specified 8-15+ nodes as extraction threshold, don't extract 2-3 node groups + - What's unclear: What about 4-7 node groups? Gray area between "too small" and "meaningful" + - Recommendation: Apply pragmatic judgment: if 4-7 nodes represent cohesive user-facing outcome and are reused, extract; if single-use utility logic, keep in parent + +4. **Sub-workflow output shape standardization** + - What we know: User specified "Claude decides output shape (structured response vs raw data)" in CONTEXT.md + - What's unclear: Should all sub-workflows return same structure `{success, error, message, data}` or domain-specific shapes? + - Recommendation: Plan phase should propose standard envelope format for consistency in error handling while allowing domain-specific data payload + +5. **Rollback mechanism details** + - What we know: User specified "git commits + explicit backup files before changes" + - What's unclear: Backup file naming convention, where to store, how to automate backup creation + - Recommendation: Plan phase should define specific rollback procedure with file naming (e.g., `n8n-workflow.backup-YYYY-MM-DD.json`) + +## Sources + +### Primary (HIGH confidence) +- Existing workflow files: /home/luc/Projects/unraid-docker-manager/n8n-workflow.json, n8n-container-update.json, n8n-container-actions.json, n8n-container-logs.json - actual implementation patterns +- Phase CONTEXT.md: /home/luc/Projects/unraid-docker-manager/.planning/phases/10.1-aggressive-workflow-modularization/10.1-CONTEXT.md - locked user decisions +- [n8n Sub-workflows documentation](https://docs.n8n.io/flow-logic/subworkflows/) - official docs +- [n8n Execute Sub-workflow node](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.executeworkflow/) - official node docs +- [n8n Execute Sub-workflow Trigger](https://docs.n8n.io/integrations/builtin/core-nodes/n8n-nodes-base.executeworkflowtrigger/) - official trigger docs + +### Secondary (MEDIUM confidence) +- [n8n Sub-workflow conversion](https://docs.n8n.io/workflows/subworkflow-conversion/) - official docs (content truncated but URL verified) +- [n8n Error handling](https://docs.n8n.io/flow-logic/error-handling/) - official docs (structure verified) +- [Creating error workflows in n8n](https://blog.n8n.io/creating-error-workflows-in-n8n/) - official blog +- [Seven N8N Workflow Best Practices for 2026](https://michaelitoback.com/n8n-workflow-best-practices/) - community best practices, consistent with official docs +- [N8N Performance Optimization](https://www.wednesday.is/writing-articles/n8n-performance-optimization-maximizing-workflow-efficiency) - performance guidance verified against memory docs +- [Best Practices for Naming Your Workflows](https://spacetag.co.uk/blogs/workflow-automation/best-practices-for-naming-your-workflows-in-n8n-zapier-and-make/) - naming conventions + +### Tertiary (LOW confidence) +- [n8n community: structuring workflows for scale](https://community.n8n.io/t/best-practices-for-structuring-n8n-workflows-for-scale-and-long-term-maintainability/248671) - recent community discussion, unverified recommendations +- [n8n community: modularize workflows](https://community.n8n.io/t/best-practice-to-modularize-workflows/501) - older community thread, may not reflect current best practices + +## Metadata + +**Confidence breakdown:** +- Standard stack: HIGH - All components verified from existing workflow files and official n8n documentation +- Architecture patterns: HIGH - Patterns extracted from working sub-workflows, validated against official docs and locked user decisions +- Pitfalls: MEDIUM-HIGH - Memory/performance issues verified with official docs, other pitfalls based on community consensus and general software architecture principles +- Open questions: HIGH - Clearly identified gaps that require planning phase analysis, aligned with CONTEXT.md discretionary areas + +**Research date:** 2026-02-04 +**Valid until:** 2026-03-04 (30 days - n8n platform stable, best practices unlikely to change rapidly) + +**Note on research limitations:** +- WebFetch of n8n documentation returned truncated content (navigation structure only) +- Compensated by: analyzing existing working implementations, cross-referencing multiple community sources, using WebSearch for practical patterns +- All code examples sourced from actual project files (HIGH confidence) +- Architectural recommendations grounded in locked user decisions from CONTEXT.md (HIGH confidence)