unraid-docker-manager/.planning/phases/10-workflow-modularization/10-RESEARCH.md

Phase 10: Workflow Modularization - Research

Researched: 2026-02-04 Domain: n8n workflow modularization Confidence: HIGH (verified against official n8n documentation)

Summary

The current n8n workflow (n8n-workflow.json) is 8,485 lines with approximately 150+ nodes handling all functionality in a single monolithic workflow. This phase will break it into modular sub-workflows using n8n's built-in sub-workflow infrastructure.

n8n provides first-class support for workflow modularization through the Execute Sub-workflow node and Execute Sub-workflow Trigger node. Sub-workflows enable:

• Breaking large workflows into focused components (recommended: 5-10 nodes per sub-workflow)
• Reusing logic across multiple entry points (e.g., update flow used by text commands AND callbacks)
• Memory isolation - sub-workflow memory is released after completion
• Independent testing and debugging
• Clear data contracts between components

Primary recommendation: Use n8n's Execute Sub-workflow node to extract logical modules, starting with the duplicated update flow (DEBT-03) as the first extraction target since it's called from two places.

Standard Stack

Core n8n Nodes for Modularization

Node	Purpose	Why Standard
Execute Sub-workflow Trigger	Entry point for sub-workflows	Built-in n8n node, defines input contract
Execute Sub-workflow	Calls sub-workflows from parent	Built-in n8n node, handles data passing
Code Node	Data transformation between modules	Prepare inputs, format outputs
Switch Node	Route to different sub-workflows	Handle multiple action types

Input Data Modes

Mode	When to Use
Define using fields	Best for typed, validated inputs (recommended)
Define using JSON example	When input structure is complex/nested
Accept all data	Legacy/migration - avoid for new sub-workflows

Recommendation: Use "Define using fields" for all new sub-workflows - provides clear data contracts and automatic field population in the Execute Sub-workflow node.

Architecture Patterns

Recommended Module Structure

Based on workflow analysis, these logical sub-workflows are recommended:

Main Orchestrator (n8n-workflow.json - reduced)
├── Authentication check (inline - too small to extract)
├── Command routing (inline - just a switch node)
│
├── Sub-workflows:
│   ├── container-operations/
│   │   ├── container-action.json      # start/stop/restart a container
│   │   ├── container-update.json      # full update flow (DEBT-03 consolidation)
│   │   └── container-logs.json        # fetch and format logs
│   │
│   ├── keyboard-generation/
│   │   ├── container-list.json        # paginated container list keyboard
│   │   ├── container-submenu.json     # single container action menu
│   │   └── confirmation-dialog.json   # stop/update confirmation keyboards
│   │
│   └── batch-operations/
│       ├── batch-loop.json            # execute action on multiple containers
│       └── batch-selection.json       # selection keyboard management

Pattern 1: Container Update Sub-workflow (Priority - addresses DEBT-03)

What: Extract the entire container update sequence into a single sub-workflow Why: This logic is currently duplicated between:

1. Text command path (lines ~1656-2400)
2. Callback/inline keyboard path (lines ~3628-4010)

Input contract (Execute Sub-workflow Trigger):

{
  "containerId": "string",
  "containerName": "string",
  "chatId": "number",
  "messageId": "number (optional - for inline updates)",
  "responseMode": "string ('text' | 'inline')"
}

Output contract (returned to parent):

{
  "success": "boolean",
  "message": "string",
  "oldDigest": "string (optional)",
  "newDigest": "string (optional)"
}

Flow extracted:

1. Inspect container configuration
2. Pull latest image
3. Compare digests
4. If update needed: stop -> remove -> create -> start
5. Clean up old image
6. Return result

Pattern 2: Keyboard Generation Sub-workflow

What: Extract keyboard building logic Why: Keyboard generation is scattered throughout and could be centralized

Example input:

{
  "type": "submenu",
  "containerName": "string",
  "containerState": "running|exited|paused",
  "options": {
    "showBack": true,
    "backTarget": "list:0"
  }
}

Pattern 3: Data Contract Design

Principle: Every sub-workflow should have clearly defined inputs and outputs.

[Parent Workflow]
    │
    ▼ Input: { containerId, containerName, chatId, messageId }
[Execute Sub-workflow Node]
    │
    ▼ Processes via Execute Sub-workflow Trigger
[Sub-workflow: container-update]
    │
    ▼ Output: { success, message, details }
[Parent continues with result]

Anti-Patterns to Avoid

• Shared state via workflow static data: Use explicit input/output instead
• Over-modularization: Don't extract nodes that are only used once
• Deep nesting: Keep sub-workflow call depth to 2 maximum (parent -> child)
• Accept all data mode: Always define explicit inputs for maintainability

Don't Hand-Roll

Problem	Don't Build	Use Instead	Why
Workflow-to-workflow communication	Custom webhooks	Execute Sub-workflow node	Built-in, handles data passing, memory isolation
Reusable logic extraction	Copy-paste nodes	Sub-workflow with trigger	Single source of truth, DRY
Complex routing	Deeply nested If nodes	Switch + sub-workflows	Cleaner, testable
Input validation	Manual checks in Code node	Define input fields in trigger	Type safety, documentation

Key insight: n8n's sub-workflow infrastructure handles all the complexity of data passing, execution context, and memory management. Using Execute Sub-workflow node instead of custom solutions provides memory isolation that prevents heap exhaustion on large operations.

Common Pitfalls

Pitfall 1: Circular Sub-workflow Calls

What goes wrong: Sub-workflow A calls B which calls A, causing infinite loop Why it happens: Accidental dependency cycles during refactoring How to avoid: Document call hierarchy, keep sub-workflows single-purpose Warning signs: Workflow hangs indefinitely, memory usage spikes

Pitfall 2: Lost Context in Multi-step Flows

What goes wrong: Data from early nodes not available in sub-workflow Why it happens: Sub-workflow only receives explicit inputs, not parent context How to avoid: Design complete input contracts - if sub-workflow needs data, it must be in input Warning signs: "Cannot read property of undefined" errors in sub-workflow

Pitfall 3: Credential Scope Issues

What goes wrong: Sub-workflow can't access credentials defined in parent Why it happens: Credentials are workflow-scoped, not globally shared How to avoid: Configure same credentials in both parent and sub-workflows Warning signs: Authentication failures only when running as sub-workflow

Pitfall 4: Breaking Existing Callback Data

What goes wrong: Inline keyboard callbacks stop working after modularization Why it happens: Callback data parsing expects specific node structure/names How to avoid: Preserve callback data format, update routing to call sub-workflows Warning signs: "Callback query expired" or routing errors after deployment

Pitfall 5: Over-extraction

What goes wrong: Simple operations become 5+ sub-workflow calls, adding latency Why it happens: Applying microservice patterns too aggressively How to avoid: Only extract logic that is: (a) reused, OR (b) large/complex, OR (c) needs isolation Warning signs: Simple operations now take seconds instead of milliseconds

Code Examples

Example 1: Sub-workflow Trigger Setup

{
  "name": "Container Update",
  "nodes": [
    {
      "parameters": {
        "inputSource": "passthrough",
        "schema": {
          "schemaType": "fromFields",
          "fields": [
            { "fieldName": "containerId", "fieldType": "string" },
            { "fieldName": "containerName", "fieldType": "string" },
            { "fieldName": "chatId", "fieldType": "number" },
            { "fieldName": "messageId", "fieldType": "number" },
            { "fieldName": "responseMode", "fieldType": "string" }
          ]
        }
      },
      "name": "When executed by another workflow",
      "type": "n8n-nodes-base.executeWorkflowTrigger",
      "typeVersion": 1.1
    }
  ]
}

Example 2: Calling Sub-workflow from Parent

{
  "parameters": {
    "source": "database",
    "workflowId": "container-update-workflow-id",
    "mode": "once",
    "options": {
      "waitForSubWorkflow": true
    }
  },
  "name": "Execute Update",
  "type": "n8n-nodes-base.executeWorkflow",
  "typeVersion": 1.2
}

Example 3: Preparing Sub-workflow Input (Code Node)

// In parent workflow, prepare input for sub-workflow
const containerData = $('Match Container').item.json;
const triggerData = $('Telegram Trigger').item.json;

return {
  json: {
    containerId: containerData.matches[0].Id,
    containerName: containerData.matches[0].Name,
    chatId: triggerData.message.chat.id,
    messageId: triggerData.message.message_id,
    responseMode: 'text'
  }
};

Example 4: Converting Callback Path to Use Sub-workflow

// Instead of duplicated update logic, call sub-workflow
const callbackData = $('Parse Callback Data').item.json;

return {
  json: {
    containerId: callbackData.containerId,
    containerName: callbackData.containerName,
    chatId: callbackData.chatId,
    messageId: callbackData.messageId,
    responseMode: 'inline'  // Sub-workflow handles response format
  }
};

State of the Art

Old Approach	Current Approach	When Changed	Impact
Monolithic workflows	Modular sub-workflows	n8n 1.0+	Better maintainability, memory efficiency
Copy-paste nodes	Execute Sub-workflow node	Core feature	DRY, single source of truth
Accept all data	Define using fields	2025 best practice	Type safety, documentation
Manual webhook calls	Native sub-workflow execution	Core feature	Built-in context, error handling

Current n8n sub-workflow features (2026):

• Execute Sub-workflow Trigger node with schema definition
• Sub-workflow conversion from canvas context menu (select nodes -> right-click -> Create sub-workflow)
• Memory isolation for sub-workflows
• Sub-workflow executions don't count toward n8n Cloud quotas
• Wait for completion toggle for async patterns

Open Questions

1. Sub-workflow File Organization

What we know: n8n stores workflows as individual entities in its database What's unclear: Whether to export sub-workflows as separate JSON files in git or keep them n8n-internal Recommendation: Export as separate files for version control, document workflow IDs in main workflow

2. Testing Sub-workflows

What we know: Can use Manual Trigger + test data for isolated testing What's unclear: Best practice for automated testing Recommendation: Create test input nodes in each sub-workflow, manually validate before deployment

3. Migration Strategy

What we know: Can convert existing nodes to sub-workflow via context menu What's unclear: Whether to do incremental migration or big-bang refactor Recommendation: Incremental - extract one sub-workflow at a time, test, then proceed

Specific Extraction Plan

Based on workflow analysis, recommended extraction order:

Priority 1: Container Update (addresses DEBT-03)

• Why first: Addresses explicit tech debt requirement, duplicated in two places
• Lines affected: ~1656-2400 (text path) + ~3628-4010 (callback path)
• Expected reduction: ~750 lines from main workflow
• Risk: Medium - update flow is critical, needs thorough testing

Priority 2: Container Simple Actions (start/stop/restart)

• Why second: Simpler than update, similar pattern of text+callback duplication
• Expected reduction: ~200 lines
• Risk: Low - simple operations

Priority 3: Keyboard Generation

• Why third: Scattered throughout, centralizing improves consistency
• Expected reduction: ~300 lines
• Risk: Low - UI only, easy to test

Priority 4: Batch Loop (if time permits)

• Why last: Complex state management, higher risk
• Expected reduction: ~400 lines
• Risk: Medium-High - batch operations have edge cases

Sources

Primary (HIGH confidence)

• n8n Sub-workflows Documentation - Official docs on sub-workflow concepts
• Execute Sub-workflow Node - Official node documentation
• Execute Sub-workflow Trigger - Official trigger documentation

Secondary (MEDIUM confidence)

• n8n Best Practices 2026 - Community best practices guide
• When to Use Sub-workflows - n8n Community guidance
• n8n Expert Best Practices - Expert recommendations

Tertiary (LOW confidence)

• Modularizing n8n Workflows - Blog post on modularization patterns

Metadata

Confidence breakdown:

• Standard stack: HIGH - Based on official n8n documentation
• Architecture patterns: MEDIUM - Based on workflow analysis + best practices
• Pitfalls: MEDIUM - Community patterns, verified against documentation
• Extraction plan: MEDIUM - Based on codebase analysis

Research date: 2026-02-04 Valid until: 2026-03-04 (30 days - n8n stable, patterns unlikely to change)