diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md index f2d8ad4..f301b5a 100644 --- a/.planning/PROJECT.md +++ b/.planning/PROJECT.md @@ -2,7 +2,7 @@ ## What This Is -A conversational Telegram bot that lets you manage Docker containers on your Unraid server using natural language. Think JARVIS for your homelab — ask questions, get status updates, and control containers all from your phone without needing to open a laptop. +A Telegram bot that lets you manage Docker containers on your Unraid server. Control containers from your phone without needing to open a laptop — check status, view logs, start/stop/restart/update containers via simple keyword commands. ## Core Value @@ -23,15 +23,15 @@ When you get a container update notification or notice a service is down, you ca - [ ] Restart a container by name - [ ] Update a container (pull new image, recreate) - [ ] View container logs with configurable line count (default 50) -- [ ] Ask conversational questions ("what's using the most memory?") - [ ] Bot only responds to your Telegram user ID ### Out of Scope - Taking over Unraid notifications — keep existing notification system, this bot is for control - Deploying new containers — manage existing only, not create new ones -- Local LLM — N100 can't handle it well, using Claude API instead +- Natural language understanding — simple keyword matching sufficient for v1, Claude API adds complexity - Proactive monitoring/notifications — bot is reactive (you ask, it answers) for v1 +- Resource queries — "what's using the most memory?" deferred to future version ## Context @@ -54,7 +54,7 @@ When you get a container update notification or notice a service is down, you ca - **Platform**: Must run on Unraid (Docker-based deployment) - **Orchestration**: Use n8n for workflow orchestration (already running) -- **LLM**: Claude API for natural language understanding +- **Matching**: Keyword/substring matching for container names - **Auth**: Single user only — verify Telegram user ID - **Logs**: Support configurable line count, default to 50 lines @@ -62,10 +62,10 @@ When you get a container update notification or notice a service is down, you ca | Decision | Rationale | Outcome | |----------|-----------|---------| -| Use Claude API over local LLM | N100 CPU can't run LLMs fast enough for good UX | — Pending | +| Use keyword matching over NLU | Simple substring matching works well, Claude API adds complexity for v1 | ✓ Good | | Use n8n for orchestration | Already running, handles Telegram webhooks, reduces new infrastructure | — Pending | | Manage existing containers only | Keeps scope focused, deployment is complex and rarely needed from mobile | — Pending | | Single user auth via Telegram ID | Simple security model, only one person needs access | — Pending | --- -*Last updated: 2026-01-28 after initialization* +*Last updated: 2026-01-31 after removing NLU from v1.0 scope* diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 6f52874..1c6e5e7 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1,6 +1,6 @@ # Roadmap — Unraid Docker Manager v1.0 -## Milestone: v1.0 — Conversational Docker Control +## Milestone: v1.0 — Docker Control via Telegram ### Phase 1: Foundation **Goal:** Basic Telegram ↔ n8n communication working @@ -65,17 +65,20 @@ Plans: --- -### Phase 4: Logs & Intelligence -**Goal:** View logs and answer conversational questions +### Phase 4: Logs +**Goal:** View container logs via Telegram - Implement log retrieval with configurable line count -- Integrate Claude API for natural language understanding -- Parse user intent from conversational messages -- Implement resource queries ("what's using most memory?") +- Handle long log output (truncation for Telegram limits) -**Delivers:** REQ-07 (logs), REQ-08 (conversational queries) +**Delivers:** REQ-07 (logs) -**Status:** 🔲 Not started +**Plans:** 1 plan + +Plans: +- [x] 04-01-PLAN.md — Container log retrieval with configurable lines + +**Status:** ✅ Complete (2026-01-31) --- @@ -104,7 +107,7 @@ Plans: | REQ-05 | Restart container | 3 | | REQ-06 | Update container | 3 | | REQ-07 | View logs (configurable lines) | 4 | -| REQ-08 | Conversational queries | 4 | +| REQ-08 | ~~Conversational queries~~ | Out of scope | | REQ-09 | User ID authentication | 1 | --- diff --git a/.planning/STATE.md b/.planning/STATE.md index 0de373c..7215d02 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -2,16 +2,16 @@ ## Project Reference -**Building:** Conversational Telegram bot for Docker container management on Unraid -**Core Value:** Investigate and control containers from your phone through natural conversation +**Building:** Telegram bot for Docker container management on Unraid +**Core Value:** Control containers from your phone via simple keyword commands ## Current Position -- **Milestone:** v1.0 — Conversational Docker Control -- **Phase:** 4 of 5 — Logs & Intelligence (IN PROGRESS) -- **Plan:** 1 of 2 executed -- **Status:** Phase 4 in progress - container log retrieval complete -- **Last activity:** 2026-01-30 - Completed 04-01-PLAN.md (container log retrieval) +- **Milestone:** v1.0 — Docker Control via Telegram +- **Phase:** 4 of 5 — Logs (COMPLETE) +- **Plan:** 1 of 1 executed +- **Status:** Phase 4 complete - ready for Phase 5 +- **Last activity:** 2026-01-31 - Removed NLU from scope, Phase 4 complete ## Progress @@ -19,10 +19,10 @@ Phase 1: Foundation [██████████] Complete (2/2 plans) Phase 2: Docker Integration [██████████] Complete (2/2 plans) Phase 3: Container Actions [██████████] Complete (4/4 plans) -Phase 4: Logs & Intelligence[█████░░░░░] In progress (1/2 plans) +Phase 4: Logs [██████████] Complete (1/1 plans) Phase 5: Polish & Deploy [░░░░░░░░░░] Not started -Overall: [███████░░░] 70% +Overall: [████████░░] 80% ``` ## Recent Decisions @@ -54,6 +54,7 @@ Overall: [███████░░░] 70% | Binary stream header via charCodeAt | Docker multiplexed stream uses byte 0 values 1/2, strip 8 bytes | 2026-01-30 | | Default 50 lines, cap at 1000 | Balance between useful context and Telegram limits | 2026-01-30 | | Truncate at 3800 chars | Telegram limit is 4096, leave room for header and formatting | 2026-01-30 | +| Remove NLU from v1.0 | Simple keyword matching sufficient, Claude API adds complexity | 2026-01-31 | ## Pending Todos @@ -65,11 +66,10 @@ Overall: [███████░░░] 70% ## Session Continuity -- **Last session:** 2026-01-30 -- **Stopped at:** Phase 4 checkpoint verification (04-02 Claude Intent Parsing) -- **Resume file:** `.planning/phases/04-logs-intelligence/.continue-here.md` -- **Next step:** Complete checkpoint verification, then run phase verifier -- **Resume command:** `/gsd:resume-work` +- **Last session:** 2026-01-31 +- **Stopped at:** Scope change - removed NLU from v1.0 +- **Next step:** Start Phase 5 (Polish & Deploy) +- **Resume command:** `/gsd:plan-phase 5` --- *Auto-maintained by GSD workflow* diff --git a/.planning/phases/04-logs-intelligence/.continue-here.md b/.planning/phases/04-logs-intelligence/.continue-here.md deleted file mode 100644 index 258f8e3..0000000 --- a/.planning/phases/04-logs-intelligence/.continue-here.md +++ /dev/null @@ -1,70 +0,0 @@ ---- -phase: 04-logs-intelligence -task: checkpoint-verification -total_tasks: checkpoint -status: in_progress -last_updated: 2026-01-30 ---- - - -Phase 4 execution complete but checkpoint verification is in progress. Both plans (04-01, 04-02) have been executed by agents, but the 04-02 Claude Intent Parsing plan requires human verification. During testing, multiple n8n workflow bugs were discovered and fixed. The user is actively testing the workflow and reporting issues. - -Current issue: "Send Batch Confirmation" node was showing "The resource you are requesting could not be found" error. Fixed by converting from HTTP Request to native Telegram node. - - - -- 04-01-PLAN.md: Container log retrieval - Complete -- 04-02-PLAN.md: Claude Intent Parsing - Executed, awaiting verification -- Fixed 10+ workflow bugs discovered during testing: - - Connection source names (orphaned nodes) - - Route Update Type conversion error - - Claude Intent Parser body preparation - - Multiple Switch node type validation issues - - Parse and Match routing - - Match Container Parse Intent reference - - Send Batch Confirmation credential access - - - -- Complete 04-02 checkpoint verification (user must test conversational routing) -- Verify all intent types work: view_logs, container_action, container_status, list_containers -- Run phase verification after checkpoint passes -- Proceed to Phase 5: Polish & Deploy - - - -- Native Telegram node for Send Batch Confirmation: HTTP Request credential access via URL expression wasn't resolving -- Added chatId validation in Build Batch Keyboard: Defensive error handling for clearer debugging -- Keep typeValidation "loose" for number comparisons in Switch nodes: rightValue must be actual numbers, not strings - - - -- User needs to configure Anthropic API credential in n8n (Header Auth with x-api-key) -- User needs to test batch confirmation flow to verify Send Batch Confirmation fix works - - - -The session was focused on executing Phase 4 and debugging workflow issues. Both 04-01 and 04-02 plans were executed via /gsd:execute-phase 4. The agents completed successfully but when the user started testing, they discovered multiple integration bugs between: -1. Old command-based routing (Parse Action node) -2. New intent-based routing (Parse Intent node from 04-02) - -Each bug was fixed incrementally as the user reported errors. The workflow has evolved significantly from the original plan output - expect the n8n-workflow.json to have many fixes applied. - -Key commits from this session: -- 5e7cab5: fix Send Batch Confirmation → native Telegram node -- 5d55bde: fix type validation across all Switch nodes -- aa67770: update Match Container to use Parse Intent -- f423f4a: add Prepare Claude Request node - - - -1. User should re-import workflow to n8n -2. User should configure Anthropic API credential if not done -3. User tests conversational commands via Telegram: - - "restart nginx" (single container action) - - "restart arr" (batch - multiple matches) - - "show me plex logs" (log viewing) - - "status" (list containers) -4. If all work: type "approved" to pass checkpoint -5. If issues: report the error for additional fixes - diff --git a/.planning/phases/04-logs-intelligence/04-02-PLAN.md b/.planning/phases/04-logs-intelligence/04-02-PLAN.md deleted file mode 100644 index 6dcfc7e..0000000 --- a/.planning/phases/04-logs-intelligence/04-02-PLAN.md +++ /dev/null @@ -1,253 +0,0 @@ ---- -phase: 04-logs-intelligence -plan: 02 -type: execute -wave: 1 -depends_on: [] -files_modified: [n8n-workflow.json] -autonomous: false - -user_setup: - - service: anthropic - why: "Claude API for natural language understanding" - env_vars: - - name: ANTHROPIC_API_KEY - source: "Anthropic Console -> API Keys -> Create Key" - dashboard_config: [] - -must_haves: - truths: - - "Messages are parsed by Claude for intent detection" - - "Recognized intents route to appropriate handlers" - - "Unknown intents receive helpful response" - artifacts: - - path: "n8n-workflow.json" - provides: "Claude API integration and intent routing" - contains: "anthropic" - key_links: - - from: "Telegram message" - to: "Claude API" - via: "HTTP Request node" - pattern: "api.anthropic.com" - - from: "Claude response" - to: "Switch node" - via: "intent parsing code node" - pattern: "action.*view_logs|query_stats" ---- - - -Integrate Claude API for natural language understanding and intent-first message routing. - -Purpose: Enables conversational interaction (REQ-08) by understanding user intent before execution. Users can speak naturally instead of memorizing command syntax. - -Output: Extended n8n workflow with Claude API integration, intent parsing, and routing logic. - - - -@/home/luc/.claude/get-shit-done/workflows/execute-plan.md -@/home/luc/.claude/get-shit-done/templates/summary.md - - - -@.planning/PROJECT.md -@.planning/ROADMAP.md -@.planning/STATE.md -@.planning/phases/04-logs-intelligence/04-RESEARCH.md -@n8n-workflow.json - - - - - - Task 1: Add Claude API credential and HTTP Request node - n8n-workflow.json - -Create Claude API integration in the workflow. - -1. Add "Claude Intent Parser" HTTP Request node: - - Method: POST - - URL: https://api.anthropic.com/v1/messages - - Authentication: Header Auth - - Header: x-api-key = {{$credentials.anthropicApi.apiKey}} (or use Generic Credential) - - Header: anthropic-version = 2023-06-01 - - Header: content-type = application/json - - Body (JSON): - ```json - { - "model": "claude-sonnet-4-5-20250929", - "max_tokens": 256, - "system": [ - { - "type": "text", - "text": "You are a Docker container management assistant. Parse user requests and return ONLY valid JSON.\n\nValid actions:\n- view_logs: User wants to see container logs\n- query_stats: User asks about resource usage (memory, CPU)\n- container_action: User wants to start/stop/restart/update a container\n- container_status: User asks about container status\n- list_containers: User wants to see all containers\n- unknown: Cannot determine intent\n\nRespond with JSON: {\"action\": \"\", \"container\": \"\", \"parameters\": {}}\\n\\nExamples:\\n- \"show me plex logs\" -> {\"action\": \"view_logs\", \"container\": \"plex\", \"parameters\": {\"lines\": 50}}\\n- \"what's using the most memory?\" -> {\"action\": \"query_stats\", \"container\": null, \"parameters\": {\"metric\": \"memory\", \"sort\": \"desc\"}}\\n- \"restart nginx\" -> {\"action\": \"container_action\", \"container\": \"nginx\", \"parameters\": {\"action\": \"restart\"}}\\n- \"how's sonarr doing?\" -> {\"action\": \"container_status\", \"container\": \"sonarr\", \"parameters\": {}}\\n- \"hello\" -> {\"action\": \"unknown\", \"container\": null, \"parameters\": {\"message\": \"I can help with Docker containers. Try: 'show logs', 'restart plex', or 'what's using memory?'\"}}", - "cache_control": {"type": "ephemeral"} - } - ], - "messages": [ - {"role": "user", "content": "{{$json.message}}"} - ] - } - ``` - - Options: Timeout 30000ms, Retry on fail (3 attempts) - -Note: n8n may need a "Header Auth" credential type created with the API key, OR use the body to include auth. Check n8n's HTTP Request node documentation for Anthropic compatibility. - -Alternative if Header Auth credential doesn't work well: -- Use "None" authentication -- Add x-api-key header manually via "Send Headers" option -- Value: Use expression referencing an n8n credential or environment variable - - -1. In n8n, navigate to Credentials and ensure Anthropic/Header credential is configured -2. Test the HTTP Request node manually with a simple message like "hello" -3. Check response contains valid JSON with action field - - Claude API HTTP Request node configured and able to receive intent parsing responses. - - - - Task 2: Create intent parsing and validation logic - n8n-workflow.json - -After Claude HTTP Request node, add "Parse Intent" Code node to validate and extract the structured intent. - -```javascript -// Parse and validate Claude's intent response -const response = $input.item.json; - -// Claude response structure: { content: [{ type: "text", text: "..." }] } -let intentText = ''; -try { - intentText = response.content[0].text; -} catch (e) { - return { - action: 'error', - error: 'Invalid Claude response structure', - raw: JSON.stringify(response) - }; -} - -// Parse JSON from Claude's response -let intent; -try { - // Claude might wrap JSON in markdown code blocks, strip them - const cleaned = intentText.replace(/```json\n?/g, '').replace(/```\n?/g, '').trim(); - intent = JSON.parse(cleaned); -} catch (e) { - return { - action: 'error', - error: 'Could not parse intent JSON', - raw: intentText - }; -} - -// Validate required fields -const validActions = ['view_logs', 'query_stats', 'container_action', 'container_status', 'list_containers', 'unknown']; -if (!intent.action || !validActions.includes(intent.action)) { - return { - action: 'unknown', - error: 'Invalid or missing action', - parameters: { message: 'I didn\'t understand that. Try: "show logs plex" or "restart nginx"' } - }; -} - -// Normalize container name if present -if (intent.container) { - intent.container = intent.container.toLowerCase().trim(); -} - -// Set defaults for parameters -intent.parameters = intent.parameters || {}; - -return intent; -``` - - -Test Parse Intent node with various Claude responses: -1. Valid JSON response -> extracts action and container -2. Malformed JSON -> returns error action with helpful message -3. Missing action field -> returns unknown with guidance - - Intent parsing extracts and validates Claude's response into usable workflow data. - - - - Task 3: Wire intent router to existing and new handlers - n8n-workflow.json - -Restructure workflow to use intent-first routing: - -1. After auth check (existing), route ALL messages through Claude Intent Parser first (not just unknown messages). - -2. Add "Intent Router" Switch node after Parse Intent: - - Condition 1: action == "view_logs" -> Route to logs handler (from 04-01) - - Condition 2: action == "query_stats" -> Route to stats handler (placeholder for 04-03) - - Condition 3: action == "container_action" -> Route to existing actions branch - - Condition 4: action == "container_status" -> Route to existing status branch - - Condition 5: action == "list_containers" -> Route to existing status summary - - Condition 6: action == "unknown" -> Route to Unknown Intent handler - - Condition 7: action == "error" -> Route to Error handler - -3. Create "Unknown Intent" handler: - - Telegram Send Message with helpful text from intent.parameters.message - - Default: "I can help manage your Docker containers. Try:\n- 'show logs plex'\n- 'restart sonarr'\n- 'what containers are running?'\n- 'what's using the most memory?'" - -4. Create "Stats Placeholder" handler (will be replaced in 04-03): - - Returns: "Stats queries coming soon! For now, try 'status' to see running containers." - -5. Update existing action/status handlers to use intent.container instead of re-parsing message. - -Important: Keep existing command-based routing as fallback for when Claude API is unavailable (rate limit, error). Add error handling around Claude call that falls back to pattern matching. - - -Test complete flow via Telegram: -1. "show me nginx logs" -> routes to view_logs handler -2. "restart plex" -> routes to container_action handler -3. "how's sonarr?" -> routes to container_status handler -4. "what's running" -> routes to list_containers handler -5. "hello there" -> routes to unknown with helpful message -6. "what's eating memory?" -> routes to stats placeholder - - All messages routed through Claude intent parsing with appropriate handler dispatch. - - - - - - Claude API integration with intent-first message routing - -1. Open Telegram chat with the bot -2. Send: "show me the logs for n8n" - - Expected: Should return n8n container logs (if 04-01 complete) or route correctly -3. Send: "restart plex" (or any container you can safely restart) - - Expected: Should trigger restart via existing action handler -4. Send: "hey how are you" - - Expected: Should return helpful message about available commands -5. Send: "what's using the most CPU?" - - Expected: Should return "Stats queries coming soon!" placeholder -6. Check n8n execution log to verify Claude API was called and intent parsed - - Type "approved" if conversational routing works, or describe issues - - - -After all tasks: -1. Claude API credential configured in n8n -2. HTTP Request node successfully calls Claude Messages API -3. Intent parsing extracts action, container, and parameters -4. Switch node routes to correct handlers based on intent -5. Unknown intents receive helpful guidance -6. Existing command-based functionality still works - - - -- Claude Sonnet 4.5 used for intent parsing with prompt caching -- Natural language queries parsed into structured intents -- All existing functionality (status, actions) accessible via conversation -- Graceful fallback/error handling when Claude unavailable -- Unknown intents guide users toward valid commands - - - -After completion, create `.planning/phases/04-logs-intelligence/04-02-SUMMARY.md` - diff --git a/.planning/phases/04-logs-intelligence/04-01-PLAN.md b/.planning/phases/04-logs/04-01-PLAN.md similarity index 100% rename from .planning/phases/04-logs-intelligence/04-01-PLAN.md rename to .planning/phases/04-logs/04-01-PLAN.md diff --git a/.planning/phases/04-logs-intelligence/04-01-SUMMARY.md b/.planning/phases/04-logs/04-01-SUMMARY.md similarity index 100% rename from .planning/phases/04-logs-intelligence/04-01-SUMMARY.md rename to .planning/phases/04-logs/04-01-SUMMARY.md diff --git a/.planning/phases/04-logs-intelligence/04-RESEARCH.md b/.planning/phases/04-logs/04-RESEARCH.md similarity index 100% rename from .planning/phases/04-logs-intelligence/04-RESEARCH.md rename to .planning/phases/04-logs/04-RESEARCH.md