test(10.2): complete UAT - 2 passed, 4 issues
This commit is contained in:
@@ -0,0 +1,95 @@
|
||||
---
|
||||
status: complete
|
||||
phase: 10.2-better-logging-and-log-management
|
||||
source: 10.2-01-SUMMARY.md, 10.2-02-SUMMARY.md, 10.2-03-SUMMARY.md
|
||||
started: 2026-02-08T18:00:00Z
|
||||
updated: 2026-02-08T18:10:00Z
|
||||
---
|
||||
|
||||
## Current Test
|
||||
|
||||
[testing complete]
|
||||
|
||||
## Tests
|
||||
|
||||
### 1. Basic Bot Functionality (No Regression)
|
||||
expected: Send a container command (e.g., "status" or tap an inline button). Bot responds normally with no errors or changed behavior.
|
||||
result: pass
|
||||
|
||||
### 2. Correlation ID in Execution Logs
|
||||
expected: Send a text command to the bot (e.g., "status"). Then check the n8n execution log for that execution. The data flowing through "Generate Correlation ID" node should show a correlationId field (format: timestamp-randomstring, e.g., "1770573038000-k3j8d9f2x").
|
||||
result: issue
|
||||
reported: "generate correlation IDs are not wired with connections. Additionally there are 8 orphan nodes"
|
||||
severity: major
|
||||
|
||||
### 3. Callback Correlation ID in Execution Logs
|
||||
expected: Tap an inline keyboard button (e.g., a container action button). Check the n8n execution log. The data flowing through "Generate Callback Correlation ID" node should show a correlationId field in the same format.
|
||||
result: issue
|
||||
reported: "Same as above — callback correlation ID node also not wired"
|
||||
severity: major
|
||||
|
||||
### 4. Sub-workflow Receives Correlation ID
|
||||
expected: Trigger a sub-workflow call (e.g., send "status" which calls the Status sub-workflow). In the n8n execution log for the sub-workflow execution, the input data should contain a correlationId field passed from the main workflow.
|
||||
result: issue
|
||||
reported: "Same as above — correlation IDs can't reach sub-workflows since generators aren't wired"
|
||||
severity: major
|
||||
|
||||
### 5. Structured Error Return from Sub-workflow
|
||||
expected: Trigger an action that would cause a Docker API error (e.g., try to stop a container that doesn't exist, or an action on an invalid container name). The sub-workflow should return a structured object with `success: false` and an `error` object containing `workflow`, `node`, `message` fields — visible in the n8n execution log output.
|
||||
result: pass
|
||||
|
||||
### 6. Debug Commands Removed (No False Triggers)
|
||||
expected: Send "/errors" or "/debug status" as a message to the bot. The bot should NOT crash or produce unexpected behavior. These commands were removed — the bot should either ignore them or treat them as unrecognized input.
|
||||
result: issue
|
||||
reported: "/errors gives the commands menu and /debug status returns 'no container found matching /debug'"
|
||||
severity: minor
|
||||
|
||||
## Summary
|
||||
|
||||
total: 6
|
||||
passed: 2
|
||||
issues: 4
|
||||
pending: 0
|
||||
skipped: 0
|
||||
|
||||
## Gaps
|
||||
|
||||
- truth: "Correlation ID generated and visible in execution logs for text commands"
|
||||
status: failed
|
||||
reason: "User reported: generate correlation IDs are not wired with connections. Additionally there are 8 orphan nodes"
|
||||
severity: major
|
||||
test: 2
|
||||
root_cause: ""
|
||||
artifacts: []
|
||||
missing: []
|
||||
debug_session: ""
|
||||
|
||||
- truth: "Correlation ID generated and visible in execution logs for callback commands"
|
||||
status: failed
|
||||
reason: "User reported: Same as above — callback correlation ID node also not wired"
|
||||
severity: major
|
||||
test: 3
|
||||
root_cause: ""
|
||||
artifacts: []
|
||||
missing: []
|
||||
debug_session: ""
|
||||
|
||||
- truth: "Sub-workflows receive correlationId from main workflow"
|
||||
status: failed
|
||||
reason: "User reported: Same as above — correlation IDs can't reach sub-workflows since generators aren't wired"
|
||||
severity: major
|
||||
test: 4
|
||||
root_cause: ""
|
||||
artifacts: []
|
||||
missing: []
|
||||
debug_session: ""
|
||||
|
||||
- truth: "Removed debug commands don't trigger unexpected behavior"
|
||||
status: failed
|
||||
reason: "User reported: /errors gives the commands menu and /debug status returns 'no container found matching /debug'"
|
||||
severity: minor
|
||||
test: 6
|
||||
root_cause: ""
|
||||
artifacts: []
|
||||
missing: []
|
||||
debug_session: ""
|
||||
Reference in New Issue
Block a user