fix: surface full error message in agent failure issue comments#22689
fix: surface full error message in agent failure issue comments#22689
Conversation
Expand buildEngineFailureContext() to capture more error patterns (Error:, Fatal:, FATAL:, panic:) and fall back to the last 10 non-empty lines of agent-stdio.log when no specific pattern matches. This ensures that agent failures caused by step timeouts or unexpected terminations always include useful context in the failure issue comment, rather than just 'Agent job [run_id](run_url) failed.' Also exports buildEngineFailureContext so it can be unit-tested, and adds comprehensive tests for all patterns. Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com> Agent-Logs-Url: https://github.com/github/gh-aw/sessions/1bc80351-f450-45a5-a967-a395c489db7e
There was a problem hiding this comment.
Pull request overview
Improves failure issue/comment diagnostics by extracting more actionable context from agent-stdio.log when the agent fails without a categorized error.
Changes:
- Expanded
buildEngineFailureContext()pattern matching to includeError:,Fatal:/FATAL:, andpanic:lines. - Added a fallback that surfaces the last 10 non-empty
agent-stdio.loglines when no known patterns match. - Exported
buildEngineFailureContext()and added unit tests covering patterns, deduplication, and fallback behavior.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| actions/setup/js/handle_agent_failure.cjs | Enhances engine-failure context extraction from agent-stdio.log and exports the helper for testing. |
| actions/setup/js/handle_agent_failure.test.cjs | Adds unit tests validating the new parsing patterns and the tail-lines fallback behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| let context = "\n**⚠️ Engine Failure**: The AI engine terminated unexpectedly.\n\n**Last agent output:**\n```\n"; | ||
| context += tailLines.join("\n"); |
There was a problem hiding this comment.
The fallback wraps raw tail log lines in a fenced code block. If any of the last lines begin with a fence (e.g., or ~~~, which can happen when the agent logs markdown/tool output), it can prematurely close the block and break the formatting of the entire failure comment/issue. Consider escaping/neutralizing fence lines (e.g., indenting each emitted log line by 4 spaces, choosing a fence longer than any fence appearing in the payload, or running the content through the existing markdown code-region balancer) before inserting into the fenced block.
| let context = "\n**⚠️ Engine Failure**: The AI engine terminated unexpectedly.\n\n**Last agent output:**\n```\n"; | |
| context += tailLines.join("\n"); | |
| // Neutralize any leading Markdown fence lines in the tail output so they | |
| // cannot prematurely close the surrounding fenced code block. | |
| const sanitizedTailLines = tailLines.map(line => { | |
| if (line.startsWith("```") || line.startsWith("~~~")) { | |
| return " " + line; | |
| } | |
| return line; | |
| }); | |
| let context = "\n**⚠️ Engine Failure**: The AI engine terminated unexpectedly.\n\n**Last agent output:**\n```\n"; | |
| context += sanitizedTailLines.join("\n"); |
| expect(result).toContain("Last agent output"); | ||
| expect(result).toContain("Agent interrupted"); | ||
| }); | ||
|
|
There was a problem hiding this comment.
The new fallback behavior is tested, but there’s no coverage for the markdown-safety edge case where the tailed log contains a line starting with a fence (e.g. ```), which would currently break the generated comment formatting. Adding a unit test for this case would help prevent regressions once the fallback output is hardened.
| it("falls back safely when last lines include a markdown fence", () => { | |
| const logLines = [ | |
| "Starting agent...", | |
| "Running tool: list_branches", | |
| "```bash", | |
| "echo 'hello from agent'", | |
| "```", | |
| "Agent interrupted", | |
| ]; | |
| fs.writeFileSync(stdioLogPath, logLines.join("\n") + "\n"); | |
| const result = buildEngineFailureContext(); | |
| expect(result).toContain("Engine Failure"); | |
| expect(result).toContain("Last agent output"); | |
| // Ensure the markdown fence line from the log is present in the fallback output | |
| expect(result).toContain("```bash"); | |
| expect(result).toContain("Agent interrupted"); | |
| }); |
When the agent fails without triggering a specific categorized error (code push failure, assignment error, etc.), the failure issue comment contained only
"Agent job [run_id](run_url) failed."— no actionable context. This happened becausebuildEngineFailureContext()only matched two narrow patterns inagent-stdio.log(ERROR:prefix andReconnecting...), leaving step-timeout kills and most real Copilot CLI failures silent.Changes
Expanded error pattern matching in
buildEngineFailureContext():Error: <msg>— Node.js-style errorsFatal: <msg>/FATAL: <msg>— fatal errorspanic: <msg>— Go runtime panicsAdded tail-lines fallback: when no known pattern matches (e.g. agent killed by step timeout), the last 10 non-empty lines of
agent-stdio.logare surfaced as aLast agent outputcode block — giving reviewers the agent's final activity even when there's no error prefixExported
buildEngineFailureContextand added 14 unit tests covering all patterns, deduplication, and the fallback behaviorBefore / After
Before (step-timeout failure):
After:
Warning
Firewall rules blocked me from connecting to one or more addresses (expand for details)
I tried to connect to the following addresses, but was blocked by firewall rules:
https://api.github.com/graphql/usr/bin/gh /usr/bin/gh api graphql -f query=query($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { hasDiscussionsEnabled } } -f owner=github -f name=gh-aw GO111MODULE 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git GoFiles,Compiledgit **/*.cjs 64/pkg/tool/linu--show-toplevel git(http block)https://api.github.com/orgs/test-owner/actions/secrets/usr/bin/gh gh api /orgs/test-owner/actions/secrets --jq .secrets[].name -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env json' --ignore-path ../../../.pr**/*.json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/ai-inference/git/ref/tags/v1/usr/bin/gh gh api /repos/actions/ai-inference/git/ref/tags/v1 --jq .object.sha --show-toplevel /tmp/go-build1697151324/b441/_testmain.go /usr/bin/git -json GO111MODULE x_amd64/vet git conf�� user.name Test User /usr/bin/gh -json GO111MODULE x_amd64/vet gh(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v3/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v3 --jq .object.sha -bool -buildtags ache/node/24.14.0/x64/bin/node -errorsas -ifaceassert -nilfunc /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/compile t-33�� sistency_WithImports633606976/001/main.md -trimpath /usr/bin/git -p main -lang=go1.25 git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v5/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha te '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.pret.prettierignore GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet ache�� che/go-build/ab/abfc11f840b03ef92b174a83f4a6923b-errorsas GOPROXY .cfg rkflow/js/**/*.jgit GOWORK erignore ache/go/1.25.0/x64/pkg/tool/linuremote.origin.url(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha github.repository x_amd64/vet /usr/bin/git */*.json' '!../.git GO111MODULE 64/pkg/tool/linu--show-toplevel git rev-�� --show-toplevel 64/pkg/tool/linux_amd64/vet /usr/bin/git 6366978/b402/_pkgit .cfg 64/pkg/tool/linu--show-toplevel git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v5 --jq .object.sha --show-toplevel ache/go/1.25.0/x64/pkg/tool/linutest@example.com /usr/bin/git /ref/tags/v8 node ache/go/1.25.0/x--show-toplevel git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git prettier --write 7151324/b181/vet--show-toplevel git(http block)https://api.github.com/repos/actions/checkout/git/ref/tags/v6/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha user.name Test User /usr/bin/git ./../pkg/workflogit GO111MODULE x_amd64/vet git -C nt/action/git/ref/tags/v999.999.999 status /usr/bin/unpigz .github/workflowgit 1bb61422:go.mod x_amd64/vet /usr/bin/unpigz(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha k/gh-aw/gh-aw/.github/workflows/bot-detection.md go /usr/bin/git d GO111MODULE x_amd64/vet git rev-�� --show-toplevel x_amd64/vet /usr/bin/git g_.a GO111MODULE x_amd64/vet git(http block)/usr/bin/gh gh api /repos/actions/checkout/git/ref/tags/v6 --jq .object.sha --show-toplevel eutil.test /usr/bin/git --noprofile node .cfg git rev-�� --show-toplevel ache/go/1.25.0/x64/pkg/tool/linux_amd64/vet /usr/bin/git 1740-37922/test-git **/*.cjs ache/go/1.25.0/x--show-toplevel git(http block)https://api.github.com/repos/actions/github-script/git/ref/tags/v8/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha th .prettierignore GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE tions/setup/node-nilfunc GOINSECURE GOMOD igFiles,SwigCXXF-bool go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 0/x64/bin/npx GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh api /repos/actions/github-script/git/ref/tags/v8 --jq .object.sha -json GO111MODULE odules/npm/node_-nilfunc GOINSECURE GOMOD GOMODCACHE go 0/x6�� th .prettierigno-errorsas GO111MODULE ode_modules/.bin-nilfunc GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/actions/setup-go/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-go/git/ref/tags/v4 --jq .object.sha --get remote.origin.url /usr/bin/git ./../pkg/workflogit GO111MODULE x_amd64/vet git -C /tmp/gh-aw-test-runs/20260324-131740-37922/test-823574591 status /usr/bin/git .github/workflowgit GO111MODULE x_amd64/vet git(http block)https://api.github.com/repos/actions/setup-node/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/setup-node/git/ref/tags/v4 --jq .object.sha add origin /usr/bin/git ./../pkg/workflogit GO111MODULE x_amd64/vet git rev-�� --git-dir x_amd64/vet /usr/bin/git -json GO111MODULE x_amd64/vet git(http block)https://api.github.com/repos/actions/upload-artifact/git/ref/tags/v4/usr/bin/gh gh api /repos/actions/upload-artifact/git/ref/tags/v4 --jq .object.sha 7151324/b435/sliceutil.test /tmp/go-build1697151324/b057/vet.cfg 7151324/b435/importcfg.link -json GO111MODULE 64/bin/go 4vdN3eqAa1K0H/u1ogOOgsHoxcKHcseOEC/vKfaaJsUtRlxju1NbgiD/uvljh3C4vdN3eqAa1K0H -uns�� ithub-script/git/ref/tags/v8 /tmp/go-build1697151324/b220/vet.cfg 7151324/b435/_pkg_.a -json GO111MODULE 64/bin/go /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.0.0 --jq .object.sha -unreachable=false /tmp/go-build1697151324/b089/vet.cfg 0/x64/bin/node -json GO111MODULE(http block)https://api.github.com/repos/github/gh-aw-actions/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw-actions/git/ref/tags/v1.2.3 --jq .object.sha runs/20260324-131740-37922/test-926136587/.github/workflows /tmp/go-build1697151324/b123/vet.cfg 7151324/b326/vet.cfg -json GO111MODULE /sh /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet -uns�� k/gh-aw/gh-aw/.github/workflows /tmp/go-build1697151324/b230/vet.cfg 0/x64/bin/node h ../../../.pretgit GO111MODULE es/.bin/sh /opt/hostedtoolcache/go/1.25.0/x64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/runs/1/artifacts/usr/bin/gh gh run download 1 --dir test-logs/run-1 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD erignore 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12345/artifacts/usr/bin/gh gh run download 12345 --dir test-logs/run-12345 GO111MODULE 64/pkg/tool/linu-nolocalimports GOINSECURE GOMOD erignore 64/pkg/tool/linu/tmp/go-build1697151324/b441/_testmain.go env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD ode-gyp-bin/sh 64/pkg/tool/linuTest User(http block)https://api.github.com/repos/github/gh-aw/actions/runs/12346/artifacts/usr/bin/gh gh run download 12346 --dir test-logs/run-12346 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD erignore 64/pkg/tool/linux_amd64/vet env -json GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com(http block)https://api.github.com/repos/github/gh-aw/actions/runs/2/artifacts/usr/bin/gh gh run download 2 --dir test-logs/run-2 GO111MODULE x_amd64/link GOINSECURE GOMOD erignore x_amd64/link env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE ef/N6GE9dzJuLpfUe9tz4e_/ThKvzodBlPIPkS6j74YO(http block)https://api.github.com/repos/github/gh-aw/actions/runs/3/artifacts/usr/bin/gh gh run download 3 --dir test-logs/run-3 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD erignore 64/pkg/tool/linux_amd64/vet env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/runs/4/artifacts/usr/bin/gh gh run download 4 --dir test-logs/run-4 GO111MODULE x_amd64/compile GOINSECURE GOMOD erignore x_amd64/compile env -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/runs/5/artifacts/usr/bin/gh gh run download 5 --dir test-logs/run-5 GO111MODULE 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD erignore 64/pkg/tool/linux_amd64/vet estl�� -json .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/actions/workflows/usr/bin/gh gh workflow list --json name,state,path -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env json' --ignore-path ../../../.pr--ignore-path GO111MODULE tions/setup/js/node_modules/.bin--log-level=error GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 100 GOMOD GOMODCACHE go env ./../pkg/workflow/js/**/*.json' --ignore-path GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)/usr/bin/gh gh run list --json databaseId,number,url,status,conclusion,workflowName,createdAt,startedAt,updatedAt,event,headBranch,headSha,displayTitle --workflow nonexistent-workflow-12345 --limit 6 GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 6366978/b425/_pkg_.a .cfg 64/pkg/tool/linux_amd64/vet nore GOMOD GOMODCACHE 64/pkg/tool/linutest@example.com(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.0.0 --jq .object.sha -json .cfg At,event,headBranch,headSha,displayTitle nore 75bfcd2886f5dc9brev-parse GOMODCACHE 64/pkg/tool/linux_amd64/vet env te '**/*.cjs' '**/*.ts' '**/*.json' --ignore-path ../../../.prettierignore .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v1.2.3/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v1.2.3 --jq .object.sha on' --ignore-path ../../../.prettierignore GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go env */*.ts' '**/*.json' --ignore-path ../../../.prettierignore GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v2.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha on' --ignore-path ../../../.prettierignore GO111MODULE tions/setup/js/node_modules/.bin/node GOINSECURE GOMOD GOMODCACHE FMXAZENQ7Ra6 tion�� -json GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE sh GOINSECURE GOMOD GOMODCACHE go tion�� ../pkg/workflow/-s GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v2.0.0 --jq .object.sha -json GO111MODULE n-dir/node GOINSECURE GOMOD GOMODCACHE go tion�� ../pkg/workflow/-test.timeout=10m0s GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)https://api.github.com/repos/github/gh-aw/git/ref/tags/v3.0.0/usr/bin/gh gh api /repos/github/gh-aw/git/ref/tags/v3.0.0 --jq .object.sha -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go tion�� ../pkg/workflow/-errorsas GO111MODULE x_amd64/vet GOINSECURE GOMOD GOMODCACHE x_amd64/vet(http block)https://api.github.com/repos/nonexistent/action/git/ref/tags/v999.999.999/usr/bin/gh gh api /repos/nonexistent/action/git/ref/tags/v999.999.999 --jq .object.sha -json .cfg 64/pkg/tool/linux_amd64/vet nore GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env ithout_min-integrity603508619/001 .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/nonexistent/repo/actions/runs/12345/usr/bin/gh gh run view 12345 --repo nonexistent/repo --json status,conclusion ignore GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet env 3647897819/.github/workflows .cfg 64/pkg/tool/linux_amd64/vet GOINSECURE GOMOD GOMODCACHE 64/pkg/tool/linux_amd64/vet(http block)https://api.github.com/repos/owner/repo/actions/workflows/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo node GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ode GOINSECURE GOMOD GOMODCACHE go(http block)/usr/bin/gh gh workflow list --json name,state,path --repo owner/repo 64/bin/go GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE ache/go/1.25.0/x64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/owner/repo/contents/file.md/tmp/go-build1697151324/b402/cli.test /tmp/go-build1697151324/b402/cli.test -test.testlogfile=/tmp/go-build1697151324/b402/testlog.txt -test.paniconexit0 -test.v=true -test.parallel=4 -test.timeout=10m0s -test.run=^Test -test.short=true GOINSECURE GOMOD GOMODCACHE go env -json GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go(http block)https://api.github.com/repos/test-owner/test-repo/actions/secrets/usr/bin/gh gh api /repos/test-owner/test-repo/actions/secrets --jq .secrets[].name th .prettierigno-errorsas GO111MODULE 64/bin/go GOINSECURE GOMOD GOMODCACHE go env json' --ignore-p-errorsas GO111MODULE ache/go/1.25.0/x-nilfunc GOINSECURE GOMOD GOMODCACHE go(http block)If you need me to access, download, or install something from one of these locations, you can either:
📍 Connect Copilot coding agent with Jira, Azure Boards or Linear to delegate work to Copilot in one click without leaving your project management tool.