fix(opencode): resample turns with an unmapped finish reason - #156
Open
Alezander9 wants to merge 1 commit into
Open
fix(opencode): resample turns with an unmapped finish reason#156Alezander9 wants to merge 1 commit into
Alezander9 wants to merge 1 commit into
Conversation
A provider stream that opens, delivers a first chunk, then dies without sending a finish chunk is reported by the AI SDK as finishReason "other" with null usage. "other" is not in the FinishReason literal set, so session/llm/ai-sdk.ts maps it to "unknown". The two loop guards then disagreed. The error check at prompt.ts:1295 excludes "unknown", so no error was recorded. The loop-exit check did not, so the loop broke. The turn exited cleanly mid-task with no error, no timeout and no errored span, and the run was indistinguishable from a model that gave up. Measured on the Odysseys benchmark: 0.054% of LLM calls, ~5.3% of tasks (11/200 and 12/200 on two full runs), and 100% of them terminal. "unknown" is the fallback in every mapper (openai-chat, openai-responses, anthropic-messages, gemini, bedrock-converse) and never denotes a normal completion, so exclude it from the loop-exit check as well and let the turn resample. The dead turn contributes no model messages, so the repeat request is identical to the one that was dropped. A warning log keeps the event visible when the resample succeeds.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A provider stream that opens, delivers a first chunk, then dies without sending a finish chunk is reported by the AI SDK as
finishReason: "other"with null usage."other"is not in ourFinishReasonliteral set (packages/llm/src/schema/ids.ts:39), sosession/llm/ai-sdk.ts:22maps it to"unknown".The two loop guards in
session/prompt.tsthen disagreed:"unknown"tool-calls,unknowntool-callsThe turn exited cleanly mid-task with no error, no timeout and no errored span. The run was indistinguishable from a model that gave up.
This changes the loop-exit check to exclude
"unknown"as well, so the turn resamples instead of ending the run, and adds a warning log so the event stays visible when the resample succeeds.Why
"other"means "the stream died"stepFinishReasonis initialised to"other"in the AI SDK (node_modules/ai/dist/index.mjs:7246) and is only overwritten when a real finish chunk arrives. The AI SDK still emitsfinish-stepwhen the socket closes, so"other"reaches us with the start value intact.Impact
Measured on the Odysseys benchmark with gpt-5.6-luna xhigh, bcode v0.1.20:
final_resulton the affected tasks is the agent's opening plan ("I'll work through the requested storefronts in sequence..."), never a completion.Removing the affected tasks moves that arm from 93.7 rubric / 86.0 perfect to 98.1 / 91.5 — the entire measured gap to the reference harness.
opus-5 appears unaffected on a different provider path (0 / 10,550 spans; expected 5.7 at this rate, P(0)=0.003).
Why this is safe
"unknown"never denotes a normal completion. It is the fallback branch in every mapper:openai-chat.ts:383,openai-responses.ts:528,anthropic-messages.ts:563,gemini.ts:376,bedrock-converse.ts:436. The fix therefore also covers the native runtime, not just the AI SDK bridge.step-startpart.toModelMessagesEffectproduces an assistant message with no content blocks, and the AI SDK drops empty blocks (processBlock()returns early onblock.length === 0), so no empty assistant message is sent to the provider."unknown"is not a completion; this applies the same rule at the second site rather than inventing a new one.Alternative considered
Routing this through
SessionRetryinstead (new error class,maxTotalAttempts, backoff). Rejected for now:AssistantErrorSchema, which is a public wire contract change and drags regenerated SDK output into the diff.SessionRetryresamples the same assistant message, so it would have to wipe already-streamed parts (as theOutputLengthErrorpath does viaresetOutputLimit()); loop continuation preserves any partial work instead.retryable()already returns nomaxTotalAttemptsfor transient API errors (session/retry.ts:140), and retry.ts:28 documents that as intentional.Verification
bun typecheckinpackages/opencode: clean.loop resamples when the provider closes a stream without a finish reasontotest/session/prompt.test.ts, mirroring the existingloop continues when finish is tool-callstest. It queues a reply with no finish chunk, then a normal reply, and asserts two calls and a"stop"finish.Note:
test/session/prompt.test.tsdoes not run on macOS — 31/58 tests fail on a clean tree, because the fixture's temp-diropencode.jsonis not picked up (project root resolves to/) and model resolution falls through to a liveopencode/big-pickle. CI has to validate the new test.Summary by cubic
Resamples turns that end with an unmapped finish reason ("unknown") and logs a warning. Previously, dead streams mapped to "unknown" broke the loop without recording an error; now the loop continues and retries instead of silently truncating the run.
packages/opencode/src/session/prompt.tsto exclude "unknown" (like "tool-calls") and adds a warning log with session/message IDs; adds a test inpackages/opencode/test/session/prompt.test.tsfor streams that close without a finish chunk; no migration required and normal completions are unchanged.Written for commit 1fdb393. Summary will update on new commits.