feat(web): make the agent-session Overview a verdict-first triage page - #637
Open
JeremyFunk wants to merge 4 commits into
Open
feat(web): make the agent-session Overview a verdict-first triage page#637JeremyFunk wants to merge 4 commits into
JeremyFunk wants to merge 4 commits into
Conversation
The Overview answered "what happened" but never "was that okay?" — its turn-by-turn digest was a fourth turn browser next to Traces, Flow and Transcript, and on a failed session nothing linked the page to the failing span. The page now leads with a verdict (failed / completed with findings / completed cleanly) and a findings list: discrete claims in the instrumentation's own vocabulary, each linking the span that is its evidence via the Traces view. Detectors are deterministic reads of the captured spans — grouped failure events, truncated replies (finish_reason length, deduped at the deepest reporter), the same tool called 8+ times within one turn, and 30s+ stalls inside a turn. A per-turn health strip replaces the digest for spatial orientation; the time bar and the rail stay, minus the rail's Failures section, which the findings list absorbs. "Terminal" means the group containing the span the final turn died on, not "occurred in the final turn" — a retried rate limit in the failing turn stays a non-terminal finding.
A failed tool call is recorded as a value on an Ok span — error.type tool_error with the message in gen_ai.tool.call.result and no status message — so the finding row rendered bare, answering neither what went wrong nor whether it is actionable. The evidence line now falls back to the tool call's recorded result, read tolerantly: Maple's own agent stamps a plain string, other vendors wrap the message in an object or an MCP-style content array.
Maple's own toolCallJson records a bare error string as {"result": "…"}
(and an over-budget one as {truncated, prefix}), so the finding row's
prose walk found no message under its error-shaped keys and rendered
bare. Verified against the captured spans in the warehouse: the failing
query_data call carries exactly that envelope. `result` and `prefix`
join the keys the walk tries.
…ronological The verdict sat above the two-column split, pushing the rail down and leaving dead space beside the findings — it now leads the left column, so Cost by model top-aligns with it. The time bar draws chronological occupancy intervals (a mid-session stall is a hole where it happened, not an idle block pinned left) while the legend keeps summed segments; sub-half-percent slivers stay on the bar and drop only from the legend. The shape strip's caption says "with errors" rather than "failed" — a red cell marks a turn something went wrong inside, which may well have closed cleanly, and "failed" contradicted a Completed verdict above it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
The session detail Overview now answers the question a reader opens it with — did this session work, and if not, what exactly went wrong — instead of offering a fourth way to browse the turns next to Traces, Flow and Transcript.
Failed — <label> on the final turn(with an Open failing span deep link),Completed, with N findings, orCompleted cleanly— the clean state says what the claim covers (no errors, refusals, truncated replies, stalls, or repetition).Detectors (
session-findings.ts)All deterministic reads of captured spans, no scoring or inference:
failureEvents, so verdict, list and counts cannot disagree), with prompt-growth evidence on context-window deaths where usage was reportedfinish_reasonlength/max_tokens, deduped at the deepest reporter"Terminal" means the group containing the span the final turn died on, not "occurred in the final turn" — a retried rate limit inside the failing turn stays a non-terminal finding.
Verification
/lab/agent-session: verdict and findings render over the fixture, the strip flags the failing turns, and clicking a finding lands in Traces with the span expanded and its error banner showing.Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.