Intelligence-mode runs break with strict providers (z.ai GLM / Anthropic): delta-only messages, silent empty replies, LangChain-shaped tool calls in history

[MAB2908]

Summary

Running a Bot on a strict OpenAI-compatible provider (z.ai GLM, https://api.z.ai/api/coding/paas/v4, model glm-5.3) fails in three related places, all rooted in how intelligence-mode runs and persisted history are handled. OpenAI tolerates all three; GLM (and partly Anthropic) rejects them. Net effect for the user: the agent "stalls" after every browser-executed tool call.

Deployment: openbot v0.0.4 (main @ 6826e11), @copilotkit/runtime 1.68.3, intelligence mode, agents on the managed agent-langgraph runtime.

1. Follow-up runs carry only the message delta — no user turn — and strict providers reject them

In intelligence mode, after the surface executes a frontend tool (e.g. computer_run_command) and starts the next run, the client sends only the new messages:

[ assistant(content: null, toolCalls: […]), tool(result) ]

handleIntelligenceRun fetches thread history only to compute persistedInputMessages (dedup for persistence) — it never merges history into canonicalInput.messages, so the agent receives a conversation with no user message at all.

z.ai rejects this with 400 {"code":1214,"message":"The messages parameter is illegal"}. Verified by direct API bisect: the same payload with any user message added → 200; without → 1214. Anthropic similarly requires the first message to be user.

Suggestion: hydrate canonicalInput.messages from platform history before invoking the agent (the runtime is the only side that owns full history), or document that providers must accept delta-only conversations.

Local workaround we run: agent-langgraph's toLangChainMessages injects a synthetic human "continuation" turn when the converted input contains no human message.

2. An empty model response silently ends the run mid-work

GLM occasionally returns a response with no content and no tool calls. The answer node emits it as-is; the conditional edge sees no tool calls and ends the run. The transcript just stops — no error, no event, the Bot looks dead. Our logs show RUN_STARTED → RUN_FINISHED within ~3 s with zero content events.

Suggestion: treat an empty assistant message as retryable (bounded re-ask) or surface a visible error. Local workaround: in the answer node we re-invoke up to 3 times on an empty reply, then emit a visible fallback message instead of a blank turn.

3. agent-langgraph persists LangChain-shaped tool calls on interrupted runs, which later fails AG-UI validation

When a run is interrupted, persisted history contains tool calls shaped {id, name, args} instead of AG-UI {id, type: "function", function: {name, arguments}}. Consequences on the next run over that thread:

• server-side run validation rejects the whole run: 400 invalid_literal, expected "function", path: messages.N.toolCalls.0.type — the user cannot send any further message in that thread;
• the channel transcript renderer crashes reading toolCall.function.arguments (Cannot read properties of undefined).

We found 17 such tool calls in one thread after a couple of interrupted runs.

Suggestion: serialize messages to the AG-UI shape before persistence; consider normalizing on read for existing threads. Local workarounds: a server middleware that rewrites {name, args} → {type: "function", function: …} on incoming run bodies, and a normalizing guard in the transcript projection.

Environment

• openbot v0.0.4, Docker deployment
• @copilotkit/runtime 1.68.3, intelligence mode (managed CopilotKit Intelligence)
• Model: z.ai GLM-5.3 via coding endpoint (OpenAI-compatible, BOT_RESPONSES_API=false)
• The built-in Bots on the same deployment hit the same walls

Happy to provide full request/response captures or test a patch.

[Hotragn]

Went through this against the code because problem 3 overlaps something I had just changed. Reporting
what I found rather than a patch, including a correction to something I said elsewhere.

Problem 3, browser half: fixed in #207

The transcript crash reading toolCall.function.arguments was the same defect as #44.
app/src/lib/copilot/thread-messages.ts ended stored as Message[] — a cast, not a check — so
whatever the history store held went to setMessages and then to every projection. #207 parses each
stored turn against MessageSchema from @ag-ui/core, leaves out the ones that do not parse, and
says on screen how many were left out. A tool call shaped {id, name, args} fails
AssistantMessageSchema on toolCalls[].type, so those turns no longer reach a renderer.

Problem 3, the 400: I think #207 also stops it, and I was wrong to say otherwise

On #44 I wrote that the run-validation 400 was "a separate change on the server". Reading the runtime,
I do not think that is right, and the distinction matters for whoever picks this up.

handleIntelligenceRun uses thread history for nothing but id dedup
(dist/v2/runtime/handlers/intelligence/run.mjs:116):

const history = await runtime.intelligence.getThreadMessages({ threadId: canonicalThreadId, userId });
const historicMessageIds = new Set(history.messages.map((message) => message.id));
persistedInputMessages = input.messages.filter((message) => !historicMessageIds.has(message.id));

Which confirms your problem 1 — history is never merged into canonicalInput.messages — and also means
the 400 invalid_literal, expected "function" at messages.N.toolCalls.0.type is validating
input.messages, the messages the client sent, not the persisted rows.

The client's only route to holding those turns was reading them back through the cast above. So with
#207 the browser drops them on read and never sends them again, and the thread stops being unusable
without anything changing on the server. If you can retest a wrecked thread on main once #207 lands,
that would confirm or kill this — it is inference from the code, not something I reproduced, and you
have the thread with 17 of them in it.

What #207 does not do: the persisted rows stay malformed, and those turns are gone from the
transcript rather than repaired — visibly, with a count, but gone. Anything reading that thread's
history outside the browser still sees them.

Problem 1 is the one with nowhere in this repo to fix it

Hydrating canonicalInput.messages is inside handleIntelligenceRun, which OpenBot does not own —
it comes from @copilotkit/runtime. The seam this deployment does have is the intelligence client:
getThreadMessages is called by both the run handler and the history endpoint, and OpenBot already
subclasses CopilotKitIntelligence to override it (server/src/copilot.ts,
IntelligenceKnowingANewThread, added in #189 so a thread nobody has run reads as empty rather than
500). But an override there cannot add a user turn to the run input, because the run handler never
puts history into the input at all — so the fix for problem 1 is upstream in the runtime, or a change
to what the client sends, not something that seam reaches.

Worth saying plainly since you offered to test a patch: problem 1 is not fixable in this repository
without either a runtime change or the surface sending a fuller message list, and the second is a
design decision about what a follow-up run carries rather than a bug fix.

The root cause of problem 3 is still open

Neither #207 nor anything else explains why {id, name, args} got persisted. I checked both
shipped agents and neither emits that shape: agent-langgraph/src/history.ts:74-80 and
agent-bot/src/history.ts:63 both read AG-UI's call.function.arguments and convert into their
framework's args shape for internal use — inbound, not outbound. So the write is happening somewhere
else, and it only shows up on interrupted runs, which is a strong hint it is the interrupt path
persisting framework-shaped state rather than AG-UI events.

If you still have a deployment that reproduces it, the useful capture would be the persisted rows for
one interrupted run plus which agent produced them — that would say whether this is the platform, the
runtime's interrupt handling, or a third agent implementation.