feat(analytics,proxy): routing headers visualization and routing mode query param - #503
Open
yanaSelin wants to merge 6 commits into
Open
feat(analytics,proxy): routing headers visualization and routing mode query param#503yanaSelin wants to merge 6 commits into
yanaSelin wants to merge 6 commits into
Conversation
…report EPMCDME-14083
- Append ?routing=signal or ?routing=classifier to every upstream request EPMCDME-14083
…apture (#513) Co-authored-by: Sviatoslav Likhtarchyk <Sviatoslav_Likhtarchyk@epam.com> Co-authored-by: codemie-ai <codemie.ai@gmail.com>
… Routing KPI The Routing KPI section's routedSessions filter only matched sessions with a measured judge cost or an explicitly-unmeasured (LiteLLM) session, so any Switchyard session routed by heuristic (decision-source: dimensions, no LLM classifier invoked, judgeCostUSD stays null) silently fell out of the section entirely — even though routing genuinely happened. routingCostKnown is set (true or false) whenever a session had at least one routed turn, so checking `!= null` catches that case too. EPMCDME-14083
…ctor's prefix list x-codemie-routed-model is a real Switchyard response header — confirmed against 8943 real routed turns across local transcripts, it always matches the response body's own `model` (the actually-dispatched model) — but it doesn't share a prefix with x-codemie-routing-*/x-codemie-requested-*, so the proxy's CODEMIE_ROUTING_PREFIXES list silently dropped it. x_codemie_routing_capable_model, the field usage-readers.ts and statusline.mjs relied on for the dispatched model, was consequently never populated in any of those 8943 records; both already fell back correctly to message.model, so this was latent rather than user-visible, but routedModel/extractRoutedModel now resolve from real proxy data instead of by accident. EPMCDME-14083
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Extract Switchyard/LiteLLM routing metadata from proxy response headers stored in Claude native logs, expose it in the analytics report, and add an optional
?routing=<mode>query parameter for backend routing experiments.Changes
x-codemie-routing-*andx-litellm-router-*headers inusage-readers.tssimple/middle/complex/reasoningand decision sources to kebab-casemodelTimelinewith routing metadata; aggregate classifier/judge cost and tokensCODEMIE_ROUTINGenv var / profile config to append?routing=signal|classifierto upstream requestsTesting
npm run typechecknpm run lintChecklist
mainEPMCDME-14083