You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The model router preserved upstream status codes and bodies but discarded end-to-end response metadata. In particular, a runtime returning 429/503 lost Retry-After, request correlation IDs, and cache directives, leaving OpenAI-compatible clients unable to back off correctly or correlate failures.
Root cause and invariant
Both streaming and buffered response constructors passed only ODS-generated headers. The invariant is that end-to-end upstream headers survive both response paths, while hop-by-hop fields, connection-nominated fields, stale body framing/encoding, and ODS-owned metadata never cross incorrectly.
A shared response filter now removes hop-by-hop headers, Content-Length, Content-Encoding, every token named by Connection, and case-insensitive collisions with ODS headers. Remaining headers are merged into both response paths.
Overlap check
Searched open and closed PRs for model router response headers, Retry-After, and the changed production file. Same-file open PRs are #2843 (MLX runtime family) and #2719 (internal-key auth); neither covers response metadata. Token Spy header PR #2971 affects a different proxy and implementation.
Regression coverage
A FastAPI boundary test covers buffered and streaming requests. It asserts preservation of Retry-After, an upstream request ID, and Cache-Control, while proving Connection, its nominated private header, and a stale Content-Length do not leak. The full router suite also protects existing ODS header behavior; an initial duplicate-case collision was caught and fixed before commit.
The filter preserves arbitrary end-to-end extension headers because model runtimes use provider-specific rate-limit and correlation names. Body framing is always regenerated after decoding/rewriting. Revert is isolated and requires no state migration.
Validated merge order: #2989 ? #2990 ? #2993 ? #2991 ? #2992 ? #2994 ? #2995 ? #2996 ? #2997 ? #2998. The changes are independently useful; this order only reconciles shared model-router and magic-link files.
Synthetic integration head: origin/batch/quality-ten-20260822-round2-integration at 91eb730d. The only textual conflict was the two model-router tests inserting at the same class boundary; the integration resolution retains both contracts. Magic-link changes merged cleanly.
All required GitHub checks are green across the batch. #2992 initially hit a transient openSUSE repository/mirror failure while installing rsync; a clearly labeled empty retry commit reran the unchanged tree, and openSUSE plus the full matrix passed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this matters
The model router preserved upstream status codes and bodies but discarded end-to-end response metadata. In particular, a runtime returning 429/503 lost
Retry-After, request correlation IDs, and cache directives, leaving OpenAI-compatible clients unable to back off correctly or correlate failures.Root cause and invariant
Both streaming and buffered response constructors passed only ODS-generated headers. The invariant is that end-to-end upstream headers survive both response paths, while hop-by-hop fields, connection-nominated fields, stale body framing/encoding, and ODS-owned metadata never cross incorrectly.
A shared response filter now removes hop-by-hop headers,
Content-Length,Content-Encoding, every token named byConnection, and case-insensitive collisions with ODS headers. Remaining headers are merged into both response paths.Overlap check
Searched open and closed PRs for
model router response headers,Retry-After, and the changed production file. Same-file open PRs are #2843 (MLX runtime family) and #2719 (internal-key auth); neither covers response metadata. Token Spy header PR #2971 affects a different proxy and implementation.Regression coverage
A FastAPI boundary test covers buffered and streaming requests. It asserts preservation of
Retry-After, an upstream request ID, andCache-Control, while provingConnection, its nominated private header, and a staleContent-Lengthdo not leak. The full router suite also protects existing ODS header behavior; an initial duplicate-case collision was caught and fixed before commit.Validation
pytest -q ods/extensions/services/model-router/tests/test_router.py -x— 50 passedpython -m py_compile ods/extensions/services/model-router/app/main.py ods/extensions/services/model-router/tests/test_router.pygit diff --checkTradeoffs and rollback
The filter preserves arbitrary end-to-end extension headers because model runtimes use provider-specific rate-limit and correlation names. Body framing is always regenerated after decoding/rewriting. Revert is isolated and requires no state migration.