fix(tracking): cap token estimates at the shell output truncation ceiling - #3635
Open
xantorres wants to merge 1 commit into
Open
fix(tracking): cap token estimates at the shell output truncation ceiling#3635xantorres wants to merge 1 commit into
xantorres wants to merge 1 commit into
Conversation
…ling estimate_tokens() had no ceiling, so a single large read (e.g. a 349KB file) could log tens of thousands of input tokens that no model context ever received — coding-agent shells commonly truncate captured Bash output well below that. Those inflated rows drown real savings in `rtk gain` aggregates. Clamp input/output token estimates at track/record time to tracking.estimate_cap_chars / 4 (default 30_000 chars, 0 disables the cap), using the same ceiling rounding as estimate_tokens() so the two never disagree at small remainders. RTK_TRACK_CAP_CHARS overrides the config value and is checked on every call; the config-derived fallback is resolved once per process. estimate_tokens() itself is unchanged; other callers (guard.rs, git.rs) are unaffected.
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
estimate_tokensis uncappedbytes/4: a singlertk readof a 349KB file records 87,224 input tokens. Coding-agent shells truncate captured command output (~30,000 chars is a common ceiling), so recorded savings past that point never existed in any model context, and one large row drowns thertk gainaggregate (a 0.0% efficiency meter over an otherwise honest run). CLAUDE.md already documents absolute counts as approximate — this makes the approximation honest.Fix
Clamp input and output token estimates at the ceiling inside
TimedExecution::track. Configurable:tracking.estimate_cap_charsin config.toml (default 30000, 0 disables) withRTK_TRACK_CAP_CHARSenv override taking precedence.estimate_tokensitself and its other callers are unchanged.Tests
7 new: input/output clamping, saved consistency, env override, 0-disables, config default and missing-field default. Full suite green in a clean environment.