UN-4042 [FIX] Bill the pages LLMWhisperer extracted, not every page in the file - #2266
Open
athul-rs wants to merge 1 commit into
Open
UN-4042 [FIX] Bill the pages LLMWhisperer extracted, not every page in the file#2266athul-rs wants to merge 1 commit into
athul-rs wants to merge 1 commit into
Conversation
…n the file `push_usage_details` opened the input file and counted its pages, so an adapter configured with `pages_to_extract = "1-5"` billed all 199 pages of a 199-page PDF while LLMWhisperer billed the 5 it actually read. LLMWhisperer already returns the number it bills. `assemble_document` (`tasks_v2.py:1834`) computes `processed_page_count` once and uses that same value for both `whisper_metadata` (line 2032) and `page_count_total` (line 2095), and the whole `assembled_document` reaches the adapter through `/whisper-retrieve`. The V2 adapter was discarding it. - `TextExtractionResult` carries an optional `page_count` - `LLMWhispererHelper.get_processed_page_count` reads and validates it off the response - `push_usage_details` prefers it, falling back to counting the file when an adapter reports nothing Scoped to PDFs. LLMWhisperer also applies its 3000-words rule to sheets and text where Unstract hardcodes 1 page; adopting that raises those bills and is tracked separately in UN-4043. The billed count can now only go down or stay equal, never up. CoverForce, July 2026: 1,622 documents, 24,078 pages charged and never extracted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zLkLjqzwoN5Lm8thPqV9p
|
Contributor
|
| Filename | Overview |
|---|---|
| unstract/sdk1/src/unstract/sdk1/adapters/x2text/dto.py | Adds an optional, backward-compatible page-count field to extraction results. |
| unstract/sdk1/src/unstract/sdk1/adapters/x2text/llm_whisperer_v2/src/helper.py | Safely reads a positive non-boolean integer from LLMWhisperer response metadata. |
| unstract/sdk1/src/unstract/sdk1/adapters/x2text/llm_whisperer_v2/src/llm_whisperer_v2.py | Propagates the validated processed-page count into the adapter result. |
| unstract/sdk1/src/unstract/sdk1/x2txt.py | Uses reported counts for PDF usage and preserves full-file and non-PDF fallback behavior. |
| unstract/sdk1/tests/test_x2txt_page_usage.py | Covers metadata validation, PDF fallback accounting, non-PDF behavior, and end-to-end propagation. |
Sequence Diagram
sequenceDiagram
participant LW as LLMWhisperer V2
participant A as X2Text Adapter
participant X as X2Text
participant U as Usage Audit
LW-->>A: whisper_metadata.processed_page_count
A-->>X: TextExtractionResult(page_count)
alt Valid count and PDF
X->>U: Report processed page count
else Missing count and PDF
X->>X: Count pages with pdfplumber
X->>U: Report full PDF page count
else Non-PDF
X->>U: Report one page
end
Reviews (1): Last reviewed commit: "UN-4042 [FIX] Bill the pages LLMWhispere..." | Re-trigger Greptile
Contributor
Unstract test resultsPer-group results
Critical paths
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
push_usage_detailsopened the input file and counted its pages, so an adapter configured withpages_to_extract = "1-5"billed all 199 pages of a 199-page PDF while LLMWhisperer billed the 5 it actually read.Real case — CoverForce, 27 July 2026: a 199-page PDF, 5 pages requested, 5 processed, completed successfully. LLMWhisperer billed 5. Unstract billed 199.
Across July for that org alone: 1,622 documents, 24,078 pages charged and never extracted.
Why this approach
LLMWhisperer already returns the number it bills, and it was reaching the adapter and being thrown away.
assemble_document()computesprocessed_page_countonce (unstract-llm-whisperer/backend/tasks_v2.py:1834) and uses that same local in both places:tasks_v2.py:2032whisper_metadata— what this PR readstasks_v2.py:2095page_count_total=— what LLMWhisperer billsThere is no second derivation, so the two meters cannot drift. The whole
assembled_documentalready travels to the SDK via/whisper-retrieve→client_v2.py:621→helper.make_request→ the V2 adapter, which kept onlywhisper_hashandline_metadata.No LLMWhisperer-side change is required. Re-implementing the page-range rule inside the SDK was considered and rejected — it would be a second implementation to keep in sync, which is the bug we are fixing.
Change
TextExtractionResultcarries an optionalpage_count. Additive with a default, so the five adapters that report nothing are unaffected.LLMWhispererHelper.get_processed_page_count()reads and validates it off the response.boolis rejected explicitly, sinceisinstance(True, int)isTrueand would silently bill 1 page.push_usage_details()prefers it and falls back to counting the file when it is absent or unusable.Side effect: PDFs on the LLMWhisperer path are no longer re-read and parsed by
pdfplumber. One fewer full-file read and parse per document.Scope
PDFs only. LLMWhisperer also applies its 3,000-words-per-page rule to spreadsheets and text (
tasks_v2.py:1836-1857) where Unstract hardcodes 1 page. Adopting that here would fix a second gap but would raise those bills the day it ships, so it stays in UN-4043 where account owners can be warned first.pages_to_extract=1-5, 199 pagesInvariant: the billed count can only go down or stay equal. Never up.
Reviewer notes
A second behaviour change beyond
pages_to_extract.processed_page_count = len(page_nos)counts pages actually assembled. If a single page fails inside an otherwise-successful document,processed < totaleven with no range configured, so that document now bills less than before. This is correct — we bill what we extracted, matching LLMWhisperer — and it only ever reduces, but it is worth knowing about.Revenue impact is real and intended. CoverForce July would move 55,721 → ~32,189, a 42% reduction for that account. Every customer using the "Page number(s) or range to extract" setting is affected.
LLMWhisperer V1 is not fixed. It has the same bug and the same setting, but its API returns raw text with no page count — the response carries nothing to read. It is deprecated in its own schema (
llm_whisperer/src/static/json_schema.json:9). Left alone deliberately.Not in this PR: the dead
MimeType.TEXTbranch atx2txt.py:110-114reads the whole file into memory, assignstext_extraction_result, and is overwritten unconditionally by the next line. Real and confirmed inert by coverage, but unrelated — kept out to keep this diff reviewable.Import churn: the two import-block edits are from the pinned pre-commit
ruff; those files were already failing lint onmain.Testing
New
unstract/sdk1/tests/test_x2txt_page_usage.py— 20 tests, all passing. Every changed line is covered.pdfplumber, unchanged0,-3,True,"5",5.0→ all rejected, falls backX2Text.process()forwards the adapter's count end to endLLMWhispererV2.process()lifts it fromwhisper_metadata, and returnsNonewhen absentFull SDK suite: 494 passed. The 4 failures in
test_llm_compat.pyare pre-existing —pytest-asynciois not installed in the dev venv — and reproduce identically on a stashed tree.Jira: UN-4042 · follow-up: UN-4043
🤖 Generated with Claude Code
https://claude.ai/code/session_019zLkLjqzwoN5Lm8thPqV9p