Sprint 4 [FIX] Backend fixes: UN-2900, UN-2902, UN-2953, UN-3038, UN-3133, UN-3176, UN-3333 - #2257
Open
hari-kuriakose wants to merge 12 commits into
Open
Sprint 4 [FIX] Backend fixes: UN-2900, UN-2902, UN-2953, UN-3038, UN-3133, UN-3176, UN-3333#2257hari-kuriakose wants to merge 12 commits into
hari-kuriakose wants to merge 12 commits into
Conversation
…OCR threshold UN-2953: validate_adapter_permissions read adapter ids straight out of tool_meta and added them unconditionally, so a tool instance holding "" for an adapter id made the JSON schema validator compare "" against the UUID enum and raise. The error was logged but not handled, repeating every validation pass until the pod stopped answering health checks. Skips empty/missing ids and uses .get() so a missing key no longer raises KeyError. Also initialises adapter_id per iteration -- previously a disabled entry could re-add the previous loop's id. UN-3038: push_usage_details billed len(pdf.pages) for every PDF, ignoring the adapter's pages_to_extract range, so a 5-page extraction from a 100-page document was charged 100 pages. Narrows the count to the selected pages, handling ranges, open-ended ranges, overlaps and out-of-range values, and falling back to the full count when the setting is absent or unparseable so usage is never under-reported. UN-3333: adds word_confidence_threshold to the LLMWhisperer v2 adapter schema (number, default 0.3, 0.0-1.0) so it is configurable from the adapter UI. The parameter is implemented in the LLMWhisperer backend but was never exposed. Not added to the v1 schema, which predates the OCR tuning parameters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
KeywordTableIndex's retriever caps results with `num_chunks_per_query` (default 10), not `similarity_top_k`. The code passed similarity_top_k, which as_retriever accepts and ignores, so the configured limit never took effect -- a profile set to 3 chunks still retrieved 10. Passes num_chunks_per_query instead. Filed as a frontend ticket, but the setting was being displayed correctly; only the retriever ignored it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
_sanitize_for_bigquery documents a 15-significant-figure cap for PARSE_JSON compatibility, but derived a DECIMAL-place count from the value's magnitude. That only equals 15 significant figures for values >= 1. For 0.0053325 the magnitude is -2, so it asked for 17 decimals -- more precision than the safe zone permits -- and the value was returned unchanged. Formats with `.15g` so the significant-figure limit applies at any magnitude. Verified round-tripping for small values, Unix timestamps, large mantissas and binary-artifact values such as 0.1 + 0.2. NOTE: this corrects a real precision defect but is NOT confirmed to be the whole of the reported failure. The ticket's rejected value (0.0053325) is already representable and survives sanitization unchanged, so reproducing the BigQuery-side error needs the customer's table and PARSE_JSON expression. Flagged for follow-up rather than closed on this commit alone. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
litellm's ContextWindowExceededError derives from BadRequestError, so it was caught by the generic openai.APIError branch in parse_litellm_err and wrapped as a plain SdkError reading "Error from <provider>." plus the raw 400 text. Users hitting the token limit saw a generic failure and had to read worker logs to find the cause (reported on execution f308a67f-02a1-437e-8531-05e067a94e02). Adds a ContextWindowExceededError SdkError subclass, maps it ahead of the generic wrap, and returns early so the actionable guidance (reduce chunk size, limit pages extracted, or use a larger-context model) is not overwritten by the generic tail. The provider's own text is kept in a code block underneath for support. Other litellm errors are unchanged. Context from the source thread: the customer's PRIMARY complaint there was inconsistent JSON structure across questions, which Jagadeesh identified as a prompting issue, not this one. Only the mis-surfaced token-limit error is in scope for UN-3133. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
…traction
Single pass builds ONE combined prompt -- every field declared up front in a
single JSON schema, answered in one LLM call -- so no prompt's output exists to
feed another prompt's variable. The runtime reflects that: the enterprise
single_pass_extraction plugin calls the shared replacement service with
structured_output={}, and both replace_static_variable and
replace_dynamic_variable return the prompt UNCHANGED when their lookup misses.
The literal {{...}} is then sent to the LLM, silently degrading the answer.
This is not specific to custom_data, despite the ticket title. CUSTOM_DATA is
in fact the ONLY variable type that survives single pass, because it resolves
from the tool's own custom_data and never consults the variable map. STATIC and
DYNAMIC variables both fail on their own, with no custom_data involved:
static : "check {{invoice_number}}" -> "check {{invoice_number}}"
dynamic : "via {{https://.../x[cust_id]}}" -> unchanged
custom : "{{custom_data.client.name}}" -> "Acme GmbH" (works)
Adds find_unresolvable_single_pass_variables() and surfaces the result per
prompt as single_pass_unresolvable_variables when the tool has single-pass
enabled. Warning only -- deliberately NOT a save-time block, because existing
projects may already carry this combination and users toggle single pass on and
off; a hard refusal would break them retroactively and be order-dependent.
Classification pairs with VariableReplacementService in the worker, which keeps
its own copy of the variable regexes; noted in the docstring since drift would
make validation and runtime disagree.
Known gap: this covers Prompt Studio authoring, not an already-exported tool
running single pass via API deployment, which keeps failing silently until the
tool is re-saved. That argues for pairing this with placeholder-stripping at
runtime later, not for widening this change.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
Follow-up to f45d4e0, which computed the warning only in CustomToolSerializer.to_representation -- so it appeared on tool fetch but not on the prompt save response, and the UI had no fresh value after an edit. Moves the computation to ToolStudioPromptSerializer as a SerializerMethodField. That serializer is what the prompt CRUD view returns AND what CustomToolSerializer nests per prompt, so one implementation now covers both page load and save. Removes the duplicated assignment from CustomToolSerializer. Adds select_related("tool_id") to the prompt query in CustomToolSerializer.to_representation: the new field reads the parent tool's single_pass_extraction_mode, which would otherwise be one query per prompt -- the exact N+1 CustomToolListSerializer's docstring calls out. Verified: single-pass off -> [] regardless of content; on -> static and dynamic variables reported, custom_data excluded, empty prompt safe (5/5). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
Every google.api_core BadRequest was mapped to ColumnMissingException, whose message tells the user to "make sure all the columns exist in your table as per the destination DB configuration". BigQuery also returns BadRequest for VALUE-level failures -- including this ticket's "cannot round-trip through string representation; error in PARSE_JSON expression" -- so users were sent to check a schema that was never wrong. That misdirection is likely why this was filed as a datatype-conversion bug. Adds BigQueryValueException and discriminates before wrapping: prefers the structured errors[] payload, falls back to message signatures for the round-trip / PARSE_JSON / invalid-JSON cases BigQuery does not tag. Anything unrecognised falls through to the existing ColumnMissingException, so this only narrows messages that were already wrong. Verified 7/7 including the ticket's verbatim error text and two genuine missing-column messages that must NOT be reclassified. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011sXFBEu2GHatXq2CV2ShPF
for more information, see https://pre-commit.ci
) #2 _sanitize_for_bigquery: the UN-3176 comment said the old magnitude-derived decimal count over-preserved precision for values BELOW 1, citing 0.0053325 as a value "passed through unchanged". That is inverted. For magnitude m < 0 the value carries |m| leading zeros after the point, so `15 - m` decimals preserves exactly 15 significant figures; the old form was already correct for every m <= 15. It over-preserves only ABOVE 10^15, where the count floors at 0 and the full integer part is emitted. A 200k-sample sweep over magnitudes 10^-12 to 10^+20 shows old and new differ only at 10^15 and above -- and old(0.0053325) == new(0.0053325) exactly. The code change is right; the stated cause was not. #5 _is_value_error: the docstring claimed it "prefers the structured errors payload" because "invalidQuery covers the value-level rejections". Neither is true -- the message text is checked first and returns before the payload is reached, and `reason` is never inspected at all. Restated to describe what the function does. The e.errors loop is kept and is provably live: str() of a google.api_core BadRequest is just "400 <message>" and omits the payload, so a marker present only there is still matched. #6 serializers.py: drop a stray blank line added by the diff. No behaviour change.
) Adversarial verification refuted the causal story in the previous commit. It claimed the old and new forms agree below 10^15. They do not: when |x| sits just below a power of ten, log10 returns an exact integer (the true value is within half an ULP), so the derived magnitude is one too large and the old form emitted 14 significant figures, not 15. Reproduced against the shipped function -- old(9.99999999999999e-05) == 0.0001 while new() preserves all 15 digits, and the same happens in every decade below 1e-4. That is the second false account of this line in as many commits, so this one stops narrating the old form's failure mode and states only the invariant that matters: `g` asks for significant figures, the old form asked for decimal places, and the two are not the same quantity. _is_value_error: the "str(e) does not include the payload" claim was true for the path this code sees but stated absolutely. GoogleAPICallError.__str__ (api-core 2.24.2) folds an errors entry into the string only when it exposes .code/.message ATTRIBUTES; BigQuery's REST path fills errors with plain dicts, which fail that hasattr filter. Says that instead. Verified the loop is still load-bearing: a marker present only in the dict payload is absent from str(e) ('400 Schema mismatch on insert') and the function still returns True. No behaviour change. unstract/connectors tests/databases: 36 passed.
The PR shipped no tests. These pin the three behaviours whose regressions would be silent, and each was mutation-checked -- reverted or neutered the fix, confirmed the test fails, restored. BigQuery BadRequest discrimination (UN-3176), in the existing test_bigquery_db.py alongside the Forbidden/NotFound cases: - a value-level BadRequest routes to BigQueryValueException, and the message no longer tells the user to check columns that were never wrong - the same, with the marker present ONLY in the structured errors payload. GoogleAPICallError.__str__ folds an entry into the string only when it has .code/.message ATTRIBUTES, and BigQuery's REST path supplies plain dicts, so this is reachable solely through the payload loop. Deleting that loop fails this test and nothing else -- verified. - a genuine schema BadRequest still routes to ColumnMissingException, so the discrimination cannot drift into matching everything. _sanitize_for_bigquery (UN-3176): the two values where the old magnitude- derived decimal count and `:.15g` actually disagree -- 1234567890123456.0 at the high end, and 9.99999999999999e-05 at the low end, where log10 returns an exact integer, inflating the magnitude and costing a significant figure. A round number like 3.14159 passes under both forms and would prove nothing. Plus the NaN/Inf/zero guards and nested-structure recursion. Page-range billing (UN-3038): _parse_pages_to_extract over single pages, ranges, open-ended ranges, overlaps, inverted ranges and out-of-range values; the four degenerate configs that must fall back to the full count rather than bill zero; and one test asserting the count Audit actually receives, because asserting on _get_billable_page_count alone still passes when the call is dropped from push_usage_details -- which is the only place the number becomes a bill. Confirmed: removing that call site fails only the new test. No new test files -- both suites extend files already in the repo. unstract/connectors tests/databases: 44 passed (was 36). unstract/sdk1: 570 passed (was 554); the 11 failures are pre-existing and byte-identical to the base commit (missing pytest-asyncio, network-bound bedrock tests).
|
hari-kuriakose
marked this pull request as ready for review
August 31, 2026 20:51
Contributor
|
| Filename | Overview |
|---|---|
| backend/prompt_studio/prompt_studio_core_v2/prompt_variable_service.py | Adds ordered detection of variables that single-pass extraction cannot resolve while exempting custom-data variables. |
| backend/prompt_studio/prompt_studio_v2/serializers.py | Exposes unresolved single-pass variables on prompt serialization for both detail and save responses. |
| backend/tool_instance_v2/tool_instance_helper.py | Makes adapter-ID collection tolerant of missing and empty metadata values before access validation. |
| unstract/connectors/src/unstract/connectors/databases/bigquery/bigquery.py | Corrects significant-figure sanitization and introduces targeted classification of BigQuery value errors. |
| unstract/connectors/src/unstract/connectors/databases/exceptions.py | Adds a user-facing exception for BigQuery row-value rejection. |
| unstract/sdk1/src/unstract/sdk1/exceptions.py | Maps LiteLLM context-window overflow to a distinct actionable SDK error. |
| unstract/sdk1/src/unstract/sdk1/x2txt.py | Parses configured page selections and reports the resulting PDF page count to usage accounting. |
| unstract/sdk1/src/unstract/sdk1/adapters/x2text/llm_whisperer_v2/src/static/json_schema.json | Adds a bounded OCR word-confidence threshold to LLMWhisperer v2 configuration. |
| workers/executor/executors/retrievers/keyword_table.py | Applies the requested retrieval limit using KeywordTableIndex’s effective chunk-count parameter. |
Reviews (1): Last reviewed commit: "Merge branch 'main' into un-sprint4-D-ba..." | Re-trigger Greptile
hari-kuriakose
requested review from
chandrasekharan-zipstack and
kirtimanmishrazipstack
August 31, 2026 20:57
Contributor
Unstract test resultsPer-group results
Critical paths
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sprint 4 — backend fixes
Six tickets investigated and fixed across the backend and SDK. Grouped onto one branch because they were worked as a single sprint-4 investigation pass; each commit is self-contained and can be reviewed independently.
be9d7747da198d0eaa0df6f8aa4f086af45d4e0fc830880f1f8a4a7cNotes for reviewers
UN-3133 — litellm's
ContextWindowExceededErrorsubclassesBadRequestErrorand thereforeopenai.APIError, soparse_litellm_errcaught it in the generic branch and wrapped it as a plain "Error from <provider>" plus raw 400 text. It now maps ahead of the generic wrap, so the actionable message (reduce chunk size / limit pages / larger-context model) survives.UN-3176 —
bigquery.pymapped everygoogle.api_core.exceptions.BadRequesttoColumnMissingException, so a value-level failure told users to "make sure all the columns exist". Adds a_is_value_errordiscriminator; unrecognised shapes still fall through toColumnMissingException, so this only narrows messages that were already wrong.UN-2900 — the ticket title points at the one thing that works: under single pass,
custom_dataresolves correctly while every other variable type silently fails. This surfaces the unresolvable ones as an authoring-time warning rather than a save-time block, since existing projects may already carry the combination and blocking would break them retroactively.Caveats
single_pass_unresolvable_variablesfield; that lives on the C branch, which is inert without this one.🤖 Generated with Claude Code
https://claude.ai/code/session_0197gRFp4QFxu2VzxhTEozGn