Skip to content

fix(prices): batch get_multiple — one request per 20 codes, not per code (api#7240) - #97

Merged
karlwaldman merged 2 commits into
mainfrom
fix/7240-batch-get-multiple
Aug 23, 2026
Merged

fix(prices): batch get_multiple — one request per 20 codes, not per code (api#7240)#97
karlwaldman merged 2 commits into
mainfrom
fix/7240-batch-get-multiple

Conversation

@karlwaldman

Copy link
Copy Markdown
Member

Closes #7240 (filed as api#7240).

The defect

get_multiple() looped over get(), issuing one HTTP request per commodity. The REST API accepts up to 20 codes in one request that counts once against quota.

So the method whose entire purpose is fetching several prices cost up to 20× more quota than writing the call by hand. Through it, the free plan was 50 code-reads a day; through the raw API it is 1,000.

That is a plausible contributor to a measured result: sdk-python converts at 2.2% while people using plain Python and ignoring our SDK convert at 4.2% (api#7214). Our SDK made the free plan twenty times harder to live inside than not using it.

Verified against production before writing the fix

 20 codes -> HTTP 200
 21 codes -> HTTP 400  "Too many commodity codes requested (max: 20, requested: 21)"

And one multi-code call writes one api_requests row — confirmed by making a marked 7-code call and finding a single row for it. Response shape: 1 code → flat data; 2+ → data.prices[].

Both clients

The async one was worse: asyncio.gather fanned out one request per code concurrently, which also trips the 60-per-60s rate limit on a long list. It now gathers chunks — 25 codes are 2 concurrent requests, not 25.

Failure contract preserved

The API rejects the whole request when any code in it is unknown, so a chunk failure does not say which code was at fault. That chunk — and only that chunk — is retried per code to rebuild the per-code failure list. A one-code chunk has nothing to narrow down and is recorded directly rather than refetched.

Live smoke against production

 10 codes  -> 1 request,  10 prices    (was 10 requests)
  1 code   -> 1 request,   1 price     BRENT_CRUDE_USD $93.60
 30 codes  -> 2 requests, 29 prices    (was 30 requests)
 bad code  -> 3 requests,  1 ok + 1 reported failure

Tests

Four new tests assert request count, not just results — this regression is invisible without that. And two existing tests asserted call_count == 2 for two codes, encoding the defect; they now assert 1 and carry a note explaining why they changed.

645 passed, 42 skipped, 0 failed
coverage 63.38% (gate 50%)

Repo-wide ruff/black findings are pre-existing and unchanged — prices.py and async_client.py are 0 before, 0 after.

Behaviour change worth noting in release notes

Existing users' quota consumption drops — in their favour, but it is a change.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JKAExynd9zoKwt6rYA66EA

…ode (api#7240)

get_multiple() looped over get(), issuing one HTTP request per commodity. The
REST API accepts up to 20 codes in ONE request that counts ONCE against quota,
so the method whose whole purpose is fetching several prices cost up to 20x
more quota than writing the call by hand.

Through this method the free plan was 50 code-reads a day. Through the raw API
it is 1,000. A plausible contributor to a measured result: sdk-python converts
at 2.2% while people using plain Python and ignoring the SDK convert at 4.2%
(api#7214). Our SDK made the free plan twenty times harder to live inside than
not using it.

Verified against production before writing the fix (2026-08-23):
  20 codes -> 200; 21 -> 400 "Too many commodity codes requested (max: 20)"
  one multi-code call writes ONE api_requests row
  1 code -> flat data object; 2+ -> data.prices[]

Both clients fixed. The async one was worse: asyncio.gather fanned out one
request PER CODE concurrently, which also trips the 60-per-60s rate limit on a
long list. It now gathers CHUNKS, so 25 codes are 2 concurrent requests.

Failure contract preserved. The API rejects the whole request when any code in
it is unknown, so a chunk failure does not say which code was at fault — that
chunk (and only that chunk) is retried per code to rebuild the per-code failure
list. A one-code chunk has nothing to narrow down and is recorded directly
rather than refetched.

Live smoke against production:
  10 codes  -> 1 request,  10 prices   (was 10 requests)
   1 code   -> 1 request,   1 price
  30 codes  -> 2 requests, 29 prices   (was 30 requests)
  bad code  -> 3 requests,  1 ok + 1 reported failure

Four new tests assert REQUEST COUNT, not just results — the regression is
invisible without that. Two existing tests asserted call_count == 2 for two
codes, encoding the defect; they now assert 1 and carry a note saying why.

Repo-wide ruff/black findings are pre-existing and unchanged (prices.py and
async_client.py: 0 before, 0 after).

Closes api#7240

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKAExynd9zoKwt6rYA66EA
@coderabbitai

coderabbitai Bot commented Aug 23, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 2b0c95d9-9b87-49e4-85c5-b001d5e7a67e


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

CI's mypy step failed on the new helper with 8 errors across both clients:
'Missing type parameters for generic type dict' and incompatible argument
types for commodity/value/timestamp.

The errors are real but not new behaviour — the previous Price(**mapped_data)
form passed a dict of kwargs, which mypy cannot check, so the same looseness
was always there and simply invisible. Writing the call out explicitly exposed
it.

Typed the parameter as Dict[str, Any] and cast the three fields pydantic
validates anyway, with a comment saying why the casts are there. Not silenced
with type: ignore — the shape is genuinely Any coming off JSON, and pydantic
raises on anything actually wrong.

mypy on both files: 0 errors.
Full suite still 645 passed. Live smoke re-run after the change:
  5 codes -> 1 request, 5 prices, BRENT_CRUDE_USD $93.60

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JKAExynd9zoKwt6rYA66EA
@karlwaldman
karlwaldman merged commit 91fd7f3 into main Aug 23, 2026
7 checks passed
@karlwaldman
karlwaldman deleted the fix/7240-batch-get-multiple branch August 23, 2026 13:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant