Skip to content

docs: correct client rate-limit guidance - #896

Open
sylvesterkaczmarek wants to merge 2 commits into
openai:mainfrom
sylvesterkaczmarek:fix/client-rate-limit-docs
Open

docs: correct client rate-limit guidance#896
sylvesterkaczmarek wants to merge 2 commits into
openai:mainfrom
sylvesterkaczmarek:fix/client-rate-limit-docs

Conversation

@sylvesterkaczmarek

Copy link
Copy Markdown

Summary

Correct the OpenAIClient and OpenAIClientAsync class-level documentation so it no longer claims that each client proactively handles API rate limiting.

Fixes #644.

Problem

Both client interfaces currently say:

The client also handles rate limiting per client. This means that creating and using multiple instances at the same time will not respect rate limits.

That implies client-wide proactive rate-limit coordination. The SDK does not implement that behavior. It can retry individual requests after retryable responses, including rate-limit responses, but it does not coordinate a shared rate budget across requests or across client instances.

The existing wording can therefore lead users to believe that reusing one client is required for correct API rate-limit accounting.

Fix

Update the shared guidance in both synchronous and asynchronous client interfaces to state that:

  • reusing a client is still recommended for connection-pool/thread-pool efficiency;
  • individual requests may be retried when the API returns a retryable response;
  • the client does not proactively coordinate API rate limits across requests or client instances.

Scope

  • OpenAIClient.kt: 3 additions / 3 deletions
  • OpenAIClientAsync.kt: 3 additions / 3 deletions
  • no API, runtime, retry, or transport behavior changes

Validation

The branch is based on current upstream main at 81134d727946fb84cc48db98e6c8c53c8a8d22ab and is 0 commits behind upstream.

Risk

Minimal. Documentation only, with sync and async client guidance kept consistent.

@sylvesterkaczmarek
sylvesterkaczmarek requested a review from a team as a code owner August 18, 2026 22:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Potential misleading documentation on OpenAiClientAsync

1 participant