Add support for allowing clients to limit which finish_reasons they allow.

[hellovai]

Clients can do:

client<llm> MyClient {
  provider "openai"
  options {
    finish_reason_whitelist ["stop"]
  }
}

client<llm> MyClient {
  provider "openai"
  options {
    finish_reason_blacklist ["content_policy"]
  }
}

---

Important

Add support for client-configurable finish reason whitelists and blacklists with updated error handling and tests.

• Behavior:
  • Adds finish_reason_whitelist and finish_reason_blacklist options to client configurations for controlling acceptable finish reasons.
  • Updates PropertiesHandler to handle these options and validate finish reasons.
  • Modifies LLM response handling to check finish reasons against the whitelist/blacklist.
• Error Handling:
  • Updates error handling to raise BamlValidationError for non-terminal finish reasons.
  • Adds tests for finish reason validation in both Python and TypeScript.
• Client Implementations:
  • Integrates finish reason handling into OpenAIClient, AnthropicClient, GoogleAIClient, AwsClient, and VertexClient.
  • Updates resolve_properties functions to extract and store finish reason options.
• Testing:
  • Adds integration tests in integ-tests/python/tests/test_functions.py and integ-tests/typescript/tests/integ-tests.test.ts to verify finish reason handling.

^{This description was created by for 535156c. It will automatically update as commits are pushed.}

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
baml	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Oct 11, 2024 4:51am

[ellipsis-dev]

❌ Changes requested. Reviewed everything up to e56e099 in 2 minutes and 57 seconds

More details

• Looked at 5332 lines of code in 41 files
• Skipped 0 files when reviewing.
• Skipped posting 7 drafted comments based on config settings.

1. engine/baml-runtime/src/internal/llm_client/primitive/openai/properties/azure.rs:6

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

2. engine/baml-runtime/src/internal/llm_client/primitive/openai/properties/generic.rs:6

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

3. engine/baml-runtime/src/internal/llm_client/primitive/openai/properties/ollama.rs:6

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

4. engine/baml-runtime/src/internal/llm_client/primitive/openai/properties/openai.rs:6

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

5. engine/baml-runtime/src/internal/llm_client/primitive/vertex/vertex_client.rs:2

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

6. engine/baml-runtime/src/internal/llm_client/strategy/fallback.rs:11

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

7. engine/baml-runtime/src/internal/llm_client/strategy/roundrobin.rs:19

• Draft comment:
  Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.
• Reason this comment was not posted:
  Marked as duplicate.

Workflow ID: wflow_qu1hOzTDbk2lbddE

---

Want Ellipsis to fix these issues? Tag @ellipsis-dev in a comment. You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.

[ellipsis-dev]

Typo in module name properties_hander. It should be properties_handler. This typo is present in multiple files.

[ellipsis-dev]

👍 Looks good to me! Incremental review on 535156c in 1 minute and 31 seconds

More details

• Looked at 149 lines of code in 4 files
• Skipped 0 files when reviewing.
• Skipped posting 3 drafted comments based on config settings.

1. engine/baml-runtime/src/internal/llm_client/properties_hander.rs:78

• Draft comment:
  The error message still uses 'whitelist'. It should be updated to 'allowlist' for consistency.

                None => anyhow::bail!("finish_reason_allowlist must be an array of strings"),

• Reason this comment was not posted:
  Marked as duplicate.

2. engine/baml-runtime/src/internal/llm_client/properties_hander.rs:89

• Draft comment:
  The error message still uses 'blacklist'. It should be updated to 'denylist' for consistency.

                None => anyhow::bail!("finish_reason_denylist must be an array of strings"),

• Reason this comment was not posted:
  Marked as duplicate.

3. engine/baml-runtime/src/internal/llm_client/properties_hander.rs:98

• Draft comment:
  The error message still uses 'whitelist' and 'blacklist'. It should be updated to 'allowlist' and 'denylist' for consistency.

                "Only one of finish_reason_allowlist or finish_reason_denylist can be specified"

• Reason this comment was not posted:
  Comment looked like it was already resolved.

Workflow ID: wflow_YH5HkKmtCKnRPu3O

---

You can customize Ellipsis with 👍 / 👎 feedback, review rules, user-specific overrides, quiet mode, and more.