Vendor TinyInference and use its APIs directly - #134
Merged
senamakel merged 43 commits intoAug 30, 2026
Conversation
Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the tinyinference crate as a dependency in the harness Cargo.toml to support model registry functionality, which requires inference capabilities for the newly introduced model registry module. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…inyinference crate Replace all direct references to `tinyagents_harness::message`, `tinyagents_harness::model`, `tinyagents_harness::usage`, and `tinyagents_harness::providers` with their counterparts in the new `tinyinference` crate. This change extracts the inference-related types into a dedicated crate, clarifying the boundary between harness orchestration and model interaction. All imports, doc links, and error conversions are updated accordingly across the graph, harness, registry, and integration test crates. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add the tinyinference crate as a dependency to tinyagents-graph, tinyagents-integration-tests, and tinyagents-registry to enable inference capabilities across these packages. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed the `embeddings`, `message`, `model`, `providers`, and `usage` modules from the public API, along with their re-exports, as their functionality has been superseded by the new `model_registry` and `token_estimation` modules. This simplifies the harness crate's surface and prevents confusion between overlapping abstractions. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed the import paths for `AssistantMessage`, `Message`, and `ToolSchema` from local super and crate paths to the `tinyinference` crate, aligning the token estimation module with the project's dependency structure. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The entire embeddings module, including all provider implementations, the vector store, retriever, and related utilities, has been removed from the tinyagents-harness crate. This module was no longer used by any remaining code and its removal simplifies the crate's public API and reduces maintenance burden. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…cal model_registry Move the `ModelRegistry` and `ResolvedModelBinding` types from the external `tinyinference` crate into the harness's own `model_registry` module, updating all imports across the harness and registry crates. This decouples the harness from the inference library's type definitions, allowing the model registry to evolve independently and reducing the public API surface that the harness exposes from its dependencies. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed the local definitions of `ToolFormat`, `ToolSchema`, `ToolCall`, and `ToolDelta` in favor of re-exporting them from the `tinyinference` crate. This eliminates duplication and ensures the harness uses the canonical type definitions maintained in the shared dependency. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed the `ToolSchema`, `ToolFormat`, `ToolCall`, and related validation functions that were no longer used after the tool system was refactored. The schema validation logic has been replaced by the `tinytools` crate, making these implementations redundant. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Removed the import of `Result` and `TinyAgentsError` from the tool module, as these types are no longer used in this file. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
This change implements the `From<tinyinference::Error>` trait for `TinyAgentsError`, allowing automatic conversion between the two error types. This enables the use of the `?` operator with tinyinference errors in functions returning `TinyAgentsError`, improving error handling ergonomics throughout the codebase. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The Cargo.lock file is updated to include the new `tinyinference` crate at version 0.2.1, along with its transitive dependency `httpdate` version 1.0.3. This change also adds `tinyinference` as a dependency to several existing packages in the workspace. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…internal module Updated all token estimation calls from the external `tinyinference::message` path to the crate's own `crate::token_estimation` module, consolidating token counting logic within the harness crate and removing the direct dependency on the external estimator's internal path. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
… arguments Extract the relaxed JSON repair logic from the tinyinference provider into a shared crate module, making it available to both the structured output repair path and the tool prompt parser. The new module handles unquoted keys, redundant brace layers, and leaked chat-template quote tokens that small models frequently emit, preventing infinite retry loops when strict JSON parsing fails. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The `relaxed_json` module was previously private and is now re-exported at the crate level so that other modules within the harness crate can use it without making it part of the public API. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The change replaces the `tool_from_result` helper with an explicit `ToolMessage` construction that includes the `trusted_verbatim` and `artifact` fields from the result. This ensures that when a tool result was marked as requiring byte-for-byte fidelity, that information is carried into the transcript so the host can distinguish it from freely reshapable output. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…el implementations Update the return type of `ChatModel::invoke` and `ChatModel::stream` implementations across test files to use the fully qualified `tinyinference::Result` instead of the unqualified `Result`, aligning with a change in the trait's signature. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
… paths Wrap the model invocation future to convert its error into TinyAgentsError, and adjust the accumulator finish call and tool error return to use the proper error type conversions. This ensures consistent error handling across the agent loop. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The `CachePolicy` struct and its associated methods were removed from the cache types module because they are no longer used anywhere in the codebase, eliminating dead code and reducing maintenance burden. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…exporting Move the `CachePolicy` type from the local cache module to the `tinyinference` crate, updating all imports to reference the canonical definition. This eliminates a duplicate type definition and ensures consistent cache policy handling across the codebase. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Replace all uses of `TinyAgentsError` with the corresponding `tinyinference::Error` variants in test model implementations. This aligns the test code with the actual error type returned by the model trait, ensuring that retry logic and error handling in the agent loop correctly match on the expected error type. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Move the `CachePolicy` type import from `tinyagents_harness::cache` to `tinyinference::cache` across five integration test files, and add the missing `tinyinference::embeddings` import in the e2e embeddings test. This aligns the test code with the recent refactoring that relocated cache policy types into the tinyinference crate, ensuring the tests compile against the current crate structure. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ed helper Moves the inline construction of a tool message from the agent loop into a dedicated public function in the tool module, so that the same conversion logic can be reused by integration tests and other callers without duplicating the message shape. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…ports Move the imports for ModelRegistry and ModelSelection from tinyinference::model to tinyagents_harness::model_registry in two e2e test files, reflecting the relocation of these types to the harness crate. This keeps the imports aligned with the current module structure and avoids relying on re-exports from the model module. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Add a new `local` module that identifies local OpenAI-compatible runtimes (Ollama, LM Studio, llama.cpp, vLLM), probes them for real context windows and capabilities, and provides native API escape hatches for settings the OpenAI wire format cannot express. This fixes two critical defects: the context window was being guessed from model IDs (overstating by up to 60×, causing silent prompt truncation), and native tools were unconditionally disabled for all local runtimes, forcing the prompt-guided branch against tiny windows. The module also adds error classification for chat template rejections, missing models, context overflows, and tools unsupported responses, enabling the transport to degrade gracefully rather than fail. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The entire `local.rs` module and its accompanying `local_test.rs` have been deleted. This module contained the local OpenAI-compatible runtime identification, capability probing, and native API escape hatches that were part of an earlier approach to handling local model servers. The functionality is no longer needed as the crate has moved to a different strategy for managing local runtime interactions. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The test now destructures the error as a generic provider error and checks the error code field instead of relying on a dedicated context-overflow variant, reflecting the upstream change that consolidated error types. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reordered import statements throughout the project to follow the convention of placing external crate imports before internal crate imports, and grouping related imports together. This change is purely cosmetic and does not affect any runtime behavior, but improves code consistency and readability by establishing a uniform import ordering pattern across all modules. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Updated two integration test files to return `tinyinference::Error::Model` instead of `TinyAgentsError::Model` when simulating model failures, aligning the test error handling with the current error type used by the inference layer. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
The integration test for shared estimator charges now calls `count_tokens_approximately` from the `tinyagents_harness::token_estimation` module instead of the removed `tinyinference::message` path, aligning with the refactored token estimation API. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…s-harness to tinyinference Relocate the `ToolCall`, `ToolSchema`, `ToolDelta`, and `ToolFormat` types from the `tinyagents-harness` crate into the `tinyinference` crate, updating all imports across 44 files to reference the new location. This change aligns the type ownership with the inference layer where these types are primarily used, reducing cross-crate dependencies and clarifying the module boundaries. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Changed the visibility of re-exported tool types from `pub` to `pub(crate)` to limit their accessibility to within the crate, preventing unintended exposure of internal dependencies. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
…rence::message` Reorder `use` statements across 43 files so that `tinyinference::tool` items are imported after `tinyinference::message` items, matching the prevailing convention in the codebase. This is a purely cosmetic change with no effect on behaviour. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Remove the re-exports of ToolCall, ToolFormat, and ToolSchema from the harness crate's public API, as these types are no longer needed by consumers of the harness module. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Relocate the `ToolCall` and `ToolSchema` imports from `tinyagents_harness::tool` to `tinyinference::tool` across both goal and todo tool modules, reflecting the upstream migration of these types to the inference crate. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
ToolCall and ToolSchema were previously re-exported from tinyagents_harness but have been moved to the tinyinference crate. Update all import sites to pull these types from tinyinference::tool instead, removing the now-stale re-exports from the harness crate. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
Reordered import statements in three integration test files so that `tinyinference::tool::{ToolCall, ToolSchema}` appears after `tinyinference::message::Message` and related imports, aligning with the project's standard import ordering.
Auto-committed-on: macbook
Co-authored-by: Medulla <medulla@tinyhumans.ai>
Update the tinynference subproject to a new commit, incorporating upstream changes and fixes. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: trueWarning Your free Security trial is over. An organization admin can activate Security or dismiss this notice. Comment |
The unused import of `serde_json::Value` was removed from the tool module to clean up the code and eliminate a compiler warning about unused imports. Auto-committed-on: macbook Co-authored-by: Medulla <medulla@tinyhumans.ai>
senamakel
marked this pull request as ready for review
August 30, 2026 18:13
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
tinysweeper
Bot
added
the
priority: p3
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
tinyhumansai/tinyinferenceasvendor/tinyinferencetinyinferenceinstead of re-exporting them through TinyAgentsupstream/mainDependency
Draft until tinyhumansai/tinyinference#3 lands; this PR currently pins that PR head.
Verification