diff --git a/README.md b/README.md index 43686fab..03681ebc 100644 --- a/README.md +++ b/README.md @@ -97,7 +97,7 @@ is deliberately outside `full`: "give me the whole workspace" is not the same request as "give me the test doubles". This table says which crate each feature brings in. For what each *engine* -feature actually serves — driver class, and how many of the twenty capability +feature actually serves — driver class, and which capability families answer — see the engine table under [Using from your project](#using-from-your-project). @@ -176,17 +176,18 @@ use tinymemory::tinycortex::{provider, InMemoryMemoryStore}; let provider = Arc::new(provider(Arc::new(InMemoryMemoryStore::new()))); ``` -That is a complete embedded setup for the mandatory three families. The full -twenty-family engine (`TinycortexProvider`) additionally needs the host +That is a complete embedded setup for the mandatory families plus document +ingestion. The full +full engine (`TinycortexProvider`) additionally needs the host seams (`EmbeddingHost` et al.) installed — see `crates/tinymemory-tinycortex/tests/full_provider_conformance.rs` for the minimal working wiring. | Feature | Engine | Class | Families served | | --- | --- | --- | --- | -| `tinycortex` | TinyCortex, in-process | embedded | 3 (mandatory) via `provider`; all 18 via `TinycortexProvider` | +| `tinycortex` | TinyCortex, in-process | embedded | mandatory + document ingest via `provider`; every compiled family via `TinycortexProvider` | | `supermemory` | Supermemory, hosted | external | 3 (mandatory) | -| `mem0` | Mem0, hosted (`cloud`) or self-hosted | external | 3 (mandatory) | +| `mem0` | Mem0, hosted (`cloud`) or self-hosted | external | mandatory + conversation ingest | | `cognee` | Cognee, hosted or self-hosted | external | 3 (mandatory) | | `agentmemory` | AgentMemory, self-hosted | external | 3 (mandatory) | | `memory-git` | add-on: git-backed diff snapshots | — | requires `tinycortex` | @@ -204,7 +205,7 @@ for assistant-memory workloads; wrong for high-volume keyed storage. ## The contract `MemoryProvider` is an object-safe trait with **three mandatory** capability -families and **seventeen optional** ones. The mandatory three are supertraits, so +families and independently negotiated optional ones. The mandatory three are supertraits, so a driver missing any of them cannot be constructed; the optional seventeen are reached through `as_ingest()` / `as_tree()` / … accessors that default to `None`, so a minimal driver implements what it supports and inherits correct absence for @@ -215,6 +216,25 @@ A driver's advertised set and its reachable accessors must agree. implemented" into a detectable, testable mistake rather than a runtime surprise on the first call. +The product-facing routes are available through one router: + +```rust,ignore +use tinymemory::{MemoryApi, operations::AnswerRequest}; + +let memory = MemoryApi::new(provider.as_ref()); +let hits = memory.recall("release date", 10, &Default::default(), None).await?; + +if provider.as_answer().is_some() { + let response = memory.answer(AnswerRequest::new("When do we release?")).await?; + println!("{}", response.answer); +} +``` + +Document, conversation, learning, event, and answer support are independent +capabilities. Recall remains mandatory. See the +[operation specification](docs/specs/ingestion-retrieval-api.md) for the adapter +matrix and payload rules. + Capabilities are asked **once, at bind time, and cached**: a host filters its RPC surface and its agent-tool list from the answer, so a set that changed afterwards would not be noticed. diff --git a/crates/tinymemory-api/src/lib.rs b/crates/tinymemory-api/src/lib.rs index efb7b39a..93d8846c 100644 --- a/crates/tinymemory-api/src/lib.rs +++ b/crates/tinymemory-api/src/lib.rs @@ -67,10 +67,10 @@ //! - [`recall`]: the borrowed [`recall::RecallOpts`] and owned, serde-derived //! [`recall::OwnedRecallOpts`] recall filters (both re-exported from //! [`types`]). -//! - [`capabilities`]: the twenty [`capabilities::Capability`] families and +//! - [`capabilities`]: the [`capabilities::Capability`] families and //! the [`capabilities::Capabilities`] set negotiated at bind time. //! - [`provider`]: the driver contract — [`provider::MemoryProvider`] plus the -//! twenty capability family traits and the value types they need. +//! capability family traits and the value types they need. //! - [`null`]: [`null::NullMemoryProvider`], the reference driver a //! compiled-out or unconfigured memory subsystem binds to. //! - [`health`]: [`health::MemoryHealth`], the liveness state a driver reports. @@ -118,7 +118,7 @@ pub mod sync_events; // nothing type-checks. pub use tinymemory_bus::{ capabilities, chunks, composio, error, evidence, goals, graph, health, learning, namespace, - recall, tool_memory, tree, types, version, wire, + operations, recall, tool_memory, tree, types, version, wire, }; /// The mandatory-family composition: wrap any [`traits::Memory`] backend as a /// complete [`provider::MemoryProvider`]. diff --git a/crates/tinymemory-api/src/null.rs b/crates/tinymemory-api/src/null.rs index dffd04f2..9ac12b0d 100644 --- a/crates/tinymemory-api/src/null.rs +++ b/crates/tinymemory-api/src/null.rs @@ -55,6 +55,8 @@ use crate::capabilities::{Capabilities, Capability}; use crate::error::MemoryError; use crate::goals::GoalsDoc; use crate::health::MemoryHealth; +use crate::learning::LearningCandidate; +use crate::operations::{AnswerRequest, AnswerResponse, RawMemoryEvent}; use crate::provider::types::{ DiffReport, EntityHit, ExportPage, ExportRecord, FlushOutcome, ImportOutcome, IngestItem, IngestOutcome, MaintenanceReport, ResetOutcome, SnapshotRef, SourceItem, SourceScope, @@ -62,13 +64,15 @@ use crate::provider::types::{ use crate::provider::{ AddressBookSeedOutcome, ChunkDetail, ChunkEmbedding, ChunkQuery, CodingSessionIngestReport, CodingSessionIngestRequest, CodingSessionSource, CoverWindowQuery, EntityMatch, FacetType, - FastRetrieveQuery, MemoryChunks, MemoryCodingSessions, MemoryCore, MemoryDiff, MemoryDocuments, - MemoryEntities, MemoryGoals, MemoryGraph, MemoryIngest, MemoryMaintenance, MemoryPeople, - MemoryPortability, MemoryProfile, MemoryProvider, MemoryRecall, MemoryRetrieval, MemoryScoring, - MemorySourceSink, MemorySourceSync, MemoryToolMemory, MemoryTree, PersonHandle, - PersonInteraction, PersonRecord, PersonScore, ProfileFacet, RankedPerson, RawArchiveCoverage, - RawRebuildOutcome, ResolvedPerson, RetrievalHit, RetrievalResponse, SourceRetrievalQuery, - SourceSyncState, SourceSyncStatus, SyncAuditEntry, SyncRunOutcome, UserState, + FastRetrieveQuery, MemoryAnswer, MemoryChunks, MemoryCodingSessions, MemoryConversationIngest, + MemoryCore, MemoryDiff, MemoryDocumentIngest, MemoryDocuments, MemoryEntities, + MemoryEventIngest, MemoryGoals, MemoryGraph, MemoryIngest, MemoryLearningIngest, + MemoryMaintenance, MemoryPeople, MemoryPortability, MemoryProfile, MemoryProvider, + MemoryRecall, MemoryRetrieval, MemoryScoring, MemorySourceSink, MemorySourceSync, + MemoryToolMemory, MemoryTree, PersonHandle, PersonInteraction, PersonRecord, PersonScore, + ProfileFacet, RankedPerson, RawArchiveCoverage, RawRebuildOutcome, ResolvedPerson, + RetrievalHit, RetrievalResponse, SourceRetrievalQuery, SourceSyncState, SourceSyncStatus, + SyncAuditEntry, SyncRunOutcome, UserState, }; use crate::recall::OwnedRecallOpts; use crate::tool_memory::ToolMemoryRule; @@ -236,6 +240,47 @@ impl MemoryIngest for NullMemoryProvider { } } +#[async_trait] +impl MemoryDocumentIngest for NullMemoryProvider { + async fn ingest_document(&self, _document: IngestItem) -> Result { + unsupported(Capability::DocumentIngest) + } +} + +#[async_trait] +impl MemoryConversationIngest for NullMemoryProvider { + async fn ingest_conversation( + &self, + _messages: Vec, + ) -> Result { + unsupported(Capability::ConversationIngest) + } +} + +#[async_trait] +impl MemoryLearningIngest for NullMemoryProvider { + async fn ingest_learning( + &self, + _learning: LearningCandidate, + ) -> Result { + unsupported(Capability::LearningIngest) + } +} + +#[async_trait] +impl MemoryEventIngest for NullMemoryProvider { + async fn ingest_event(&self, _event: RawMemoryEvent) -> Result { + unsupported(Capability::EventIngest) + } +} + +#[async_trait] +impl MemoryAnswer for NullMemoryProvider { + async fn answer(&self, _request: AnswerRequest) -> Result { + unsupported(Capability::Answer) + } +} + #[async_trait] impl MemoryDocuments for NullMemoryProvider { async fn put_document(&self, _input: NamespaceDocumentInput) -> Result { diff --git a/crates/tinymemory-api/src/null_tests.rs b/crates/tinymemory-api/src/null_tests.rs index 7f65606b..6b9fd73c 100644 --- a/crates/tinymemory-api/src/null_tests.rs +++ b/crates/tinymemory-api/src/null_tests.rs @@ -261,7 +261,10 @@ fn every_optional_method_fails_with_its_advertised_family_name() { subject: None, list_unsubscribe: None, }; - assert_unsupported(block_on(driver.ingest_document(ingest)), Capability::Ingest); + assert_unsupported( + block_on(MemoryIngest::ingest_document(&driver, ingest)), + Capability::Ingest, + ); let document = NamespaceDocumentInput { namespace: "ns".into(), diff --git a/crates/tinymemory-api/src/provider/driver.rs b/crates/tinymemory-api/src/provider/driver.rs index c25769f2..3b655caf 100644 --- a/crates/tinymemory-api/src/provider/driver.rs +++ b/crates/tinymemory-api/src/provider/driver.rs @@ -60,6 +60,10 @@ use crate::provider::content::{MemoryDocuments, MemoryIngest, MemoryTree}; use crate::provider::episodic::MemoryEpisodic; use crate::provider::knowledge::{MemoryDiff, MemoryEntities, MemoryGraph}; use crate::provider::mandatory::{MemoryCore, MemoryPortability, MemoryRecall}; +use crate::provider::operations::{ + MemoryAnswer, MemoryConversationIngest, MemoryDocumentIngest, MemoryEventIngest, + MemoryLearningIngest, +}; use crate::provider::people::MemoryPeople; use crate::provider::profile::MemoryProfile; use crate::provider::records::{ @@ -220,6 +224,31 @@ pub trait MemoryProvider: MemoryCore + MemoryRecall + MemoryPortability + 'stati None } + /// Product-facing document ingestion, when advertised. + fn as_document_ingest(&self) -> Option<&dyn MemoryDocumentIngest> { + None + } + + /// Product-facing conversation ingestion, when advertised. + fn as_conversation_ingest(&self) -> Option<&dyn MemoryConversationIngest> { + None + } + + /// Product-facing learning ingestion, when advertised. + fn as_learning_ingest(&self) -> Option<&dyn MemoryLearningIngest> { + None + } + + /// Product-facing event ingestion, when advertised. + fn as_event_ingest(&self) -> Option<&dyn MemoryEventIngest> { + None + } + + /// Agentic grounded answers, when advertised. + fn as_answer(&self) -> Option<&dyn MemoryAnswer> { + None + } + /// Whether `capability` is actually **reachable** on this driver. /// /// This is the implementation-side truth, as opposed to @@ -253,6 +282,11 @@ pub trait MemoryProvider: MemoryCore + MemoryRecall + MemoryPortability + 'stati Capability::SourceSync => self.as_source_sync().is_some(), Capability::CodingSessions => self.as_coding_sessions().is_some(), Capability::Scoring => self.as_scoring().is_some(), + Capability::DocumentIngest => self.as_document_ingest().is_some(), + Capability::ConversationIngest => self.as_conversation_ingest().is_some(), + Capability::LearningIngest => self.as_learning_ingest().is_some(), + Capability::EventIngest => self.as_event_ingest().is_some(), + Capability::Answer => self.as_answer().is_some(), } } } diff --git a/crates/tinymemory-api/src/provider/mod.rs b/crates/tinymemory-api/src/provider/mod.rs index 2d7a658d..daf018d1 100644 --- a/crates/tinymemory-api/src/provider/mod.rs +++ b/crates/tinymemory-api/src/provider/mod.rs @@ -1,4 +1,4 @@ -//! The memory driver contract: [`MemoryProvider`] plus the twenty capability +//! The memory driver contract: [`MemoryProvider`] plus its capability //! family traits a driver may implement. //! //! ## Shape @@ -58,7 +58,8 @@ //! //! ## Reference implementation //! -//! [`crate::null::NullMemoryProvider`] implements all twenty families: +//! [`crate::null::NullMemoryProvider`] implements every family directly for +//! conformance testing: //! `/dev/null` semantics for the mandatory three, and //! [`crate::error::MemoryError::Unsupported`] for the other seventeen, which it //! does not advertise. It is what a compiled-out or unconfigured memory subsystem @@ -72,6 +73,7 @@ pub mod driver; pub mod episodic; pub mod knowledge; pub mod mandatory; +pub mod operations; pub mod people; pub mod profile; pub mod records; @@ -104,6 +106,11 @@ pub use driver::MemoryProvider; pub use episodic::{ConversationSegment, EpisodicEvent, EpisodicTurn, EventKind, MemoryEpisodic}; pub use knowledge::{MemoryDiff, MemoryEntities, MemoryGraph, INBOUND_SCAN_LIMIT}; pub use mandatory::{MemoryCore, MemoryPortability, MemoryRecall}; +pub use operations::{ + AnswerCitation, AnswerRequest, AnswerResponse, AnswerStep, MemoryAnswer, + MemoryConversationIngest, MemoryDocumentIngest, MemoryEventIngest, MemoryLearningIngest, + RawMemoryEvent, +}; pub use people::{ AddressBookSeedOutcome, MemoryPeople, PersonHandle, PersonInteraction, PersonRecord, PersonRef, PersonScore, RankedPerson, ResolvedPerson, diff --git a/crates/tinymemory-api/src/provider/operations.rs b/crates/tinymemory-api/src/provider/operations.rs new file mode 100644 index 00000000..fb8d2c98 --- /dev/null +++ b/crates/tinymemory-api/src/provider/operations.rs @@ -0,0 +1,82 @@ +//! Granular product-facing ingestion and answer operations. +//! +//! These traits deliberately split the legacy [`MemoryIngest`](super::MemoryIngest) +//! family. A connector may be excellent at conversations while having no +//! document, learning, or event model, and capability negotiation must be able +//! to express exactly that. + +use async_trait::async_trait; + +use crate::error::MemoryError; +use crate::learning::LearningCandidate; +use crate::provider::types::{IngestItem, IngestOutcome}; + +pub use crate::operations::{ + AnswerCitation, AnswerRequest, AnswerResponse, AnswerStep, RawMemoryEvent, +}; + +/// Document ingestion with driver-owned chunking and indexing. +#[async_trait] +pub trait MemoryDocumentIngest: Send + Sync { + /// Ingest one decoded document. + /// + /// # Errors + /// + /// Returns [`MemoryError::Invalid`] for rejected input and a backend error + /// when persistence or indexing fails. + async fn ingest_document(&self, document: IngestItem) -> Result; +} + +/// Ordered conversation ingestion. +#[async_trait] +pub trait MemoryConversationIngest: Send + Sync { + /// Ingest all messages belonging to one conversation. + /// + /// # Errors + /// + /// Returns [`MemoryError::Invalid`] when the batch mixes conversations or + /// contains invalid content, otherwise backend failures. + async fn ingest_conversation( + &self, + messages: Vec, + ) -> Result; +} + +/// Ingestion of already-extracted learnings. +#[async_trait] +pub trait MemoryLearningIngest: Send + Sync { + /// Persist one learning candidate and its evidence pointer. + /// + /// # Errors + /// + /// Returns [`MemoryError::Invalid`] for malformed confidence or keys, + /// otherwise backend failures. + async fn ingest_learning( + &self, + learning: LearningCandidate, + ) -> Result; +} + +/// Ingestion of raw durable events. +#[async_trait] +pub trait MemoryEventIngest: Send + Sync { + /// Persist one event. + /// + /// # Errors + /// + /// Returns [`MemoryError::Invalid`] for malformed event data, otherwise + /// backend failures. + async fn ingest_event(&self, event: RawMemoryEvent) -> Result; +} + +/// Agentic retrieval that synthesises a grounded answer. +#[async_trait] +pub trait MemoryAnswer: Send + Sync { + /// Retrieve evidence and synthesize an answer with citations. + /// + /// # Errors + /// + /// Returns [`MemoryError::Invalid`] for an empty query and backend or + /// inference errors when retrieval or synthesis fails. + async fn answer(&self, request: AnswerRequest) -> Result; +} diff --git a/crates/tinymemory-bus/README.md b/crates/tinymemory-bus/README.md index 720f57a1..c16af246 100644 --- a/crates/tinymemory-bus/README.md +++ b/crates/tinymemory-bus/README.md @@ -50,7 +50,7 @@ A host depends on `tinymemory-bus` and gets vocabulary alone. ## What is deliberately absent -**No traits.** `MemoryProvider` and the twenty capability-family traits +**No traits.** `MemoryProvider` and its capability-family traits describe what an engine must implement, not what a frame carries. They stay in `tinymemory-api`. The split is readable off the path: a name here is data, a name there is an obligation. diff --git a/crates/tinymemory-bus/src/capabilities.rs b/crates/tinymemory-bus/src/capabilities.rs index a90edf9a..47068e07 100644 --- a/crates/tinymemory-bus/src/capabilities.rs +++ b/crates/tinymemory-bus/src/capabilities.rs @@ -51,7 +51,7 @@ use crate::error::MemoryError; /// One capability family a memory driver may advertise. /// -/// The variants are exactly the twenty families of the memory contract. Each +/// The variants are exactly the capability families of the memory contract. Each /// maps to a trait family in the contract, a group of RPC methods, and a group /// of agent tools; a driver that does not advertise a family simply has that /// surface absent. @@ -116,6 +116,16 @@ pub enum Capability { /// Scoring and NLP operations: entity extraction, text embedding, and /// embedder identification. Scoring, + /// Raw document ingestion with driver-owned chunking and indexing. + DocumentIngest, + /// Ordered conversation ingestion. + ConversationIngest, + /// Durable learning-candidate ingestion. + LearningIngest, + /// Raw event ingestion. + EventIngest, + /// Agentic, grounded answer synthesis. + Answer, } impl Capability { @@ -124,7 +134,7 @@ impl Capability { /// Declaration order is also bit order in [`Capabilities`] and iteration /// order in its serialized form, so this slice is the single ordering /// authority for the whole module. - pub const ALL: [Capability; 21] = [ + pub const ALL: [Capability; 26] = [ Capability::Core, Capability::Recall, Capability::Ingest, @@ -149,6 +159,11 @@ impl Capability { Capability::SourceSync, Capability::CodingSessions, Capability::Scoring, + Capability::DocumentIngest, + Capability::ConversationIngest, + Capability::LearningIngest, + Capability::EventIngest, + Capability::Answer, ]; /// The families a driver must advertise to be bindable at all. @@ -195,6 +210,11 @@ impl Capability { Self::SourceSync => "source_sync", Self::CodingSessions => "coding_sessions", Self::Scoring => "scoring", + Self::DocumentIngest => "document_ingest", + Self::ConversationIngest => "conversation_ingest", + Self::LearningIngest => "learning_ingest", + Self::EventIngest => "event_ingest", + Self::Answer => "answer", } } @@ -245,6 +265,11 @@ impl Capability { Self::SourceSync => 18, Self::CodingSessions => 19, Self::Scoring => 20, + Self::DocumentIngest => 21, + Self::ConversationIngest => 22, + Self::LearningIngest => 23, + Self::EventIngest => 24, + Self::Answer => 25, } } diff --git a/crates/tinymemory-bus/src/capabilities_tests.rs b/crates/tinymemory-bus/src/capabilities_tests.rs index 19f92300..1fba6309 100644 --- a/crates/tinymemory-bus/src/capabilities_tests.rs +++ b/crates/tinymemory-bus/src/capabilities_tests.rs @@ -2,7 +2,7 @@ //! //! Three properties are load-bearing and each has its own test: //! -//! 1. the enum has exactly the twenty-one contract families and no more; +//! 1. the enum has exactly the twenty-six contract families and no more; //! 2. the serialized form is stable snake_case **strings**, never discriminant //! integers — a driver deployed against an older build must keep advertising //! the same set after a variant is inserted mid-enum; @@ -19,9 +19,9 @@ use super::*; use serde_json::json; #[test] -fn capability_has_exactly_the_twenty_one_contract_families() { - assert_eq!(Capability::ALL.len(), 21); - assert_eq!(Capability::all().len(), 21); +fn capability_has_exactly_the_twenty_six_contract_families() { + assert_eq!(Capability::ALL.len(), 26); + assert_eq!(Capability::all().len(), 26); let names: Vec<&str> = Capability::ALL.iter().map(|c| c.as_str()).collect(); assert_eq!( @@ -48,6 +48,11 @@ fn capability_has_exactly_the_twenty_one_contract_families() { "source_sync", "coding_sessions", "scoring", + "document_ingest", + "conversation_ingest", + "learning_ingest", + "event_ingest", + "answer", ] ); } diff --git a/crates/tinymemory-bus/src/lib.rs b/crates/tinymemory-bus/src/lib.rs index 14927f5a..b5daf75d 100644 --- a/crates/tinymemory-bus/src/lib.rs +++ b/crates/tinymemory-bus/src/lib.rs @@ -31,7 +31,7 @@ //! //! ## What is deliberately not here //! -//! **No traits.** `MemoryProvider` and the twenty capability-family traits +//! **No traits.** `MemoryProvider` and its capability-family traits //! are driver obligations: they describe what an engine must implement, not //! what a frame carries. They stay in `tinymemory-api`, which depends on this //! crate. @@ -89,6 +89,7 @@ pub mod health; pub mod learning; pub mod names; pub mod namespace; +pub mod operations; pub mod provider; pub mod recall; pub mod tool_memory; diff --git a/crates/tinymemory-bus/src/names.rs b/crates/tinymemory-bus/src/names.rs index c8020fd7..989de73f 100644 --- a/crates/tinymemory-bus/src/names.rs +++ b/crates/tinymemory-bus/src/names.rs @@ -64,6 +64,12 @@ pub mod methods { pub const INGEST_CHAT: &str = "IngestChat"; /// `IngestEmail` — ingest email. pub const INGEST_EMAIL: &str = "IngestEmail"; + /// `IngestLearning` — ingest one learning candidate. + pub const INGEST_LEARNING: &str = "IngestLearning"; + /// `IngestEvent` — ingest one raw event. + pub const INGEST_EVENT: &str = "IngestEvent"; + /// `Answer` — synthesize a grounded answer. + pub const ANSWER: &str = "Answer"; // Namespace-scoped document storage and retrieval. /// `PutDocument` — put document. @@ -364,7 +370,7 @@ pub mod methods { /// The order matters: `tinybus`'s `Interface::members()` returns declaration /// order, and the module compares the two sequences directly rather than as /// sets, so a reordering is caught alongside an addition or a removal. -pub const METHODS: [&str; 138] = [ +pub const METHODS: [&str; 141] = [ methods::DRIVER_ID, methods::CAPABILITIES, methods::HEALTH, @@ -503,6 +509,9 @@ pub const METHODS: [&str; 138] = [ methods::RUNTIME_SUMMARIZE, methods::RUNTIME_REBUILD, methods::FLAVOUR_PROFILE, + methods::INGEST_LEARNING, + methods::INGEST_EVENT, + methods::ANSWER, ]; #[cfg(test)] diff --git a/crates/tinymemory-bus/src/names_tests.rs b/crates/tinymemory-bus/src/names_tests.rs index 4fab5ba9..f1adcdc6 100644 --- a/crates/tinymemory-bus/src/names_tests.rs +++ b/crates/tinymemory-bus/src/names_tests.rs @@ -139,7 +139,7 @@ fn the_runtime_tree_doors_hold_the_wire_slots_they_were_released_in() { // reason the summariser-door test above gives: member order is wire order, // and an assertion measured from the end moves silently under the next // append — which is exactly the edit this exists to catch. - assert_eq!(METHODS.len(), 138); + assert_eq!(METHODS.len(), 141); assert_eq!(METHODS[131], methods::RUNTIME_BUFFER_WRITE); assert_eq!(METHODS[132], methods::RUNTIME_READ_NODE); assert_eq!(METHODS[133], methods::RUNTIME_READ_CHILDREN); @@ -147,4 +147,10 @@ fn the_runtime_tree_doors_hold_the_wire_slots_they_were_released_in() { assert_eq!(METHODS[135], methods::RUNTIME_SUMMARIZE); assert_eq!(METHODS[136], methods::RUNTIME_REBUILD); assert_eq!(METHODS[137], methods::FLAVOUR_PROFILE); + + // The granular ingestion and answer doors landed after the runtime-tree + // round and must not renumber any of its released slots. + assert_eq!(METHODS[138], methods::INGEST_LEARNING); + assert_eq!(METHODS[139], methods::INGEST_EVENT); + assert_eq!(METHODS[140], methods::ANSWER); } diff --git a/crates/tinymemory-bus/src/operations.rs b/crates/tinymemory-bus/src/operations.rs new file mode 100644 index 00000000..48fafc7c --- /dev/null +++ b/crates/tinymemory-bus/src/operations.rs @@ -0,0 +1,130 @@ +//! High-level ingestion and answer payloads. +//! +//! The lower-level provider families expose the engine's storage primitives. +//! These values describe the product-facing routes that connectors negotiate: +//! document, conversation, learning, and event ingestion, plus grounded answer +//! synthesis. Recall keeps using [`crate::recall`] and [`crate::types::MemoryEntry`]. + +use chrono::{DateTime, Utc}; +use serde::{Deserialize, Serialize}; + +use crate::provider::types::SourceScope; +use crate::recall::OwnedRecallOpts; +use crate::types::MemoryTaint; + +/// One raw event supplied by an application or connector. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub struct RawMemoryEvent { + /// Stable idempotency key. + pub id: String, + /// Logical event namespace. + pub namespace: String, + /// Open event-type vocabulary, such as `calendar_changed` or `tool_call`. + pub event_type: String, + /// Human-readable event content indexed for recall. + pub content: String, + /// When the event occurred, when known. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub occurred_at: Option>, + /// Session associated with the event, when any. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub session_id: Option, + /// Connector-defined structured data retained with the event. + #[serde(default)] + pub metadata: serde_json::Value, + /// Provenance assigned by the host. + #[serde(default)] + pub taint: MemoryTaint, +} + +/// A grounded, agentic answer request. +#[derive(Clone, Debug, Serialize, Deserialize)] +pub struct AnswerRequest { + /// The question to answer. + pub query: String, + /// Maximum number of memories the answering agent may retrieve. + #[serde(default = "default_answer_limit")] + pub limit: usize, + /// Recall filters applied before synthesis. + #[serde(default)] + pub recall: OwnedRecallOpts, + /// Optional per-turn source allowlist. + #[serde(default)] + pub scope: Option, + /// Optional caller guidance for tone, format, or focus. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub instructions: Option, +} + +fn default_answer_limit() -> usize { + 12 +} + +impl AnswerRequest { + /// Construct a request with conservative retrieval defaults. + #[must_use] + pub fn new(query: impl Into) -> Self { + Self { + query: query.into(), + limit: default_answer_limit(), + recall: OwnedRecallOpts::default(), + scope: None, + instructions: None, + } + } +} + +/// One memory cited by an answer. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub struct AnswerCitation { + /// Stable driver record id. + pub id: String, + /// Namespace containing the record, when the backend exposes it. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub namespace: Option, + /// Human-readable key or title. + pub key: String, + /// Retrieved text supplied to the answering agent. + pub content: String, + /// Backend relevance score, when available. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub score: Option, +} + +/// Observable retrieval work performed while producing an answer. +#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)] +pub struct AnswerStep { + /// Stable operation name, such as `recall` or `synthesise`. + pub operation: String, + /// Short, content-free description safe for logs and user interfaces. + pub detail: String, +} + +/// A grounded answer and the retrieval evidence behind it. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub struct AnswerResponse { + /// Synthesised prose answer. + pub answer: String, + /// Memories made available to the answering agent, in rank order. + #[serde(default)] + pub citations: Vec, + /// High-level execution trace; never contains prompts or credentials. + #[serde(default)] + pub steps: Vec, + /// Model identifier used for synthesis, when the provider exposes it. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub model: Option, +} + +#[cfg(test)] +mod tests { + use super::AnswerRequest; + + #[test] + fn answer_request_defaults_bound_retrieval() { + let request = AnswerRequest::new("what did we decide?"); + assert_eq!(request.limit, 12); + assert_eq!(request.query, "what did we decide?"); + assert!(request.scope.is_none()); + } +} diff --git a/crates/tinymemory-bus/src/version.rs b/crates/tinymemory-bus/src/version.rs index b6f6232d..fa62e1bd 100644 --- a/crates/tinymemory-bus/src/version.rs +++ b/crates/tinymemory-bus/src/version.rs @@ -60,7 +60,7 @@ /// added to a family a driver may already advertise** (negotiation is /// family-granular, not method-granular, so that case cannot be made minor-safe /// by negotiation alone). -pub const CONTRACT_VERSION: (u16, u16) = (4, 0); +pub const CONTRACT_VERSION: (u16, u16) = (4, 1); /// Whether a driver speaking `remote` can be bound against this build. /// diff --git a/crates/tinymemory-bus/src/version_tests.rs b/crates/tinymemory-bus/src/version_tests.rs index 866d7b0f..8e1a2b8b 100644 --- a/crates/tinymemory-bus/src/version_tests.rs +++ b/crates/tinymemory-bus/src/version_tests.rs @@ -7,7 +7,7 @@ use super::*; #[test] -fn contract_version_is_four_zero() { +fn contract_version_is_four_one() { // (4, 0): the six runtime-tree members and `flavour_profile` were added to // `Tree` — a family a driver may ALREADY advertise. The rule makes that a // major bump and not a minor one, and the reason is the whole point of the @@ -28,7 +28,10 @@ fn contract_version_is_four_zero() { // constant. All of that was wrong by this rule; those releases and their // hosts moved in lockstep so nothing was bound across the gap, but it is // drift, not precedent. This round declines to extend it. - assert_eq!(CONTRACT_VERSION, (4, 0)); + // + // (4, 1): the five granular operation capabilities are new families, so + // capability negotiation makes their addition minor-safe. + assert_eq!(CONTRACT_VERSION, (4, 1)); } #[test] diff --git a/crates/tinymemory-module/src/lib.rs b/crates/tinymemory-module/src/lib.rs index 5fb4693b..542a9bc7 100644 --- a/crates/tinymemory-module/src/lib.rs +++ b/crates/tinymemory-module/src/lib.rs @@ -693,6 +693,11 @@ mod exports { "RuntimeSummarize", "RuntimeRebuild", "FlavourProfile", + // Granular ingestion and agentic retrieval, appended so all + // previously released TinyBus member slots stay stable. + "IngestLearning", + "IngestEvent", + "Answer", ], signals = [], // The host's embedder is deliberately NOT declared as `requires`. That diff --git a/crates/tinymemory-module/src/service/mod.rs b/crates/tinymemory-module/src/service/mod.rs index 1d713d13..97c3036e 100644 --- a/crates/tinymemory-module/src/service/mod.rs +++ b/crates/tinymemory-module/src/service/mod.rs @@ -166,6 +166,8 @@ use tinymemory_api::chunks::Chunk; use tinymemory_api::error::MemoryError; use tinymemory_api::goals::GoalsDoc; use tinymemory_api::health::MemoryHealth; +use tinymemory_api::learning::LearningCandidate; +use tinymemory_api::operations::{AnswerRequest, AnswerResponse, RawMemoryEvent}; use tinymemory_api::provider::types::{ ChunkEntityOccurrence, DiffReport, EntityHit, EntityOccurrence, ExportPage, ExportRecord, FlushOutcome, ForgetOutcome, ForgetSelector, ImportOutcome, IngestItem, IngestOutcome, @@ -623,15 +625,15 @@ impl MemoryService { } async fn ingest_document(&self, item: IngestItem) -> BusResult { - require_family!(self, as_ingest, Capability::Ingest) + require_family!(self, as_document_ingest, Capability::DocumentIngest) .ingest_document(item) .await .map_err(|error| into_bus_error(&error)) } async fn ingest_chat(&self, messages: Vec) -> BusResult { - require_family!(self, as_ingest, Capability::Ingest) - .ingest_chat(messages) + require_family!(self, as_conversation_ingest, Capability::ConversationIngest) + .ingest_conversation(messages) .await .map_err(|error| into_bus_error(&error)) } @@ -2187,6 +2189,34 @@ impl MemoryService { .await .map_err(|error| into_bus_error(&error)) } + + /// Ingest one learning candidate through the granular capability added + /// after the runtime-tree doors. Kept at the interface tail to preserve + /// every previously released wire slot. + async fn ingest_learning(&self, learning: LearningCandidate) -> BusResult { + require_family!(self, as_learning_ingest, Capability::LearningIngest) + .ingest_learning(learning) + .await + .map_err(|error| into_bus_error(&error)) + } + + /// Ingest one raw event through the granular event capability. Appended + /// here so older member indices remain stable. + async fn ingest_event(&self, event: RawMemoryEvent) -> BusResult { + require_family!(self, as_event_ingest, Capability::EventIngest) + .ingest_event(event) + .await + .map_err(|error| into_bus_error(&error)) + } + + /// Produce a grounded answer through the granular answer capability. + /// Appended here so older member indices remain stable. + async fn answer(&self, request: AnswerRequest) -> BusResult { + require_family!(self, as_answer, Capability::Answer) + .answer(request) + .await + .map_err(|error| into_bus_error(&error)) + } } /// The response-size ceiling for a method that returns a list of entries. diff --git a/crates/tinymemory-module/tests/module_e2e.rs b/crates/tinymemory-module/tests/module_e2e.rs index 766f1fe4..fe50f5b1 100644 --- a/crates/tinymemory-module/tests/module_e2e.rs +++ b/crates/tinymemory-module/tests/module_e2e.rs @@ -734,6 +734,11 @@ const EXPECTED_METHODS: &[&str] = &[ "RuntimeSummarize", "RuntimeRebuild", "FlavourProfile", + // Granular ingestion and agentic retrieval, appended after all previously + // released wire slots. + "IngestLearning", + "IngestEvent", + "Answer", ]; #[tokio::test] diff --git a/crates/tinymemory-remote/src/conformance_test.rs b/crates/tinymemory-remote/src/conformance_test.rs index 019ea80c..c1e7d2c9 100644 --- a/crates/tinymemory-remote/src/conformance_test.rs +++ b/crates/tinymemory-remote/src/conformance_test.rs @@ -28,6 +28,8 @@ use axum::extract::{Path, Query, State}; use axum::routing::{delete, get, post, put}; use axum::{Json, Router}; use serde_json::{json, Value}; +use tinymemory_api::capabilities::Capability; +use tinymemory_api::provider::{MemoryCore, MemoryProvider}; use crate::{mem0_provider, Mem0Memory}; @@ -183,6 +185,39 @@ async fn mem0_upholds_the_contract() { tinymemory_conformance::assert_provider(Arc::new(provider)).await; } +#[tokio::test] +async fn mem0_routes_conversation_ingestion_without_claiming_other_ingest_kinds() { + let endpoint = mem0_backend().await; + let provider = mem0_provider(Mem0Memory::new(&endpoint, None).expect("client")); + assert!(provider + .capabilities() + .contains(Capability::ConversationIngest)); + assert!(!provider.capabilities().contains(Capability::DocumentIngest)); + + let messages = vec![serde_json::from_value(json!({ + "source": "conversation", + "source_id": "thread-1", + "author": "user", + "content": "I prefer terse answers" + })) + .expect("conversation item")]; + let outcome = provider + .as_conversation_ingest() + .expect("conversation route") + .ingest_conversation(messages) + .await + .expect("ingest conversation"); + assert_eq!(outcome.written, 1); + assert_eq!( + provider + .list(Some("conversation:thread-1"), None, None) + .await + .expect("list") + .len(), + 1 + ); +} + /// The suite's write-path assertions only run when the driver retains, so a /// double that silently dropped writes would let the whole run pass vacuously. /// This pins that the Mem0 double is genuinely retaining. diff --git a/crates/tinymemory-remote/src/graph_provider.rs b/crates/tinymemory-remote/src/graph_provider.rs index 07eed6be..10d699e1 100644 --- a/crates/tinymemory-remote/src/graph_provider.rs +++ b/crates/tinymemory-remote/src/graph_provider.rs @@ -18,17 +18,17 @@ use async_trait::async_trait; use tinymemory_api::capabilities::{Capabilities, Capability}; use tinymemory_api::error::MemoryError; use tinymemory_api::health::MemoryHealth; -use tinymemory_api::mandatory::MemoryTraitProvider; use tinymemory_api::provider::types::{ExportPage, ExportRecord, ImportOutcome, SourceScope}; use tinymemory_api::provider::{ - MemoryCore, MemoryGraph, MemoryPortability, MemoryProvider, MemoryRecall, + MemoryAnswer, MemoryConversationIngest, MemoryCore, MemoryDocumentIngest, MemoryEventIngest, + MemoryGraph, MemoryLearningIngest, MemoryPortability, MemoryProvider, MemoryRecall, }; use tinymemory_api::recall::OwnedRecallOpts; use tinymemory_api::types::{MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary}; -/// A [`MemoryTraitProvider`] augmented with a native [`MemoryGraph`]. +/// A [`MemoryProvider`] augmented with a native [`MemoryGraph`]. pub struct GraphMemoryProvider { - mandatory: MemoryTraitProvider, + base: Arc, graph: Arc, } @@ -37,7 +37,7 @@ impl std::fmt::Debug for GraphMemoryProvider { // `dyn MemoryGraph` is not `Debug`; the mandatory half already renders // safely (see `MemoryTraitProvider`'s own impl). f.debug_struct("GraphMemoryProvider") - .field("mandatory", &self.mandatory) + .field("driver_id", &self.base.driver_id()) .finish_non_exhaustive() } } @@ -45,8 +45,11 @@ impl std::fmt::Debug for GraphMemoryProvider { impl GraphMemoryProvider { /// Compose `mandatory` with a native `graph` implementation. #[must_use] - pub fn new(mandatory: MemoryTraitProvider, graph: Arc) -> Self { - Self { mandatory, graph } + pub fn new(base: impl MemoryProvider, graph: Arc) -> Self { + Self { + base: Arc::new(base), + graph, + } } } @@ -61,17 +64,17 @@ impl MemoryCore for GraphMemoryProvider { session_id: Option<&str>, taint: MemoryTaint, ) -> Result<(), MemoryError> { - self.mandatory + self.base .store(namespace, key, content, category, session_id, taint) .await } async fn get(&self, namespace: &str, key: &str) -> Result, MemoryError> { - self.mandatory.get(namespace, key).await + self.base.get(namespace, key).await } async fn forget(&self, namespace: &str, key: &str) -> Result { - self.mandatory.forget(namespace, key).await + self.base.forget(namespace, key).await } async fn list( @@ -80,11 +83,11 @@ impl MemoryCore for GraphMemoryProvider { category: Option<&MemoryCategory>, session_id: Option<&str>, ) -> Result, MemoryError> { - self.mandatory.list(namespace, category, session_id).await + self.base.list(namespace, category, session_id).await } async fn namespaces(&self) -> Result, MemoryError> { - self.mandatory.namespaces().await + self.base.namespaces().await } } @@ -97,7 +100,7 @@ impl MemoryRecall for GraphMemoryProvider { opts: &OwnedRecallOpts, scope: Option<&SourceScope>, ) -> Result, MemoryError> { - self.mandatory.recall(query, limit, opts, scope).await + self.base.recall(query, limit, opts, scope).await } } @@ -108,37 +111,52 @@ impl MemoryPortability for GraphMemoryProvider { cursor: Option<&str>, limit: usize, ) -> Result { - self.mandatory.export_page(cursor, limit).await + self.base.export_page(cursor, limit).await } async fn import_records( &self, records: Vec, ) -> Result { - self.mandatory.import_records(records).await + self.base.import_records(records).await } } #[async_trait] impl MemoryProvider for GraphMemoryProvider { fn driver_id(&self) -> &str { - self.mandatory.driver_id() + self.base.driver_id() } fn capabilities(&self) -> Capabilities { - Capabilities::from_iter([ - Capability::Core, - Capability::Recall, - Capability::Portability, - Capability::Graph, - ]) + self.base.capabilities().with(Capability::Graph) } async fn health(&self) -> MemoryHealth { - self.mandatory.health().await + self.base.health().await } fn as_graph(&self) -> Option<&dyn MemoryGraph> { Some(self.graph.as_ref()) } + + fn as_document_ingest(&self) -> Option<&dyn MemoryDocumentIngest> { + self.base.as_document_ingest() + } + + fn as_conversation_ingest(&self) -> Option<&dyn MemoryConversationIngest> { + self.base.as_conversation_ingest() + } + + fn as_learning_ingest(&self) -> Option<&dyn MemoryLearningIngest> { + self.base.as_learning_ingest() + } + + fn as_event_ingest(&self) -> Option<&dyn MemoryEventIngest> { + self.base.as_event_ingest() + } + + fn as_answer(&self) -> Option<&dyn MemoryAnswer> { + self.base.as_answer() + } } diff --git a/crates/tinymemory-remote/src/lib.rs b/crates/tinymemory-remote/src/lib.rs index b45898d8..9f2b596c 100644 --- a/crates/tinymemory-remote/src/lib.rs +++ b/crates/tinymemory-remote/src/lib.rs @@ -14,6 +14,7 @@ mod common; mod graph_provider; pub mod mem0; mod mem0_graph; +mod mem0_provider; pub mod supermemory; pub use agentmemory::{AgentMemoryMemory, AGENTMEMORY_API_ENDPOINT, AGENTMEMORY_DRIVER_ID}; @@ -22,6 +23,7 @@ pub use cognee_graph::CogneeGraph; pub use graph_provider::GraphMemoryProvider; pub use mem0::{Mem0Memory, MEM0_API_ENDPOINT, MEM0_DRIVER_ID}; pub use mem0_graph::Mem0Graph; +pub use mem0_provider::Mem0Provider; pub use supermemory::{SupermemoryMemory, SUPERMEMORY_API_ENDPOINT, SUPERMEMORY_DRIVER_ID}; use std::sync::Arc; @@ -36,8 +38,8 @@ pub fn supermemory_provider(memory: SupermemoryMemory) -> MemoryTraitProvider { /// Wrap a Mem0 HTTP backend as a bound TinyMemory provider. #[must_use] -pub fn mem0_provider(memory: Mem0Memory) -> MemoryTraitProvider { - MemoryTraitProvider::new(Arc::new(memory), MEM0_DRIVER_ID) +pub fn mem0_provider(memory: Mem0Memory) -> Mem0Provider { + Mem0Provider::new(memory) } /// Wrap a Cognee HTTP backend as a bound TinyMemory provider. @@ -97,8 +99,8 @@ pub fn cognee_api_graph_provider( #[must_use] pub fn mem0_graph_provider(memory: Mem0Memory) -> GraphMemoryProvider { let memory: Arc = Arc::new(memory); - let mandatory = MemoryTraitProvider::new(Arc::clone(&memory), MEM0_DRIVER_ID); - GraphMemoryProvider::new(mandatory, Arc::new(Mem0Graph::new(memory))) + let provider = Mem0Provider::from_memory(Arc::clone(&memory)); + GraphMemoryProvider::new(provider, Arc::new(Mem0Graph::new(memory))) } #[cfg(test)] diff --git a/crates/tinymemory-remote/src/mem0_provider.rs b/crates/tinymemory-remote/src/mem0_provider.rs new file mode 100644 index 00000000..495d02f5 --- /dev/null +++ b/crates/tinymemory-remote/src/mem0_provider.rs @@ -0,0 +1,196 @@ +//! Capability-accurate Mem0 provider composition. + +use std::sync::Arc; + +use async_trait::async_trait; +use sha2::{Digest, Sha256}; +use tinymemory_api::capabilities::{Capabilities, Capability}; +use tinymemory_api::error::MemoryError; +use tinymemory_api::health::MemoryHealth; +use tinymemory_api::mandatory::MemoryTraitProvider; +use tinymemory_api::provider::types::{ + ExportPage, ExportRecord, ImportOutcome, IngestItem, IngestOutcome, SourceScope, +}; +use tinymemory_api::provider::{ + MemoryConversationIngest, MemoryCore, MemoryPortability, MemoryProvider, MemoryRecall, +}; +use tinymemory_api::recall::OwnedRecallOpts; +use tinymemory_api::traits::Memory; +use tinymemory_api::types::{MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary}; + +use crate::common::encode; +use crate::mem0::Mem0Memory; +use crate::MEM0_DRIVER_ID; + +/// Mem0 exposed as mandatory storage/recall plus conversation ingestion. +pub struct Mem0Provider { + mandatory: MemoryTraitProvider, +} + +impl std::fmt::Debug for Mem0Provider { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter + .debug_struct("Mem0Provider") + .finish_non_exhaustive() + } +} + +impl Mem0Provider { + /// Wrap a native Mem0 client. + #[must_use] + pub fn new(memory: Mem0Memory) -> Self { + Self::from_memory(Arc::new(memory)) + } + + pub(crate) fn from_memory(memory: Arc) -> Self { + Self { + mandatory: MemoryTraitProvider::new(memory, MEM0_DRIVER_ID), + } + } +} + +#[async_trait] +impl MemoryCore for Mem0Provider { + async fn store( + &self, + namespace: &str, + key: &str, + content: &str, + category: MemoryCategory, + session_id: Option<&str>, + taint: MemoryTaint, + ) -> Result<(), MemoryError> { + self.mandatory + .store(namespace, key, content, category, session_id, taint) + .await + } + + async fn get(&self, namespace: &str, key: &str) -> Result, MemoryError> { + self.mandatory.get(namespace, key).await + } + + async fn forget(&self, namespace: &str, key: &str) -> Result { + self.mandatory.forget(namespace, key).await + } + + async fn list( + &self, + namespace: Option<&str>, + category: Option<&MemoryCategory>, + session_id: Option<&str>, + ) -> Result, MemoryError> { + self.mandatory.list(namespace, category, session_id).await + } + + async fn namespaces(&self) -> Result, MemoryError> { + self.mandatory.namespaces().await + } +} + +#[async_trait] +impl MemoryRecall for Mem0Provider { + async fn recall( + &self, + query: &str, + limit: usize, + opts: &OwnedRecallOpts, + scope: Option<&SourceScope>, + ) -> Result, MemoryError> { + self.mandatory.recall(query, limit, opts, scope).await + } +} + +#[async_trait] +impl MemoryPortability for Mem0Provider { + async fn export_page( + &self, + cursor: Option<&str>, + limit: usize, + ) -> Result { + self.mandatory.export_page(cursor, limit).await + } + + async fn import_records( + &self, + records: Vec, + ) -> Result { + self.mandatory.import_records(records).await + } +} + +#[async_trait] +impl MemoryConversationIngest for Mem0Provider { + async fn ingest_conversation( + &self, + messages: Vec, + ) -> Result { + let Some(first) = messages.first() else { + return Ok(IngestOutcome::default()); + }; + let conversation_id = first.source_id.clone(); + if conversation_id.trim().is_empty() { + return Err(MemoryError::Invalid( + "conversation id must not be empty".to_string(), + )); + } + if messages + .iter() + .any(|item| item.source_id != conversation_id || item.content.trim().is_empty()) + { + return Err(MemoryError::Invalid( + "conversation batches must contain one conversation and non-empty messages" + .to_string(), + )); + } + + let mut ids = Vec::with_capacity(messages.len()); + for (index, message) in messages.into_iter().enumerate() { + let namespace = message + .namespace + .unwrap_or_else(|| format!("conversation:{conversation_id}")); + let mut digest = Sha256::new(); + digest.update(conversation_id.as_bytes()); + digest.update(index.to_le_bytes()); + digest.update(message.content.as_bytes()); + if let Some(timestamp) = message.timestamp { + digest.update(timestamp.to_rfc3339().as_bytes()); + } + let key = format!("message-{}", encode(digest.finalize())); + self.store( + &namespace, + &key, + &message.content, + MemoryCategory::Conversation, + Some(&conversation_id), + message.taint, + ) + .await?; + ids.push(key); + } + + Ok(IngestOutcome { + written: u32::try_from(ids.len()).unwrap_or(u32::MAX), + ids, + ..IngestOutcome::default() + }) + } +} + +#[async_trait] +impl MemoryProvider for Mem0Provider { + fn driver_id(&self) -> &str { + MEM0_DRIVER_ID + } + + fn capabilities(&self) -> Capabilities { + Capabilities::mandatory().with(Capability::ConversationIngest) + } + + async fn health(&self) -> MemoryHealth { + self.mandatory.health().await + } + + fn as_conversation_ingest(&self) -> Option<&dyn MemoryConversationIngest> { + Some(self) + } +} diff --git a/crates/tinymemory-tinycortex/src/conformance_test.rs b/crates/tinymemory-tinycortex/src/conformance_test.rs index 067ddf6f..59bac634 100644 --- a/crates/tinymemory-tinycortex/src/conformance_test.rs +++ b/crates/tinymemory-tinycortex/src/conformance_test.rs @@ -11,7 +11,7 @@ //! `tinycortex::memory::Memory` backend. It needs nothing but the backend, so //! the suite runs against it here with the engine's own `InMemoryMemoryStore`. //! -//! [`crate::engine::TinycortexProvider`] serves all twenty families, and +//! [`crate::engine::TinycortexProvider`] serves every compiled family, and //! needs a `MemoryClient` — which needs the host's process-global seams //! (`set_embedding_host` and friends) installed before it will open. A test //! that installs a process global is order-dependent, which `AGENTS.md` rules diff --git a/crates/tinymemory-tinycortex/src/document_provider.rs b/crates/tinymemory-tinycortex/src/document_provider.rs new file mode 100644 index 00000000..dd8d489c --- /dev/null +++ b/crates/tinymemory-tinycortex/src/document_provider.rs @@ -0,0 +1,153 @@ +//! Lightweight TinyCortex composition with document ingestion. + +use async_trait::async_trait; +use tinymemory_api::capabilities::{Capabilities, Capability}; +use tinymemory_api::error::MemoryError; +use tinymemory_api::health::MemoryHealth; +use tinymemory_api::mandatory::MemoryTraitProvider; +use tinymemory_api::provider::types::{ + ExportPage, ExportRecord, ImportOutcome, IngestItem, IngestOutcome, SourceScope, +}; +use tinymemory_api::provider::{ + MemoryCore, MemoryDocumentIngest, MemoryPortability, MemoryProvider, MemoryRecall, +}; +use tinymemory_api::recall::OwnedRecallOpts; +use tinymemory_api::types::{MemoryCategory, MemoryEntry, MemoryTaint, NamespaceSummary}; + +/// TinyCortex's lightweight provider: mandatory storage plus document ingest. +pub struct TinycortexDocumentProvider { + mandatory: MemoryTraitProvider, +} + +impl std::fmt::Debug for TinycortexDocumentProvider { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter + .debug_struct("TinycortexDocumentProvider") + .finish_non_exhaustive() + } +} + +impl TinycortexDocumentProvider { + pub(crate) fn new(mandatory: MemoryTraitProvider) -> Self { + Self { mandatory } + } +} + +#[async_trait] +impl MemoryCore for TinycortexDocumentProvider { + async fn store( + &self, + namespace: &str, + key: &str, + content: &str, + category: MemoryCategory, + session_id: Option<&str>, + taint: MemoryTaint, + ) -> Result<(), MemoryError> { + self.mandatory + .store(namespace, key, content, category, session_id, taint) + .await + } + + async fn get(&self, namespace: &str, key: &str) -> Result, MemoryError> { + self.mandatory.get(namespace, key).await + } + + async fn forget(&self, namespace: &str, key: &str) -> Result { + self.mandatory.forget(namespace, key).await + } + + async fn list( + &self, + namespace: Option<&str>, + category: Option<&MemoryCategory>, + session_id: Option<&str>, + ) -> Result, MemoryError> { + self.mandatory.list(namespace, category, session_id).await + } + + async fn namespaces(&self) -> Result, MemoryError> { + self.mandatory.namespaces().await + } +} + +#[async_trait] +impl MemoryRecall for TinycortexDocumentProvider { + async fn recall( + &self, + query: &str, + limit: usize, + opts: &OwnedRecallOpts, + scope: Option<&SourceScope>, + ) -> Result, MemoryError> { + self.mandatory.recall(query, limit, opts, scope).await + } +} + +#[async_trait] +impl MemoryPortability for TinycortexDocumentProvider { + async fn export_page( + &self, + cursor: Option<&str>, + limit: usize, + ) -> Result { + self.mandatory.export_page(cursor, limit).await + } + + async fn import_records( + &self, + records: Vec, + ) -> Result { + self.mandatory.import_records(records).await + } +} + +#[async_trait] +impl MemoryDocumentIngest for TinycortexDocumentProvider { + async fn ingest_document(&self, document: IngestItem) -> Result { + if document.source_id.trim().is_empty() || document.content.trim().is_empty() { + return Err(MemoryError::Invalid( + "document source id and content must not be empty".to_string(), + )); + } + let namespace = document + .namespace + .unwrap_or_else(|| format!("document:{}", document.source_id)); + let key = document + .source_ref + .map_or_else(|| document.source_id.clone(), |source| source.value); + self.store( + &namespace, + &key, + &document.content, + MemoryCategory::Core, + None, + document.taint, + ) + .await?; + Ok(IngestOutcome { + written: 1, + ids: vec![format!("{namespace}/{key}")], + ..IngestOutcome::default() + }) + } +} + +#[async_trait] +impl MemoryProvider for TinycortexDocumentProvider { + fn driver_id(&self) -> &str { + self.mandatory.driver_id() + } + + fn capabilities(&self) -> Capabilities { + Capabilities::mandatory().with(Capability::DocumentIngest) + } + + async fn health(&self) -> MemoryHealth { + self.mandatory.health().await + } + + fn as_document_ingest(&self) -> Option<&dyn MemoryDocumentIngest> { + Some(self) + } +} diff --git a/crates/tinymemory-tinycortex/src/engine/mod.rs b/crates/tinymemory-tinycortex/src/engine/mod.rs index 95a2e5ab..d5c92c8d 100644 --- a/crates/tinymemory-tinycortex/src/engine/mod.rs +++ b/crates/tinymemory-tinycortex/src/engine/mod.rs @@ -39,6 +39,9 @@ use tinymemory_api::provider::types::{ }; // Diff-family value types, used only by the `MemoryDiff` impl below — which is // compiled out without the git-backed snapshot store. +use tinymemory_api::operations::{ + AnswerCitation, AnswerRequest, AnswerResponse, AnswerStep, RawMemoryEvent, +}; #[cfg(feature = "memory-git")] use tinymemory_api::provider::types::{ChangeKind, DiffReport, SnapshotRef, SourceChange}; use tinymemory_api::provider::{ @@ -46,12 +49,13 @@ use tinymemory_api::provider::{ ChunkScoreSignals, CodingSessionIngestReport, CodingSessionIngestRequest, CodingSessionSource, ConversationSegment, CoverWindowQuery, DegradedCapabilities, Diagnosis, DiagnosisCounters, DiagnosisFailure, DiagnosisStage, EntityMatch, EpisodicEvent, EpisodicTurn, EventKind, - FacetType, FastRetrieveQuery, MemoryChunks, MemoryCodingSessions, MemoryCore, MemoryDiff, - MemoryDocuments, MemoryEntities, MemoryEpisodic, MemoryGoals, MemoryGraph, MemoryIngest, - MemoryMaintenance, MemoryPeople, MemoryPortability, MemoryProfile, MemoryProvider, - MemoryRecall, MemoryRetrieval, MemoryScoring, MemorySourceSink, MemorySourceSync, - MemoryToolMemory, MemoryTree, PersonHandle, PersonInteraction, PersonRecord, PersonScore, - ProfileFacet, RankedPerson, RawArchiveCoverage, RawRebuildOutcome, ResolvedPerson, + FacetType, FastRetrieveQuery, MemoryAnswer, MemoryChunks, MemoryCodingSessions, + MemoryConversationIngest, MemoryCore, MemoryDiff, MemoryDocumentIngest, MemoryDocuments, + MemoryEntities, MemoryEpisodic, MemoryEventIngest, MemoryGoals, MemoryGraph, MemoryIngest, + MemoryLearningIngest, MemoryMaintenance, MemoryPeople, MemoryPortability, MemoryProfile, + MemoryProvider, MemoryRecall, MemoryRetrieval, MemoryScoring, MemorySourceSink, + MemorySourceSync, MemoryToolMemory, MemoryTree, PersonHandle, PersonInteraction, PersonRecord, + PersonScore, ProfileFacet, RankedPerson, RawArchiveCoverage, RawRebuildOutcome, ResolvedPerson, RetrievalHit, RetrievalResponse, SourceIngestQuery, SourceIngestStatus, SourceRetrievalQuery, SourceSyncState, SourceSyncStatus, SourceTotal, SyncAuditEntry, SyncFreshness, SyncRunOutcome, UserState, @@ -748,6 +752,177 @@ impl MemoryIngest for TinycortexProvider { } } +#[async_trait] +impl MemoryDocumentIngest for TinycortexProvider { + async fn ingest_document(&self, document: IngestItem) -> Result { + MemoryIngest::ingest_document(self, document).await + } +} + +#[async_trait] +impl MemoryConversationIngest for TinycortexProvider { + async fn ingest_conversation( + &self, + messages: Vec, + ) -> Result { + MemoryIngest::ingest_chat(self, messages).await + } +} + +#[async_trait] +impl MemoryLearningIngest for TinycortexProvider { + async fn ingest_learning( + &self, + learning: tinymemory_api::learning::LearningCandidate, + ) -> Result { + if learning.key.trim().is_empty() || learning.value.trim().is_empty() { + return Err(MemoryError::Invalid( + "learning key and value must not be empty".to_string(), + )); + } + if !learning.initial_confidence.is_finite() + || !(0.0..=1.0).contains(&learning.initial_confidence) + { + return Err(MemoryError::Invalid( + "learning confidence must be between 0 and 1".to_string(), + )); + } + + let class = serde_json::to_value(learning.class) + .ok() + .and_then(|value| value.as_str().map(str::to_owned)) + .ok_or_else(|| MemoryError::Invalid("learning class is invalid".to_string()))?; + let namespace = format!("learning:{class}"); + let key = learning.key.clone(); + let content = serde_json::to_string(&learning) + .map_err(|error| Self::other("encode learning", error))?; + self.store( + &namespace, + &key, + &content, + MemoryCategory::Core, + None, + MemoryTaint::Internal, + ) + .await?; + tinymemory_core::learning_candidate::global().push(learning); + Ok(IngestOutcome { + written: 1, + ids: vec![format!("{namespace}/{key}")], + ..IngestOutcome::default() + }) + } +} + +#[async_trait] +impl MemoryEventIngest for TinycortexProvider { + async fn ingest_event(&self, event: RawMemoryEvent) -> Result { + if event.id.trim().is_empty() + || event.namespace.trim().is_empty() + || event.event_type.trim().is_empty() + || event.content.trim().is_empty() + { + return Err(MemoryError::Invalid( + "event id, namespace, type, and content must not be empty".to_string(), + )); + } + let event_id = event.id.clone(); + let namespace = format!("event:{}", event.namespace); + let content = serde_json::to_string(&event) + .map_err(|error| Self::other("encode raw event", error))?; + self.store( + &namespace, + &event_id, + &content, + MemoryCategory::Daily, + event.session_id.as_deref(), + event.taint, + ) + .await?; + Ok(IngestOutcome { + written: 1, + ids: vec![event_id], + ..IngestOutcome::default() + }) + } +} + +#[async_trait] +impl MemoryAnswer for TinycortexProvider { + async fn answer(&self, request: AnswerRequest) -> Result { + if request.query.trim().is_empty() { + return Err(MemoryError::Invalid( + "answer query must not be empty".to_string(), + )); + } + if request.limit == 0 { + return Err(MemoryError::Invalid( + "answer retrieval limit must be greater than zero".to_string(), + )); + } + + let memories = self + .recall( + &request.query, + request.limit, + &request.recall, + request.scope.as_ref(), + ) + .await?; + let citations: Vec = memories + .iter() + .map(|memory| AnswerCitation { + id: memory.id.clone(), + namespace: memory.namespace.clone(), + key: memory.key.clone(), + content: memory.content.clone(), + score: memory.score, + }) + .collect(); + let context = citations + .iter() + .enumerate() + .map(|(index, citation)| format!("[{}] {}", index + 1, citation.content)) + .collect::>() + .join("\n\n"); + let instructions = request.instructions.as_deref().unwrap_or(""); + let (chat, model) = tinymemory_core::chat::build_chat_runtime(&self.config) + .map_err(|error| Self::other("build answer agent", error))?; + let prompt = tinymemory_core::chat::ChatPrompt { + system: format!( + "You are a grounded memory-answering agent. Answer only from the retrieved \ + memories. Cite supporting memories as [n]. Say when the memories do not contain \ + enough information. Additional caller instructions: {instructions}\n\nRetrieved \ + memories:\n{context}" + ), + user: request.query, + temperature: 0.1, + kind: "memory_answer", + max_tokens: None, + }; + let answer = chat + .chat_for_text(&prompt) + .await + .map_err(|error| Self::other("answer synthesis", error))?; + + Ok(AnswerResponse { + answer, + citations, + steps: vec![ + AnswerStep { + operation: "recall".to_string(), + detail: format!("retrieved {} memories", memories.len()), + }, + AnswerStep { + operation: "synthesise".to_string(), + detail: "generated a grounded answer".to_string(), + }, + ], + model: Some(model), + }) + } +} + #[async_trait] impl MemoryGraph for TinycortexProvider { async fn kv_get( @@ -2764,6 +2939,21 @@ impl MemoryProvider for TinycortexProvider { fn as_scoring(&self) -> Option<&dyn MemoryScoring> { Some(self) } + fn as_document_ingest(&self) -> Option<&dyn MemoryDocumentIngest> { + Some(self) + } + fn as_conversation_ingest(&self) -> Option<&dyn MemoryConversationIngest> { + Some(self) + } + fn as_learning_ingest(&self) -> Option<&dyn MemoryLearningIngest> { + Some(self) + } + fn as_event_ingest(&self) -> Option<&dyn MemoryEventIngest> { + Some(self) + } + fn as_answer(&self) -> Option<&dyn MemoryAnswer> { + Some(self) + } } // ── Source sync ────────────────────────────────────────────────────────────── diff --git a/crates/tinymemory-tinycortex/src/lib.rs b/crates/tinymemory-tinycortex/src/lib.rs index 4f7a5aac..b5bab609 100644 --- a/crates/tinymemory-tinycortex/src/lib.rs +++ b/crates/tinymemory-tinycortex/src/lib.rs @@ -18,9 +18,7 @@ //! backend as a TinyMemory //! [`Memory`](tinymemory_api::traits::Memory). //! - [`provider`] — the one call that turns a TinyCortex backend into a -//! mandatory-only driver, by pairing [`TinycortexMemory`] with -//! [`MemoryTraitProvider`]. Enough when a host wants store, recall and -//! export and nothing else. +//! lightweight driver serving the mandatory families and document ingest. //! - [`engine`] — [`TinycortexProvider`](engine::TinycortexProvider), the whole //! engine behind the contract: trees, chunks, entities, the graph, goals, //! tool-memory, ingestion, sources, maintenance, people, retrieval, profile, @@ -28,7 +26,8 @@ //! //! ## Two drivers, and why both //! -//! [`provider`] advertises Core, Recall and Portability. That used to be the +//! [`provider`] advertises Core, Recall, Portability and DocumentIngest. The +//! mandatory-only composition used to be the //! only thing here, and it was the reason anything wanting a summary tree or a //! diff ledger reached past the contract to the engine directly: the families //! existed, but not through `MemoryProvider`. Issue #18 §C3 lifted those @@ -49,9 +48,11 @@ //! [`engine::advertised_capabilities`] and not just the accessor — a build //! without the git-backed snapshot store must not claim a diff ledger. +mod document_provider; pub mod engine; mod memory; +pub use document_provider::TinycortexDocumentProvider; pub use memory::TinycortexMemory; use std::sync::Arc; @@ -75,19 +76,19 @@ pub use tinycortex; /// The engine's simplest backend, re-exported for first-run and test wiring: /// `provider(Arc::new(InMemoryMemoryStore::new()))` is a complete embedded -/// setup for the mandatory three families. +/// setup for the mandatory families and document ingestion. pub use tinycortex::memory::store::InMemoryMemoryStore; /// Wrap a TinyCortex backend as a bound memory driver. /// -/// The returned provider advertises the mandatory three families and nothing -/// else; see the crate docs. +/// The returned provider advertises the mandatory families plus document +/// ingestion; see the crate docs. #[must_use] -pub fn provider(memory: Arc) -> MemoryTraitProvider { - MemoryTraitProvider::new( +pub fn provider(memory: Arc) -> TinycortexDocumentProvider { + TinycortexDocumentProvider::new(MemoryTraitProvider::new( Arc::new(TinycortexMemory::new(memory)), TINYCORTEX_DRIVER_ID, - ) + )) } #[cfg(test)] diff --git a/crates/tinymemory-tinycortex/src/memory_test.rs b/crates/tinymemory-tinycortex/src/memory_test.rs index c618d120..02eeefd4 100644 --- a/crates/tinymemory-tinycortex/src/memory_test.rs +++ b/crates/tinymemory-tinycortex/src/memory_test.rs @@ -7,6 +7,7 @@ #![allow(clippy::expect_used, clippy::panic)] use tinycortex::memory::store::InMemoryMemoryStore; +use tinymemory_api::capabilities::Capability; use tinymemory_api::provider::{audit_provider, MemoryCore, MemoryPortability, MemoryProvider}; use tinymemory_api::types::{MemoryCategory, MemoryTaint, GLOBAL_NAMESPACE}; @@ -24,10 +25,40 @@ async fn the_adapter_reports_the_engine_backend_name() { } #[tokio::test] -async fn a_driver_over_the_engine_advertises_exactly_the_mandatory_three() { +async fn a_lightweight_driver_advertises_document_ingestion() { let driver = crate::provider(engine()); audit_provider(&driver).expect("advertised capabilities match the accessors"); assert_eq!(driver.driver_id(), TINYCORTEX_DRIVER_ID); + assert!(driver.capabilities().contains(Capability::DocumentIngest)); + assert!(driver.as_document_ingest().is_some()); + assert!(!driver + .capabilities() + .contains(Capability::ConversationIngest)); +} + +#[tokio::test] +async fn lightweight_document_ingestion_routes_into_the_embedded_store() { + let driver = crate::provider(engine()); + let document = serde_json::from_value(serde_json::json!({ + "source": "upload", + "source_id": "handbook", + "content": "The release train leaves on Friday." + })) + .expect("document item"); + let outcome = driver + .as_document_ingest() + .expect("document route") + .ingest_document(document) + .await + .expect("ingest document"); + + assert_eq!(outcome.written, 1); + let stored = driver + .get("document:handbook", "handbook") + .await + .expect("get document") + .expect("stored document"); + assert_eq!(stored.content, "The release train leaves on Friday."); } #[tokio::test] diff --git a/crates/tinymemory-tinycortex/tests/full_provider_conformance.rs b/crates/tinymemory-tinycortex/tests/full_provider_conformance.rs index 9b35af83..632230d4 100644 --- a/crates/tinymemory-tinycortex/tests/full_provider_conformance.rs +++ b/crates/tinymemory-tinycortex/tests/full_provider_conformance.rs @@ -1,7 +1,7 @@ -//! The conformance suite over the FULL twenty-family driver (#18 §E1/§E3). +//! The conformance suite over the full driver (#18 §E1/§E3). //! -//! `conformance_test.rs` (in-lib) covers `crate::provider` — the mandatory -//! three families over any engine backend. This target covers +//! `conformance_test.rs` (in-lib) covers `crate::provider` — the lightweight +//! provider over any engine backend. This target covers //! [`tinymemory_tinycortex::engine::TinycortexProvider`], which the in-lib //! test cannot: the provider needs a `MemoryClient`, and a `MemoryClient` //! needs the host's process-global embedding seam installed. A process global @@ -141,6 +141,61 @@ async fn the_full_provider_actually_retains() { ); } +#[tokio::test(flavor = "multi_thread")] +async fn learning_and_raw_event_routes_persist_recallable_records() { + use tinymemory_api::operations::RawMemoryEvent; + use tinymemory_api::provider::{MemoryCore, MemoryProvider}; + use tinymemory_api::types::MemoryTaint; + + let workspace = tempfile::tempdir().expect("workspace"); + let provider = provider_over(workspace.path()); + let learning = serde_json::from_value(serde_json::json!({ + "class": "tooling", + "key": "package_manager", + "value": "pnpm", + "cue_family": "explicit", + "evidence": {"type": "episodic", "episodic_id": 7}, + "initial_confidence": 0.95, + "observed_at": 1_700_000_000.0 + })) + .expect("learning candidate"); + provider + .as_learning_ingest() + .expect("learning route") + .ingest_learning(learning) + .await + .expect("ingest learning"); + assert!(provider + .get("learning:tooling", "package_manager") + .await + .expect("get learning") + .is_some()); + + let event = RawMemoryEvent { + id: "evt-1".into(), + namespace: "calendar".into(), + event_type: "meeting_rescheduled".into(), + content: "The architecture review moved to Friday".into(), + occurred_at: None, + session_id: Some("session-1".into()), + metadata: serde_json::json!({"calendar_id": "work"}), + taint: MemoryTaint::ExternalSync, + }; + provider + .as_event_ingest() + .expect("event route") + .ingest_event(event) + .await + .expect("ingest event"); + let stored = provider + .get("event:calendar", "evt-1") + .await + .expect("get event") + .expect("stored event"); + assert_eq!(stored.taint, MemoryTaint::ExternalSync); + assert!(stored.content.contains("meeting_rescheduled")); +} + /// The maintenance diagnostics answer from the store, not from their defaults. /// /// `store_stats`, `queue_stats` and `latest_queue_failure` are defaulted on diff --git a/crates/tinymemory/src/lib.rs b/crates/tinymemory/src/lib.rs index dfba2bc4..58626d27 100644 --- a/crates/tinymemory/src/lib.rs +++ b/crates/tinymemory/src/lib.rs @@ -146,6 +146,8 @@ pub use tinymemory_documents as documents; pub use tinymemory_conformance as conformance; pub mod registry; +pub mod routing; +pub use routing::MemoryApi; // Typed surfaces for the sections the namespace convention names — // conversations, learnings, documents — plus a section-aware recall. Documented @@ -157,8 +159,8 @@ pub mod sections; // glob so the crate's own surface is visible in one place and rustdoc links // resolve — and so adding a module to the contract is a deliberate act here too. pub use tinymemory_api::{ - capabilities, chunks, error, goals, health, namespace, null, provider, recall, tool_memory, - traits, tree, types, + capabilities, chunks, error, goals, health, namespace, null, operations, provider, recall, + tool_memory, traits, tree, types, }; pub use tinymemory_api::{is_compatible, CONTRACT_VERSION}; diff --git a/crates/tinymemory/src/routing.rs b/crates/tinymemory/src/routing.rs new file mode 100644 index 00000000..eb8f0959 --- /dev/null +++ b/crates/tinymemory/src/routing.rs @@ -0,0 +1,158 @@ +//! One high-level router over a negotiated memory provider. + +use tinymemory_api::capabilities::Capability; +use tinymemory_api::error::MemoryError; +use tinymemory_api::learning::LearningCandidate; +use tinymemory_api::operations::{AnswerRequest, AnswerResponse, RawMemoryEvent}; +use tinymemory_api::provider::types::{IngestItem, IngestOutcome, SourceScope}; +use tinymemory_api::provider::MemoryProvider; +use tinymemory_api::recall::OwnedRecallOpts; +use tinymemory_api::types::MemoryEntry; + +/// Routes the six product-facing memory operations through one provider. +/// +/// Optional operations fail with a typed [`MemoryError::Unsupported`] naming +/// the absent capability. Recall is always callable because it is mandatory on +/// [`MemoryProvider`]. +#[derive(Clone, Copy)] +pub struct MemoryApi<'a> { + provider: &'a dyn MemoryProvider, +} + +impl std::fmt::Debug for MemoryApi<'_> { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + formatter + .debug_struct("MemoryApi") + .field("driver_id", &self.provider.driver_id()) + .field("capabilities", &self.provider.capabilities()) + .finish() + } +} + +impl<'a> MemoryApi<'a> { + /// Bind the router to a provider. + #[must_use] + pub fn new(provider: &'a dyn MemoryProvider) -> Self { + Self { provider } + } + + /// Ingest a document through the provider's document route. + /// + /// # Errors + /// + /// Returns `Unsupported(document_ingest)` when the route is absent, + /// otherwise the provider's validation or backend error. + pub async fn ingest_document( + &self, + document: IngestItem, + ) -> Result { + self.provider + .as_document_ingest() + .ok_or_else(|| MemoryError::unsupported(Capability::DocumentIngest))? + .ingest_document(document) + .await + } + + /// Ingest an ordered conversation. + /// + /// # Errors + /// + /// Returns `Unsupported(conversation_ingest)` when absent, otherwise the + /// provider's error. + pub async fn ingest_conversation( + &self, + messages: Vec, + ) -> Result { + self.provider + .as_conversation_ingest() + .ok_or_else(|| MemoryError::unsupported(Capability::ConversationIngest))? + .ingest_conversation(messages) + .await + } + + /// Ingest one extracted learning. + /// + /// # Errors + /// + /// Returns `Unsupported(learning_ingest)` when absent, otherwise the + /// provider's error. + pub async fn ingest_learning( + &self, + learning: LearningCandidate, + ) -> Result { + self.provider + .as_learning_ingest() + .ok_or_else(|| MemoryError::unsupported(Capability::LearningIngest))? + .ingest_learning(learning) + .await + } + + /// Ingest one raw event. + /// + /// # Errors + /// + /// Returns `Unsupported(event_ingest)` when absent, otherwise the + /// provider's error. + pub async fn ingest_event(&self, event: RawMemoryEvent) -> Result { + self.provider + .as_event_ingest() + .ok_or_else(|| MemoryError::unsupported(Capability::EventIngest))? + .ingest_event(event) + .await + } + + /// Run deterministic ranked recall. + /// + /// # Errors + /// + /// Returns the provider's validation or backend error. + pub async fn recall( + &self, + query: &str, + limit: usize, + options: &OwnedRecallOpts, + scope: Option<&SourceScope>, + ) -> Result, MemoryError> { + self.provider.recall(query, limit, options, scope).await + } + + /// Run agentic grounded answer synthesis. + /// + /// # Errors + /// + /// Returns `Unsupported(answer)` when absent, otherwise the provider's + /// retrieval or inference error. + pub async fn answer(&self, request: AnswerRequest) -> Result { + self.provider + .as_answer() + .ok_or_else(|| MemoryError::unsupported(Capability::Answer))? + .answer(request) + .await + } +} + +#[cfg(test)] +mod tests { + use super::MemoryApi; + use tinymemory_api::capabilities::Capability; + use tinymemory_api::error::MemoryError; + use tinymemory_api::null::NullMemoryProvider; + use tinymemory_api::operations::AnswerRequest; + + #[tokio::test] + async fn absent_optional_routes_return_the_named_capability() { + let provider = NullMemoryProvider::new(); + let api = MemoryApi::new(&provider); + let result = api.answer(AnswerRequest::new("question")).await; + if let Err(error) = result { + assert!(matches!( + error, + MemoryError::Unsupported { + capability + } if capability == Capability::Answer.as_str() + )); + } else { + assert!(result.is_err(), "answer route should be absent"); + } + } +} diff --git a/docs/specs/README.md b/docs/specs/README.md index a8286ae3..7066fa11 100644 --- a/docs/specs/README.md +++ b/docs/specs/README.md @@ -1,5 +1,7 @@ # Specifications +- [Granular ingestion and retrieval API](ingestion-retrieval-api.md) + Specifications define what the system must do before implementation details take over. Create one for behavior that changes a public API, crosses module boundaries, introduces a durable data format, or has meaningful operational diff --git a/docs/specs/ingestion-retrieval-api.md b/docs/specs/ingestion-retrieval-api.md new file mode 100644 index 00000000..0ea33545 --- /dev/null +++ b/docs/specs/ingestion-retrieval-api.md @@ -0,0 +1,67 @@ +# Granular ingestion and retrieval API + +## Purpose + +TinyMemory exposes six product-facing memory operations: + +1. document ingestion; +2. conversation ingestion; +3. learning ingestion; +4. event ingestion; +5. recall; and +6. answer. + +They are capability-negotiated independently. A connector must never advertise +an operation merely because it implements a neighbouring one. + +## Contract + +Document and conversation ingestion accept the existing `IngestItem` wire +shape. This preserves source identity, ownership, timestamps, provenance taint, +and citations without adding a parallel payload model. Conversations are +ordered batches and every item must share one `source_id`. + +Learning ingestion accepts `LearningCandidate`, including its cue family, +confidence, and typed evidence pointer. Event ingestion accepts +`RawMemoryEvent`: an open event type, idempotency key, searchable content, +timestamp, metadata, session, and provenance taint. + +Recall remains the mandatory deterministic ranked-retrieval mechanism. Answer +is optional: it retrieves evidence, asks a configured inference route to +synthesise grounded prose, and returns the answer together with citations and a +content-free execution trace. + +The five optional operation capabilities are appended to the capability bit +order as `document_ingest`, `conversation_ingest`, `learning_ingest`, +`event_ingest`, and `answer`. Existing capability indices do not move. + +## Adapter matrix + +| Adapter | Document | Conversation | Learning | Event | Recall | Answer | +| --- | --- | --- | --- | --- | --- | --- | +| TinyCortex lightweight provider | yes | no | no | no | yes | no | +| Full TinyCortex/Cortex provider | yes | yes | yes | yes | yes | yes | +| Mem0 | no | yes | no | no | yes | no | +| Supermemory, Cognee, AgentMemory | no | no | no | no | yes | no | +| Null | no | no | no | no | yes, empty | no | + +The full embedded provider uses the native document and chat canonicalisation +pipelines. Learnings are stored durably under `learning:` and also +enter the candidate buffer. Raw events are stored under `event:` for +ordinary recall while retaining their structured envelope. Answer uses +the host-provided chat route and never owns credentials. + +Mem0 stores each ordered message through its native memory endpoint with a +deterministic key and conversation namespace. It deliberately advertises no +other ingestion capability. + +## Failure and safety rules + +- Empty identifiers, empty content, invalid confidence, and zero answer limits + are `MemoryError::Invalid`. +- Provenance taint is passed through document and conversation routes. +- Source allowlists are applied inside recall before answer synthesis. +- An answer response exposes retrieved evidence but never exposes prompts, + credentials, or hidden model reasoning. +- Unsupported operations are absent from capability negotiation and provider + accessors; callers do not discover them by invoking a failing method.