Changes
✨ New Features
- This library now integrates the logging thanks to #126 by @maumar
GetIndex
now returnsIndex<RestTransport>
as a default. After analysis in light of current Pinecone's HTTP/2 stack configuration, it appears to be the better default choice. You can find detailed explanation here: https://github.com/neon-sunset/Pinecone.NET?tab=readme-ov-file#rest-vs-grpc-transport- (.NET 6+) In addition to previously added automatic batching of
Fetch
andUpsert
operations,Delete
is now batched too which is necessary for large deletes as Pinecone limits the request size to 1000 IDs at a time. Much likeFetch
andUpsert
, it is also parallelized. - (.NET 6+) Existing parallelized batching has been enhanced with automatic batch size selection according to vector size to ensure the requests are kept under Pinecone's 2MiB limit. The default parallelism was tuned to reduce heap size impact while retaining existing throughput.
- (.NET 6+) Rich aggregated exception types have been added to parallelized batched operations to allow recovery from partial failure (i.e. where only some of the batches have failed) that preserve successfully retrieved/processed data. Please find the usage example here: https://github.com/neon-sunset/Pinecone.NET?tab=readme-ov-file#advanced
- This update also adds
List
,ListAll
andListPaginated
operations for the recently introducedlist
API call to read or iterate through vector IDs in the PineconeIndex<T>
using specified arguments
🐛 Bug Fixes
- fix: add missed trimmer annotation (f82b555)
🚀 Enhancements
- The library was updated to remain in sync with latest API and official client changes by Pinecone
- The library has switched to using
ReadOnlyMemory<float>
as the abstraction for vector values in order to allow Semantic Kernel and other users reliant onROM<float>
to pass them directly instead of allocating large arrays GrpcTransport
is now configured to take advantage of client-side load balancing, which improves throughput and reduces latency under high concurrency scenarios. The users are still encouraged to useRestTransport
instead, as noted above.- refactor: improve UX of the constructor overload that accepts an HttpClient instance (beb28b7)
📦 Dependencies
- chore(deps): bump Google.Protobuf from 3.27.1 to 3.27.2 in the nuget group (PR #121) by @dependabot (bot)
- chore(deps): bump the nuget group with 3 updates (PR #123) by @dependabot (bot)
- chore(deps): bump the nuget group with 3 updates (PR #125) by @dependabot (bot)
🧰 Misc
- feat: add a convenience method for constructing MetadataValue (aa68e65)
- Disable pod-based index tests until the test environment is switched to a paid tier (c64d63d)
- test: disable remaining pod-based index tests (e384265)
- Further modernization effort (d125d26)
- Further refinement (56de307)
- test: make sure the vector values are compared by sequence (c11c916)
- perf!: switch to RestTransport as the default transport type for GetIndex (e00bfe4)
- test: see if bigger delay fixes the error (c7cc61a)
- test: handle the condition where the namespace is not present in the namespace list if there are no more vectors associated with it (feabe70)
- feat: align impl. with API changes, document collection types, make internal types non-record - there is no need to spend extra IL/AOT size on that (8276c3c)
- test: bump up the delay interval back - too flakey (8cd8b48)
- docs: update examples (edfb41d)
- misc: error message whitespace (7aa2b07)
- docs: add parallel failure recovery example, nudge users to cache client and index, and write detailed guidance on transport selection (e104210)
- docs: formatting (ac88382)
Full Changelog: 2.1.1...3.0.0
Published with dotnet-releaser