Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The index does not get smaller after removing the vector #2963

Open
Amphetaminewei opened this issue Oct 16, 2024 · 1 comment
Open

The index does not get smaller after removing the vector #2963

Amphetaminewei opened this issue Oct 16, 2024 · 1 comment

Comments

@Amphetaminewei
Copy link

I'm using chroma 0.5.0 by PersistentClient and I've noticed that the index doesn't get smaller when I delete the vector.
By consulting the data, i found that the HNSW index only supports increments and does not support vector deletion. What is chroma's current policy for removing vectors?
In my usage scenario, my database is deployed on a local PC, and I record the vector data of all the text files of the user in the database and update it as the user adds/deletes files, so if there is no deletion scenario, my database may grow indefinitely and occupy more and more memory of the user.

@tazarov
Copy link
Contributor

tazarov commented Oct 20, 2024

@Amphetaminewei, you are right on the HNSW growing. We have similar issue open - #2594 and a PR to address this #2621.

I am also working on a bit of tooling to rebuild HNSW indices.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants