Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

No indexing in batch when using ingestion api #2760

Open
JCTaz9 opened this issue Oct 10, 2024 · 0 comments
Open

No indexing in batch when using ingestion api #2760

JCTaz9 opened this issue Oct 10, 2024 · 0 comments

Comments

@JCTaz9
Copy link

JCTaz9 commented Oct 10, 2024

We use the indexing API to embed a lot of documents with a structure like in the documentation (by section). https://docs.danswer.dev/backend_apis/ingestion
It takes a very long time to embed all of our documents because for every document, it's making one api call. Would it be possible to embed in batch when using the ingestion api?

Right now it takes 13 days embedding what would take a few hours with the files connectors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant