Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Using "lang":"painless" in the _msearch has poor performance #12184

Open
masterwang22327 opened this issue Feb 6, 2024 · 4 comments
Open

Using "lang":"painless" in the _msearch has poor performance #12184

masterwang22327 opened this issue Feb 6, 2024 · 4 comments
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc

Comments

@masterwang22327
Copy link

Is your feature request related to a problem? Please describe

I have a multi-index and multi-field knn vector search scenario, and the data volume reaches millions. In order to perform full-text search, I plan to use the "_msearch" interface to implement this function in one request. After studying the official documentation, I found that only the "script_score" request method is supported. In the request body I used "lang":"painless". Below is my detailed request body example, a search scenario with three indexes and nine fields.
企业微信截图_17072069355853
When I used this request body to query opensearch, the cashback response time and QPS were very low, so I gradually increased the vector query volume for comparative experiments.
企业微信截图_17072071789730
When I conducted the stress test experiment, some configurations were as follows.
企业微信截图_17072074814052
企业微信截图_17072076281734

Describe the solution you'd like

I want to achieve a corresponding performance of 50QPS and 3 seconds in a multi-index and multi-vector search scenario. How should I optimize the hardware configuration or optimize the query body?

Related component

Search

Describe alternatives you've considered

How should I optimize the hardware configuration or optimize the query body?

Additional context

No response

@masterwang22327 masterwang22327 added enhancement Enhancement or improvement to existing feature or request untriaged labels Feb 6, 2024
@github-actions github-actions bot added the Search Search query, autocomplete ...etc label Feb 6, 2024
@peternied peternied changed the title [Feature Request] <Using "lang":"painless" in the "_msearch" interface has poor performance> Using "lang":"painless" in the _msearch has poor performance Feb 7, 2024
@peternied
Copy link
Member

[Triage - attendees 1 2 3]
@masterwang22327 Thanks for filing this issue. Would it be possible to get a text of the query to aid in analyzing the issue. Making a pull request would be useful to help diagnose as well.

@masterwang22327
Copy link
Author

hello peternied.
my text of the query is as followed。
opensearch1

@peternied
Copy link
Member

@masterwang22327 Thanks, I wasn't clear: please copy/paste the full text of request/response as opposed to a screenshot of text.

@masterwang22327
Copy link
Author

@peternied It was delayed for a while due to technical issues.I put the request body and return body into two txt files respectively.
git_embedding_msearch_query_body.txt
git_embedding_msearch_response_body.txt

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Enhancement or improvement to existing feature or request Search Search query, autocomplete ...etc
Projects
Status: Later (6 months plus)
Development

No branches or pull requests

2 participants