Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimization - Skip processing non-existing fields in FieldsWillMerge #5125

Merged
merged 1 commit into from
Oct 22, 2024

Conversation

francisbeaudoin
Copy link
Contributor

@francisbeaudoin francisbeaudoin commented Oct 15, 2024

Context

It was observed that if a query includes thousands of non-existing fields, it takes a lot of processing time within the FieldsWillMerge#find_conflicts_within.

What

The iteration loop represents n(n-1^2)/2, where n is the number of duplicated fields being queried, and the underlying call to #find_conflict is using considerable time. For example, a query with 2000 times the same non-existing field i.e.query { a a ... 2000 times } takes ~6000 ms to process and skipping these fields reduces the response time to ~500ms.

Optimization

As far as I can tell, skipping non-existing fields from the FieldsWillMerge rule doesn't alter the business logic.

The only caveat of this optimization, is that if a service using this library was relying on the Schema#validate_timeout to safeguard from a potentially malicious query, it may no longer reach the configured timeout but rather instead return multiple undefinedField errors, which can be mitigated by the Schema#validate_max_errors.

@rmosolgo rmosolgo added this to the 2.3.18 milestone Oct 15, 2024
@francisbeaudoin
Copy link
Contributor Author

As an alternative, we may be able to further reduce the processing time by not adding the fields entirely here by conditionality adding on unless definition.nil? but I'm not clear on all of the implications hence if it makes sense.

fields << Field.new(node, definition, owner_type, parents)

@rmosolgo
Copy link
Owner

Thanks for this improvement!

@rmosolgo rmosolgo merged commit ec80ea5 into rmosolgo:master Oct 22, 2024
14 of 15 checks passed
@rmosolgo rmosolgo modified the milestones: 2.3.18, 2.3.19 Oct 24, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants