Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Low-hanging perf improvements in highlighting code #196

Merged
merged 2 commits into from
Feb 6, 2024

Conversation

stellarhoof
Copy link
Member

@stellarhoof stellarhoof commented Feb 5, 2024

Use F.pushOn in more places and a few perf tweaks in hot code paths.

Copy link

changeset-bot bot commented Feb 5, 2024

🦋 Changeset detected

Latest commit: 462eaf0

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
contexture-elasticsearch Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@stellarhoof stellarhoof force-pushed the feature/highlighting-perf-improvements branch from 21436b6 to 1237b3a Compare February 5, 2024 23:01
@stellarhoof stellarhoof self-assigned this Feb 5, 2024
Copy link
Member

@dubiousdavid dubiousdavid left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It may be worth looking at some of these functions/flows to determine if a simple for loop could be much more performant.

(acc, field, path) => {
let subFields = getFieldSubFields(field)
for (let k in subFields) acc[`${path}.${k}`] = subFields[k]
return acc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutating the accumulator and then returning it feels kind of silly to me. Why not just use a for loop. It's easy to read and clearly indicates the imperative approach you're taking here.

@@ -6,7 +6,7 @@ import { groupStats } from './groupStatUtils.js'
// [1, 2, 3] -> [{to: 1}, {from: 1, to: 2}, {from: 2, to: 3}, {from: 3}]
let boundariesToRanges = _.flow(
F.mapIndexed((to, i, list) => F.compactObject({ from: list[i - 1], to })),
(arr) => F.push({ from: _.last(arr).to }, arr)
(arr) => F.pushOn(arr, { from: _.last(arr).to })
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I actually prefer F.push here. Mixing non-mutating with mutating in the same flow is kind of awkward for me. If you want to mutate an array with push you should just loop over the array and do that.

@@ -148,7 +151,7 @@ export let getAllHighlightFields = _.memoize((schema) => {
let collectKeysAndValues = (f, coll) =>
F.reduceTree()(
(acc, val, key) =>
f(val) ? F.push(val, acc) : f(key) ? F.push(key, acc) : acc,
f(val) ? F.pushOn(acc, val) : f(key) ? F.pushOn(acc, key) : acc,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same kind of thing here. Using reduce with a mutating call in the reducing function feels a little awkward to me. Why not just walk the tree and and push onto an array?

@stellarhoof stellarhoof force-pushed the feature/highlighting-perf-improvements branch from 1d609cc to 462eaf0 Compare February 6, 2024 15:36
Copy link
Contributor

github-actions bot commented Feb 6, 2024

Coverage after merging feature/highlighting-perf-improvements into main will be

90.59%

Coverage Report
FileStmtsBranchesFuncsLinesUncovered Lines
packages/client/src
   exampleTypes.js93.78%85.71%40%97.80%109, 123, 126–133
   index.js98.96%93.85%100%100%211, 213, 284, 73
   lens.js100%100%100%100%
   listeners.js100%100%100%100%
   mockService.js95.65%83.33%100%100%22, 30
   node.js96.75%96%87.50%97.78%72, 76, 76
   reactors.js99.12%100%90.91%100%
   serialize.js100%100%100%100%
   subquery.js97.59%88.89%100%98.55%66, 66
   traversals.js100%100%100%100%
   types.js92.59%83.33%100%95.24%12–13
   validation.js78.79%83.33%100%77.78%20–26
packages/client/src/actions
   index.js100%100%100%100%
   wrap.js96.67%85.71%100%97.92%41, 41
packages/client/src/exampleTypes
   pivot.js82.87%74.29%87.50%83.70%10–13, 138–139, 14, 140–149, 15, 150–161, 167–168, 21–22, 227, 23, 243, 257–263, 48, 53–57, 8–9, 9
packages/client/src/util
   futil.js67.31%100%57.14%65.85%18–19, 24–28, 31, 36–41
   tree.js100%100%100%100%
packages/export/src
   csv.js92.73%80%50%95.83%25, 45–46
   index.js0%100%0%0%1–5
   utils.js100%100%100%100%
packages/export/src/nodes
   index.js0%100%0%0%1–5
   pivot.js98.43%82.35%100%100%52, 56, 89
   results.js100%100%100%100%
   terms_stats.js100%100%100%100%
packages/provider-elasticsearch/src
   index.js81.82%52%80%88.24%104–112, 12–13, 29, 53, 53–57, 62, 65–66, 73, 75, 87, 91, 91
   schema.js92.52%92.86%80%93.18%52–55, 85–87
   types.js100%100%100%100%
packages/provider-elasticsearch/src/example-types
   index.js100%100%100%100%
   schemaMapping.js100%100%100%100%
   testUtils.js98.88%92.31%100%100%12
packages/provider-elasticsearch/src/example-types/filters
   bool.js100%100%100%100%
   date.js84.62%66.67%50%88.24%16, 16, 6–8
   dateRangeFacet.js69.79%100%75%66.67%56–83
   exists.js100%100%100%100%
   facet.js92.80%86.67%66.67%94.39%39, 64, 64–69
   geo.js100%100%100%100%
   number.js52.05%100%50%50%22–29, 33–56
   query.js100%100%100%100%
   tagsQuery.js86.47%90.63%85.71%85.71%103–120, 132, 141–145, 151–153
   tagsText.js100%100%100%100%
   text.js95.73%86.67%100%98.82%11, 22, 40, 71, 71
packages/provider-elasticsearch/src/example-types/legacy
   cardinality.js93.33%100%50%100%
   dateHistogram.js95.45%100%50%100%
   esTwoLevelAggregation.js92.11%86.67%66.67%93.75%34–36, 83–87
   matchStats.js97.06%100%50%100%
   rangeStats.js100%100%100%100%
   smartIntervalHistogram.js97.44%100%50%100%
   statistical.js90.91%100%50%100%
   terms_stats.js97.73%100%66.67%100%
packages/provider-elasticsearch/src/example-types/metricGroups
   dateIntervalGroupStats.js100%100%100%100%
   dateRangesGroupStats.js98.28%100%75%100%
   fieldValuePartitionGroupStats.js87.76%100%40%92.31%34–36
   fieldValuesDelta.js97.44%100%66.67%100%
   fieldValuesGroupStats.js99.07%100%83.33%100%
   groupStatUtils.js86.05%100%75%85.71%20–24
   numberIntervalGroupStats.js100%100%100%100%
   numberRangesGroupStats.js97.87%100%66.67%100%
   percentilesGroupStats.js82.46%75%50%85.71%16–18, 24–26, 43–44
   pivot.js85.62%89.89%71.43%85.65%107, 151–152, 159, 166, 408–410, 460–470, 473, 478, 480, 483, 486, 534–535, 563–619, 75–81, 84–91
   stats.js86.96%100%50%94.12%16
   tagsQueryGroupStats.js100%100%100%100%
packages/provider-elasticsearch/src/example-types/metricGroups/pivotData
   columnResponse.js100%100%100%100%
   columnResult.js100%100%100%100%
   pivotResponse.js100%100%100%100%
   pivotResponseWithFilteredFieldValueGroup.js100%100%100%100%
packages/provider-elasticsearch/src/example-types/results
   index.js18.60%100%0%19.51%10–40, 8–9
packages/provider-elasticsearch/src/example-types/results/highlighting
   request.js98.80%97.83%100%98.98%40–42
   response.js100%100%100%100%
   search.js31.15%100%0%31.67%20–60
   testSchema.js100%100%100%100%
   util.js100%100%100%100%
packages/provider-elasticsearch/src/schema-data
   aliases.js100%100%100%100%
   mapping-with-non-objects.js100%100%100%100%
   mapping-with-types.js100%100%100%100%
   mapping-without-types.js100%100%100%100%
   schema-with-types.js100%100%100%100%
   schema-without-types.js100%100%100%100%
packages/provider-elasticsearch/src/utils
   dateUtil.js88.37%85.71%60%90.54%13–15, 18, 64–67
   elasticDSL.js100%100%100%100%
   fields.js100%100%100%100%
   futil.js99.39%100%92.86%100%
   keywordGenerations.js100%100%100%100%
   luceneQueryUtils.js100%100%100%100%
   regex.js100%100%100%100%
   results.js100%100%100%100%
   smartInterval.js83.33%33.33%100%92.86%11, 11, 9
packages/provider-mongo/src
   index.js93.33%71.43%83.33%96.77%10, 33, 45, 9
   types.js100%100%100%100%
packages/provider-mongo/src/example-types
   bool.js100%100%100%100%
   date.js79.34%90.91%55.56%80.20%54–58, 61–69, 7, 70–71, 8–9, 90, 90
   dateHistogram.js94.20%62.50%100%98.31%19, 19, 25, 25
   exists.js100%100%100%100%
   facet.js98.95%95.74%100%99.56%123–124, 124
   index.js100%100%100%100%
   mongoId.js100%100%100%100%
   number.js100%100%100%100%
   results.js97.74%92.31%100%99.01%180, 59, 61, 64–66
   statistical.js100%100%100%100%
   tagsText.js100%100%100%100%
   termsStats.js100%100%100%100%
   text.js100%100%100%100%
packages/react/src
   FilterAdder.js0%100%0%0%1, 10–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–52, 6–9
   FilterAdder.stories.js0%100%0%0%1, 10–19, 2, 20–23, 3–9
   FilterButtonList.js0%100%0%0%1, 10, 100–109, 11, 110–119, 12, 120–129, 13, 130–139, 14, 140–149, 15, 150–159, 16, 160–168, 17–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–59, 6, 60–69, 7, 70–79, 8, 80–89, 9, 90–99
   FilterButtonList.stories.js0%100%0%0%1, 10–19, 2, 20–29, 3, 30–39, 4, 40–49, 5, 50–56, 6–9
   FilterList.js0%100%0%0%1, 10, 100–109, 11, 110–119, 12, 120–129, 13, 130–139, 14, 140–149, 15, 150–159, 16, 160–169, 17,

@dubiousdavid dubiousdavid merged commit dac1b32 into main Feb 6, 2024
4 checks passed
@dubiousdavid dubiousdavid deleted the feature/highlighting-perf-improvements branch February 6, 2024 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants