Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

IBX-152: Optimized LSE word indexer #176

Merged
merged 1 commit into from
Aug 8, 2023
Merged

IBX-152: Optimized LSE word indexer #176

merged 1 commit into from
Aug 8, 2023

Conversation

adamwojs
Copy link
Member

@adamwojs adamwojs commented Apr 7, 2021 •
edited by alongosz
Loading

Question Answer
JIRA issue IBX-152
Type bug
Target Ibexa version v3.3+
BC breaks no

Description

Currently word indexer in LSE inserts each unique word to database using separate transaction what have significant performance impact:

image

Enclosing all word insertions queries into single transaction fixes this issue:

image

Other changes introduced in this PR:

  • There is no point of passing 1 for each stmt. as it's might be default value for object_count column but it's rather cosmetic change.

Further optimizations

  • Use bulk inserts for ezsearch_word and ezsearch_object_word_link data when supported in DBAL or create our own abstraction.

Notes for QA

  1. Create clean installation with LSE
  2. Create folder with long description
  3. Publish content
Test data example Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed commodo feugiat pharetra. Duis faucibus ultrices ex, malesuada rhoncus felis lobortis in. Vestibulum vehicula dolor sed lectus malesuada sagittis. Vivamus lobortis elementum eros, at maximus orci varius et. Donec aliquam venenatis ex sit amet semper. Proin dapibus et sapien in ullamcorper. Suspendisse ornare in dolor a dapibus. Duis suscipit aliquet nulla, commodo lobortis diam pellentesque quis. Duis tempor maximus turpis, non fermentum purus tincidunt elementum. Nulla facilisi. Proin nec quam turpis.

Vestibulum ante ipsum primis in faucibus orci luctus et ultrices posuere cubilia curae; Nulla tincidunt tortor id fringilla dapibus. Cras fringilla accumsan neque ornare imperdiet. Pellentesque fringilla ipsum sapien, sed posuere orci sagittis ac. In sodales, dui vitae placerat interdum, eros nisi ornare sapien, nec consectetur massa nibh eu erat. Nullam ornare quam condimentum mi malesuada, non tincidunt ante euismod. Pellentesque tempor feugiat sapien, eget dapibus quam varius quis. Duis egestas, enim ac ornare tempor, augue velit consectetur odio, vel vestibulum sem turpis non ante. Praesent sem nisi, lobortis vitae feugiat nec, tempus sed arcu. Integer interdum urna ut arcu vestibulum fermentum. Suspendisse ex nulla, dapibus vitae aliquet nec, blandit et tortor.

Fusce elit ante, sodales eu blandit et, gravida nec quam. Nullam eget nisl ut magna mollis pulvinar non ut arcu. Suspendisse elementum efficitur orci, at auctor lectus commodo maximus. Etiam aliquam posuere augue, a varius est viverra non. Quisque dictum, erat in aliquet efficitur, enim ex aliquet tellus, at vulputate augue ipsum ut purus. Fusce imperdiet, lacus a malesuada aliquam, lorem nibh semper nulla, at interdum arcu ante vitae nunc. Proin aliquet eleifend luctus.

Aliquam varius sollicitudin sapien, at egestas odio porta interdum. Morbi fringilla in lacus pharetra maximus. Phasellus tincidunt quam a mauris pharetra, in fermentum neque consequat. Aliquam pretium massa ut lacus egestas, ut interdum nunc semper. Maecenas sed metus bibendum, luctus velit quis, consectetur justo. Nunc feugiat mi et lorem vestibulum aliquam. Phasellus sodales, risus vel sodales tincidunt, felis eros ultrices nulla, ac venenatis elit odio sit amet lorem. Pellentesque ut tellus a purus iaculis aliquam. Aenean nec fermentum libero, eu imperdiet diam. Sed iaculis rutrum risus ac fermentum.

Etiam suscipit nisi eget urna sagittis, non aliquet turpis commodo. Proin sed mollis massa. Nam quis euismod ligula. Quisque a interdum odio. Ut fermentum porta nisl, ut pharetra nunc tincidunt at. Etiam vel purus augue. Praesent quis neque non odio iaculis posuere id in augue. Donec ullamcorper mauris sit amet scelerisque finibus. Nunc molestie dui sit amet metus tempor tempor. Fusce sagittis aliquet pharetra. Proin ultricies lacinia euismod. Mauris sapien erat, porttitor non mattis nec, pharetra at urna. Aliquam luctus ligula sit amet tortor feugiat, et luctus est hendrerit. Mauris eget sagittis lectus.

Cras rutrum fermentum nulla vitae tempus. Aliquam elementum tincidunt tincidunt. Ut sit amet libero ut ex posuere lobortis at quis felis. Vivamus condimentum ante ut ante fringilla, id accumsan ex tempus. Nam eleifend, ligula eu imperdiet ultricies, nunc est convallis ex, non gravida libero justo ac nisi. In dictum non lorem vel vehicula. Donec sapien urna, posuere at arcu at, porta lobortis velit.

Donec at finibus est. Fusce quis diam diam. Suspendisse non pharetra nisi. Nunc porttitor consectetur eros, vel malesuada sapien sagittis at. Ut maximus diam vel erat sodales sodales. Duis maximus nisi non justo suscipit, eget sodales ex mollis. Duis magna augue, suscipit interdum tempor ac, volutpat quis nunc. Donec condimentum convallis quam, eget facilisis nulla mollis ut. Curabitur id risus erat. Pellentesque habitant morbi tristique senectus et netus et malesuada fames ac turpis egestas.

Proin mollis luctus lobortis. Suspendisse suscipit fermentum ultricies. Mauris tempus quam dolor, non facilisis velit luctus ac. Cras lacinia convallis neque elementum consequat. Cras non nisl velit. Cras id odio vitae lorem elementum pretium. Donec congue lobortis consectetur. Vestibulum non scelerisque quam, sit amet commodo tortor. Sed porta ligula nulla, eu volutpat est bibendum ut. Mauris pretium felis lacus, vel aliquet tellus rhoncus eget.

Nunc a felis malesuada, tempus nisi at, semper massa. Maecenas malesuada facilisis ipsum a accumsan. Nam a aliquet orci. Morbi non ultrices nulla. Maecenas maximus ut nisl posuere scelerisque. Curabitur nisl quam, vestibulum at sodales ut, pretium et mi. Praesent eget ipsum nibh. Duis venenatis orci sed efficitur ornare. Mauris at nisi ultrices, iaculis mi sit amet, pellentesque tellus. Etiam vitae leo diam. Donec accumsan dui eu neque cursus luctus. Nunc id laoreet quam, et ullamcorper nisi.

Ut quam arcu, feugiat non condimentum cursus, pretium vel est. Aenean consequat justo sit amet felis faucibus, vel varius felis euismod. Sed in semper lacus. Nunc eu nisi id diam rutrum malesuada. Duis tellus diam, sagittis quis pellentesque ac, congue vitae ipsum. Aenean lobortis convallis erat ut auctor. Donec a quam dictum, pretium risus nec, gravida mauris. Praesent consectetur orci congue, lobortis lacus eu, interdum lorem. Mauris imperdiet fermentum neque tristique laoreet. Etiam porttitor felis sit amet augue accumsan, sit amet auctor lacus auctor. Praesent molestie dolor id massa dapibus, sit amet tempus elit mattis. Suspendisse rhoncus diam felis, vel lacinia ligula venenatis ut. Nam suscipit elementum feugiat.

Duis eu quam vel mi sodales lobortis. Praesent eleifend nunc et ante dapibus dapibus. Praesent tincidunt pretium pretium. Quisque lobortis purus sed fermentum maximus. Vestibulum sagittis blandit sem vel convallis. Maecenas suscipit dolor et elementum vehicula. Proin venenatis sapien at ornare efficitur. Curabitur iaculis nec massa et interdum. Fusce eu nisl a lacus bibendum condimentum ac sed arcu. Nulla id urna in nunc sodales laoreet nec nec tortor. Aenean ac sollicitudin lectus. Sed mollis facilisis metus non consectetur. Phasellus vel lectus at tortor posuere lobortis. Donec nec sagittis sem. Donec non sagittis velit.

Nullam viverra enim tellus, sed bibendum risus feugiat rhoncus. Nulla gravida orci nisi, nec suscipit nisl scelerisque vel. Nulla facilisi. In dictum, enim ac faucibus ultricies, leo metus facilisis elit, nec facilisis velit tortor et lorem. Curabitur congue dui enim, at sodales sem luctus at. Donec viverra faucibus ante, sagittis ultrices risus iaculis sed. Aliquam faucibus quis diam eu scelerisque. Sed imperdiet risus eu finibus vehicula. Pellentesque vulputate purus eu diam volutpat faucibus. Aliquam.

Checklist:

  • Provided PR description.
  • Tested the solution manually.
  • Provided automated test coverage.
  • Checked that target branch is set correctly (master for features, the oldest supported for bugs).
  • Ran PHP CS Fixer for new PHP code (use $ composer fix-cs).
  • Asked for a review (ping @ezsystems/php-dev-team).

@adamwojs adamwojs changed the title Wrapped LSE word indexer optimization Apr 7, 2021
webhdx
webhdx approved these changes Apr 8, 2021
View reviewed changes
Copy link
Contributor

@webhdx webhdx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good but please start creating tickets or at least do one combined for all changes. These optimizations are very important and should be trackable in the future.

@lserwatka
Copy link
Member

Yes for this one and for the one in Workflow we need a story.

@adamwojs
Copy link
Member Author

adamwojs commented Apr 8, 2021

OK, I will create tickets for them.

@alongosz alongosz changed the title LSE word indexer optimization Optimized LSE word indexer Apr 8, 2021
@adamwojs adamwojs changed the title Optimized LSE word indexer IBX-152: Optimized LSE word indexer Apr 8, 2021
@adamwojs
Copy link
Member Author

adamwojs commented Apr 8, 2021

As agreed internally we will postpone minor performance improvements which requires schema changes for 4.x.

@adamwojs adamwojs requested a review from alongosz April 8, 2021 10:18
@adamwojs adamwojs self-assigned this Apr 8, 2021
@adamwojs adamwojs added Improvement Changes not fixing or changing behavior Ready for QA labels Apr 8, 2021
@sonarcloud
Copy link

sonarcloud bot commented Apr 8, 2021

Kudos, SonarCloud Quality Gate passed!

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@alongosz alongosz changed the base branch from 1.2 to 1.3 July 28, 2023 14:53
@sonarcloud
Copy link

sonarcloud bot commented Jul 28, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

tomaszszopinski
tomaszszopinski approved these changes Aug 8, 2023
View reviewed changes
Copy link

@tomaszszopinski tomaszszopinski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

QA approved on IbexaDXP 3.3 experience

@alongosz alongosz merged commit 576d699 into 1.3 Aug 8, 2023
27 checks passed
@alongosz alongosz deleted the lse_ft_opt branch August 8, 2023 12:46
@alongosz alongosz mentioned this pull request Aug 8, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Steveb-p Steveb-p Steveb-p approved these changes

@alongosz alongosz alongosz approved these changes

@ciastektk ciastektk ciastektk approved these changes

@webhdx webhdx webhdx approved these changes

@konradoboza konradoboza konradoboza approved these changes

@tomaszszopinski tomaszszopinski tomaszszopinski approved these changes

Assignees

@adamwojs adamwojs

@kisztof kisztof

@mikadamczyk mikadamczyk

@alongosz alongosz

Labels
Improvement Changes not fixing or changing behavior QA approved
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.