Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: lazily load plugins [APE-1422] #1681

Merged
merged 13 commits into from
Sep 29, 2023
Merged

Conversation

antazoey
Copy link
Member

@antazoey antazoey commented Sep 27, 2023

What I did

This shaves almost a second off of time ape. None of the plugins are loaded if they are not needed (but they are loaded if they are needed).

How I did it

Use iterators mostly.

How to verify it

Checklist

  • All changes are completed
  • New test cases have been added
  • Documentation has been updated

@vany365 vany365 changed the title perf: lazily load plugins perf: lazily load plugins [APE-1422] Sep 27, 2023
fubuloubu
fubuloubu previously approved these changes Sep 27, 2023
src/ape/plugins/__init__.py Show resolved Hide resolved
@fubuloubu
Copy link
Member

seeing about a 12% performance improvement with this on my machine 🚀

fubuloubu and others added 2 commits September 28, 2023 08:23
* feat(query): add timestamp to ContractLog

fixes: ApeWorX#1612

* docs: add docstring to timestamp property of ContractLog

* test: add test for ContractLog.timestamp
@antazoey
Copy link
Member Author

antazoey commented Sep 28, 2023

seeing about a 12% performance improvement with this on my machine 🚀

My goal right now is to not load plugins for any of the help-commands e.g. ape run --help
I think ape ledger still does because of some of account tools, so I'm going to fix that now

  • fix account tool lazy stuff Edit: turns out it was related to networks and config loading. fixed now! fast as heck

but disclaimer here: if you run any cmd that loads plugins, there shouldnt be any performance gains by this PR alone. in the future, we want to load only plugins needed but thatll likely require a bigger refactor

@antazoey
Copy link
Member Author

seeing about a 12% performance improvement with this on my machine 🚀

You will see a bigger one on commands like ape run --help (sub command helps).

Main: ape run 4.26s user 0.75s system 121% cpu 4.102 total
Branch: ape run 2.45s user 0.54s system 144% cpu 2.071 total

src/ape/cli/choices.py Outdated Show resolved Hide resolved
_T = TypeVar("_T") # _LazySequence generic.


class _LazySequence(Sequence[_T]):
Copy link
Member Author

@antazoey antazoey Sep 29, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at this time, this is only really used in internal stuff in the framework for performance reasons; i tried using collections.deque() but it wasnt lazily enough and so i went with this but if there is way to make deque not iterate at all upon init, that would be nice, or an alternative data structure

@antazoey antazoey merged commit 7a8e273 into ApeWorX:main Sep 29, 2023
15 checks passed
@antazoey antazoey deleted the perf/lazy-plugins branch September 29, 2023 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants