Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimisation hook for array/bag/dataframe #89

Open
3 tasks
multimeric opened this issue Aug 11, 2022 · 2 comments
Open
3 tasks

Optimisation hook for array/bag/dataframe #89

multimeric opened this issue Aug 11, 2022 · 2 comments

Comments

@multimeric
Copy link
Contributor

I just realised that the current hook that we use to insert graphchain into dask is delayed_optimize, which only actually applies for "raw" Delayed instances, but doesn't actually apply to e.g. the DataFrame API, which is a very popular part of dask. As described here, these types have their own optimisation hooks that we would need to implement. I will look in to writing one for the DataFrame API. I think we should be able to integrate it fairly easily into the current codebase.

  • array_optimize
  • dataframe_optimize
  • bag_optimize
@lsorber
Copy link
Member

lsorber commented Aug 11, 2022

That would be great, thank you @multimeric!

@multimeric
Copy link
Contributor Author

Okay so upon investigation, it seems that all the hooks use the same API, so we're all good in that regard, and it was just an unrelated bug that I was hitting.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants