Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Epic][CPU] Enable predictable performance for matrix multiplication using data-tiling and micro-kernels #13216

Open
dcaballe opened this issue Apr 21, 2023 · 0 comments
Assignees
Labels
codegen/llvm LLVM code generation compiler backend codegen Shared code generation infrastructure and dialects epic Tracking issues for related deliverables (as in Agile dev) p1

Comments

@dcaballe
Copy link
Contributor

dcaballe commented Apr 21, 2023

This EPIC tracking all the micro-kernel related work.

Tasks related to core functionality.

- [x] Enable use of microkernel through plugins
- [x] Enable use of microkernels through bitcode linking (#12112 ) : WIP PR : #13460
- [ ] Benchmark performance on models of data tiling approach using microkernels as plugins (until #12112 is addressed)
- [ ] #11360
- [ ] Avoid needing to unpack the results of the GEMM, by writing results of GEMM in unpacked form
- [ ] Improve distribution of CPU code-generation to threads.
- [ ] https://github.com/openxla/iree/issues/14431

Blocking issues with data-tiling on e2e workloads

- [ ] https://github.com/openxla/iree/issues/14414
- [ ] https://github.com/openxla/iree/issues/14398
- [ ] https://github.com/openxla/iree/issues/14252

Amortizing costs related to packing/unpacking by using propagation

### Tasks
- [ ] Add a early matmul -> data tiling pass to preprocessing stage
- [ ] Add DataLayoutPropagation pass to preprocessing or flow stage

Enable propagation in models where packing can be done without padding, starting with MobileBertfp32.

Some supports are already developed in -upstream. To enable it for MobileBert, we need

### Tasks
- [ ] #13530 
- [ ] #13531 
- [ ] #13532 

Evaluating propagation in models where packing needs padding as well.

This needs more investigation. Propagation of tensor.pack ops with padding values is very challenging. We are able to propagate it through some limited cases. Need more investigation for this part.

@dcaballe dcaballe added the codegen Shared code generation infrastructure and dialects label Apr 21, 2023
@aaron-schneider aaron-schneider added the epic Tracking issues for related deliverables (as in Agile dev) label Apr 26, 2023
@MaheshRavishankar MaheshRavishankar added codegen/llvm LLVM code generation compiler backend codegen Shared code generation infrastructure and dialects and removed codegen Shared code generation infrastructure and dialects labels May 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
codegen/llvm LLVM code generation compiler backend codegen Shared code generation infrastructure and dialects epic Tracking issues for related deliverables (as in Agile dev) p1
Projects
None yet
Development

No branches or pull requests

4 participants