support amd/rocm

[ehartford]

when I run pip install megablocks I get this:

      clang: error: unsupported option '--ptxas-options=-v'
      clang: error: unsupported option '--generate-code=arch=compute_90,code=sm_90'

[tgale96]

Hi Eric, there are a few places where we still use CUDA kernels. They're not difficult to replace with Torch ops/Triton kernels, but I'm a bit buried in various other tasks currently so I haven't gotten around to it. Contributions are welcome if you have the bandwidth and I am happy to assist however I can!

[ehartford]

Fair; but beyond my skill level. I will ask my network and see if I can find any insight.

[Muennighoff]

Have you tried hipify (https://github.com/ROCm/HIPIFY) to port the cuda kernels?