Triton Support In Forge Build?

[ArmadstheDoom]

So a few days ago, triton was finally ported to windows and made accessible via this repo: https://github.com/woct0rdho/triton-windows

I don't know too much about this sort of thing, but I know that triton was for a long time not able to be used; if this can be used, will it be added into the main forge build in order to take advantage of its optimizations? Maybe it's impossible, I don't know, but I'd figure that I'd ask.

[evelryu]

Just see this on Reddit. It looks to improve speed a lot.
https://www.reddit.com/r/StableDiffusion/comments/1g45n6n/triton_3_wheels_published_for_windows_and_working/

[ArmadstheDoom]

Yeah that's where I heard about it too. That's why I figured I'd ask if it would be added into forge at some point in the future, since it does still require a python environment (and I don't know enough to know how to install it myself and get forge to use it to test it).

But if it is something good, I'd like to see it get added, if only because right now we get 'triton not found' errors at startup haha.

[evelryu]

Yes, people say it's easy to install, but I don't want to risk breaking my forge.

[ArmadstheDoom]

Same. I don't know enough about python to try it. But if it IS good, I hope they'll see this and decide to add it.

[yamfun]

hoping for this too

[HMRMike]

Not enough time to test properly but this is the first impression.
Already had Forge with torch 2.4.0+cu124
Already installed xformers a while ago (xformers-0.0.28.dev893+cu124-cp310-cp310-win_amd64.whl into the main Forge folder, activate venv, pip install the wheel)- Got a small speed boost with Flux back then.

Installed Triton and dependencies per the instructions at https://github.com/woct0rdho/triton-windows, including cuda 12.5 (careful with the bloatware and old driver nvidia shoves in the installer), and MSVC+Windows 10 SDK through custom Visual Studio Build Tools installer settings. Made sure it's all in PATH and so on.
Installed triton-3.1.0-cp310-cp310-win_amd64.whl (same process as with xformers): missing triton error disappeared, and no errors about missing SDK or CUDA (they will appear if not installed).
All paths were checked, triton test script ran well. Seems everything is in order.

1024x1024 with Flux Dev Q8, t5xxl Q8, 20 steps Euler/Simple, rtx3090.
it/s stayed the same, zero effect. But generating the first image (after empty memory) was sped up significantly- about 1:40-1:35 minutes vs 2 minutes previously, not sure whether Triton had anything to do with it.

[ArmadstheDoom]

This sounds interesting, but is definitely too complicated for someone like me to understand, since I have no idea what to install where or what a wheel is, or what you mean by empty memory.

I'm too afraid to screw up my forge install, that's why I hope they'll just made it a part of forge.

[HMRMike]

Yeah it's quite an annoying install right now with all the capricious dependencies.
I'm nowhere near as well versed in all that stuff as the devs, so maybe I missed some vital step, hopefully the generous giants who lent us their shoulders will assist with this salad of software bits once more.

I assume Triton is supposed to take some time to compile stuff, but it does nothing right now.