-
Notifications
You must be signed in to change notification settings - Fork 45
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Got error in ZigZagRingFlashAttnVarlenFunc #46
Comments
fixed in #47. the reason of the bug is: #44 (comment) |
Thanks for your reply! I have tried your latest commit and sadly it did not run well in my case. The program will get stuck. I think the reason is the attention mask for _flash_attn_varlen_forward is different across ranks. Do you possibly know how to address this? |
Perhaps we should send cu_seqlens_k and max_seqlen_in_batch_k along with k and v to other ranks. |
hmm... are you using the lastest main branch of the repo? I've just given it another try, it should works with:
And as for attention mask is different across ranks, it is by design. |
The text was updated successfully, but these errors were encountered: