Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Maybe warn if dot reduces a dim that isn't in common between args? #20

Open
dlwh opened this issue Aug 24, 2023 · 0 comments
Open

Maybe warn if dot reduces a dim that isn't in common between args? #20

dlwh opened this issue Aug 24, 2023 · 0 comments

Comments

@dlwh
Copy link
Member

dlwh commented Aug 24, 2023

Someone did:

attn_output = hax.dot("key_pos", attn, v)

but forgot that v had a pos dimension not a key_pos dimension (it should have had a key_pos but they forgot to rename. attn had both, so this produced a result with the right shape, but it didn't do what they wanted.

This is a valid operation and it's not inconceivable someone could want it: it's like einsum("ij,je->je, attn, v) or hax.sum(attn, "key_pos") * v. However it seems like an uncommon thing? should we maybe issue a warning? or have a "paranoid debug" mode that logs warnings for things like this?

@dlwh dlwh changed the title Maybe warn if Maybe warn if dot reduces a dim that isn't in common between args? Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant