Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do you support the bias of mlp? #156

Open
maobenz opened this issue Oct 9, 2024 · 1 comment
Open

Do you support the bias of mlp? #156

maobenz opened this issue Oct 9, 2024 · 1 comment

Comments

@maobenz
Copy link

maobenz commented Oct 9, 2024

Thanks for your work. i check the code and find the mlp only have w1 and w2. Does the sparse mlp support the bias? thanks a lot! i want to initialize the mlp with our original mlp with bias. Do you have some ideas how to do it?

@mvpatel2000
Copy link
Contributor

Unfortunately, at this time no, but you are welcome to open a PR :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants