Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

请问能解释一下全连接层和lstm这样初始化的原因吗? #5

Open
qq31415926 opened this issue Sep 28, 2022 · 0 comments

Comments

@qq31415926
Copy link

qq31415926 commented Sep 28, 2022

全连接层初始化代码
bias = np.sqrt(6.0 / (input_linear.weight.size(0) + input_linear.weight.size(1))) nn.init.uniform_(input_linear.weight, -bias, bias) if input_linear.bias is not None: input_linear.bias.data.zero_()
lstm层初始化代码
for ind in range(0, input_lstm.num_layers): weight = eval('input_lstm.weight_ih_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias) weight = eval('input_lstm.weight_hh_l' + str(ind)) bias = np.sqrt(6.0 / (weight.size(0) / 4 + weight.size(1))) nn.init.uniform_(weight, -bias, bias)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant