Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LevenbergMarquardt error for LSTM #266

Open
ghost opened this issue Feb 15, 2020 · 3 comments
Open

LevenbergMarquardt error for LSTM #266

ghost opened this issue Feb 15, 2020 · 3 comments
Assignees
Labels

Comments

@ghost
Copy link

ghost commented Feb 15, 2020

I'm trying to implement a simple lstm network with LevenbergMarquardt algorithm but I get an error as follow:

Cannot use 'training-updates_4/compute_jacobian/while/gradients/f_count_1' as input to 'training-updates_4/compute_jacobian/while/gradients/f_count' because they are in different while loops. See info log for more details.

here is my code:

 def create_dataset(X, y, time_steps=1):
     Xs, ys = [], []
     for i in range(len(X) - time_steps):
         v = X[i:(i + time_steps)]
         Xs.append(v)
         ys.append(y[i + time_steps])
     return np.array(Xs), np.array(ys)

 time_steps = 3
 train_x,train_y = create_dataset(inp_arr,out_arr,time_steps)

The shapes of train_x and train_y will be (87596, 3, 6) and (87596, 1) respectively. My network is as follow:

 n_time_steps = train_x[0].shape
 network = algorithms.LevenbergMarquardt(
        [ 
             layers.Input((3,6)),
             layers.LSTM(100,only_return_final=False),
             layers.LSTM(100),
             layers.Linear(1),
         ],
         verbose=True
     )
 network.train(train_x,train_y,epochs = 10)

am I doing something wrong?

@itdxer
Copy link
Owner

itdxer commented Feb 17, 2020

Hi, Can you also tell me python's version and versions of the following libraries: NeuPy, tensorflow?

@itdxer itdxer self-assigned this Feb 17, 2020
@itdxer itdxer added the bug label Feb 17, 2020
@ghost
Copy link
Author

ghost commented Feb 17, 2020

Hi, Can you also tell me python's version and versions of the following libraries: NeuPy, tensorflow?

Hi, thanks for replying to me. well, when I install the NeuPy, during the installation, it deletes the tensorflow and installs the "1.13.2" version of tensorflow and I have "0.8.2" version of the Neupy. I'm using python version 3 on google Colab.

@itdxer
Copy link
Owner

itdxer commented Feb 23, 2020

Sorry for the delay, I just tried to investigate and fix this issue, but it looks like it's not that easy (at least with tensorflow v1). Is it possible for you to use different optimizer?

Levenberg-Marquardt optimizer won't work for your dataset and architecture. It will require building N*M matrix where N is number of samples and M is number of parameters. Probably you won't be able to fit such a matrix in your computer anyway

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant