Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Neuralnet providing different results for same predictors of different arrangements #39

Open
javakh opened this issue Jun 7, 2022 · 1 comment

Comments

@javakh
Copy link

javakh commented Jun 7, 2022

I'm using the 'neuralnet' package for creating a neural network for prediction purposes. However, I'm getting different network and output results if I change the order of predictors in the same dataset. For example, say the order of predictors initially x1,x2,x3,x4,x5. But if I make it x1,x5,x3,x4,x2, I'm getting different networks and results (Shown in the attached screenshot).
I'm attaching two sets of data ( same data with different order of the predictor) and the screenshots of the different outputs I'm getting using the 'neuralnet' package. I'm also providing here the code I've used. Please check it and let me know what is wrong with it.

Code:

library(caret)
bob_post=read.csv('Set1.csv')
bob_post_scaled=as.data.frame(scale(bob_post,center=T,scale=T))
set.seed(123)
indexes = createDataPartition(bob_post$CF, p = .7, list = F)
train = bob_post_scaled[indexes, ]
test = bob_post_scaled[-indexes, ]
library(neuralnet)
set.seed(123)
model_ann <- neuralnet( CF~.,data=train , hidden = c(4,1), linear.output = T, algorithm = "rprop+")
plot(model_ann)
ann.pred_train = predict(model_ann, train)
ann.pred_test = predict(model_ann, test)
print(c(cor(ann.pred_train,train$CF),cor(ann.pred_test,test$CF)))

library(caret)
bob_post=read.csv('Set2.csv')
bob_post_scaled=as.data.frame(scale(bob_post,center=T,scale=T))
set.seed(123)
indexes = createDataPartition(bob_post$CF, p = .7, list = F)
train = bob_post_scaled[indexes, ]
test = bob_post_scaled[-indexes, ]
library(neuralnet)
set.seed(123)
model_ann <- neuralnet( CF~.,data=train , hidden = c(4,1), linear.output = T, algorithm = "rprop+")
plot(model_ann)
ann.pred_train = predict(model_ann, train)
ann.pred_test = predict(model_ann, test)
print(c(cor(ann.pred_train,train$CF),cor(ann.pred_test,test$CF)))

Set1.csv
Set2.csv
two_dataset
Output
Neuralnet_set1
Neuralnet_set2

@mnwright
Copy link
Member

Isn't that just random variation? For different column orders, weights get initialized differently which gives you different results. That should be similar to re-running with same column order but different seeds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants