Tutorial: neural network in R by examples (using the package neuralnet)

Tutorial: Create a neural network in R

Let's start with the basics. 

If you want to start learning how does it work a NN in R, start searching for some packages supporting you. 

Googling a bit you will wee that there are two main packages: 

• Neuralnet: specific for neural networks
• Caret: generic machine learning package, containing a lot of machine learning algorithm, supporting very well the neural networks.

In this post I will show some investigation I made on the package "Neuralnet".
In this next post I show the investigation I made on the package "Caret".

Basic AND and OR functions

If you go on the help of R, typing: 

 ?neuralnet  

You can get the complete documentation.
Going quickly to the example, you find the following code:

 AND <- c(rep(0,7),1)  
 OR <- c(0,rep(1,7))  
 binary.data <- data.frame(expand.grid(c(0,1), c(0,1), c(0,1)), AND, OR)  
 net <- neuralnet(AND+OR~Var1+Var2+Var3, binary.data, hidden=0,   
             rep=10, err.fct="ce", linear.output=FALSE)  

Let's have a look inside to the returned object "net":

 > typeof(net)  
 [1] "list"  

 > summary(net)  
                   Length   Class      Mode    
 call                  7    -none-     call    
 response             16    -none-     numeric   
 covariate            24    -none-     numeric   
 model.list            2    -none-     list    
 err.fct               1    -none-     function  
 act.fct               1    -none-     function  
 linear.output         1    -none-     logical   
 data                  5    data.frame list    
 net.result           10    -none-     list    
 weights              10    -none-     list    
 startweights         10    -none-     list    
 generalized.weights  10    -none-     list    
 result.matrix       110    -none-     numeric   
 

> head(net)  
 $call  
 neuralnet(formula = AND + OR ~ Var1 + Var2 + Var3, data = binary.data,   
   hidden = 0, rep = 10, err.fct = "ce", linear.output = FALSE)  
 $response  
  AND OR  
 1  0 0  
 2  0 1  
 3  0 1  
 4  0 1  
 5  0 1  
 6  0 1  
 7  0 1  
 8  1 1  
 $covariate  
    [,1] [,2] [,3]  
 [1,]  0  0  0  
 [2,]  1  0  0  
 [3,]  0  1  0  
 [4,]  1  1  0  
 [5,]  0  0  1  
 [6,]  1  0  1  
 [7,]  0  1  1  
 [8,]  1  1  1  
 $model.list  
 $model.list$response  
 [1] "AND" "OR"   
 $model.list$variables  
 [1] "Var1" "Var2" "Var3"  
 $err.fct  
 function (x, y)   
 {  
   -(y * log(x) + (1 - y) * log(1 - x))  
 }  
 <environment: 0x00000000174034c8>  
 attr(,"type")  
 [1] "ce"  
 $act.fct  
 function (x)   
 {  
   1/(1 + exp(-x))  
 }  
 <environment: 0x00000000174034c8>  
 attr(,"type")  
 [1] "logistic"  

> print(net)
10 repetitions were calculated.

           Error Reached Threshold Steps
7  0.07270325501    0.007209205002   237
2  0.08361412743    0.008216337601   210
1  0.08425544114    0.008840366736   228
4  0.08667354432    0.009692218237   226
6  0.08736308608    0.009046804494   236
10 0.08767067498    0.009009176337   240
3  0.08817255652    0.009546552076   214
8  0.09165608530    0.009676470949   221
5  0.09170859687    0.008872336865   213
9  0.09517374836    0.009993935991   223

How can I use the NN once I have trained it?

 > test <- data.frame(expand.grid(c(0,1), c(0,1), c(0,1)))  
 > predicted <- compute(net, test) #Run them through the neural network  
 > print(predicted$net.result)  
            [,1]       [,2]  
 [1,] 0.000000005510800906 0.00003437586348  
 [2,] 0.000008322603907184 0.99999887725263  
 [3,] 0.000010232856460365 0.99999418995760  
 [4,] 0.015219103082846375 1.00000000000000  
 [5,] 0.000009115152595497 0.99998044588564  
 [6,] 0.013579325087889940 1.00000000000000  
 [7,] 0.016644285280624567 1.00000000000000  
 [8,] 0.962352882489281747 1.00000000000000  
 >   

What is very cool about this package is the graphical representation:

 > plot(net)

generate this plot:

Square root function

There is a great tutorial in this page: 

http://gekkoquant.com/2012/05/26/neural-networks-with-r-simple-example/

Let's have a look at the code: 

 install.packages('neuralnet')  
 library("neuralnet")  

 #Going to create a neural network to perform sqare rooting  
 #Type ?neuralnet for more information on the neuralnet library  
 #Generate 50 random numbers uniformly distributed between 0 and 100  
 #And store them as a dataframe  
 traininginput <- as.data.frame(runif(50, min=0, max=100))  
 trainingoutput <- sqrt(traininginput)  

 #Column bind the data into one variable  
 trainingdata <- cbind(traininginput,trainingoutput)  
 colnames(trainingdata) <- c("Input","Output")  

 #Train the neural network  
 #Going to have 10 hidden layers  
 #Threshold is a numeric value specifying the threshold for the partial  
 #derivatives of the error function as stopping criteria.  
 net.sqrt <- neuralnet(Output~Input,trainingdata, hidden=10, threshold=0.01)  

 #Test the neural network on some training data  
 testdata <- as.data.frame((1:10)^2) #Generate some squared numbers  
 net.results <- compute(net.sqrt, testdata) #Run them through the neural network  

Let's have a look at the reuslt:

 > cleanoutput <- cbind(testdata,sqrt(testdata),  
 +           as.data.frame(net.results$net.result))  
 > colnames(cleanoutput) <- c("Input","Expected Output","Neural Net Output")  
 > print(cleanoutput)  
   Input Expected Output Neural Net Output  
 1   1                 1       1.292537723  
 2   4                 2       2.004110690  
 3   9                 3       3.004860581  
 4   16                4       4.001313250  
 5   25                5       4.996608428  
 6   36                6       6.003918278  
 7   49                7       6.997395208  
 8   64                8       7.995504751  
 9   81                9       9.009677415  
 10  100              10       9.959406366  
 >   

Having a look at the graphical representation:

Square root function with more nodes in the hidden layer

And what if I wanted to increase the number of nodes in the hidden layer?
Please consider that this is only to see how to tune the network. Use an hidden layer for the Sqrt function is totally useless, and I will show it later with the results.

Anyway, to add more layers, it is simply needed to use the value:    "hidden = c(x, y, z)"
where:
- x: number of perceptrons in the first hidden layer
- y: number of perceptrons in the second hidden layer

- z: number of perceptrons in the third hidden layer

 traininginput <- as.data.frame(runif(50, min=0, max=100))  
 trainingoutput <- sqrt(traininginput)  
 trainingdata <- cbind(traininginput,trainingoutput)  
 colnames(trainingdata) <- c("Input","Output")  
 net.sqrt <- neuralnet(Output~Input,trainingdata, hidden=c(10, 10, 10), threshold=0.01)  

Let's plot now the network:

Let's check now the results (you will see, adding these hidden layer did not improve the results at all):

 > cleanoutput <- cbind(testdata,sqrt(testdata),  
 +           as.data.frame(net.results$net.result))  
 > colnames(cleanoutput) <- c("Input","Expected Output","Neural Net Output")  
 > print(cleanoutput)  
   Input Expected Output Neural Net Output  
 1   1                1    0.591036793  
 2   4                2    2.070495859  
 3   9                3    3.000115410  
 4   16               4    3.999594071  
 5   25               5    4.997643979  
 6   36               6    6.034835331  
 7   49               7    6.997600383  
 8   64               8    7.997306744  
 9   81               9    8.999801209  
 10  100             10    9.982023499  
 >