Tutorial: neural network in R by examples (using the package neuralnet)

Tutorial: Create a neural network in R

Let's start with the basics. 

If you want to start learning how does it work a NN in R, start searching for some packages supporting you. 

Googling a bit you will wee that there are two main packages: 

• Neuralnet: specific for neural networks
• Caret: generic machine learning package, containing a lot of machine learning algorithm, supporting very well the neural networks.

In this post I will show some investigation I made on the package "Neuralnet".
In this next post I show the investigation I made on the package "Caret".

Basic AND and OR functions

If you go on the help of R, typing: 

 ?neuralnet  

You can get the complete documentation.
Going quickly to the example, you find the following code:

 AND <- c(rep(0,7),1)  
 OR <- c(0,rep(1,7))  
 binary.data <- data.frame(expand.grid(c(0,1), c(0,1), c(0,1)), AND, OR)  
 net <- neuralnet(AND+OR~Var1+Var2+Var3, binary.data, hidden=0,   
             rep=10, err.fct="ce", linear.output=FALSE)  

Let's have a look inside to the returned object "net":

 > typeof(net)  
 [1] "list"  

 > summary(net)  
                   Length   Class      Mode    
 call                  7    -none-     call    
 response             16    -none-     numeric   
 covariate            24    -none-     numeric   
 model.list            2    -none-     list    
 err.fct               1    -none-     function  
 act.fct               1    -none-     function  
 linear.output         1    -none-     logical   
 data                  5    data.frame list    
 net.result           10    -none-     list    
 weights              10    -none-     list    
 startweights         10    -none-     list    
 generalized.weights  10    -none-     list    
 result.matrix       110    -none-     numeric   
 

> head(net)  
 $call  
 neuralnet(formula = AND + OR ~ Var1 + Var2 + Var3, data = binary.data,   
   hidden = 0, rep = 10, err.fct = "ce", linear.output = FALSE)  
 $response  
  AND OR  
 1  0 0  
 2  0 1  
 3  0 1  
 4  0 1  
 5  0 1  
 6  0 1  
 7  0 1  
 8  1 1  
 $covariate  
    [,1] [,2] [,3]  
 [1,]  0  0  0  
 [2,]  1  0  0  
 [3,]  0  1  0  
 [4,]  1  1  0  
 [5,]  0  0  1  
 [6,]  1  0  1  
 [7,]  0  1  1  
 [8,]  1  1  1  
 $model.list  
 $model.list$response  
 [1] "AND" "OR"   
 $model.list$variables  
 [1] "Var1" "Var2" "Var3"  
 $err.fct  
 function (x, y)   
 {  
   -(y * log(x) + (1 - y) * log(1 - x))  
 }  
 <environment: 0x00000000174034c8>  
 attr(,"type")  
 [1] "ce"  
 $act.fct  
 function (x)   
 {  
   1/(1 + exp(-x))  
 }  
 <environment: 0x00000000174034c8>  
 attr(,"type")  
 [1] "logistic"  

> print(net)
10 repetitions were calculated.

           Error Reached Threshold Steps
7  0.07270325501    0.007209205002   237
2  0.08361412743    0.008216337601   210
1  0.08425544114    0.008840366736   228
4  0.08667354432    0.009692218237   226
6  0.08736308608    0.009046804494   236
10 0.08767067498    0.009009176337   240
3  0.08817255652    0.009546552076   214
8  0.09165608530    0.009676470949   221
5  0.09170859687    0.008872336865   213
9  0.09517374836    0.009993935991   223

How can I use the NN once I have trained it?

 > test <- data.frame(expand.grid(c(0,1), c(0,1), c(0,1)))  
 > predicted <- compute(net, test) #Run them through the neural network  
 > print(predicted$net.result)  
            [,1]       [,2]  
 [1,] 0.000000005510800906 0.00003437586348  
 [2,] 0.000008322603907184 0.99999887725263  
 [3,] 0.000010232856460365 0.99999418995760  
 [4,] 0.015219103082846375 1.00000000000000  
 [5,] 0.000009115152595497 0.99998044588564  
 [6,] 0.013579325087889940 1.00000000000000  
 [7,] 0.016644285280624567 1.00000000000000  
 [8,] 0.962352882489281747 1.00000000000000  
 >   

What is very cool about this package is the graphical representation:

 > plot(net)

generate this plot:

Square root function

There is a great tutorial in this page: 

http://gekkoquant.com/2012/05/26/neural-networks-with-r-simple-example/

Let's have a look at the code: 

 install.packages('neuralnet')  
 library("neuralnet")  

 #Going to create a neural network to perform sqare rooting  
 #Type ?neuralnet for more information on the neuralnet library  
 #Generate 50 random numbers uniformly distributed between 0 and 100  
 #And store them as a dataframe  
 traininginput <- as.data.frame(runif(50, min=0, max=100))  
 trainingoutput <- sqrt(traininginput)  

 #Column bind the data into one variable  
 trainingdata <- cbind(traininginput,trainingoutput)  
 colnames(trainingdata) <- c("Input","Output")  

 #Train the neural network  
 #Going to have 10 hidden layers  
 #Threshold is a numeric value specifying the threshold for the partial  
 #derivatives of the error function as stopping criteria.  
 net.sqrt <- neuralnet(Output~Input,trainingdata, hidden=10, threshold=0.01)  

 #Test the neural network on some training data  
 testdata <- as.data.frame((1:10)^2) #Generate some squared numbers  
 net.results <- compute(net.sqrt, testdata) #Run them through the neural network  

Let's have a look at the reuslt:

 > cleanoutput <- cbind(testdata,sqrt(testdata),  
 +           as.data.frame(net.results$net.result))  
 > colnames(cleanoutput) <- c("Input","Expected Output","Neural Net Output")  
 > print(cleanoutput)  
   Input Expected Output Neural Net Output  
 1   1                 1       1.292537723  
 2   4                 2       2.004110690  
 3   9                 3       3.004860581  
 4   16                4       4.001313250  
 5   25                5       4.996608428  
 6   36                6       6.003918278  
 7   49                7       6.997395208  
 8   64                8       7.995504751  
 9   81                9       9.009677415  
 10  100              10       9.959406366  
 >   

Having a look at the graphical representation:

Square root function with more nodes in the hidden layer

And what if I wanted to increase the number of nodes in the hidden layer?
Please consider that this is only to see how to tune the network. Use an hidden layer for the Sqrt function is totally useless, and I will show it later with the results.

Anyway, to add more layers, it is simply needed to use the value:    "hidden = c(x, y, z)"
where:
- x: number of perceptrons in the first hidden layer
- y: number of perceptrons in the second hidden layer

- z: number of perceptrons in the third hidden layer

 traininginput <- as.data.frame(runif(50, min=0, max=100))  
 trainingoutput <- sqrt(traininginput)  
 trainingdata <- cbind(traininginput,trainingoutput)  
 colnames(trainingdata) <- c("Input","Output")  
 net.sqrt <- neuralnet(Output~Input,trainingdata, hidden=c(10, 10, 10), threshold=0.01)  

Let's plot now the network:

Let's check now the results (you will see, adding these hidden layer did not improve the results at all):

 > cleanoutput <- cbind(testdata,sqrt(testdata),  
 +           as.data.frame(net.results$net.result))  
 > colnames(cleanoutput) <- c("Input","Expected Output","Neural Net Output")  
 > print(cleanoutput)  
   Input Expected Output Neural Net Output  
 1   1                1    0.591036793  
 2   4                2    2.070495859  
 3   9                3    3.000115410  
 4   16               4    3.999594071  
 5   25               5    4.997643979  
 6   36               6    6.034835331  
 7   49               7    6.997600383  
 8   64               8    7.997306744  
 9   81               9    8.999801209  
 10  100             10    9.982023499  
 >   

Basic SQL Operation in R

I want to have in R the equivalent of most of the basic operations normally performed in SQL.
In this post it will follow a sniplet in SQL and immediately after the correspondent in R.

Topics Covered:
- Distinct
- Where
- Inner / outer joins
- Group by


Before starting with the Pure R syntax, just keep in mind that R is providing a very useful package called SQLDF. Through this package it is possible to perform a simple SQL query over tables / data frames.

 # installs everything you need to use sqldf with SQLite  
 # including SQLite itself  
 install.packages("sqldf")  
 # shows built in data frames  
 data()   
 # load sqldf into workspace  
 library(sqldf)  
 sqldf("select * from iris limit 5")  
 sqldf("select count(*) from iris")  
 sqldf("select Species, count(*) from iris group by Species")  
 # create a data frame  
 DF <- data.frame(a = 1:5, b = letters[1:5])  
 sqldf("select * from DF")  
 sqldf("select avg(a) mean, variance(a) var from DF") # see example 15  

Source: http://code.google.com/p/sqldf/

WHERE

 SELECT *   
 FROM df1   
 WHERE product = "Toaster"  

In R:

 df1 = data.frame(CustomerId=c(1:6),Product=c(rep("Toaster",3),rep("Radio",3))) ;  
 df <- df1[df1$Product=="Toaster",];  

DISTINCT

the select distinct in SQL:

 select distinct x  
 from my_table;  

The equivalent in R is:

 > x <- list(a=c(1,2,3), b = c(2,3,4), c=c(4,5,6))  
 > xx <- unlist(x)  
 > xx  
 a1 a2 a3 b1 b2 b3 c1 c2 c3   
  1 2 3 2 3 4 4 5 6   
 > unique(xx)  
 [1] 1 2 3 4 5 6  

INNER / OUTER JOINS

Having in SQL the following query:

 select *   
 from product [left] [right] [outer] join countries  
     on (product.customer_id = countries.customer_id)  

In R:

 df1 = data.frame(CustomerId=c(1:6),Product=c(rep("Toaster",3),rep("Radio",3)))  
 df2 = data.frame(CustomerId=c(2,4,6),State=c(rep("Alabama",2),rep("Ohio",1)))  
 > df1  
  CustomerId Product  
       1 Toaster  
       2 Toaster  
       3 Toaster  
       4  Radio  
       5  Radio  
       6  Radio  
 > df2  
  CustomerId  State  
       2 Alabama  
       4 Alabama  
       6  Ohio  
 #Outer join:   
 merge(x = df1, y = df2, by = "CustomerId", all = TRUE)  
 #Left outer:   
 merge(x = df1, y = df2, by = "CustomerId", all.x=TRUE)  
 #Right outer:   
 merge(x = df1, y = df2, by = "CustomerId", all.y=TRUE)  
 #Cross join:   
 merge(x = df1, y = df2, by = NULL)  

Source:
http://stackoverflow.com/questions/1299871/how-to-join-data-frames-in-r-inner-outer-left-right

GROUP BY

For the Group By function there are many options.
Let's start with the most basic one:

Having in SQL the following snipplet:

 CREATE TABLE my_table (  
  a varchar2(10 char),   
  b varchar2(10 char),   
  c number  
 );  
 SELECT a, b, mean(c)  
 FROM my_table  
 GROUP BY a, b  

In R:

 grouped_data <- aggregate(my_table, by=list(my_table$a, my_table$b, FUN=mean);  

Alternatively:

 > mydf  
  A B  
 1 1 2  
 2 1 3  
 3 2 3  
 4 3 5  
 5 3 6  
 > aggregate(B ~ A, mydf, sum)  
  A B  
 1 1 5  
 2 2 3  
 3 3 11  

If your data are large, I would also recommend looking into the "data.table" package.

  
 > library(data.table)  
 > DT <- data.table(mydf)  
 > DT[, sum(B), by = A]  
   A V1  
 1: 1 5  
 2: 2 3  
 3: 3 11  

And finally the most recommended ddply function:

 > DF <- data.frame(A = c("1", "1", "2", "3", "3"), B = c(2, 3, 3, 5, 6))  
 > library(plyr)  
 > DF.sum <- ddply(DF, c("A"), summarize, B = sum(B))  
 > DF.sum  
  A B  
 1 1 5  
 2 2 3  
 3 3 11  

Source:
http://stackoverflow.com/questions/18799901/data-frame-group-by-column