{"id":538,"date":"2018-12-28T20:43:58","date_gmt":"2018-12-29T00:43:58","guid":{"rendered":"http:\/\/dominionsw.com\/wordpress\/?p=538"},"modified":"2019-11-25T07:51:31","modified_gmt":"2019-11-25T11:51:31","slug":"classifying-tree-seedlings-with-machine-learning-part-4","status":"publish","type":"post","link":"https:\/\/www.dominionsw.com\/?p=538","title":{"rendered":"Classifying tree seedlings with Machine Learning, Part 4."},"content":{"rendered":"

\"\"<\/a><\/p>\n

Image Recognition with the Matlab Deep Learning Toolbox<\/h1>\n

After some time in the Artificial Intelligence Doghouse, Neural Networks regained status after Alex Krizhevsky, Ilya Sustkever, and Goeffrey Hinton won the\u00a0Large Scale Visual Recognition Challenge in 2012 using their Neural Network, now known as AlexNet.<\/p>\n

Details can be found in their paper<\/a>.<\/p>\n

The Matlab Deep Learning Toolbox makes it really easy to develop our own Deep Learning network to use for our tree seedling classification task. And, as it turns out, we can take advantage of the work done by the winners above by using their trained network to bootstrap our own network using a technique called Transfer Learning.<\/em><\/p>\n

Overview of Deep Learning Neural Networks<\/h2>\n

A Neural Network is designed to mimic the way a biological neuron “Activates” on a given stimulus. How close it comes to recreating an actual biological neuron I will leave to the Neuroscientists to decide. But that is the concept. These neurons are then connected into “Layers”. The “Deep” part of “Deep Learning” refers to the fact that there are many layers of neurons in a particular network, therefore creating a “Deep” Neural Network.<\/p>\n

Let’s take a look at the typical diagram for a section of a Neural Network:<\/p>\n

\"\"<\/a><\/p>\n

The circles on the left are the activations. The lines connecting to the circle on the right are the weights. The circle on the right is the output which is the weighted sum of the activations plus a bias, b, plugged in as a parameter to\u00a0\u03c3(), which is a function known as a Rectifier which simply sets everything < 0 to 0. We’ll skip over the bias and the sigma function, as they are important but not essential to the basic concept.<\/p>\n

These diagrams certainly didn’t mean all that much to me at first; as a programmer, I immediately thought “what does this look like in code”?<\/p>\n

Well, it turns out that if you know a bit about image processing filters, it is easy to understand the basic concepts of convolutional neural networks.<\/p>\n

It turns out that for the image specific tasks, like classifying photos, the activations are the input pixels. The weights are the values in an image filter. So, for instance, a filter could be a 3 by 3 edge detection filter,<\/p>\n\n\n\n\n\t\n\t\n\t
-1<\/td>0<\/td>1<\/td>\n<\/tr>\n
-1<\/td>0<\/td>1<\/td>\n<\/tr>\n
-1<\/td>0<\/td>1<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n\n

or any other type of image filter. The activations are the pixels of the input image, and so one is left with the “classic” image processing task of sliding a filter across an image, taking the dot product and setting a new pixel – potentially a new activation in the next layer.<\/p>\n

\"\"<\/a><\/p>\n

Now, here’s where the “Learning” part of Deep Learning comes in. Instead of first specifying what filters to use, we tell the network what result we want, and it keeps adjusting the filters until it gets that result, or close to it. This feedback is known as back-propagation. So, let’s say we want to train the network to recognize an image with the label “Cat”. We keep feeding the images we want to be recognized as a Cat, and the algorithm keeps updating the filters until they agree that the images can be classified as a Cat. This is a bit of a simplification, but it is the basic essence of what is happening during the training process.<\/p>\n

Once the network is trained properly, the filters will then be able to process an unseen before image as a “Cat”.\u00a0\"\"<\/a><\/p>\n

With Matlab, we can actually look at the filters in the first layer of AlexNet.<\/p>\n

Load AlexNet<\/p>\n

net = alexnet;<\/pre>\n
Let’s look at the Layers:<\/p>\n
net.Layers<\/p>\n
ans = \r\n  25x1 Layer<\/a> array with layers:\r\n\r\n     1   'data'     Image Input                   227x227x3 images with 'zerocenter' normalization\r\n     2   'conv1'    Convolution                   96 11x11x3 convolutions with stride [4  4] and padding [0  0  0  0]\r\n     3   'relu1'    ReLU                          ReLU\r\n     4   'norm1'    Cross Channel Normalization   cross channel normalization with 5 channels per element\r\n     5   'pool1'    Max Pooling                   3x3 max pooling with stride [2  2] and padding [0  0  0  0]\r\n     6   'conv2'    Convolution                   256 5x5x48 convolutions with stride [1  1] and padding [2  2  2  2]\r\n     7   'relu2'    ReLU                          ReLU\r\n     8   'norm2'    Cross Channel Normalization   cross channel normalization with 5 channels per element\r\n     9   'pool2'    Max Pooling                   3x3 max pooling with stride [2  2] and padding [0  0  0  0]\r\n    10   'conv3'    Convolution                   384 3x3x256 convolutions with stride [1  1] and padding [1  1  1  1]\r\n    11   'relu3'    ReLU                          ReLU\r\n    12   'conv4'    Convolution                   384 3x3x192 convolutions with stride [1  1] and padding [1  1  1  1]\r\n    13   'relu4'    ReLU                          ReLU\r\n    14   'conv5'    Convolution                   256 3x3x192 convolutions with stride [1  1] and padding [1  1  1  1]\r\n    15   'relu5'    ReLU                          ReLU\r\n    16   'pool5'    Max Pooling                   3x3 max pooling with stride [2  2] and padding [0  0  0  0]\r\n    17   'fc6'      Fully Connected               4096 fully connected layer\r\n    18   'relu6'    ReLU                          ReLU\r\n    19   'drop6'    Dropout                       50% dropout\r\n    20   'fc7'      Fully Connected               4096 fully connected layer\r\n    21   'relu7'    ReLU                          ReLU\r\n    22   'drop7'    Dropout                       50% dropout\r\n    23   'fc8'      Fully Connected               1000 fully connected layer\r\n    24   'prob'     Softmax                       softmax\r\n    25   'output'   Classification Output         crossentropyex with 'tench' and 999 other classes<\/pre>\n
We can take a look at the 2nd layer and visualize the image filters. There are 96 filters, 11 pixels by 11 pixels by 3 pixels (one set of pixels each for R, G, B colors).<\/p>\n
We offset the pixels all into the positive range so we can see them more easily, then write them as a tiff file.<\/p>\n
layer = net.Layers(2);\r\nimg = imtile(layer.Weights);\r\nmn = min(min(min(img)));\r\nimg = img - mn;\r\n\r\nimgu = uint8(img.*255);\r\n\r\nimwrite(imgu,'Tiles.tif','tif');<\/pre>\n
\"\"<\/a><\/p>\n
This is pretty interesting. 96 image filters that were “learned” from the 1.2 million images with 1000 different labels. I also notice the cyan and green colors, which remind me of the Lab color space from a previous post.<\/p>\n
There a lot of details that one can go into here, of course. We’ll skip ahead to how we can use this network to recognize our own images.<\/p>\n
Transfer Learning<\/h2>\n
It turns out that we can retrain an existing network to classify our seedlings using what is known as “Transfer Learning”. Using the Matlab Deep Learning Toolbox, we can do this by replacing two of the layers in the original AlexNet with our own.<\/p>\n
First, we load the network, and set up a path to our Labeled image folders.<\/p>\n
net = alexnet;\r\nimagePath = 'C:\\Users\\rickf\\Google Drive\\Greenstand\\TestingAndTraining';<\/pre>\n
 <\/p>\n
We can set up our datastore, and use a Matlab helper function to pick 80 percent of the images for training. In addition, the\u00a0augmentedImageDatastore() function let’s us scale the images to the required 227×227 inputs that AlexNet requires.<\/p>\n
imds = imageDatastore(imagePath,'IncludeSubfolders',true,'LabelSource','foldernames');\r\n[trainIm, testIm] = splitEachLabel(imds, 0.8,'randomized')\r\n\r\ntrainData = augmentedImageDatastore([227 227], trainIm)\r\ntestData = augmentedImageDatastore([227 227], testIm)<\/pre>\n
If we look above we can see that the final layers in the AlexNet network are set up for 1000 different labels.<\/p>\n
23 'fc8' Fully Connected 1000 fully connected layer \r\n24 'prob' Softmax softmax \r\n25 'output' Classification Output crossentropyex with 'tench' and 999 other classes<\/pre>\n
What we need to do is to replace layer 23 with a fully connected 6 input layer (for our 6 labels),
\nand replace 25 with a new classifier that will accept those 6 inputs from the softmax layer.<\/p>\n
layer(23) = fullyConnectedLayer(6,'Name','fcfinal');\r\nlayer(end) = classificationLayer('Name','treeClassification');<\/pre>\n
Set up training options, and train the network.<\/div>\n
<\/div>\n
opts = trainingOptions('adam','InitialLearnRate',0.0001,'Plots','training-progress','MaxEpochs',10);\r\n[net,info] = trainNetwork(trainData,layer,opts);<\/pre>\n
\"\"<\/a><\/p>\n
We specified the option to plot progress, which is shown above.<\/p>\n
The final results can be seen in our confusion matrix.<\/p>\n
\"\"<\/a><\/p>\n
Much better results that the Support Vector Machine, and no color segmentation was needed!<\/p>\n
I\u00a0 would suspect with more training data we can get the Broadleaf and Fernlike classifications to improve. We’ll try that, and then put the network to a test of classifying unseen new images in the next post.<\/p>\n
 <\/p>\n","protected":false},"excerpt":{"rendered":"
Image Recognition with the Matlab Deep Learning Toolbox After some time in the Artificial Intelligence Doghouse, Neural Networks regained status after Alex Krizhevsky, Ilya Sustkever, and Goeffrey Hinton won the\u00a0Large Scale Visual Recognition Challenge in 2012 using their Neural Network, now known as AlexNet. Details can be found in their paper. The Matlab Deep Learning […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[],"class_list":["post-538","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"_links":{"self":[{"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/posts\/538","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=538"}],"version-history":[{"count":18,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/posts\/538\/revisions"}],"predecessor-version":[{"id":609,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=\/wp\/v2\/posts\/538\/revisions\/609"}],"wp:attachment":[{"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=538"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=538"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.dominionsw.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=538"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}