Author: Kathryn Rose Krobock

Introduction

For this week’s assignment, we are training a CIFAR-10 CNN.  Before doing anything involving actual training, I first wanted to understand the CIFAR-10 dataset because I’ve never worked with it before.  I read that the dataset contains 60,000 images in total (the training set is only 50,000), with each image at 32×32. These are broken down into 10 classes: airplanes, cars, birds, cats, deer, dogs, frogs, horses, ships, and trucks. Each class contains 6,000 images. 

Machine Specs

Tests

I ran 3 different tests to test the relationship between epochs and batch size, and its effect on accuracy. Test 1 is high epoch, high batch. Test 2 is low epoch, high batch. Test 3 is low epoch, low batch. 

1: (Epochs: 100, batch size: 2048, 20 hours)

The first test I did (not sensibly) was to just run the program.  It was really painful but I wanted to see it through to see what it could do. I knew this was going to take forever to complete, so I tried running it when I was asleep. I forgot that my computer also goes to sleep very quickly when I’m not using it. I had to come back to this a lot to wake my computer up, so as a result, this took me 20 hours.  This resulted in a 0.6326 accuracy.  I was pretty surprised at this, considering the testing we did in class resulted in much higher accuracy, and much faster. 

2: (epochs: 10, batch size: 2048, 25 minutes)

Thinking that it was the high number of epochs that took so long to process (even if it would have been continuous), I assumed that lowering them to 10 would go very fast. It definitely cut down on the total amount of time, but it seemed to be processing at about the same rate as the first test. The final result is a 0.4093 accuracy.

3: (Epochs: 10, Batch size: 256, 20 minutes)

Finally, I decreased the batch size as well, meaning testing low epoch, low batch. This had surprised me that despite testing a much lower batch size than in test 2, it took about the same amount of time. Even more surprising was the fact that it resulted in a 0.5453 accuracy, meaning that the lower batch size produced a more accurate result.

Conclusion

I’m still a bit confused about the different aspects that go into CNN training, and especially how their relationships with each other affect the outcomes. But at the very least I learned the amount of time it would take to try to train the CIFAR-10 dataset. I’m still not totally sure why it took so much more time for a less accurate result than in class, especially since the Fashion-MNIST dataset has an almost equal number of images as CIFAR-10. On quick evaluation, I see that each image in Fashion-MNIST is only 28×28, whereas CIFAR-10 is 32×32. I wonder if that has something to do with it, but that difference still seems pretty small for such a big gap in the results.

I think that there is no doubt that artificial intelligence neural networks are, at the very least, inspired by the brain/neuron structure. Even without reading any McCulloch and Pitts, I think it makes sense if we assume that 1) we cannot create anything that is completely new, but we can build off of previous discoveries and knowledges, and 2) developers over the years did not have any other established system of learning on which to base their work, aside from biological neural networks (human or otherwise). But, as a lot of classmates have pointed out, the key word is “inspired”; this does not imply that the structures are exactly the same, and it does not imply that the processes of learning are the same either.

The structures of these neural networks are similar, but different. After receiving an input, the processing of biological and artificial structures (in cell bodies and hidden layers, respectively) vary; this is in part due to the fact that most artificial neural networks are built in a way that accomplishes a single task, whereas biological can learn entirely new tasks. In that same vein, the ways AI learn and the ways humans learn are also different. Actually, while we can categorize different ways of AI learning, is it possible to do this for human learning? Is it supervised or unsupervised? While I was researching I also came across the terms inductive, deductive, transductive learning, but I still don’t know nearly enough about any of these or even enough about neuroscience to try to speculate on how they tie into human learning. One thing that’s clear, though, is that humans learn constantly and continuously; even if AI learned through a similar method, I don’t know if it would be possible for it to function with the same degree of complexity as a human, and I don’t think that outcome would be very predictable or controllable.

Ultimately, I think that artificial neural networks are definitely inspired by biological neural networks. However, the networks themselves, as well as the way AI and humans learn, can be vastly different. I think this conversation is interesting since other questions inevitably come out of it—will we ever be able to construct an ANN so that AI can learn like a human? Is this even the goal?

This week’s assignment turned out to be bit less successful than I hoped. I wanted to use the assignment as an opportunity to build off of an earlier project I did using image classification. For that project, I loosely trained the model with images of the alphabet in American Sign Language (ASL). *note that I’m not very knowledgable about ASL and even worse at signing (I had to reference an alphabet guide) but here’s a video for reference of the last project:

Despite the fact that I only trained it with 24 of the 26 letters (J and Z require motion while image classification training requires still images), it was incredibly time consuming to have to retrain it every time I opened the project. Previously when I was doing this project, I didn’t know that a save/load model had recently been developed, so I figured I should try to implement that in this project to make it more efficient.

I referenced Daniel Shiffman’s video on the Save/Load model, and for the first part it does seem to be working; after training, the model.json and model.weights.bin files download and open to look like the demonstration in the video. It’s only when I try to load them back into the program that it stops working. In localhost, it stops running at “loading model.” My terminal shows this:

I think there’s probably a relatively simple explanation to this or something that I’m overlooking, and I think I should continue to work on it and make sure I can make this work. If I could get it to work, I think the save/load model would be extremely helpful in developing a larger project, and especially if I wanted to move forward with something similar to the image classification model.

Code:

Training & saving model code: https://github.com/katkrobock/aiarts/tree/master/train_saveModel

Loading model code: https://github.com/katkrobock/aiarts/tree/master/loadModel

Reference:

Daniel Shiffman / The Coding Train video: https://www.youtube.com/watch?v=eU7gIy3xV30