For all those who’re newly exposed to Machine Learning, it’s quite common to run into the a simple but puzzling question: why is this thing called a Neural Network, and what does it have to do with the brain?
Despite a rather opposite first impression, after building a more composed knowledge set on this matter, I’ve come to believe that the relationship between Neural Network and the prototype of its name — the human brain, is circumscribed down to a figurative and superficial level; in other words, other than a certain level of resemblance ( graphically and cognitively ) between a Neural Network graph and the brain’s neurons, these two entities can’t be compared or related to each other on any deeper level.
*Due to research topic choices, the Neural Network discussed here is limited to Convolutional Neural Networks, and will be referred to as CNN.
Other than the obvious resemblance in their names, confusions on whether Neural Network is related to the human Brain is also fostered by the common graphics used to explain a CNN. As shown above, nodes and arrows composes a graphic that almost immediately spurs images of how neurons are connected to each other through synapses in the brain. (Even a vocabulary with “neuron” and “synapse” is popularly used in many of these graphs and educational blogs. ) However, apart from making explanation to new-bees easier by borrowing concepts from a more well-know field, the nodes and lines in this scheme actually mean very different things. In the brain, “nodes” are neurons, which are mini “data processors” (herewith borrowing concepts the other way around) that makes a meaning out of information fed to them, and the “lines” would be synapses, which acts like information tunnels and transporters. Whereas in CNN, things are quite the opposite: “nodes” are data units (contents to be processed), and the “lines” represent the processing functions used to take actions on the data. In a word, roles of cognitive symbols and their true meanings in these two schemes are quite literally the opposite of each other.
On a higher level, we can also see how the NN and the brain is different and should be understood as such. Brandon Rohrer’s blog on CNN provides a solid and well-written introduction to CNN. Below are some snapshots from his video explaining the mechanism of a CNN. Through convolution layer(s), pooling layer(s) and ReLU layer(s), we reach a fully connected layer, where a 2D array (from any data that can reasonably take a “graph-like” 2D array form) has already been processed into a 1D array, we have feature points that can “vote” for and determine a final outcome of the input data.
TBC.
Reference
A ‘Brief’ History of Neural Nets and Deep Learning
Inputs and Outputs: How the brain allows us tp interact with the world