Tag: #aiarts04

From what I’ve read about brain neural network and artificial neural network (ANN), I think that the first ANN was definitely inspired by the how the brain neural network functions, but as ML progresses, the ML implementation begins to diverge significantly from how our brain works, and the link between gradually weakens.

I’ll start with the similarities between our brain’s neural network and ANN. Their structures are similar in some ways. They both have their organizations to receive input, for the brain is the dendrites and the ANN the incoming connections. Then the soma in our brain’s neuron “sum up” inputs from previous steps and make a non-linear decision whether to activate the neuron and fire an output, in ANN the operation is called “the linear calculation and the activation functions”. Finally, the axon in the neuron carries the action signal and delivers it to the next layer, the output connections in ANN perform similar tasks. Also, when discussing CNN (convolutional neural networks), it is inspired by the visual pathway: the representation of the images is similar to that in the visual cortex, it starts from studying simple patterns to more complex shapes. Another thing that relates ANN with the brain is plasticity, the experiences we have strengthen or weaken the existing connection in our brains, and therefore affects our memory, ANN, too, iteratively modifies the weights of the network parameters based on inputs.

However, there are significant differences between them, our brain neural network is way more powerful and efficient than ANN and ML nowadays. Brains neurons are much stronger than artificial neurons in terms of numbers and internal complexity, chemical and electric mechanisms are more nuanced and robust in neurons compared to the artificial ones, a single brain cell can complete complex (multiple) composite tasks whereas an artificial neuron can only perform one task at a time. The components of ANN is parameters, weights, the liner and activation functions are basic and crude compared to the functioning of the brain network. The overall network architecture also varies significantly. For ANN, each layer is only connected to the previous and the next layers while for the network of neurons, there are tens of thousands of dendrites crossing “layers” and regions in numerous directions. Human brains don’t use backpropagation that leverages the chain rule over partial derivatives of an error function.

Nevertheless, the brain neural network continuously inspires the innovation of ML and AI. Power efficiency is what our brain exceeds ANN the most, the effort our brain takes performing the same task with an AI system is far less than what the latter needs. Expecting to train models with a smaller set of training examples is also a goal, currently some “built-in” models that allow “intuitive” understanding which simplifies the training process. When the brain is building a connection, it is always through unsupervised processes, while the AI we have today only perform supervised ones, if we successfully create an AI system that can train itself unsupervised, it will be more intelligent and human-like.

Neuroscience and AI have always been very separate in my mind. The motivation behind calling it a “neural network” is to create the illusion, or make stronger connections to this idea that the computer has “a brain”. The easiest way to explain the complexities of computers and answer the questions of why and how to the general population is to use magic blanket phrases like “oh it works like a brain making connections between things and processing complicated information”. 

The distinction is how the brain receives and processes information. Input comes in from the sensory organs which take a physical stimulus (like light or heat) and create electrochemical signals that are in a “language” the brain understands. (It is essentially like data processing.) Then these signals are put through a filter… because humans are surrounded by constant stimulus. It’s like being able to pick out and follow your mom’s voice in a massive crowd at the grocery store or like seeing individual fruit on a bush as opposed to the image just bluring together. 

Once the information is processed and filtered, that same filter decides how important the information is. Let’s say you are in an emergency situation and you are processing information… the color of the firefighter’s shoes is probably not important enough to be the primary focus in that situation. The information then gets stored based on importance… so think short-term memory, long-term memory, etc. Links are then formed based on relevance to other information already stored. 

Let’s compare a Neural Network to this. Consider this idea that there is a big data/ training set of data… that information to some degree gets inputed in a way the computer understands. Kind of like how the physical stimulus are translated into electrochemical signals. I think the closest comparison would be coding the Network since coding is traditionally done in coding language so the computer understands and this coding language is not necessarily flowing like normal conversation. 

This data then can get filtered. Maybe one way it does that is in the form of arrays. It makes information organized and can potentially have connections hard coded into it. (Think about this in terms of maybe a word being tied to a definition, there is a way to do that within an array for example). 

The clear difference here between this neural network and say the brain is the prioritizing of information. This difference is so massive that it overrides all the similarities previously stated. The data is not put in a hierarchical form. The data is there, categorized… but there is no situational input to show which data is more important. Think back to that emergency example, the detail of the firefighter’s shoes would probably be forgotten, but in the case of a computer, that data would still be stored. The computer doesn’t “forget” like a brain would. 

Citations:

http://www.teach-nology.com/teachers/methods/info_processing/