As the name Neural Network suggests, the idea is somehow borrowed from the human body/brain. To make it clear, the Neural Network is trying to simulate the structure of the brain and how neurons are connected. Previously, the concept of a neural network is also called the multi(layer) perceptron, which suggests that every node in the Neural Network will represent or capture a feature and by combining those features, detailed results will be the outcome. This is basically how the multi-perceptron works. However, the human brain works in a similar but different mechanism,
Neural Network
Essentially, the Neural Network is doing feature extractions and feature combination and use these features to further deliver the output. And the CNN, RNN and more serve as an extraction mechanism for different kind of tasks. Furthermore, in practice, the Neural Network does not have a generic solution to all tasks. That is to say, for specific tasks, customized models are needed.
Human Brain
On the other hand, the brain works similarly in a way that does the feature extractions and feature combination by hands, eyes, and ears. However, the brain can do continuous learning, which keeps yield solutions to different tasks,