Neural Networks

- Overview

All recent advances in AI in recent years have been attributed to deep learning. Without deep learning, we wouldn't have self-driving cars, chatbots, or personal assistants like Alexa and Siri. The Google Translate app will continue to be as primitive as it was 10 years ago (before Google switched to neural networks for this app), and Netflix or Youtube have no idea which movies or TV shows we like or dislike. Behind all these technologies are neural networks. 

Deep learning is a subset of machine learning, and machine learning is a subset of artificial intelligence. Artificial intelligence is a general term referring to technologies that enable computers to mimic human behavior. Machine learning represents a set of algorithms trained on data that make this possible. 

On the other hand, deep learning is just a type of machine learning, inspired by the structure of the human brain. Deep learning algorithms try to draw conclusions similar to humans by constantly analyzing data with a given logical structure. To achieve this, deep learning uses a multi-layered algorithmic structure called a neural network. 

- Neural Networks and Deep Learning

Neural networks mirror the behavior of the human brain, enabling computer programs to recognize patterns and solve common problems in the fields of artificial intelligence, machine learning, and deep learning.

Deep Learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural networks. Deep Learning is Large Neural Networks.

The core of deep learning is that we now have fast enough computers and enough data to actually train large neural networks. Very large neural networks that we can now have, and huge amounts of data that we have access to, it is the time that deep learning is taking off. 

As we construct larger neural networks and train them with more and more data, their performance continues to increase. This is generally different to other machine learning techniques that reach a plateau in performance. For most flavors of the old generations of learning algorithms, performance will plateau. Deep learning is the first class of algorithms that is scalable. Performance just keeps getting better as you feed them more data.

- Deep Neural Networks and Living Brains

A new computational model predicts how information deep inside the brain could flow from one network to another, and how neural network clusters can self optimize over time. Researchers at the Cyber-Physical Systems Group at the USC Viterbi School of Engineering, in conjunction with the University of Illinois at Urbana-Champaign, have developed a new model of how information deep in the brain could flow from one network to another and how these neuronal network clusters self-optimize over time. 

Their work, chronicled in the paper “Network Science Characteristics of Brain-Derived Neuronal Cultures Deciphered From Quantitative Phase Imaging Data,” is believed to be the first study to observe this self-optimization phenomenon in in vitro neuronal networks, and counters existing models. 

Their findings can open new research directions for biologically inspired artificial intelligence, detection of brain cancer and diagnosis and may contribute to or inspire new Parkinson’s treatment strategies.

Similarly, researchers have demonstrated that the deep neural networks most proficient at classifying speech, music and simulated scents have architectures that seem to parallel the brain’s auditory and olfactory systems. Such parallels also show up in deep nets that can look at a 2D scene and infer the underlying properties of the 3D objects within it, which helps to explain how biological perception can be both fast and incredibly rich. All these results hint that the structures of living neural systems embody certain optimal solutions to the tasks they have taken on.

- Linear and Nonlinear Models

A neural network is a computing system made up of many interconnected units called neurons that can process and learn from data.

Neural network can be regarded as a nonlinear model. As long as there are enough data and neurons, it can approximate any function. 

Neural networks can also be composed of multiple layers, where each layer performs a linear or nonlinear transformation on the input of the previous layer.

The output layer produces the final prediction or classification.