Artificial Neuron

- Overview

An artificial neuron is a connection point in an artificial neural network. Artificial neural networks, like biological neural networks in the human body, have a layered architecture where each network node (connection point) has the ability to process input and forward output to other nodes in the network. In artificial and biological architectures, nodes are called neurons, and connections are characterized by synaptic weights, which represent the importance of the connection. As new data is received and processed, synaptic weights change, and this is how learning occurs. 

Artificial neurons are modeled after the hierarchical arrangement of neurons in a biological sensory system. For example, in the visual system, light input passes through neurons in successive layers of the retina, then to neurons in the thalamus of the brain, and then to neurons in the visual cortex of the brain. 

As neurons pass signals through more and more layers, the brain gradually extracts more information until it is confident that it can recognize what a person is seeing. In artificial intelligence, this fine-tuning process is called deep learning. 

[Neuron and Synapse - GeeksforGeeks]

- Artificial And Biological Neuron

We have heard of the latest advancements in the field of DL due to the usage of different neural networks. At the most basic level, all such neural networks are made up of artificial neurons that try to mimic the working of biological neurons. Understanding how these artificial neurons compare to the structure of biological neurons in our brains and if possibly this could lead to a way to improve neural networks further. 

An Artificial Neural Network (ANN) is an information processing paradigm that is inspired the brain. ANNs, like people, learn by example. An ANN is configured for a specific application, such as pattern recognition or data classification, through a learning process. Learning largely involves adjustments to the synaptic connections that exist between the neurons.  

In artificial and biological networks, as neurons process the input they receive, they decide whether or not the output should be passed as input to the next layer. The decision of whether to send information is called bias, and it is determined by an activation function built into the system. 

For example, the artificial neuron may only pass the output signal to the next layer if the sum of its inputs (actually voltages) exceeds a certain threshold. Because activation functions can be linear or non-linear, neurons will typically have a wide range of convergence and divergence. Divergence is the ability of one neuron to communicate with many other neurons in the network while convergence is the ability of one neuron to receive input from many other neurons in the network.  

- Artificial Neuron (Perception)

A neuron is a mathematical function that is used as a model of a biological neuron in a neural network. The mathematical representation of a neuron is called an artificial neuron, or perceptron, and is made up of three components:

• Inputs: Also represented as x1, x2, ..., xn, these are the input values that the neuron receives. These inputs can represent features, measurements, or outputs from other neurons in the network.
• Weights: Also represented as w1, w2, ..., wn, these represent the strength or importance of each input in influencing the neuron's output. Weights can be positive, negative, or zero.
• Activation function: This is the non-linear function that takes the value from the inputs and becomes the neuron's output.

The artificial neuron receives one or more inputs, applies weights to these inputs, and sums them to produce an output. The value is then passed to the activation function to become the neuron's output. 

A neural network is also a mathematical function. The simplest case is a single input node, a weight, and an output node. When multiple layers are added, the neural network becomes a composition of functions as the signal passes from layer to layer. 

An artificial neuron (also referred to as a perceptron) is a mathematical function. It takes one or more inputs that are multiplied by values called “weights” and added together.

- Neural Network Activation Functions

The activation function of a node in an ANN is a function that calculates the output of the node based on its individual inputs and their weights. Neural networks can represent a wide variety of functions with appropriate weights. 

Activation functions can be linear and non-linear, although the most useful ones are non-linear. Non-linear activation functions play a vital role in neural networks and other deep and algorithmic learning models.

Nontrivial problems can be solved using only a few nodes if the activation function is nonlinear.

Please refer to the following for more details:

• Wikipedia: Activation Function

- Artificial Neuron Example

An artificial neuron is a mathematical function conceived as a model of biological neurons, a neural network. Artificial neurons are elementary units in an artificial neural network. Neural networks are multi-layer networks of neurons (the blue and magenta nodes in the chart above) that we use to classify things, make predictions, etc. 

Above is the diagram of a simple neural network with five inputs, 5 outputs, and two hidden layers of neurons. Starting from the left, we have:

• The input layer of our model in orange.
• Our first hidden layer of neurons in blue.
• Our second hidden layer of neurons in magenta.
• The output layer (a.k.a. the prediction) of our model in green.

The arrows that connect the dots shows how all the neurons are interconnected and how data travels from the input layer all the way through to the output layer. 

Later we will calculate step by step each output value. We will also watch how the neural network learns from its mistake using a process known as backpropagation.