Basic of Neural Network

Introduction to Artificial neural network (ANN)

We will come to know about the details of ANN later but first of all we should know why we should go for ANN. How will it be beneficial to our engineering works?

Prompt answer of the question is there are some real life problems that does not have any kind of direct linear relationship between the inputs and outputs. But we some time need to solve that kind of problems of estimation or classification like sales forecasting, industrial process control, customer research, data validation, risk management, target marketing etc.

ANN is mostly used for fuzzy, difficult problems that don't yield to traditional algorithmic approaches. IOWs, there are more "suitable" solutions for computers, but sometimes those solutions don't work, and in those cases one approach is a neural network

Basic Analogy 

What is the Human brain made of?

The bulk of the brain is made up of structural cells termed glial cells and astrocytes. Lying in amongst these cells are neurons, specialized cells that conduct electrical impulses along their processes. It has been estimated that the average human brain contains about 100 billion neurons and, on average, each neuron is connected to 1000 other neurons. This results in the generation of vast and complex neural networks that are the mainstay of the brain's processing capabilities.

What is a neuron?

Neurons are the basic data processing units, the 'chips', of the brain. Each neuron receives electrical inputs from about 1000 other neurons. Impulses arriving simultaneously are added together and, if sufficiently strong, lead to the generation of an electrical discharge, known as an action potential (a 'nerve impulse'). The action potential then forms the input to the next neuron in the network.

Recognition & Memory

Our brain is capable of recognizing different objects and keeps the information in memory. For everything, neurons are responsible.

Example of how neuron works

If someone ask you what comes after 5 what will be your answer. It must be 6. How do you come to know that? That is where the neuron / brain work. You are trained about the number systems that’s why you are able to think that after 5 it will be 6 not 7. But if you were being trained in a way that after 5 it will be 9 then your answer must be 9. 

Similarly if we can build a model that can be trained as our brain then it can be used in many aspect of computational machine learning.

Artificial Neural Network

In machine learning and computational neuroscience, an artificial neural network, often just named a Neural Network, is a mathematical model inspired by biological neural networks. A neural network consists of an interconnected group of artificial neurons, and it processes information using a connectionist approach to computation. In most cases a neural network is an adaptive system changing its structure during a learning phase. Neural networks are used for modelling complex relationships between inputs and outputs or to find patterns in data. 

Neural networks do not perform miracles. But if used sensibly they can produce some amazing results.

A Simple Neuron

An artificial neuron is a device with many inputs and one output. The neuron has two modes of operation; the training mode and the using mode. In the training mode, the neuron can be trained to fire (or not), for particular input patterns. In the using mode, when a taught input pattern is detected at the input, its associated output becomes the current output. If the input pattern does not belong in the taught list of input patterns, the firing rule is used to determine whether to fire or not.

Network Architecture

The commonest type of artificial neural network consists of three groups, or layers, of units: a layer of "input" units is connected to a layer of "hidden" units, which is connected to a layer of "output" units. 

1. The activity of the input units represents the raw information that is fed into the network. 

2. The activity of each hidden unit is determined by the activities of the input units and the weights on the connections between the input and the hidden units. 

3. The behavior of the output units depends on the activity of the hidden units and the weights between the hidden and output units. 

This simple type of network is interesting because the hidden units are free to construct their own representations of the input. The weights between the input and hidden units determine when each hidden unit is active, and so by modifying these weights, a hidden unit can choose what it represents.

Network Training

After construction of network, network have to be trained accordingly using training data. So that any test data can produce expected output result. Training can be of two types,

1. Supervised Training

In supervised training, both the inputs and the outputs are provided. The network then processes the inputs and compares its resulting outputs against the desired outputs. Errors are then propagated back through the system, causing the system to adjust the weights which control the network. This process occurs over and over as the weights are continually tweaked. The set of data which enables the training is called the "training set." During the training of a network the same set of data is processed many times as the connection weights are ever refined.

2. Unsupervised or Adaptive Training 

The other type of training is called unsupervised training. In unsupervised training, the network is provided with inputs but not with desired outputs. The system itself must then decide what features it will use to group the input data. This is often referred to as self-organization or adaption.

Basically there are two types of problem to solved:

1. Classification Problems : Here netwrok is used to classify a set of data into several groups or classes. 

• Example: PNN ( Probabilistic Neural Network) ,SOM (Self organizing map), Perceptron, MLP ( Multilayer Perceptron) etc. 

2.  Approximation function or Regression Problems : Here network predicts/estimates the output of the input data.

• Example: : GRNN (General regression neural network), RBF (Radial basis function) etc.

Note: Some of the neural networks can be used for both classification as well as approximation.