Perceptron learning in Neural Network

Introduction

In ANN it is a supervised learning algorithm basically used for linearly separable classification problems. Here the system can learn concepts and can respond the output as TRUE (1) or FALSE (0) value.

Perceptron is a single layer network whose weight and bias can be trained to produce correct output when trained with input dataset. This training is called Perceptron Learning Rule.

The perceptron has a great ability to generalize from its training data and work with randomly distributed connections. Perceptrons are especially suited for simple problems in pattern classification.

Fig: Perceptron neural network

Training

Data from a training set are presented to the network one after another. If the network's output is correct, no change is made. Otherwise, the weights and biases are updated using the perceptron learning rule. An entire pass through all of the input training data is called an epoch. When such an entire pass of the training set has occurred without error, training is complete.

At this time any input training data may be presented to the network and it will respond with the correct output data. If a data P not in the training set is presented to the network, the network will tend to exhibit generalization by responding with an output similar to target vectors for input vectors close to the previously unseen input data P.

The Learning Rule

Let’s say,

Input data is: [X₁, X₂, X_3, ----------------------X_N].

Output data is: [Y₁, Y2, Y3, ---------------------Y_N]. Where Y_i= {0, 1}

Factor of input data: P is the no of factor in data [X₁₁, X₁₂, X₁₃, ------------, X_1P]

Weight is: W    [W₁, W₂, W_3, ----------------------W_P].

Bias is: B

Threshold is: TH

Learning rate is: LR

ALGORITHM

Set random weights to W of dimension 1xP;

Set Bias B of dimension 1x1;

Do {

Correct=0;

Epoch++;

   For I = 1 to N {

Activation = 0;

 For J = 1 to P

 {

 Activation = Activation + W_J*X_IJ;

 }

Activation = Activation + B;

If Activation>TH

Result=1;

Else

Result=0;

If Result == Y_I

                        Correct++;

Else {

                For K = 0 to P

 {

 W_K = W_K + LR * (Y_I – Result) *X_IK;

}

B=B+ LR * (Y_I – Result);

}

   }

If Epoch>1000

                Break;

} While (Correct<N)

Example: OR GATE FUNCTION

TH	LR	B=0
0.5	0.1	0
Input		output	Initial Weights				SUM	OUTPUT IF(S>TH) THEN 1  ELSE 0	ERROR	CORRECTION	Final Weights
X1	X2	Y	W1	W2	C1 (X1*W1)	C2(X2*W2)	S(C1+C2)	N	E(Y-N)	C(R*E)	W1(W1+X1*C)	W2(W2+X2*C)
0	0	0	0	0	0	0	0	0	0	0	0	0
0	1	1	0	0	0	0	0	0	1	0.1	0	0.1
1	0	1	0	0.1	0	0	0	0	1	0.1	0.1	0.1
1	1	1	0.1	0.1	0.1	0.1	0.2	0	1	0.1	0.2	0.2
0	0	0	0.2	0.2	0	0	0	0	0	0	0.2	0.2
0	1	1	0.2	0.2	0	0.2	0.2	0	1	0.1	0.2	0.3
1	0	1	0.2	0.3	0.2	0	0.2	0	1	0.1	0.3	0.3
1	1	1	0.3	0.3	0.3	0.3	0.6	1	0	0	0.3	0.3
0	0	0	0.3	0.3	0	0	0	0	0	0	0.3	0.3
0	1	1	0.3	0.3	0	0.3	0.3	0	1	0.1	0.3	0.4
1	0	1	0.3	0.4	0.3	0	0.3	0	1	0.1	0.4	0.4
1	1	1	0.4	0.4	0.4	0.4	0.8	1	0	0	0.4	0.4
0	0	0	0.4	0.4	0	0	0	0	0	0	0.4	0.4
0	1	1	0.4	0.4	0	0.4	0.4	0	1	0.1	0.4	0.5
1	0	1	0.4	0.5	0.4	0	0.4	0	1	0.1	0.5	0.5
1	1	1	0.5	0.5	0.5	0.5	1	1	0	0	0.5	0.5
0	0	0	0.5	0.5	0	0	0	0	0	0	0.5	0.5
0	1	1	0.5	0.5	0	0.5	0.5	0	1	0.1	0.5	0.6
1	0	1	0.5	0.6	0.5	0	0.5	0	1	0.1	0.6	0.6
1	1	1	0.6	0.6	0.6	0.6	1.2	1	0	0	0.6	0.6
0	0	0	0.6	0.6	0	0	0	0	0	0	0.6	0.6
0	1	1	0.6	0.6	0	0.6	0.6	1	0	0	0.6	0.6
1	0	1	0.6	0.6	0.6	0	0.6	1	0	0	0.6	0.6
1	1	1	0.6	0.6	0.6	0.6	1.2	1	0	0	0.6	0.6
0	0	0	0.6	0.6	0	0	0	0	0	0	0.6	0.6
0	1	1	0.6	0.6	0	0.6	0.6	1	0	0	0.6	0.6
1	0	1	0.6	0.6	0.6	0	0.6	1	0	0	0.6	0.6
1	1	1	0.6	0.6	0.6	0.6	1.2	1	0	0	0.6	0.6

Limitations

Perceptron networks have several limitations.

1.       First, the output values of a perceptron can take on only one of two values (True or False).

2.       Second, perceptrons can only classify linearly separable sets of vectors. If a straight line or plane can be drawn to separate the input vectors into their correct categories, the input vectors are linearly separable and the perceptron will find the solution.

3.        If the vectors are not linearly separable learning will never reach a point where all vectors are classified properly.

MULTICLASS PERCEPTRON NETWORKS:

Let be X = {x1X2… xn} the training data and the objects belong to C classes, c(x) = {1 to C}. Each class has its own linear classifier: W and B.

The classification rule becomes,

c(x) = argmax (wi,xi) for all i=1 to C

Using compact notation

Where wi0 = (wi1,………….,wid, bi)

Where x0 = (x1,……….,xd,1).

MULTI-CLASS PERCEPTRON ALGORITHM

Let’s say

Input data X= {X1, X2…, Xn}

Output data Y= {Y1, Y2…, Yn} where Y belongs to Y(X) = {1 to C}.

Weight W = {W1, W2, ----------, WC}

Bias B = {B1, B2, ---------, BC}

For all x in X

 I = c(x)

g = Wi * x;

error=false

for j = 1 ……C, j not equal to I,

 if g < wjx then

error=true

wj = wj-LR* x 

end for

if error then wi = wi + LR*x

end for all

In case of miss-classification the algorithm modifies the w associated to the correct class and the w associated to current selected class whose wjx is higher than the value for the correct class.