Perceptron hands on

The perceptron is the simplest form of a neural network. It consists of a layer of entries one neuron and one output. The entries are linked with the neuron in a way that each link has a weight. The neuron receives the weighted sum of its entries plus a value of its own called bias, i.e. for each row of the dataset the perceptron receives, the vector [w0,w1,…wn] is computed: The value w0 corresponds to the bias of the perceptron, while the values w1 to wn correspond to the weights of the links of its entries. For a perceptron of two entries and a data set X = [[x1,y1], [x2,y2],…, [xn , yn]] the perceptron receives the value w0+w1*x1+w2*x2 for each one of the entries of the dataset. For every value received the perceptron computes one output: There are several ways to compute the output, the simplest one is the step function y=S(x), where y = 1 if x >= 0, 0 otherwise. The output function of the perceptron is called activation function.

The dataset is accompanied with a vector which is called target. The role of the perceptron is to evaluate the weights vector in such a way that for each one of the entries the output will be as close to the target as possible. The process of evaluating the weights for this purpose is called training the perceptron.

Schematically, a perceptron of n entries looks like that:

In the code following we define a perceptron which gets a list of four entries X = [[1,1],[0,0],[0,1],[1,0]] ant a target T=[1,1,0,0]

Defining the class Perceptron

We see that the perceptron predicted correctly the target vector. Now we are going to make another implementation of a perceptron in order to linearly separate a bigger set of data. We fist generate four clusters of forty normally distributed samples around each one of the points (0,0), (0,1), (1,0) and (1,1) of the plane. The samples around the points (0,0), (0,1) and (1,1) have 0 as target, while the samples around the point (1,1) have 1. Follows the code.

The meaning of the bias now becomes clear: A perceptron without bias would had constraint the separating line to pass from the origin of the plane.

Follows the two subsets separated by a straight line.

The red points correspond to the samples having target 0, while the blue points correspond to targets 1.

During the training the perceptron approched gradually the optimum solution. The learning curve is show in the following graphic.

In a next note we’ll see a perceptron acting in a real problem of diagnosis.

Related posts:

I saunter along the banks of some dark boundless ocean in the multiverse that is my mind. The gentle undulating waves that once lulled me into the depths of its waters, now burble to spur me back…

Functions are the building blocks of a Javascript program. A Function contains a set of statements which performs a specific task. The function is basically wrapping a piece of program in a value. A…

Fury was originally released as a basic Monero clone that was forked off of the latest Monero code. The only two goals when we started were to have an anonymous coin that would take the best parts…