Using the Hopfield neural network to solve the simplest problem

Currently, a rather interesting area of ​​programming is the use of neural networks. Despite all the prospects of this idea, most of the implementations that I met were associated with various analyzes of statistical samples and the prediction of the possible value of a parameter.

Talking about artificial intelligence and cumbersome expert systems is all well and good, but how can we bring this whole theory to life, to our appropriate problems?

A little bit about the task

It is required to write a program that will recognize reference images in a “noisy picture”.
')
Neural network selection

In this article I will not consider the theoretical basis of neural networks, I think this information on the Internet is enough =) But still without theoretical material it will be difficult to understand what I am writing.

And so, the Hopfield neural network is the best solution for solving our problem.

It consists of a single layer of neurons, the number of which is simultaneously the number of inputs and outputs of the network. Each neuron is connected by synapses with all other neurons, and also has one input synapse, through which the signal is input. The output signals, as usual, are formed on axons.

This network requires a certain set of binary signals at the input, which are considered exemplary. The network should be able to select (“remember” by partial information) the corresponding pattern (if any) from an arbitrary non-ideal signal applied to its input, or otherwise produce the most similar image.

In general, any signal can be described by the vector X = {xi: i = 1, 2, ..., n},
n is the number of neurons in the network and the dimension of the input and output vectors. Each xi element is either +1 or -1.
There is a vector describing the kth sample. When the network recognizes (or “remembers”) a sample based on the data presented to it, all outputs will contain it, i.e. Y is the vector of network output values: Y = {i: i = 1, 2, ..., n}. Otherwise, if the noise is too strong at the outputs will be "garbage".

At the network initialization stage, the weights of synapses are set as follows:

Here i and j are indices of presynaptic and postsynaptic neurons, respectively;

The network operation algorithm is as follows (t iteration number):
1. An unknown signal is sent to the network inputs. In fact, it is entered directly by setting the values:


therefore, the designation on the network diagram of the input synapses in an explicit form is purely conditional. Zero in the bracket to the right of yi means zero iteration in the network loop.
2. Calculate the new state of neurons.

and new values ​​of axons

3. Check whether the output values ​​of axons have changed during the last iteration. If yes, go to step 2, otherwise (if the outputs are stabilized) - end. At the same time, the output vector is a sample that is best combined with the input data.

Thus, when a new vector is supplied, the network moves from a vertex to a vertex until it stabilizes. The stable vertex is determined by the network weights and the current inputs. If from vertex to vertex, until stabilized. The stable vertex is determined by the network weights and the current inputs. If the input vector is partially incorrect or incomplete, the network stabilizes at the vertex nearest to the desired one.
It is proved that a sufficient condition for the stable operation of such a network is the fulfillment of the following conditions:
Wij = Wji, Wii = 0.

As I said, sometimes the network can not conduct recognition and outputs a non-existent image. This is due to the problem of limited network capabilities. For the Hopfield network, the number of memorized images N must not exceed a value approximately equal to 0.15n. In addition, if the two images A and B are very similar, they will probably cause cross associations in the network, i.e. the presentation of the vector A to the inputs of the network will result in the appearance of the vector B at its outputs and vice versa.

Let's start writing the program

And so a binary sequence is necessary for a neural network (in our case, -1/1).
Let "1" is the black color of a pixel, and "-1" is white,
this way we can convert the image into a sequence.

For speed and accuracy of recovery, we will use the following scheme:
A picture of 100x100 pixels and each pixel has its own neuron.
Thus, the network will consist of 10,000 neurons.

Let's start with the presentation of the elementary unit - the neuron:

//
private class Neuron
{
//
public void ChangeState()
{
if (s > 0) y = 1;
if (s < 0) y = -1;
if (s == 0) y = 0;
}
public int s;
public int x; //
public int y; //
public int index;
}

* This source code was highlighted with Source Code Highlighter .

Now consider the neural network itself, I think for a better understanding, you should give the main part of the listing =)

namespace Neurons
{
class NetHopfild
{
//
private class Neuron
{
...
}

private const int size = 10000;
public NetHopfild()
{
// ,
mass = new Neuron[size];
for ( int i = 0; i < size; i++)
{
mass[i] = new Neuron();
}
matrix_of_connect = new int [size, size];
last_y = new int [size];
//
for ( int i = 0; i < size; i++)
{
mass[i].index = i;
}
for ( int i = 0; i < size; i++)
{
for ( int j = 0; j < size; j++)
{
matrix_of_connect[i,j] = 0;
}
}
}

public void Initialise( int []input)
{
for ( int i = 0; i < size; i++)
{
for ( int j = 0; j < size; j++)
{
if (i == j) matrix_of_connect[i,j] = 0;
else matrix_of_connect[i,j] += input[i] * input[j];
}
}
}

public void FindImage( int []input)
{
for ( int i = 0; i < size; i++) //
{
mass[i].x = input[i];
last_y[i] = mass[i].y;
}

for ( int k = 0; k < size; k++) // S
{
mass[k].s = 0;
for ( int i = 0; i < size; i++)
{
mass[k].s = mass[k].s + matrix_of_connect[i,mass[k].index] * mass[i].x;
}
mass[k].ChangeState();
}
bool flag = true ;
//
for ( int i = 0; i < size; i++)
{
if (last_y[i]!=mass[i].y)
{
flag = false ;
break ;
}
}
if (flag == false )
{
int [] temp = new int [size];
for ( int i = 0; i < size; i++)
{
temp[i] = mass[i].y;
}
// , ,
FindImage(temp);
}
}
private Neuron []mass;
private int [,] matrix_of_connect;
private int [] last_y;
// ( )
public int [] Synapses
{
...
}
};
}

* This source code was highlighted with Source Code Highlighter .

And so the network is ready, now we will work with it.

In this case, I will not describe the entire source code, I will describe only the main points.

Convert the image into a binary sequence

private int [] BitmapToIntArray()
{
// 1 -1
Color currentPixel;
int [] temp = new int [size];
int index = 0;
for ( int i = 0; i < picture.Height; i++)
{
for ( int j = 0; j < picture.Width; j++)
{
currentPixel = picture.GetPixel(j, i);
currentPixel.ToArgb();
if (currentPixel == Color.FromArgb(255, 255, 255))
{
temp[index] = -1;
}
else
{
temp[index] = 1;
}
index++;
}
}
return temp;
}

* This source code was highlighted with Source Code Highlighter .

results

Let there be two standard images.


We train our neural network



Similarly, we teach the second image.



Interfere with the picture:



Download the distorted picture into the program:



Click "Restore" in a few seconds:



As you can see, having located two emoticons after learning, the one for which the distorted design is most suitable. You can play with the recovery process for a long time, but I think it’s enough for this article =) Experiment, you get really interesting results;)

And since our network operates only with binary sequences, I think it will not be difficult to rewrite this program, for example, to restore typos in the words =) and other goodies =)

For those who are interested, I post an archive with images and noises , the program itself

And of course, if someone completely imbued with the idea and need sorts, write in the comments, I will post it =)

UPD: Corrected the passion in the code =) which indicated: graninas , Nagg , Peregrinus