The variant of synchronous impulse neural network with feedback

A warning

I warn you, the reasoning at the moment is purely theoretical. But it’s a very beautiful theory. Or I'm going crazy. I am interested in your opinion about my reasoning - is it worth starting a practical implementation?
Frankly speaking, the reason for which all further reasoning was born is that I failed one bad idea - to teach a neural network to play Tetris myself. Teaching the network with the teacher is unrealistic for a long time, because the teacher is me, and there are definitely no right options for the movement of the figures in Tetris. A genetic network learning algorithm has been applied. The maximum that my population learned during the day of the virtual process of evolution and natural selection is to lay down the falling figures in three columns (in fact, she also learned to turn the figures so that they were more “horizontal” than “vertical”, but learn how to turn the figures so that the lines were removed - it turned out not to the forces).

What I don't like about classic neural networks (a la perceptrons)

As a rule, when considering certain applications of neural networks, they imply the use of analog neural networks, like perceptrons, etc.
Such neural networks took from the real, living, organisms only the most superficial form and essence. It seems like the planes took the idea of ​​wings from the birds. Although they are used completely differently (but with the same purpose). In fact, various variations of perceptrons are a non-linear function of the sum of the same non-linear functions y = k1 * f (k2 * f (...) + k3 * f (....) + ...), and the learning algorithms simply select values coefficients k.
Due to the too large difference between real neural networks and artificial ones, it is necessary to invent various surrogate methods of network training. Naturally, nowhere in nature there is no learning of the network by the “back-propagation method of error”, as well as no “learning without a teacher” in its pure form.
I would like to implement a more believable neural network and experiment with it.

Basic principles of the functioning of "my" neural network

In accordance with my knowledge that neuroscientists are currently aware of the functioning of real neural networks, the basic principles should be as follows:

1. Transmission of information in the form of pulses
2. Information coding - frequency (possibly modulated in amplitude)
3. Ability to self-organize due to creation of new synaptic connections
4. Ability to learn by specified criteria
5. Ability to use some network parameters for optimization by genetic algorithms
6. The possibility of implementing all three known types of memory of real networks - short-term, medium-term, long-term

I will explain the points.

1. Transmission of information in the form of pulses

This, I think, everyone knows - signals between living neurons are transmitted in the form of pulses.

2. Frequency coding information

The intensity of the signal from the receptors and further, in the living neural network, is encoded by the pulse frequency. For example, the stronger the taste sensation in your tongue, the more often the impulses from the corresponding receptors arrive. The amplitude of these pulses does not depend on the intensity of the stimulus (but depends on the "fatigue" of the nervous system).
Those. in addition to the neurons proper, an essential part of the neural network is the receptors and the so-called “chemical synapses”. Receptors are an analogue of the ADC, the intensity of exposure is encoded into pulses of a certain frequency. Chemical synapses are analogous to a DAC, depending on the frequency of the pulses, they release more or less different substances that, for example, expand or constrict the vessels. Here it must be said that the terms ADC and DAC are not entirely correct, rather it is ACP and CHAP (analog-frequency and frequency-analog converters).

3. Ability to self-organize and create new synaptic connections

It is known that new synaptic connections are constantly being created in living organisms. The number of neurons we have is how much is given by nature and “will not grow anymore”, but the number of synaptic connections the brain can build up. And in advance it is not set from where and where these communications appear. In particular, it is one of the mechanisms of memory.

4. Ability to self-learning according to specified criteria

No one will argue that animals learn by trial and error. It is a cross between teaching with a teacher and learning without a teacher. In fact - the teacher is the effect of certain actions. Invented neural network should be able to self-learn. Some criteria of course are set in advance, but only very general criteria. Something like instincts.

5. The ability to use some network parameters for optimization by genetic algorithms

It is known that there is a genetic memory. At a minimum, we are able to control our lungs from birth. breathe. We do not learn this either with the teacher or without. This is a genetic memory. The network should be able to transmit some basic information to its descendants.

6. The possibility of implementing all three known types of memory of real networks - short-term, medium-term, long-term

Here I must say that there is also an instantaneous memory, lasting up to a second. But instant memory is the memory of receptor neurons, and has no particular relation to the projected network.
But the three main types of memory must be implemented.
Short-term memory (in humans - tens of seconds) - due to the presence in the network of feedback, and the formation of a closed cycle of neurons, in which the pulses move "in a circle."
Medium-term memory (in humans - a few days) - due to increased synaptic connections with repeated passage of the pulse. This type of memory is also implemented in classical artificial neural networks, with various types of training.
Long-term memory (in humans - for a long time, maybe even for a lifetime) - due to the formation of new synaptic connections and the involvement of previously unused neurons in the work.
There is also a theory that there is a super-long-lasting “non-erasable” memory, such a ROM. Those. memories that will never be forgotten. Presumably, its mechanism is associated with a change in the internal structure of the neuron, when it gives some preference to a certain synaptic connection, i.e. connects to it an "additional amplifier", which is never removed again. For simplification, we can do without super-long-term memory.
')

General scheme of network functioning

The general algorithm of the network is as follows:
1. Initialization. There are only neurons. There is no connection.
2. Select some (any) neurons as input. They are constantly giving input signals (coded frequency).
3. Select some (any) neurons as a weekend. With them constantly remove the output information.
4. In the loop:
4.1. Another tact of the network (transmission of pulses).
4.2. Check the need to create new connections and create them if necessary.
4.3. Check the need to remove unused links and remove them (to reduce the resources used).

Gradually, new synaptic connections will appear and some information will come out.

Here it is convenient to explain why the network is called synchronous.
In synchronous operation, we calculate the value at the output of the neuron, but so far we simply remember, and do not set the output to a new state. After processing all neurons, we give the command to set new values ​​at the outputs of all neurons.
With asynchronous operation - at the output of the neuron we set a new value after its calculation.
In principle, asynchronous networks are closer to reality. But in fact, it makes sense to implement them only in hardware, i.e. when each neuron is assigned its own computing unit.

Pulse Neural Network Training

And so we launched the network created at the previous stage. She began to give out some sort of messy information. And we can not yet say - what do we want from it. How to train such a neural network?
I see here three ways, which are some analogs of teaching with a teacher, teaching without a teacher, and combined - “my own teacher”.
It should be noted that internally self-learning of the network occurs automatically and constantly, due to the adjustment of communication parameters and the emergence of new connections.
So, ways to train such a network.

Teaching with a teacher

The teacher can make the network understand whether it behaves well or badly; Does it give the output we want to receive from it? You can implement it in the form of a certain function, the approximate algorithm of which is very simple.
Parameter - EMOTION_ of the TEACHER. The parameter range is from -1 to 1, where -1 = very bad, 1 = very good, 0 = no evaluation.
For all connections, set the weight of the connection W = W + W * EMOTION_CHAPTER * k, where k is the learning coefficient (can be adjusted, for example, genetically).
Everything.

Thus, with a positive value of the EMOTION_CHARTER parameter, we strengthen bonds, and strong bonds (leading to the correct result) are strengthened more strongly. If the EMOTION_READER parameter is negative, we weaken the bonds, and strong bonds (leading to an incorrect result) are weakened more.

Teaching without a teacher

Here the concept is not yet fully developed, but there are some sketches. If it will be interesting - I will tell in the following articles.

Self teacher

This method of self-study is possible after some time we have trained the network with the teacher. It is implemented simply.
We make some additional output of the network, let's call it INTERNAL_EMOTION, feed it to the same input where EMOTION_READER is fed (let's add them). And in the process of learning with the teacher we follow the inner emotion of the network. Suppose the teacher gives a signal of +0.5 (positive emotion), if the network performed the correct action and it had no internal emotions. The teacher gives a signal of 0 if the network performed the correct action, but she had a negative emotion, and the teacher gives a signal of 1 (very good) if the network performed the correct action and she had a positive emotion. Those. we additionally train the network on the rule “the right action is a good inner emotion”.
Since we connected the INTERNAL_EMOTION exit to the same input to which the TEACHER EMOTION was fed, and at the same time taught “what is good and what is bad,” theoretically, the network itself will manage its further training, even without the participation of the teacher.


This concludes the introductory part.
Details further, if these reflections at least someone interested.