In the footsteps of intelligence

In the footsteps of intelligence

From the moment of the first attempts to simulate the processes occurring in the human brain, science has gone through many steps that bring us closer to AI. But the human brain does so far unaffordable and poorly monitored work on the continuous processing of the flow of sensory information. I will try to sensibly tell the main milestones in the evolution of artificial neural networks (INS).

Man always wanted to just understand how he thinks. And he applied for this task the standard methods of science: analysis, observation, experiments. But such a study gave only descriptive knowledge, which is very far from reaching the severity of the physical sciences. I had to move away from the direct question “How do we think?” And find out, try to find an answer to the question “What does a person think?”. It turned out that the whole thing is in the brain, consisting of many similar nerve cells - neurons. In addition, there were many of these cells, but there were even more connections between them. Modeling the experimentally observed properties of the neuron gave some repetition of the results of human thinking. It remained to link this to the picture of the functioning of the entire human nervous system already at the level of physiology and psychology, but this is where the problems began.

The first models of ANN are direct distribution networks, without feedback. Their output was completely determined by the current values ​​of the sensors and coefficients (weights) at each input of each neuron. In addition, the method of teaching with the teacher was used, when the scales of the inputs of the neurons were adjusted so that the network would produce a tolerance from the required results for the training set of input values. Such a method was biologically implausible.

The first to solve the problem of learning network without a teacher. The basic principle of such training is that if the sending and receiving neurons are simultaneously active (output = 1), then the weight of the connection of these neurons increases. Biologically acceptable here is the logic of learning a neuron within the boundaries of the neuron itself and the repetition of the connection property (synapse) of biological neurons. The neuron became an integral logical self-organizing unit inside the ANN.
')
The second was solved by the problem of the influence of sequences on the INS result, first simply adding the output from the ANN to the incoming signal (Hopfield network), and then to the recurrent ANN by adding connections from internal neurons (the Elman network) or from the INS output (Jordan network) with a single delay. Recurrent input constraints are usually called context for input data. Recurrent ANNs can already be used in control systems of moving objects, as well as used for storing sequences.

The next property of memory was its simultaneous plasticity and stability. The principle of stability-plasticity is that new images are remembered in such a form that the previously remembered ones are not modified or forgotten. All previous types of ANNs were unable to repeat this property. The answer of science to this problem was adaptive resonance theory (ART). Its meaning comes down to the fact that if the network decides that the incoming set is not similar to one already memorized, then it adds another output from the network for the new image. If the network recognizes in the input set a known set within the selected similarity boundary, then a further training is provided to the recognized image inside the network. The inconsistency between stability-plasticity and ART is that a change in the recognizable set still occurs. A special sequence of input sets with a deviation smaller than the similarity limit can be completely retrained by the INS. But the addition of new neurons to the network, that is, new neurons brings us closer to the repetition of biological neural networks, throughout the life of a person new neurons are formed. The areas of the brain that are responsible for new or constantly trained human skills are condensed by new neurons.

The next problem was the recognition of the image regardless of the position in the incoming set. This problem is solved by cognitrons and neocognitrons - multi-level hierarchical self-organizing networks. The main property of such networks is to limit the area of ​​connections to the number of inputs from the previous layer of neurons, which leads us to the similarity of the structure of the visual cortex. As well as the presence of negative inputs in neurons, before that negative connections were not used in the ANN, although the presence of such connections was proved in biological neurons. Also interesting is the way to check the completion of training INS. The network is launched in the reverse direction, on the supply of just one signal to one of the outputs, the INS provides a memorized image, for recognition of which this output is responsible.

The idea of ​​limiting the area of ​​connections for self-organizing neurons is that when learning you can go crazy if you try to memorize everything, it is therefore natural that you need to abstract, highlighting the most common and important properties for all the phenomena and objects around us, creating the smallest possible number of exits from the layer of neurons and then again use these abstractions for self-learning of neurons of the next level.