Neural Networks Based on Homeostatic Neurons: Self-organization and Purposeful Behavior

Introduction

In this topic I want to talk about the model of an unusual neural network, in the development of which I was fortunate to participate. This model was developed about a year ago (about authorship is written in the last section), however, its research after that stopped due to a shortage of time (employment in their own projects). Nevertheless, I will describe it here in the hope that some thoughts will seem interesting to readers and will sprout possible further research in this direction.

At once I will make a reservation that this model does not pretend in any way to the prototype of AI. Rather, we wanted to explore the possibility of self-organization and the prospects for the emergence of integral purposeful behavior in a dynamic system of "selfish" (homeostatic) neurons.
It seems to me that for a more complete understanding of the logic of the model, it will be useful to read my topic about the Theory of Functional Systems , but again this is up to you.

Go!

')

Biological basis

This model was built on the basis of the theory of functional systems PK Anokhin and the theory of homeostasis.

Homeostasis is self-regulation, the ability of an open system to maintain the constancy of its internal state through coordinated reactions aimed at maintaining dynamic equilibrium. Walter Canon, who originally coined the term, called homeostasis "the wisdom of the body" [1].

The simplest biological motivations arise when the homeostatic equilibrium is disturbed in the neurons of the respective physiological zones (disturbance of energy, oxygen, or osmotic imbalance). However, there is a kind of universal unit for assessing biological motivation [2]. The existence of this unit is connected with the fact that the satisfaction of any motivation causes pleasure and, accordingly, the removal of an imbalance in the homeostatic system, which gives rise to this motivation. Therefore, modeling homeostasis is a direct path to the creation of an artificial system that has some semblance of motivation, i.e. system with its own purpose and ways to achieve it. There are many factors leading to a mismatch in the system of homeostasis, and there is a hierarchy of such factors according to the degree of their importance for cell survival. To maintain factors of the highest level in a stable range, homeostasis changes the optimal levels of regulation of the factors of the lower level. Those. restoration of the function may occur not through the return of the distorted parameters to the norm, but in a detour, through changes in the ratio between these parameters. In particular, excessive excitation damages neurons, but there are some factors that restore homeostasis (like cAMP, interleukin-1, thyrotropin-releasing hormone). Consequently, compensatory homeostasis can, at an intermediate stage, excite neurons on the way to the search for a new equilibrium point. In fig. 1. shows a diagram of a two-level homeostasis in a neuron. The red color in the figure shows conventionally the damage of the neuron, and the green color shows the recovery. Damage is an external factor causing misalignment.


Fig. 1. Homeostasis of the neuron
(LoR - low level homeostasis, HoR - high level homeostasis, LS - local mismatch sensor, R - reward reducing mismatch level)

A set of homeostasis models is known in the literature [3,4], but they are not focused on the self-organization of neural networks and purposeful behavior. Modeling of targeted behavior, including on the basis of neural networks and reinforcement learning, also has a rich history and is widely covered in the literature. However, within these models, the homeostasis of neurons and networks in general was not considered.

Actually, based on the foregoing, a hierarchy of models has been formulated that will make up a complete system:
- a model of a homeostatic neuron, combining the key characteristics of known models and the basic principles of the considered paradigm;
- models of homeostatic neural network based on homeostatic neurons;
- a model of the simplest organism controlled by a homeostatic neural network and its interaction with the environment;
- modeling the evolutionary development of the simplest organisms in a changing external environment.

Model of homeostatic neuron and neural network

Since the main task is to analyze the paradigm of a homeostatic neuron as a possible basis for the mechanism of self-organization of a neural network and its ability to autonomous behavior, it is necessary to pay particular attention to this key primary model. The model of a homeostatic neuron implies that each neuron seeks to maintain its optimal internal state — homeostasis; moreover, when the current state mismatches with the optimal one, the neuron forms a response aimed at returning to the optimal state. As mentioned earlier, a change in the set of factors can lead to a disruption of the neuron equilibrium, however, in the model situation, we introduce some single endogenous assessment of the state of the neuron q (t) , which has a predetermined optimal value q ^opt . We also introduce an indicator of the magnitude of the internal energy e (t) , reflecting the capabilities of a neuron in generating signals x (t) and maintaining homeostasis of its own activity.

The desire of the neuron to homeostasis is expressed in the activity aimed at eliminating the mismatch q ^opt- q (t) , the more pronounced the larger the mismatch. The activity of a neuron requires energy expenditure, therefore it can take different forms depending on the amount of energy e (t) available .

Thus, we have formulated some requirements for the model of the “egoistic” neuron, i.e. such a neuron that has one goal - to maintain its state close to optimal. However, each neuron cannot exist in isolation, but must function in a complete neural network. Thus, we can introduce the concept of a vector of action on a specific neuron from the rest of the network:

This effect changes the endogenous assessment of the neuron for the next time step:

This function may look different, but the simplest case is


It turns out that above we have introduced all the characteristics of the neuron, as well as the method for determining the effect on the neuron from the rest of the neurons. Now we introduce the concept of the neuron mismatch g (t) , the meaning of which we have already touched on above, as well as the actual concept of the action of the neuron and the output signal. Neuron mismatch:

Now we define the action of the neuron , which it chooses on the basis of the current mismatch:


At this stage it is necessary to stop in more detail, since on the one hand it is one of the key ones in the whole model, and on the other hand, the very concept of action accumulates several biological principles and our reflections, which were mentioned above.

The first thing to notice is that any effect on a neuron from other neurons leads to a deviation from its homeostatic equilibrium. Accordingly, all the actions of our "selfish" neuron should be aimed at returning to this balance. This is the principle of low level homeostasis . Here there are several possible cases that were actually considered when drawing up the action function.

The action of a neuron primarily depends on the degree of its mismatch. If the neuron is far from the point of homeostatic equilibrium, then it performs an action that immediately returns it to equilibrium — spike generation. If it is close to equilibrium, then another mechanism of homeostasis is activated — slow recovery. A neuron spends energy on every action: a large spike on a spike, but not a lot on recovery. About the replenishment of neuron energy will be discussed below.

In addition, neuron energy may not be enough to generate a spike, even when it is needed, which means the choice of recovery until the moment when the energy is replenished.

Thus, the action of a neuron is a function of the input signal, endogenous state and energy storage. Actually this ends the formal model of the neuron (including the lower level homeostasis) and the network as a whole. In fig. 2 shows a schematic representation of the “egoistic” neuron.


Fig. 2. "Selfish" neuron

Model of the body

The neural network is a part of the whole organism, but for its full definition it is necessary to introduce some more additional concepts. The first of them is the common pool of energy E (t) , from which the energies of all the neurons of the network are recovered. If e (t) <e ^min (optimal energy of the neuron), then

Now we need to define the concept of higher level homeostasis .
We introduce the concept of endogenous assessment of the state of the entire network:

In this case, we assume that there is some optimal state Q ^opt :

Then we define that the higher level homeostasis comes into effect in two different cases:
- after establishing stability (in the case if a cyclic process is observed in the network, but the state does not reach the optimal value)
- in the event of a “need arising” (in the case of a large mismatch of the endogenous assessment of the network state)

In this case, the parameters q _i ^opt change randomly, and the magnitude of the change in each value is proportional to both the mismatch of the endogenous evaluation of the network state and the mismatch of the endogenous evaluation of the specific neuron. Thus, when the system is unable to reach the optimal state and if the life of the entire network is threatened, the mechanism of higher level homeostasis is activated. Schematically, such a model of the organism is shown in Fig. 3


Fig. 3. Model of the body

Here, it is probably necessary to cite some experimental data by which it will be possible to judge the qualitative dynamics of such a model (without a higher level homeostasis). We have composed a network of 10 neurons with randomly distributed weights according to the distribution of W _ij ~ N (0,1) and with various optimal values ​​of the endogenous state and energy reserve. At the same time, at the time of initialization, 9 neurons were in equilibrium, and one of the neurons was mismatched. In fig. 4-5 shows the time dependences of the endogenous assessment of the state and energy of each neuron in such a system.


Fig. 4. The graph of the dynamics of the endogenous assessment of the state of neurons


Fig. 5. The graph of the dynamics of the energy of neurons

The figures show that as a result, the whole system comes to the position of homeostatic equilibrium, however, our experiments show that there is a range of parameters where it is impossible to achieve homeostatic equilibrium and a self-oscillatory process begins in the system, which indicates the need to include higher level homeostasis.

Interaction with the environment

Imagine that we put such an organism on Wednesday. Suppose that our body has only one need - the need to maintain a sufficient level of total energy, which is spent on homeostasis and metabolic processes. In this case, it is necessary that the body can make a decision at the right moment on the replenishment of the energy level. That is, for example, press a button to feed him.

We thought about the introduction of macroparameters of the body, the change of which determines its needs (there may be several), as well as the consideration of a set of effectors that allow the body to change the values ​​of its macroparameters due to the interaction with the external environment. The basis of the proposed approach to the problem is the transfer of information about the values ​​of macroparameters directly to homeostatic neurons, which links the internal homeostasis of the neuron with the satisfaction of the corresponding need. Thus, the homeostasis of the second level is responsible for the efficiency of the system as a whole, and the first level is responsible for the effective line of behavior of the organism when interacting with the environment.

Let me explain by example. To do this, assume that there are two specialized neurons in the structure of the neural network.

The first neuron is specialized with respect to the need for energy replenishment, that is, it enters into a mismatch if the level of total energy is insufficient. That is, it has an additional input to which the level of total energy deficiency ( EE _min ) goes. The second neuron is specialized in performing the action of energy replenishment, that is, at the moment when it generates a spike, the total energy is replenished from the external environment. This system is shown in Fig. 6


Fig. 6. The interaction of the organism with the environment
the red neuron is specialized for the need for energy replenishment, blue for the performance of the energy replenishment action)

Thus, the mismatch in the body is made at the moment when he feels the need for food. In turn, neurons try to eliminate this mismatch. Until a spike on a neuron specialized in performing an action is observed, energy will not be replenished, and the signal of a lack of energy will continue to flow into the system. Thus, at some point, the spike is generated on the second specialized neuron and the stock of total energy is replenished, because of which a neuron that is specialized with regard to the need for energy replenishment ceases to be inconsistent.

Of course, this is a very model situation, however, it shows by what logic such an organism can function in an environment where some decisions need to be made.

Continuous model

Exclusively for those who are thinking about how to make this discrete model in time continuous, I present our version based on the FitzHugh-Nagumo model of neurons.

Conclusion

Of course, this model can not qualify for at least some kind of completeness. However, it seems to me that some thoughts in it are inherent, quite interesting and can be the object of research of readers. In particular, the idea of ​​disagreement of a neuron as a motivation to perform an action, the definition of a mismatch - as the ratio of the internal parameters of the neuron with the microenvironment, as well as homeostasis - as the basis for eliminating the mismatch and for self-organizing effect.

Also, with this topic, I wanted to draw the attention of readers to the fact that even in fairly simple models there can be interesting self-organization effects, which in the long run can lead to holistic behavior.

I note that in this review I did not touch on the non-trivial issue of introducing the learning process into this model, as well as evolution itself. All this remains the subject of further possible research.

Authorship

Maxim Komarov (Nizhny Novgorod University), Daniil Kanevsky (VMiK MSU), Sergey Kulivets (IPU RAS), as well as your humble servant took part in the development of this model. Also, Lev E. Tsitolovsky (Bar-Ilan University, Israel) and Vladimir Georgievich Red'ko (NIISI RAS) for their support and tracking the direction of our thoughts when we were blessed with great thanks. Each of us brought our thoughts and ideas into the model from the areas we are dealing with.

Bibliography

[1] . Cannon WB Physical Review, 1928, 9: 399-431.
[2] . Cabanac M. Journal of Theoretical Biology, 1992,155: 173-200.
[3] . Davis GW Annual Review of Neuroscience, 29: 307-323, 2006.
[4] . Marder A., ​​Prinz AA BioEssays, 24: 1145-1154, 2002