The basic principles of learning algorithms Hierarchical Temporal Memory

Good day!
Recently, in Habré, quite often there are articles in which the authors describe modern theories and approaches to the construction of artificial intelligence and neural networks. However, examples of specific implementation provides a rather meager amount. Let's try to fill this gap. In this article I will describe only the basic theoretical and practical aspects used in writing the working model of the algorithms provided by Numenta Inc.

The basis for the implementation was the document available here . You can also find a Russian translation there, but for the sake of completeness, I personally would recommend using the original.

HTM Architecture

The network's architecture, as the author assures, is based on the structure of the cerebral cortex, or more precisely, on the part of it that is called the neocortex . The basic units, neuron cells , are organized into columns , which in turn constitute sections (region). From the plots in the future it is possible to build the very hierarchy, after which the network is named.
The picture from the official site gives a general idea of ​​the structure of the plots:


The structure of the cell, by analogy with a natural neuron, provides for the presence of dendritic segments (consisting of a certain set of dendrites) and synapses - junctions between the axon of another cell and its own dendrite. Each synapse in the HTM is actually represented by "strength" (permanence) - a real number from 0 to 1. If the strength of the synapse is less than a specified value, then the synapse is considered "non-existent", otherwise it is valid. This is the concept of potential synapse (potential synapse).
Dendritic segments of a single cell are divided into distal and proximal. Proximals are used to obtain input information, distal ones are used to connect cells to each other, within this area. In the proposed Numenta architecture, each cell column has one common proximal segment, which contributes to an increase in speed. Each dendritic segment (both distal and proximal) contains a number of potential synapses.
The input bits of information are represented as a layer of cells capable of forming synapses with proximal dendrites.
On the basis of this architecture, learning algorithms operate, consisting of two phases: spatial (spatial) and temporal (temporal, temporal?) Groupers.
')

Spatial Grouper

The spatial grouper includes three phases: overlap, inhibition, and learning.
In the overlap phase, for each column, a value is calculated that reflects the “coverage” of the proximal dendritic segment by the synapses of the “single” bits of the input information of the input information. A synapse is considered active; its strength is greater than the set value and the information bits to which it leads leads to 1.

1. public void SOverlap ( ) {
2. for ( int i = 0 ; i < xDimension * yDimension ; i ++ ) {
3. overlap [ i ] = 0.0 ;
4. for ( Synapse synapse : connectedSynapses ( i ) ) {
5. overlap [ i ] + = input ( time, synapse. c , synapse. i ) ;
6. }
7. if ( overlap [ i ] < minOverlap )
8. overlap [ i ] = 0.0 ;
9. else
10. overlap [ i ] * = boost [ i ] ;
11. }
12. }

In the suppression phase, a list of active columns is calculated. The columns with the maximum “coverage” suppress their neighbors, the number of active columns in the region is regulated by a user-defined parameter.

1. public void SInhibition ( ) {
2. activeColumns. add ( new ArrayList < Integer > ( ) ) ;
3. for ( int i = 0 ; i < xDimension * yDimension ; i ++ ) {
4. Double minLocalActivity = kthScore ( neighbors ( i ) , desiredLocalActivity ) ;
6. if ( overlap [ i ] > 0.0 && overlap [ i ] > = minLocalActivity ) {
7. activeColumns. get ( time ) . add ( i ) ;
8. }
9. }
10. }

In the learning phase, the strength of the active synapses of the proximal dendritic segment of each active column increases, inactive, and accordingly decreases. Then a mechanism is applied to ensure the regular operation of all the columns in the region. In general terms, it consists in the fact that we have a history of activation of each of the columns (in the form of a moving average ) and increase the "coverage" of the column in case of its long idle time.

1. public void SLearning ( ) {
2. for ( Integer c : activeColumns. get ( time ) ) {
3. for ( Synapse s : potentialSynapses. get ( c ) ) {
4. if ( input ( time, s. c , s. i ) > 0 ) {
5. s. permanence + = permanenceInc ;
6. s. permanence = Math . min ( s. permanence , 1.0 ) ;
7. } else {
8. s. permanence - = permanenceDec ;
9. s. permanence = Math . max ( s. permanence , 0.0 ) ;
10. }
11. }
12. }
14. for ( int i = 0 ; i < xDimension * yDimension ; i ++ ) {
15. minDutyCycle [ i ] = 0.01 * maxDutyCycle ( neighbors ( i ) ) ;
16. activeDutyCycle [ i ] = updateActiveDutyCycle ( i ) ;
17. boost [ i ] = boostFunction ( activeDutyCycle [ i ] , minDutyCycle [ i ] ) ;
18. overlapDutyCycle [ i ] = updateOverlapDutyCycle ( i ) ;
20. if ( overlapDutyCycle [ i ] < minDutyCycle [ i ] ) {
21. increasePermanences ( i, 0.1 * connectedPerm ) ;
22. }
23. }
25. inhibitionRadius = averageReceptiveFieldSize ( ) ;
26. }

At this point the spatial grouper ends. As a result, we have a list of active columns, which we still need.

Time grouper

The time grouper, as well as the spatial one, consists of three phases: calculating the activity states (active) and learning (learn) of each cell, calculating the prediction states, and applying the changes in the network structure expected at the previous steps — adding new synapses / distal dendritic segments.
In the first phase, for each active column from the list inherited from the spatial grouper, it is calculated whether this activity was predicted by any of the cells in the column. If such a cell is found, it alone becomes active; otherwise, all cells in the column become active. According to a certain rule, a cell is selected for training; if the rule is not fulfilled, the most suitable cell from the column is trained.
In the next phase, based on the activity of the segments, the cells are calculated in a predictive state and the segments are amplified, the activity of which led to a successful prediction at the last time step.
At the last stage, all changes that were proposed in the two previous phases are applied to the learning cells. New segments or potential synapses are added to the list of existing ones for this segment.

1. public void TLearning ( ) {
2. for ( int c = 0 ; c < xDimension * yDimension ; c ++ ) {
3. for ( int i = 0 ; i < cellsPerColumn ; i ++ ) {
4. if ( learnState. get ( time ) . get ( c ) . get ( i ) ) {
5. adaptSegments ( segmentUpdateList. get ( c ) . get ( i ) , true ) ;
6. // segmentUpdateList.get © .get (i) .clear ();
7. } else if ( ! predictiveState. get ( time ) . get ( c ) . get ( i ) && predictiveState. get ( time - 1 > 0 ? time - 1 : 0 ) . get ( c ) . get ( i ) ) {
8. adaptSegments ( segmentUpdateList. get ( c ) . get ( i ) , false ) ;
9. // segmentUpdateList.get © .get (i) .clear ();
10. }
11. segmentUpdateList. get ( c ) . get ( i ) . clear ( ) ;
12. }
13. }
14. }

Conclusion

We assume that the general idea of ​​the learning algorithms presented in the article is sufficient for understanding the basic principles. I can invite all interested in specific implementation on github . The project presented there is far from ideal in terms of compliance with the HTM architecture, but, practically, corresponds exactly to the pseudo-code excerpts presented in Numenovo pdf, and the description of functions and data.