Primitive prediction in the INS
It is better to debug the allocation of conditioned reflexes with objective control. Create an artificial world in which the INS will live. In this world, it is possible to carry out some actions and get a return, taking into account whether the actions were correct. You can start with the most simple conditions: there are only 2 actions, with the names "1" and "2". The world sends different combinations of signs, in response, you can do actions 1 or 2, in the case of the correct answer, the world sends reinforcements - the "+" symbol. For example, the world sent A, and the INS responded by doing action 2, then the world will send her +. After the sign B you need to do 1, then it will be + too. The world can send "junk" signs of T1 ... TN, which do not affect anything, but force the INS to separate the important from the unimportant.
To be able to easily combine different sets of neural networks and different types of worlds, you need to make their interfaces the same, and in the GUI add the ability to choose what to start with.
The program interface is easiest to do based on strings. At each cycle of the neural network, it requests an array of strings from the “world” - these are the identifiers that are considered to have worked in this cycle. This is a fairly universal solution, which is suitable for many cases: if one line is equal to one letter, then the INS can read texts, if one line is equal to one event identifier of the “Trash3” type, then the INS can live in artificial worlds like the one described above, or more complex. You can even work with a primitive vision, where each pixel has one line, but for a large resolution it is more efficient to use a different approach. Primitive sounds can also be processed by allocating a separate line for each frequency from a small set.
')
With the complexity of the ANN algorithms, it is possible to complicate the worlds.
Since in this world, the INS perceives not the letters directly, but the neurons to which activation from an unknown line has arrived, she will not be able to understand that “the letter T means garbage”, she will have to memorize the consequences of neuron triggering.
... can be created in the following ways:
1) if a new sensory string, unknown for ANN, arrived, then immediately create a sensory neuron / cluster for it. In the future, the INS will analyze it as another sign, the number of which it does not care.
2) for some system neurons (such as positive emotions), you will have to query and create them in advance so that the algorithms know where to get the data.
3) if sensory neurons are created “on demand”, an uncomfortable situation can occur when sensory neurons are intermixed with normal ones. In this case, it may be convenient to create all sensory neurons in advance. For example, if it is known that the INS will work with ordinary texts, then the whole alphabet can be requested in advance.
In order to automatically determine which patterns the INS could find and which not, you need to add prediction. Then the INS should choose those actions that will lead to positive signals. According to the statistics of the operation, it will be possible to build graphs of learning speed, automatically determine the passing of tests, etc.
INS predicts each option and chooses the one that will give the best result. This method is suitable for both primitive worlds with a small number of actions (for debugging other algorithms), and for more complex ANNs, when the INS will narrow down the range of actions to some small list that is relevant in this situation and choose between them.
In the most primitive case, the algorithm is organized as follows: after the next step of recognition, enter the cycle of possible actions, select the next action, “go into virtuality”; activate the selected action and remember it, as if it happens in reality. See which chains were recognized - if the selected chains found patterns, then they will work. To do so in several steps, so that the activation proceeds before its consequences - positive or negative emotions. Remember the amount of emotion for all time forecasting. If there is only one emotion - a positive one - then this is simple, otherwise you will have to analyze the vector of different emotions and choose which is better - a bit of negative and how much positive there is or something else.
More realistic algorithms can be obtained if the INS conducts forecasting in parallel with the rest of thinking, without the need to completely “go into virtuality”. It is better to return to these algorithms later.
After the action with the best characteristic is found, the system “goes out of virtuality” and makes it in reality. What is important - after returning from fantasy, it is necessary to restore the state of the neurons, as if the system was not just in dreams, but continues to perceive what has just happened. This is important because otherwise the system will not be able to recognize the complex features (Hierarchical Temporal Memory).
1) the first method - a vlob, on C ++ - remember the activation of all neurons, make a prediction, restore the activation.
2) “neural approach”: slow down the current activations, go back a few steps along the memory chain, and then recall them. That is, since all the events of the hippocampus are entered into a single chain, then you have to go back from the last memorization point of the hippocampus, and run 1VTs, by analogy with what was considered in the previous article.
3) “mathematical approach”: in the INS from the end of the cycle there will be no need to restore the context, since the information is stored in a different way, and the context is not lost.
After that, it is necessary to combine forecasting and highlighting generalizations together, and then look at the graphs of the success of the operation (the number of positive outcomes from all possible), the speed of learning, find out whether the INS could learn all the patterns (if the world is limited in the number of such patterns).
Each neuron or cluster and each connection between them is convenient to save the creation time, as described earlier. Initially, this time of creation was equal to the tact of the neural network, and it was just a number. After the complexity of the algorithms, the INS could spend several tacts of thinking for one tact of the external world (for example, perception, prediction, selection), therefore these cycles became out of sync. After a series of consecutive complications, the solution was obtained:
For example, line 136.2 means - the tick of the world 136, and the third tact of thinking within this tick. After that, it became very easy to see that this neurocluster was created during such and such events, and it was created in sleep / prediction / perception mode. You can click the mouse wheel on such a timestamp (as a hyperlink), and the program will show all debugging information - for example, the window with the I / O log scrolls to display those events, the tables with neuroclusters scroll to the clusters that were created at that time, and etc.
Continued - HTM and text
Front end
To be able to easily combine different sets of neural networks and different types of worlds, you need to make their interfaces the same, and in the GUI add the ability to choose what to start with.
The program interface is easiest to do based on strings. At each cycle of the neural network, it requests an array of strings from the “world” - these are the identifiers that are considered to have worked in this cycle. This is a fairly universal solution, which is suitable for many cases: if one line is equal to one letter, then the INS can read texts, if one line is equal to one event identifier of the “Trash3” type, then the INS can live in artificial worlds like the one described above, or more complex. You can even work with a primitive vision, where each pixel has one line, but for a large resolution it is more efficient to use a different approach. Primitive sounds can also be processed by allocating a separate line for each frequency from a small set.
')
With the complexity of the ANN algorithms, it is possible to complicate the worlds.
Since in this world, the INS perceives not the letters directly, but the neurons to which activation from an unknown line has arrived, she will not be able to understand that “the letter T means garbage”, she will have to memorize the consequences of neuron triggering.
Sensory Neurons / Clusters
... can be created in the following ways:
1) if a new sensory string, unknown for ANN, arrived, then immediately create a sensory neuron / cluster for it. In the future, the INS will analyze it as another sign, the number of which it does not care.
2) for some system neurons (such as positive emotions), you will have to query and create them in advance so that the algorithms know where to get the data.
3) if sensory neurons are created “on demand”, an uncomfortable situation can occur when sensory neurons are intermixed with normal ones. In this case, it may be convenient to create all sensory neurons in advance. For example, if it is known that the INS will work with ordinary texts, then the whole alphabet can be requested in advance.
Forecasting
In order to automatically determine which patterns the INS could find and which not, you need to add prediction. Then the INS should choose those actions that will lead to positive signals. According to the statistics of the operation, it will be possible to build graphs of learning speed, automatically determine the passing of tests, etc.
Brute force prediction algorithm
INS predicts each option and chooses the one that will give the best result. This method is suitable for both primitive worlds with a small number of actions (for debugging other algorithms), and for more complex ANNs, when the INS will narrow down the range of actions to some small list that is relevant in this situation and choose between them.
In the most primitive case, the algorithm is organized as follows: after the next step of recognition, enter the cycle of possible actions, select the next action, “go into virtuality”; activate the selected action and remember it, as if it happens in reality. See which chains were recognized - if the selected chains found patterns, then they will work. To do so in several steps, so that the activation proceeds before its consequences - positive or negative emotions. Remember the amount of emotion for all time forecasting. If there is only one emotion - a positive one - then this is simple, otherwise you will have to analyze the vector of different emotions and choose which is better - a bit of negative and how much positive there is or something else.
More realistic algorithms can be obtained if the INS conducts forecasting in parallel with the rest of thinking, without the need to completely “go into virtuality”. It is better to return to these algorithms later.
Context recovery
After the action with the best characteristic is found, the system “goes out of virtuality” and makes it in reality. What is important - after returning from fantasy, it is necessary to restore the state of the neurons, as if the system was not just in dreams, but continues to perceive what has just happened. This is important because otherwise the system will not be able to recognize the complex features (Hierarchical Temporal Memory).
1) the first method - a vlob, on C ++ - remember the activation of all neurons, make a prediction, restore the activation.
2) “neural approach”: slow down the current activations, go back a few steps along the memory chain, and then recall them. That is, since all the events of the hippocampus are entered into a single chain, then you have to go back from the last memorization point of the hippocampus, and run 1VTs, by analogy with what was considered in the previous article.
3) “mathematical approach”: in the INS from the end of the cycle there will be no need to restore the context, since the information is stored in a different way, and the context is not lost.
After that, it is necessary to combine forecasting and highlighting generalizations together, and then look at the graphs of the success of the operation (the number of positive outcomes from all possible), the speed of learning, find out whether the INS could learn all the patterns (if the world is limited in the number of such patterns).
Virtual time
Each neuron or cluster and each connection between them is convenient to save the creation time, as described earlier. Initially, this time of creation was equal to the tact of the neural network, and it was just a number. After the complexity of the algorithms, the INS could spend several tacts of thinking for one tact of the external world (for example, perception, prediction, selection), therefore these cycles became out of sync. After a series of consecutive complications, the solution was obtained:
class NeuroTime {
public:
typedef quint32 Iter;
typedef quint16 Cycle;
protected:
BrainState * _state;
Iter _iter; // in world ticks
Cycle _cycle; // calls f-th and the creation / deletion of cells / c within the current iteration - peace or sleep, is indifferent.
Cycle _nnChanges; // creation / deletion of cells / sv within the current iteration
...
For example, line 136.2 means - the tick of the world 136, and the third tact of thinking within this tick. After that, it became very easy to see that this neurocluster was created during such and such events, and it was created in sleep / prediction / perception mode. You can click the mouse wheel on such a timestamp (as a hyperlink), and the program will show all debugging information - for example, the window with the I / O log scrolls to display those events, the tables with neuroclusters scroll to the clusters that were created at that time, and etc.
Continued - HTM and text
Source: https://habr.com/ru/post/367519/
All Articles