Working with metasecond structures in Python - MetaNet library

When you see the only solution, ask others.

In this article I would like to talk about some of the prerequisites for the emergence of a tool for modeling metasystems.

Learning automation

Initially, there was a problem of automating the learning of artificial neural networks with certain time constraints. On the way of its solution, an approach to the use of opposed neural networks was proposed [1]. The point is to train two networks, one as usual:


')
And the second on the reverse reference outputs:



Where Is the reference set of outputs, y is the network output, ε is the deviation value (error), N is the number of neurons in a given layer, L is the number of the output layer, and n is the point in time. Then, to determine the saturation of one network or hit the local minimum - we could compare it with the opposite one. And in the case of a significant difference, do not waste time on the next training. But a new question arises - how to produce such an analysis? Then we decided to determine the accuracy of the network responses by the value of the variance of the responses. The expediency of such a decision was due to empirical observation — when it is saturated or hit the local minimum — the output values ​​differ slightly. Having built models, we have found that in some cases this approach is justified. But then another question appeared - and how to determine which network of input vectors is wrong about the network? Or - how should the degree of confidence in the responses of the network, for example, in classification problems, change when approaching specific classes of recognizable objects?

Confidence in network responses

Intuitively, we simply decided to entrust this task to another National Assembly by constructing it in such a way that its inputs are the states of all neurons of the observed network.

Then, having trained the NA, we can form a bank of its answers on the test sample. Knowing the errors and the network state matrix - to train the metaset so that it would classify errors during the operation of the core network.



As a result, we received the following schedule:



The presented graphs give reason to believe that with the help of a metanet you can build an analogue of confidence intervals. MetaOut in Figure 2 represents the output of the metasnet. It can be seen that the closer the sample sample is to the one on which the core network is mistaken, the higher the metasnet estimates the error. Err on the graph is the modulus of the difference between the target output and what the core network provides. It is worth noting that the data was provided to the network sequentially. Thus, the graph can be divided into 3 areas (by subclasses of the Iris set) [2].

Metas

But in reality, my free time occupied the issue of knowledge in the NA [3]. Turning to the question of knowledge, one cannot but turn to I. Kant. Exploring the nature of knowledge, Kant defined it as a judgment. Judging about the subject synthetically, we attach to it a sign, or trait, that is not directly in it. If we judge the subject analytically, we confine ourselves to its concept and do not add anything to it that would not be in it. Thus, it is possible to define knowledge as a synthetic judgment, since we call knowledge only that which expands our knowledge of the subject. Further, synthetic thinking can be divided into the knowledge borrowed from experience (a posteriori) and independent (a priori). Of course, the possibility of a priori synthetic knowledge was considered in detail by Kant from the point of view of philosophy; we took the liberty to consider the possibility of the latter from the point of view of neural networks (NN). For more information, you can refer to the work [3]. The main idea of ​​using meta networks is to try to explain the work of the National Assembly. Thus, by introducing a mutual correspondence between the multitude of states of the NN and the activity of an element of the metaset, we can design networks, and by introducing an unambiguous one, we can try to trace the logic of the output of a network response.

The second question related to this topic is consciousness. A number of questions arose when Thomas Metzinger appeared in the outlook. The question was whether its model of consciousness can be represented as the activity of an element of the metaset.

Seeing that the existing solutions for modeling the NA to adapt to the task is quite difficult - it was decided to write a small library. This is how the MetaNet project appeared. Now he is in the stage of deep alpha. Let us examine one example. First, create multilayer perceptrons:

out_net = FeedforwardNetwork(2, [4], 2) inp_net_1 = FeedforwardNetwork(2, [4], 2) inp_net_2 = FeedforwardNetwork(2, [4], 2) 

We will train them on XOR, after which we will create a metaset -

 metanet = MetaNet() metanet.add_input_net(inp_net_1) metanet.add_input_net(inp_net_2) metanet.add_output_net(out_net) metanet.connect_nets(inp_net_1, out_net) metanet.connect_nets(inp_net_2, out_net) metanet.connect_nets(out_net, inp_net_1) metanet.connect_nets(out_net, inp_net_2) 

Generally speaking - now there are two functions - test and simulate. The first propagates the signal until all the output networks are activated (you can set the maximum number of iterations). The second - allows you to walk the signal as the user allows.



If the signal [[1.0, 0.0], [0.0, 0.0]] is applied to such a metanet, then the first network will give 1, the second 0 - and the output network, respectively 1. So far we have not implemented the neuron-neuron connection between the networks, since By default, the connections are injective — we added one dummy output, which is always 0. When the signal passes, the neurons in the post-network are assigned the largest value between the current state and the proposed state.

When using the test function, the network, despite cyclic connections, will return the output when out_net is activated. If you use the simulate function, you can observe the following picture of the states of the network outputs

	t1	t2	t3	t4
inp_net_1	[ten]	[0, 0]	[ten]	[0, 0]
inp_net_2	[0, 0]	[0, 0]	[ten]	[0, 0]
out_net	[0, 0]	[ten]	[0, 0]	[0, 0]

It is worth noting that by passing the signal through the network - the inputs and outputs of this network are reset.

findings

The article shows the context in which the MetaNet library is being developed. I would like to ask habrahabr users to criticize both ideas and the library. I would like at an early stage of development to take into account the possible paths of development, which I probably overlook now. With regards to the main requirements for the code - readability. I would like to make a tool for research, but losing sight of the performance is also not an option. Now using, for example, mnist, the learning time increases to unacceptably large values.

The library is available at the following link .

Literature

• Otradnov K.K., Evtushenko G.I. Opposite artificial neural networks / Collection of scientific papers on the materials of the III International Scientific and Practical Conference "Theoretical and applied aspects of modern science." - c. 141-144. - 2014.
• Evtushenko G.I., Otradnov K.K. Metaseteva approach to the identification of errors of artificial neural networks / Collection of scientific papers based on the materials of the VII International Scientific and Practical Conference "Theoretical and applied aspects of modern science." -with. 32-35 -2015.
  
• Evtushenko G.I., Otradnov K.K. On the problem of cognition in neural networks / Collection of scientific papers based on the materials of the VII International Scientific and Practical Conference "Theoretical and applied aspects of modern science." -with. 28-32 -2015.