Development of a "universal" library for creating a neural network (Start)

Prehistory

I, a mere mortal, did not even think about AI as a universal helper, I calmly codilized the AWPs on this field (C / C ++), and everything would be nothing, if not one BUT.
Somehow I met a person on a forum dedicated to the subject of coding for AWP, for quite a long time he explained a lot to me, helped me, + he himself did quite a few interesting things. But as if we were talking about AI, it was then that I was interested in his answer in the style “I have a computer on my computer that what you are interested in is a good debugger, task manager + bot for ICQ, which has already passed the Turing test more than once” . I, of course, didn’t believe how completely adequate a person He refused to give the source code, he promised to give the finished program only if I hacked into his computer and got access to the file system. Several years have passed since then, and the topic has been raised more than once on his AI account. Although I didn’t manage to get a prototype, I’m not sure that he has one at all, but it’s not important, because each of our communication gave birth to the idea of ​​my own AI.

Start

To begin with, it was naturally necessary to compile a list of everything necessary for development, when I chose my favorite C / C ++ as a basis, a friend suggested to me that “you usually don’t write your intellects, and you need to choose the declared language for this”. I didn’t like this idea right away because I’d have to re-learn a new language, completely different in principle, and I decided to take a chance.
So C / C ++ was chosen.
Further, the case stood behind the small (as I then thought) choice of the library at which the development would begin, and here the second pitfall was met. I reviewed many articles on the INS, re-digged many libraries, but did not find the one I needed because either the library was not flexible enough, or it was difficult to understand, or did not satisfy the speed of work. It was certainly another BUT about him later.
So the task was to write your library.

Go

Wednesday I chose Rad Studio , feeling a ton of hate. I will try to explain myself:
I need a fast “form-spat” to test, so as not to spend a lot of time on the interface
It was necessary to ease the transfer to Delphi. many acquaintances chose it as their main language, and Rad Studio is their main medium
I was not going to badly nail the code to the VCL for its easier transfer to gcc, MS VS.
I will try to use the minimum set of tools for easier cross-platform transfer.

So, the main structure of each network is

• Connections
• Neurons
• Layers

And for the task I had to somehow untie them from each other but at the same time make them work together. Immediately, an idea came to mind, Classes, but after a little thought, after weighing everything + for and –– against it, they had to be abandoned. The network must be designed for the number of neurons / layers / connections limited only by memory, + the speed of work must be quite high, so that this network does not become a brake, + the flexibility of the network should allow simple OnLine work with both layers and each neuron in this layer, and with the connections of each neuron separately (This is exactly what was missing in any free library I found).
Coherent lists were chosen for this, and it was decided to use pointers. Let's take a look at the frame structure, we will not fill it yet ...

typedef struct _link{ }LINK; typedef struct _neuron{ struct _neuron *prev,*next; }NEURON; typedef struct _n_layer{ struct _n_layer *prev,*next; }NLAYER; 

Now the idea is how to tie it all up?
After a little thought, I decided to do this: 1 connection block connects 2 neurons;> each communication instance should be easily accessible and relate only to 2 mind specific neurons, but a neuron can have an unlimited number of connections (I tried to make a library without software limitations). Neurons, in turn, for simpler work should be in a layer, it was possible not to use layers, but working with such clusters of neurons would be extremely difficult. But not only the layer should have a pointer to the neurons belonging to it, but each neuron should have a pointer to the layer in which it is located, but this is already needed for a simpler and faster technical work with the network.
')
So the structure migrated to another state more thoughtful, I think.

 typedef struct _link{ void *from,*to; //  LINKL }LINK; typedef struct _link_list{ LINK *link; void *Neuron; //  __ struct _link_list *next,*prev; }LINKL; typedef struct _neuron{ LINKL *in,*out; void *Layer; //  __ struct _neuron *prev,*next; }NEURON; typedef struct _n_layer{ NEURON *first,*end; struct _n_layer *prev,*next; }NLAYER; 

At this stage, I am writing handlers to remove / add an item to the list.
But again, this structure is not filled and it lacks one very important point ...
Let's stop and think, unless you look somewhere you can't think?
Or when you hear something you can't see? The answer is no.
That is why the idea to make threads appeared!

I have not seen this even in paid libraries.
How should this work? It's very simple, all you need is to add 1 parameter to each branch of the main building, this will be the state parameter - it will indicate whether further work is possible with one or another network element.
And again our structure is modified.

 #define STATE_NOT_READY 0 #define STATE_READY 1 typedef struct _link{ void *from,*to; }LINK; typedef struct _link_list{ LINK *link; char state; void *Neuron; struct _link_list *next,*prev; }LINKL; typedef struct _neuron{ char state; LINKL *in,*out; void *Layer; struct _neuron *prev,*next; }NEURON; typedef struct _n_layer{ NEURON *first,*end; char state; struct _n_layer *prev,*next; }NLAYER; 

Voila, the frame of the neuro network is ready.

Now let's move on to more tasty buns, and in order not to turn out to be a hollow sound, I am putting to you to study a ready-made library written by me with respect to the respected Ilya90 respect to him!

NewNeuroLib Tonich

Pass: XabraZabra

Conclusion

This article describes my approach to creating a “smart” neural network, now it only remains to figure out how to fill each branch, most likely I will choose a double data type, but 3 things are not clear to me and until I understand them, the prototype will not see the light .

1. (decided) According to which condition a new connection is created and / or how it is chosen from which neuron to which one it will lead.
2. (decided) By what condition is a new neuron created (from what and why is it created)
3. (partially decided) When should a neuron "shoot"? When are all the link weights placed or not necessary, and if so, what will the function look like that can work with a constant different number and value of the link weights?

If these points are resolved in the next release, I will analyze the work of this network and provide the full code of the example.

If there is interest, I can lay out an example of use, but without the SaveLoad library (I haven’t received a Nobel for it yet)

Welcome to the comments!

Materials studied

Of course, many articles on Habré.
FANN-library on which focused the most.
The site from which I read a lot of articles.
A very interesting article that made me think ((: