Learning neural network geometry
Good day, users of Habr and just guests. I would like to share with you the experience of working with neural networks.
Let's start with the background. I study in the first year of the Novosibirsk University, the summer session is suitable and I had to take course work. The topic of the course work I could choose is quite simple and easy to understand program in c ++, whether it is a calculator of long numbers or a keyboard simulator. But, having experience webdev, I decided that the theme of my course work will be "Simplest neural networks for pattern recognition". Immediately I remembered the wonderful FANN library, and without hesitation I set to work. I decided to teach the neuron to distinguish 3 figures: a circle, a square and a triangle.
I would like to break up my further description into several key points:
')
As soon as the library was installed (I think there are no problems for anyone, for there are so many lessons on installing FANN), I began to squeeze and paw it. Tested functions, studied the theory of neurons and neural networks in general.
The only thing I could find on this topic is a couple of lessons on Habré, which are outdated because the functions and syntax of FANN has changed.
Of all these lessons, one can single out the main thing. That without which it is impossible to work normally with neural networks in general.
First and foremost, you need to understand what a neural network is. Understand the principle. I would love to write everything here, but there are already so many lessons on this topic. I can advise: Neural networks for beginners .
The second and important - neural networks are both simple and complex.
Well, in general, probably everything. I will describe further how the neural network is built and describe my first successful experience.
My joy knew no bounds when my neural network gave an adequate response. I originally created the class. In this class, I used a fuck function that teaches a neural network to determine a shape. More precisely, it simply adds it to the "line for study."
The principle of this function is very simple. Cycle enumerates each pixel and writes it to an array.
For figures, colors are not important, so I decided, once again not to confuse the network, to make all the images black and white. Then I realized that the images can be of different sizes, so I created a second function that changes the image size to 16x16 pixels.
Immediately, in order not to deviate from the topic, I post the code that is responsible for processing the photo that needs to be recognized. The principle is the same.
The most interesting begins - the training of the neural network. I decided to implement the following. in the following way:
Here I will analyze in more detail some points.
$ ann = fann_create_standard_array (3, array (256, 128, $ num));
We create a neural network with 3 layers of neurons.
1 layer - 256 neurons, so to say the input layer. Why 256 is very simple - 16 pixels * 16 pixels = 256 pixels.
2 layer - 128 neurons, so to speak intermediate layer.
Layer 3 - the number of neurons, equal to the number of figures in the "queue". This is the layer that the neural network returns.
Second moment.
Fann_train ($ ann, $ tonn [$ key] [$ s], $ arout [$ key]) - training the neural network.
Here I think everything is clear, this training is in a cycle, since the neural network works on probabilities and in order to achieve a result, cyclical training is needed.
Well, in the end, we save all of our "base" in the file base / base.ann. This is done for quick use of the network in the future without learning.
Next in the class is the f-Ia "think" and f-Ia processing of the results. They look like this:
Everything seems to be clear here, I don’t think there will be any difficulties. Just want to clarify, the second function simply takes the maximum result and returns the array index.
This is how learning happens in practice:
In the folders for training, I uploaded photos of these figures.
Well, the very recognition of the image, nothing complicated:
There is also the usual form, I think it is so obvious. After starting the network, I got the right answer. But sometimes mistakes still occur, which are corrected only by “feeding” the network with new and new images of figures.
The end result can be seen in my github repository .
Neural networks produced a very mixed result, more similar to random numbers.
As decided: I just reduced the image and made them black and white. If your neural network gives out different numbers, then it has learned the wrong way, however trite it may sound. The algorithm, according to which you will build arrays with which you will stuff your network in the future, will come up with you.
Neural networks gave 99% wrong answer.
How decided: fann_train. The problem was exactly here. I made two mistakes while trying to train the network.
1 time I downloaded all the figures at once with one function. This can not be done, you need to do so, as I did in my project.
2 times I tried to train the network without a loop, only once drove the train. So, too, can not be done, because the network is not enough data.
It seems that I have not met more errors. Useful links that helped me in creating this course work:
Let's start with the background. I study in the first year of the Novosibirsk University, the summer session is suitable and I had to take course work. The topic of the course work I could choose is quite simple and easy to understand program in c ++, whether it is a calculator of long numbers or a keyboard simulator. But, having experience webdev, I decided that the theme of my course work will be "Simplest neural networks for pattern recognition". Immediately I remembered the wonderful FANN library, and without hesitation I set to work. I decided to teach the neuron to distinguish 3 figures: a circle, a square and a triangle.
I would like to break up my further description into several key points:
')
- My attempts to deal with the library
- The first successful experience
- Final result
- The cones that I filled
My attempts to deal with the library
As soon as the library was installed (I think there are no problems for anyone, for there are so many lessons on installing FANN), I began to squeeze and paw it. Tested functions, studied the theory of neurons and neural networks in general.
The only thing I could find on this topic is a couple of lessons on Habré, which are outdated because the functions and syntax of FANN has changed.
Of all these lessons, one can single out the main thing. That without which it is impossible to work normally with neural networks in general.
First and foremost, you need to understand what a neural network is. Understand the principle. I would love to write everything here, but there are already so many lessons on this topic. I can advise: Neural networks for beginners .
The second and important - neural networks are both simple and complex.
Well, in general, probably everything. I will describe further how the neural network is built and describe my first successful experience.
The first successful experience
My joy knew no bounds when my neural network gave an adequate response. I originally created the class. In this class, I used a fuck function that teaches a neural network to determine a shape. More precisely, it simply adds it to the "line for study."
/** * * * @param string $name * @param string $dir * @param array &$newh * * @return array */ public function addFigure($name, $dir, &$tonn) { $j = 0; $d = dir($dir); while($entry = $d->read()) { if ( preg_match("/jpg/", $entry) ) { $im = $this->imageresize($dir . $entry,16,16,75); imagefilter($im, IMG_FILTER_GRAYSCALE); $cur_array = array(); $cnt = 0; for($y=0; $y<16; $y++) { for($x=0; $x < 16; $x++) { $rgb = imagecolorat($im, $x, $y) / 16777215; $rgb = round($rgb, 6); $cur_array[$cnt] = $rgb; $cnt++; } } imagedestroy($im); $tonn[$name][$j] = $cur_array; $j++; } } return $tonn; } The principle of this function is very simple. Cycle enumerates each pixel and writes it to an array.
For figures, colors are not important, so I decided, once again not to confuse the network, to make all the images black and white. Then I realized that the images can be of different sizes, so I created a second function that changes the image size to 16x16 pixels.
Immediately, in order not to deviate from the topic, I post the code that is responsible for processing the photo that needs to be recognized. The principle is the same.
/** * * * @param string $img * * @return array */ public function input($img) { $Input = array(); $im = $this->imageresize($img,16,16,75); imagefilter($im, IMG_FILTER_GRAYSCALE); $cnt = 0; for($y=0; $y<16; $y++) { $count = 0; for($x=0; $x < 16; $x++) { $rgb = imagecolorat($im, $x, $y) / 255; $rgb = round($rgb, 6); $Input[$cnt] = $rgb; $cnt++; } } $count = 0; imagedestroy($im); return $Input; } The most interesting begins - the training of the neural network. I decided to implement the following. in the following way:
public function teach($tonn) { $num = count($tonn); $ann = fann_create_standard_array(3, array(256, 128, $num)); for ($i=0; $i < 10000; $i++) { $j= 0; $arout = array(); foreach ($tonn as $key => $value) { for($k = 0; $k<count($tonn);$k++){ if($k==$j){ $arout[$key][$k] = 1; }else{ $arout[$key][$k] = 0; } } $j++; for ($s=0; $s < count($tonn[$key]); $s++) { fann_train($ann, $tonn[$key][$s], $arout[$key]); } } } fann_save($ann,"base/base.ann"); fann_destroy($ann); return "base/base.ann"; } Here I will analyze in more detail some points.
$ ann = fann_create_standard_array (3, array (256, 128, $ num));
We create a neural network with 3 layers of neurons.
1 layer - 256 neurons, so to say the input layer. Why 256 is very simple - 16 pixels * 16 pixels = 256 pixels.
2 layer - 128 neurons, so to speak intermediate layer.
Layer 3 - the number of neurons, equal to the number of figures in the "queue". This is the layer that the neural network returns.
Second moment.
Fann_train ($ ann, $ tonn [$ key] [$ s], $ arout [$ key]) - training the neural network.
Here I think everything is clear, this training is in a cycle, since the neural network works on probabilities and in order to achieve a result, cyclical training is needed.
Well, in the end, we save all of our "base" in the file base / base.ann. This is done for quick use of the network in the future without learning.
Next in the class is the f-Ia "think" and f-Ia processing of the results. They look like this:
/** * * * @param string $file * @param string $img * * @return array */ public function think($file, $img) { $ann = fann_create_from_file($file); $output = fann_run($ann, $img); return $output; } /** * * * @param array $output * * @return array */ public function showResult($output) { foreach($output as $k => $v) { if( ! is_numeric($v)) return array(); if( ! isset($max)) { $max = $v; continue; } if($v > $max) $max = $v; } $arr = array(); foreach($output as $k => $v) { if($v == $max) $arr[] = $k; } return $arr; } Everything seems to be clear here, I don’t think there will be any difficulties. Just want to clarify, the second function simply takes the maximum result and returns the array index.
This is how learning happens in practice:
$nn = new NeuralNetwork; /* */ if(isset($_GET['teach'])){ $tonn = array(); $nn->addFigure("triangle", "teach/triangle/", $tonn); $nn->addFigure("circle", "teach/circle/", $tonn); $nn->addFigure("square", "teach/square/", $tonn); $ann = $nn->teach($tonn); } In the folders for training, I uploaded photos of these figures.
Well, the very recognition of the image, nothing complicated:
if(isset($_POST['checkfigure'])){ $uploaddir = 'uploads/'; $uploadfile = $uploaddir . basename($_FILES['getimage']['name']); move_uploaded_file($_FILES['getimage']['tmp_name'], $uploadfile); copy("uploads/".$_FILES['getimage']['name'], "thumbs/".$_FILES['getimage']['name']); $Input = $nn->input("thumbs/".$_FILES['getimage']['name']); $output = $nn->think("base/base.ann", $Input); $result = $nn->showResult($output); switch ($result[0]) { case '0': $answer = ""; break; case '1': $answer = ""; break; case '2': $answer = ""; break; default: $answer = ""; break; } echo $answer; } There is also the usual form, I think it is so obvious. After starting the network, I got the right answer. But sometimes mistakes still occur, which are corrected only by “feeding” the network with new and new images of figures.
Final result
The end result can be seen in my github repository .
The cones that I filled
Neural networks produced a very mixed result, more similar to random numbers.
As decided: I just reduced the image and made them black and white. If your neural network gives out different numbers, then it has learned the wrong way, however trite it may sound. The algorithm, according to which you will build arrays with which you will stuff your network in the future, will come up with you.
Neural networks gave 99% wrong answer.
How decided: fann_train. The problem was exactly here. I made two mistakes while trying to train the network.
1 time I downloaded all the figures at once with one function. This can not be done, you need to do so, as I did in my project.
2 times I tried to train the network without a loop, only once drove the train. So, too, can not be done, because the network is not enough data.
It seems that I have not met more errors. Useful links that helped me in creating this course work:
Source: https://habr.com/ru/post/328698/
All Articles