Create original captcha using neural networks. Part 1
Like all programmers, recently I had a desire to invent my own “bicycle”. So how to reinvent your CMS, Framework, etc. no longer relevant, then my gaze turned on the captcha. It would seem that here you can come up with the original, which only a captcha does not exist: 2D-picture, 3D-picture, sound captcha, the choice of the “correct” picture. But it occurred to me that the creators of the captch think somehow one-sidedly, that is, everyone wants to get a uniquely correct answer from the user (and usually in the test field), and in a simple form, the server only needs to compare the answer with the original data! So I decided to fix this case and create my own “smart” captcha.
My idea is simple: display simple shapes to the user in the form of a captcha, for example a square, a circle, a triangle, and ask the user to draw these very shapes. Of course, 3 figures are not enough, but you can combine and display them in any quantity. And on the server to receive the user's response (and as you already guess), to recognize these images using a neural network (that's why I stopped at 3 figures so far to facilitate the neural network, using only 3 outputs).
Without thinking, I started to implement this idea, and I am going to describe this process in articles ...
Let's start the implementation from the client side.
Paying tribute to fashion, I will use javascript framework jQuery, and I confess, it is pretty much easier for our task. In order to further facilitate the task will be used as a container for drawing.
Once I have already decided to use jQuery, then for the sake of the beauty of the code and convenience, we arrange everything in the form of a plugin, and this experience will also be useful for beginners. In its completed form it will look like this somewhere:
Let's start with an empty "blanks" for the plugin.
Here is a typical “blank” plugin for jquery, which allows you to call internal methods and pass parameters to them, for example:
Let's do our main method init. In it we will build a table with N columns, and in two lines, at the top we will display the pictures, and at the bottom the user will draw them. Hang up all the necessary handlers and put everything in the DOM document.
')
Let's start with the construction of the table. I didn't go the easy way (insert ready html code), knowing what the $ () function is capable of, I built the table on the fly. It turned out a rather non-trivial way to build a table with a variable number of columns:
I hope my method will be useful to someone, especially if you finish the code so that any number of lines will be added.
Now the most interesting thing: to implement the ability to draw on canvas.
The main thing is to get the canvas context object:
And then for drawing lines we need only these methods:
It remains only to properly stick them into the events mousedown, mouseup, mousemove, which we are doing, along the way performing some more actions:
Now the init method of our plugin is ready, so that everything will work, let's add global variables and a function to the plugin:
I will explain why these add. actions that when the user draws in the canvas, all the coordinates for which were drawn (the mouse went) were entered into the binary matrix - pixCanvas [Ncanvas] [x] [y] (“1” - coordinate “drawn”, “0” - not)
Of course, it was much easier to save the image in the canvas in PNG using the method:
It returns the image of the canvas in base64, in the form of data: URL, which could be sent to the server and decoded, but it turns out that the PNG image on the server must be further processed in order to receive input data on our neural network.
It is easier on the client side to immediately create a binary matrix and send it to the server in the following way:
Although of course, you can do without ajax - to create hidden input on the form and put the json string there.
Now we collect all the code for the plugin together:
Of course, much is still “raw”, and I finish everything all the time, but I think the idea is clear. You can see the live work of the plugin here . Another “flaw” of this code is that it only works in current and modern browsers (for example, because of the same canvas or “JSON.stringify ()”), although if you like, you can easily redo everything under IE8-like browsers ...
As further practice showed, the choice of the 100x100 square gives the number of inputs to the neural network - 10,000, which is pretty much a lot. Most likely, the squares for captcha will be smaller (at least 50x50), everything will depend on the processing speed on the server ...
As a neural network implementation in Google, I found “ANN - Artificial Neural Network for PHP 5.x”.
At first it was interesting to use a “live” implementation example in PHP 5.3 using neimspaces, but when I tried to train a network with 10 inputs and 3 outputs, it took more than 10 minutes, I can imagine how much a real sample will be trained. In general, you may even have to reinvent your bike. You can of course take everyone’s well-known FANN library, but firstly it’s hardly friendly with PHP 5.3, secondly there are no compiled solutions on Windows (and they don’t need it), but the hosting provider doesn’t allow delivery, but doesn’t have a VDS yet. And if you distribute captcha, FANN will have to be included in the requirements.
Although it is not necessary to be attached to PHP on the server, the benefit of other server languages is enough ...
But back to our captcha, of course, that in this form it is easily broken by bots - just send 3 “correct” binary matrices to the server (I don’t mention pattern recognition, since the task is pretty simple). Although of course on the server, you can exclude probabilities above 0.95 from the results (it is unlikely that users can draw Picasso and only the bot can draw the mouse correctly), but for reliability, I think, the matrix should be encrypted with the key function that the server will issue ( and not necessarily the same), naturally in unreadable and packaged form.
Now, even if the bot recognizes the correct image - it will not be able to send the correct binary matrix to the server!
Then you can think of a lot more:
Of course, the answer is formed by javascript, at least some protection from primitive spam bots. Until someone deliberately engages in burglary of just this captcha, it is unlikely that some spam bot will bypass it, especially if you foresee the replacement / brute force of the crypto algorithm (which can be invented a great many).
The article turned out, of course, raw, yet the process of writing captcha is still going on and new thoughts constantly arise in my head. So this article is definitely not the last one, and I am ready to listen to your comments and suggestions on how to enhance the protection of the captcha, the “correct” implementation, as well as on the methods of its hacking.
PS Pictures on captcha are drawn in Paint'e just for example! Algorithm generation is not yet invented.
Introduction
My idea is simple: display simple shapes to the user in the form of a captcha, for example a square, a circle, a triangle, and ask the user to draw these very shapes. Of course, 3 figures are not enough, but you can combine and display them in any quantity. And on the server to receive the user's response (and as you already guess), to recognize these images using a neural network (that's why I stopped at 3 figures so far to facilitate the neural network, using only 3 outputs).
Without thinking, I started to implement this idea, and I am going to describe this process in articles ...
Implementation
Let's start the implementation from the client side.
Paying tribute to fashion, I will use javascript framework jQuery, and I confess, it is pretty much easier for our task. In order to further facilitate the task will be used as a container for drawing.
Once I have already decided to use jQuery, then for the sake of the beauty of the code and convenience, we arrange everything in the form of a plugin, and this experience will also be useful for beginners. In its completed form it will look like this somewhere:
<!DOCTYPE html> <html> <head> <title>Demo NNCaptcha</title> <script src="http://code.jquery.com/jquery-latest.js"></script> <script type="text/javascript" src="nnCaptcha.js"></script> </head> <body> <div id="nnCaptcha"></div> <script type="text/javascript"> $(document).ready(function(){ // $("#nnCaptcha").nnCaptcha(); }); </script> </body> </html> Let's start with an empty "blanks" for the plugin.
(function($) { // var methods = { // init:function(params) { var options = $.extend({}, defaults, params); } }; $.fn.nnCaptcha = function(method){ if (this.length != 1) { $.error('not 1 element!'); return; } // if ( methods[method] ) { // , // , // this return methods[ method ].apply( this, Array.prototype.slice.call( arguments, 1 )); } else if ( typeof method === 'object' || ! method ) { // , // return methods.init.apply( this, arguments ); } else { // $.error( ' "' + method + '" ' ); } }; })(jQuery); Here is a typical “blank” plugin for jquery, which allows you to call internal methods and pass parameters to them, for example:
$("#nnCaptcha").nnCaptcha('reset',1); // reset 1 ( “”) Let's do our main method init. In it we will build a table with N columns, and in two lines, at the top we will display the pictures, and at the bottom the user will draw them. Hang up all the necessary handlers and put everything in the DOM document.
')
Let's start with the construction of the table. I didn't go the easy way (insert ready html code), knowing what the $ () function is capable of, I built the table on the fly. It turned out a rather non-trivial way to build a table with a variable number of columns:
// table this.append('<table class="tbCaptcha"><tbody></tbody></table>'); // 2 countCanvas this.find("table.tbCaptcha").find('tbody') .append(function($){ var tr = $('<tr>'); for (i=0;i<countCanvas;i++) tr.append($('<td>') .append($('<img>') .attr('src', 'nnCaptcha.php?image=get') ) ) return tr; }($) ).append(function($){ var tr = $('<tr>'); for (i=0;i<countCanvas;i++) tr.append($('<td>') .append($('<canvas>') .attr('class', 'captcha') ) ) return tr; }($) ); I hope my method will be useful to someone, especially if you finish the code so that any number of lines will be added.
Now the most interesting thing: to implement the ability to draw on canvas.
The main thing is to get the canvas context object:
var ctx = canvas.getContext("2d"); And then for drawing lines we need only these methods:
beginPath(); // moveTo(x, y); // lineTo(x, y); // stroke(); // It remains only to properly stick them into the events mousedown, mouseup, mousemove, which we are doing, along the way performing some more actions:
// canvas this.find("canvas.captcha").each(function(i) { this.width = width; this.height = height; var ctx = this.getContext("2d"); var elem = this; elCanvas[i] = elem; var drawing = false; pixCanvas[i] = createArrayPix(); // $(this).bind("mousedown.nnCaptcha",function(e){ // canvas var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; // canvas pixCanvas[i][x][y] = 1; ctx.beginPath(); ctx.strokeStyle = options.selectedColor; ctx.lineWidth = options.selectedWidth; ctx.moveTo(x, y); drawing = true; elem.style.cursor = 'crosshair'; }); $(this).bind("mouseup.nnCaptcha",function(e){ if (drawing) { var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; ctx.lineTo(x, y); drawing = false; pixCanvas[i][x][y] = 1; } }); $(this).bind("mousemove.nnCaptcha",function(e){ if (drawing) { var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; ctx.lineTo(x, y); ctx.stroke(); ctx.moveTo(x, y); pixCanvas[i][x][y] = 1; } elem.style.cursor = 'crosshair'; }); }); Now the init method of our plugin is ready, so that everything will work, let's add global variables and a function to the plugin:
// var defaults = { selectedColor:'#000000',selectedWidth:1 }; var countCanvas = 3; var elCanvas = []; var width = 100; var height = 100; var pixCanvas = []; function createArrayPix(){ var a=new Array (width); for (i = 0; i < width; i++) { a[i]=new Array (height); for (j = 0; j < height; j++) { a[i][j] = 0; } } return a; } I will explain why these add. actions that when the user draws in the canvas, all the coordinates for which were drawn (the mouse went) were entered into the binary matrix - pixCanvas [Ncanvas] [x] [y] (“1” - coordinate “drawn”, “0” - not)
Of course, it was much easier to save the image in the canvas in PNG using the method:
canvas.toDataURL("image/png"); It returns the image of the canvas in base64, in the form of data: URL, which could be sent to the server and decoded, but it turns out that the PNG image on the server must be further processed in order to receive input data on our neural network.
It is easier on the client side to immediately create a binary matrix and send it to the server in the following way:
ajax:function(n) { $.post("test.php", { matrix: JSON.stringify(pixCanvas[n]) } ); } Although of course, you can do without ajax - to create hidden input on the form and put the json string there.
Now we collect all the code for the plugin together:
(function($) { // var defaults = { selectedColor:'#000000',selectedWidth:1 }; var countCanvas = 3; var elCanvas = []; var width = 100; var height = 100; var pixCanvas = []; function createArrayPix(){ var a=new Array (width); for (i = 0; i < width; i++) { a[i]=new Array (height); for (j = 0; j < height; j++) { a[i][j] = 0; } } return a; } // var methods = { // init:function(params) { // , var options = $.extend({}, defaults, params); // table this.append('<table class="tbCaptcha"><tbody></tbody></table>'); // 2 countCanvas this.find("table.tbCaptcha").find('tbody') .append(function($){ var tr = $('<tr>'); for (i=0;i<countCanvas;i++) tr.append($('<td>') .append($('<img>') .attr('src', 'nnCaptcha.php?image=get') ) ) return tr; }($) ).append(function($){ var tr = $('<tr>'); for (i=0;i<countCanvas;i++) tr.append($('<td>') .append($('<canvas>') .attr('class', 'captcha') ) ) return tr; }($) ); // canvas this.find("canvas.captcha").each(function(i) { this.width = width; this.height = height; var ctx = this.getContext("2d"); var elem = this; elCanvas[i] = elem; var drawing = false; pixCanvas[i] = createArrayPix(); $(this).bind("mousedown.nnCaptcha",function(e){ var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; pixCanvas[i][x][y] = 1; ctx.beginPath(); ctx.strokeStyle = options.selectedColor; ctx.lineWidth = options.selectedWidth; ctx.moveTo(x, y); drawing = true; elem.style.cursor = 'crosshair'; }); $(this).bind("mouseup.nnCaptcha",function(e){ if (drawing) { var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; ctx.lineTo(x, y); drawing = false; pixCanvas[i][x][y] = 1; } }); $(this).bind("mousemove.nnCaptcha",function(e){ if (drawing) { var offset = $(elem).offset(); var x = e.pageX - offset.left; var y = e.pageY - offset.top; ctx.lineTo(x, y); ctx.stroke(); ctx.moveTo(x, y); pixCanvas[i][x][y] = 1; } elem.style.cursor = 'crosshair'; }); }); return this; }, test:function(n,x,y) { alert(pixCanvas[n][x][y]); }, ajax:function(n) { $.post("test.php", { name: JSON.stringify(pixCanvas[n]) } ); }, // reset:function(n) { //alert(n); if (n !== undefined){ var c = elCanvas[n]; var ctx = c.getContext("2d"); ctx.clearRect(0, 0, c.width, c.height); pixCanvas[n] = createArrayPix(); } else { $.each(elCanvas,function(i) { var ctx = this.getContext("2d"); ctx.clearRect(0, 0, this.width, this.height); pixCanvas[i] = createArrayPix(); }); } } }; $.fn.nnCaptcha = function(method){ if (this.length != 1) { $.error('not 1 element!'); return; } // if ( methods[method] ) { // , // , // this return methods[ method ].apply( this, Array.prototype.slice.call( arguments, 1 )); } else if ( typeof method === 'object' || ! method ) { // , // init return methods.init.apply( this, arguments ); } else { // $.error( ' "' + method + '" ' ); } }; })(jQuery); Of course, much is still “raw”, and I finish everything all the time, but I think the idea is clear. You can see the live work of the plugin here . Another “flaw” of this code is that it only works in current and modern browsers (for example, because of the same canvas or “JSON.stringify ()”), although if you like, you can easily redo everything under IE8-like browsers ...
Generalization
As further practice showed, the choice of the 100x100 square gives the number of inputs to the neural network - 10,000, which is pretty much a lot. Most likely, the squares for captcha will be smaller (at least 50x50), everything will depend on the processing speed on the server ...
As a neural network implementation in Google, I found “ANN - Artificial Neural Network for PHP 5.x”.
At first it was interesting to use a “live” implementation example in PHP 5.3 using neimspaces, but when I tried to train a network with 10 inputs and 3 outputs, it took more than 10 minutes, I can imagine how much a real sample will be trained. In general, you may even have to reinvent your bike. You can of course take everyone’s well-known FANN library, but firstly it’s hardly friendly with PHP 5.3, secondly there are no compiled solutions on Windows (and they don’t need it), but the hosting provider doesn’t allow delivery, but doesn’t have a VDS yet. And if you distribute captcha, FANN will have to be included in the requirements.
Although it is not necessary to be attached to PHP on the server, the benefit of other server languages is enough ...
But back to our captcha, of course, that in this form it is easily broken by bots - just send 3 “correct” binary matrices to the server (I don’t mention pattern recognition, since the task is pretty simple). Although of course on the server, you can exclude probabilities above 0.95 from the results (it is unlikely that users can draw Picasso and only the bot can draw the mouse correctly), but for reliability, I think, the matrix should be encrypted with the key function that the server will issue ( and not necessarily the same), naturally in unreadable and packaged form.
Now, even if the bot recognizes the correct image - it will not be able to send the correct binary matrix to the server!
Then you can think of a lot more:
- Distortion of figures to be drawn;
- Different colors in different blocks;
- Protection by session from busting;
- Nested shapes (you need to draw one);
Of course, the answer is formed by javascript, at least some protection from primitive spam bots. Until someone deliberately engages in burglary of just this captcha, it is unlikely that some spam bot will bypass it, especially if you foresee the replacement / brute force of the crypto algorithm (which can be invented a great many).
Conclusion
The article turned out, of course, raw, yet the process of writing captcha is still going on and new thoughts constantly arise in my head. So this article is definitely not the last one, and I am ready to listen to your comments and suggestions on how to enhance the protection of the captcha, the “correct” implementation, as well as on the methods of its hacking.
PS Pictures on captcha are drawn in Paint'e just for example! Algorithm generation is not yet invented.
Source: https://habr.com/ru/post/125204/
All Articles