Sketch Image Recognition

A group of researchers from the American University of Brown and the Technical University of Berlin published their results related to the problem of computer recognition of sketches, such as those shown in the picture below. In this case, as a result, not only a long and boring document is given, but also ready-made software samples that can be downloaded and tried.









The purpose of the study was to make possible a “semantic understanding” of a sketch for a computer that must understand what is shown in the figure and suggest options. The task was seriously complicated by the fact that even human perception for such identification often does not work very well: from the created sample of 20,000 sketches, which was used to train the system, people were able to correctly figure out what was in the picture only in 73% of cases.

')

As, for example, in the picture below - what is shown here?







For the recognition of sketches by researchers, a special presentation of sketches was developed in order to facilitate their classification. As a result, it was found that the developed method can recognize the drawing with an accuracy of 56%, which, in principle, is comparable to the human result of 73%.



You can try how it works with the help of three applications - Windows , MacOS and iOS .



It looks approximately like this - in principle, it is not bad if you try to convey the truly characteristic features of the object being drawn, even with unenviable drawing abilities:











Here you can see a PDF document representing the results of the study and test sets of sketches used by researchers.