AI learned to identify items in the same style

People without problems find a stylistic similarity between completely different objects. This is a basic property on which interior design is based, for example. An experienced designer will immediately say which furniture fits into the overall style of the room, and which one does not fit. Even if the objects are made of different materials, people still feel that these objects belong to the same style.

Now the computer has learned to “feel” such things.

At the SIGGRAPH 2015 conference, a group of scientists from the University of Massachusetts, led by Evangelos Kalogerakis, presented a new software for 3D modeling, taking into account stylistic similarities. It uses complex methods of geometric comparison and machine learning to successfully emulate the human "sense of style." Thus, now people have a new tool for choosing the situation in the apartment, manipulating with 3D-models. Everyone will be his own designer.
')
Scientific work published on the website of the university.

The authors of the program explain that the stylistic similarity of objects goes beyond the simple similarity in structure. For example, in the illustration in the top row, chest B is more suitable for bed A than chest c . Among the buildings in the bottom row, according to the program, buildings A and C show a stylistic similarity.



The program takes into account three properties of objects: shape, proportion and shape.



Based on these parameters, the similarity coefficient between pairs of objects is calculated. For example, here is how the comparison in form.



Scientists say that this is the first program of this kind. For the initial data collection, they turned to crowdsourcing. Pictures in seven categories - buildings, furniture, lamps, coffee tables, architectural elements of pillars, cutlery and plates - were rated by 2,500 people, including designers and ordinary users through the Amazon Mechanical Turk service. A total of over 50,000 responses were received. People agreed with each other in assessing stylistic similarities in 85% of cases. People agreed with the result of computer analysis in 90% of cases, that is, the program is not inferior in terms of “sense of style” to a person, and even surpasses it.

Kalogerakis and colleagues hope that the developed algorithms will be used in other graphic editors and 3D modeling programs. Perhaps the houses and apartments from this will become more beautiful and harmonious.

Algorithms can be used on the Internet - in search engines for images. As they now find “similar images” in the background and outlines, they will be able to find pictures and three-dimensional objects “in the same style”. In addition, the program handles text requests, such as [find photos of the Gothic church].

The program may be interested in people from the cinema (to create harmonious scenes), the industry of computer games, virtual reality applications.

Algorithm Test Results

PS Links to program code, archive with experimental data, presentation, scientific work and accompanying materials.