Facebook has improved machine learning algorithms, users are scared
On Reddit and HN, there was a big discussion on the latest achievements of Facebook machine learning. Even experienced developers just can not understand how Facebook "guesses" the specific facts, which, like, can not know.
The final topic of discussion was geographical prompts that Facebook displays when uploading photos to the site. Often, he accurately guesses exactly where the pictures were taken, accurate to the street and the specific building. At the same time, there are no EXIF headers in the photos, there is no GPS on the camera, the files were uploaded to the site from another location (that is, geotagging does not work), and the pictures show, for example, a newborn in the maternity hospital or blurred nightclub interiors (that is, photos were taken inside building). Contextual tagging should not work either - no faces or inscriptions are recognized in the photos.
So far, there are several working hypotheses. Perhaps the methods of the recently purchased startup Color are used here. In addition, Facebook can draw some conclusions from the history of your movements on the GPS coordinates from another device (not from the camera, but from the phone) and by statistical analysis of the text on all web pages that you read, because on most notable sites Facebook Like Trojan buttons are installed, which send information about page visits, even if the user has not pressed a button.
In the case of a photograph of a newborn from the hospital, Facebook could capture data from a smartphone — most likely, a Facebook application was installed on the user's phone, which prompted the GPS coordinates of the last places the user visited.
')
In other cases, Facebook can use several photo analysis methods at once. For example, the location of this photo on Facebook accurately recognized a specific location in Costa Rica without any EXIF or GPS data. Perhaps he analyzed the neighboring photos from the same album.
In other cases, Facebook recognizes places in photos taken two years ago in a completely different country. Here you can assume the search for similar photos from other users who photographed the same landscape and published its coordinates.
In general, accumulating a huge array of statistical information about billions of users, machine learning algorithms will someday be able not only to determine the location of photographs taken, but also with a high degree of probability to predict the fate of a particular person. This is somewhat reminiscent of the Laplace demon : having information about the position and direction of all atoms in the universe, it is alleged that the future of the universe can be accurately predicted. Similarly, Facebook, having information about all the actions of an individual and the read texts, can theoretically predict the future actions of an individual - if you believe in such a degree of determinism.
As a counter, users suggest uploading photos on Facebook with fake EXIF headers to knock down an image-recognition system linked to the terrain. By coordinated efforts on the model of a Google bombing you can make them believe that object X is in the village of Gadyukino - and for everyone who uploads a photo of object X, Facebook will offer the option of Gadyukino. Suppose that the Eiffel Tower does not work, but with some rare object you can try.
The final topic of discussion was geographical prompts that Facebook displays when uploading photos to the site. Often, he accurately guesses exactly where the pictures were taken, accurate to the street and the specific building. At the same time, there are no EXIF headers in the photos, there is no GPS on the camera, the files were uploaded to the site from another location (that is, geotagging does not work), and the pictures show, for example, a newborn in the maternity hospital or blurred nightclub interiors (that is, photos were taken inside building). Contextual tagging should not work either - no faces or inscriptions are recognized in the photos.So far, there are several working hypotheses. Perhaps the methods of the recently purchased startup Color are used here. In addition, Facebook can draw some conclusions from the history of your movements on the GPS coordinates from another device (not from the camera, but from the phone) and by statistical analysis of the text on all web pages that you read, because on most notable sites Facebook Like Trojan buttons are installed, which send information about page visits, even if the user has not pressed a button.
In the case of a photograph of a newborn from the hospital, Facebook could capture data from a smartphone — most likely, a Facebook application was installed on the user's phone, which prompted the GPS coordinates of the last places the user visited.
')
In other cases, Facebook can use several photo analysis methods at once. For example, the location of this photo on Facebook accurately recognized a specific location in Costa Rica without any EXIF or GPS data. Perhaps he analyzed the neighboring photos from the same album.
In other cases, Facebook recognizes places in photos taken two years ago in a completely different country. Here you can assume the search for similar photos from other users who photographed the same landscape and published its coordinates.
In general, accumulating a huge array of statistical information about billions of users, machine learning algorithms will someday be able not only to determine the location of photographs taken, but also with a high degree of probability to predict the fate of a particular person. This is somewhat reminiscent of the Laplace demon : having information about the position and direction of all atoms in the universe, it is alleged that the future of the universe can be accurately predicted. Similarly, Facebook, having information about all the actions of an individual and the read texts, can theoretically predict the future actions of an individual - if you believe in such a degree of determinism.
As a counter, users suggest uploading photos on Facebook with fake EXIF headers to knock down an image-recognition system linked to the terrain. By coordinated efforts on the model of a Google bombing you can make them believe that object X is in the village of Gadyukino - and for everyone who uploads a photo of object X, Facebook will offer the option of Gadyukino. Suppose that the Eiffel Tower does not work, but with some rare object you can try.
Source: https://habr.com/ru/post/135048/
All Articles