Google's Faces API Example

Not so long ago, Google released version 7.8 of the Google Play Services library. Among the major innovations, there is a new Mobile Vision API consisting of two large components: Barcode API for scanning and recognizing various barcode and QR codes and the updated Faces API for searching and tracking faces in pictures. Under the cat we will get acquainted with the Faces API, consider its main features and write a small application.

The Faces API provides the ability to detect and track human faces on the finished images and streams coming from the camera of the device. It copes with this task quite tolerably:



Although in complex cases the algorithm may fail:

First kind error

Second kind error

In addition to simple tracking, the library can determine the orientation of a person in space, calculate the position of the main points of the face (nose, eyes, corners of the mouth) and produce the simplest recognition: determine the likelihood that a person’s eyes are open and the probability of a smile:
')


But enough of the introduction, let's move on to practice. Every minute, Instagram users upload 216 thousand photos . With a finger pointing to the sky, suppose that 40% of them are selfies. Same 1440 pieces per second! Let's try to join the trend and make a small application to automate self. Few mines like sour mines, so we will only shoot smiling faces. The attentive reader will ask, what's the automation? It's simple: in order not to force users to search by touch the shutter button, we will automatically take a picture after a person just winks at the camera.

As a basis we will use the official examples of the Vision API from Google. For simplicity, the issues of drawing frames from the camera are omitted. Interested people can look at the sources on the githaba .

To connect the Faces API to your project, simply add a dependency in the gradle script:

compile 'com.google.android.gms:play-services-vision:7.8.0' 

To start, make a copy of the detector. Selfies are most often individual cases, so we’ll only track the face of one person:

 FaceDetector detector = new FaceDetector.Builder(context) //       .setClassificationType(FaceDetector.ALL_CLASSIFICATIONS) //  ""    .setProminentFaceOnly(true) .setTrackingEnabled(true) .build(); 

Create the "brains" of our detector - processing algorithm. The detector needs a factory that will create an instance of the handler for each face found. Using a pile of antics in front of the phone’s front camera, it was empirically defined: a more or less adequate smile in a photo is 0.4 and higher, and a wink can be formalized like this: “the probabilities that the eyes are open, for the right and left eyes, differ by more than 0.6” . Well, let's implement this algorithm in our handler:

 private class GraphicFaceTracker extends Tracker<Face> { @Override public void onUpdate(FaceDetector.Detections<Face> detectionResults, Face face) { boolean isSmiling = face.getIsSmilingProbability() > 0.4; if (isSmiling) { float leftEye = face.getIsLeftEyeOpenProbability(); float rightEye = face.getIsRightEyeOpenProbability(); if (Math.abs(leftEye - rightEye) >= 0.6) { takeShot(); //   } } } } private class GraphicFaceTrackerFactory implements MultiProcessor.Factory<Face> { @Override public Tracker<Face> create(Face face) { return new GraphicFaceTracker(); } } 

We give our detector a copy of the factory:

  GraphicFaceTrackerFactory trackerFactory = new GraphicFaceTrackerFactory(); MultiProcessor<Face> processor = new MultiProcessor.Builder<>(trackerFactory).build(); detector.setProcessor(processor); 

Now all that is left is to simply create and launch a stream from the camera. Important! Do not forget to stop the camera correctly when we no longer need it. Otherwise, the camera will remain locked, and no application will have access to it until the phone reboots.

 CameraSource cameraSource = new CameraSource.Builder(context, detector) .setFacing(CameraSource.CAMERA_FACING_FRONT) .build().start(); 

So. Compile. We start. We try on colleagues. Everything is working. Rule the bugs. We try on colleagues. Everything is working.



In general, the Faces API leaves a pleasant impression for both the developer and the user. Developers will enjoy the simplicity and ease of development, and users will like the speed and quality of work. We are waiting in the Play Store even more applications with face recognition.

Useful links:
Google Documentation
Application sources