Geekly Articles each Day
📜 ⬆️ ⬇️

We realize video calls in an iOS application (by the example of a children's monitor and without WebRTC)

image In this post we will talk about how to write an application - baby monitor, when you install one device (tablet) near the child’s crib, and take the second (phone) with you, say to the kitchen, and occasionally look after the child through the screen .

As a new parent, I want to say that such an application saves a lot of nerves - no need to listen to every rustle or children's cry from the street, you can make sure with a glance that everything is in order with the child. A little about the technical part: the application uses our iOS video chat library, including the server part (signaling and TURN server for NAT traversal), this is all in the public domain. The video stream will work both through Wi-Fi, and through 2G / 3G / 4G. Until recently, in the appstore, there was no application for a children's video monitor that would work via the mobile Internet (apparently due to difficulties with NAT traversal), but while we prograstirovali prepared the post, one of the application leaders released a paid version with support for this functionality. In any case, the article will be useful to you if you want to file video monitoring or two-way video call in your iOS application. We specifically indicate that this is a version without WebRTC, because we are going to write about the web-compatible version (as well as about Android) separately, there are some nuances.

TK:
In our case, the application is a monitoring of young children (infants) through a mobile device running iOS. At the start, the application had to find a neighboring device, synchronize with it, and then make a video call. During the connection, the parent sees the child, and can also control the device on that side - turn on the light (flash), play a lullaby, talk there in the microphone.
')
Actually, the project is not difficult, the main difficulties lay in the implementation of 2 points:

Consider these points in more detail:

Search and sync devices


Synchronization occurs via Wi-Fi or Bluetooth. Googling, found 4 ways to do it. Here is a brief description, advantages and disadvantages:

  1. Bonjour service - synchronization via Wi-Fi. It is easy to find such a sample on the Internet. Works on iOS 6-7
  2. Core Bluetooth - works, no matter how unexpectedly it sounds, via Bluetooth with iOS 5 or higher. But here's the nuance - only Bluetooth 4 LE is supported.
  3. GameKit . Cool stuff. In principle, everything is just like a door. It works fine on ordinary bluetooth (for iPhone 4 devices and even lower). Also works Bonjour - and for WiFi networks. But there is a small drawback - deprecated since iOS 7.
  4. Multipeer Connectivity - a new framework added in iOS 7. In fact, for us it looked like an analogue of GameKit, only for iOS 7. We used it in the future.

We were able to encapsulate these services under one interface and it does not matter which of these four services we chose to use in the end. It turned out to be very convenient.

The general interface of such a service looks like this (the “BB” prefix is ​​from our application name, you can, of course, call it something different):

#import "BBDeviceModel.h" typedef enum CommonServieType { CommonServiceTypeBonjour = 0, CommonServiceTypeBluetoothLE, CommonServiceTypeGameKitWiFi, CommonServiceTypeGameKitBluetooth, CommonServiceTypeMultipeer, }CommonServieType; @protocol BBCommonServiceDelegate; @interface BBCommonService : NSObject @property (nonatomic, weak) id<BBCommonServiceDelegate> delegate; -(void) setConnectionType:(CommonServieType)type; -(void) startServerSide; -(void) stopServerSide; -(void) startSearchDevices; -(void) stopSearchDevices; -(void) selectDevice:(BBDeviceModel *)deviceModel; -(void) clean; @end @protocol BBCommonServiceDelegate <NSObject> @optional -(void) service:(BBCommonService *)servie didFindDevice:(BBDeviceModel *)device; -(void) service:(BBCommonService *)servie didRemoveDevice:(BBDeviceModel *)device; -(void) service:(BBCommonService *)servie serverDidFinishedSync:(BOOL)isFinished; -(void) service:(BBCommonService *)servie clientDidFinishedSync:(BOOL)isFinished; @end @interface BBDeviceModel : NSObject @property (nonatomic, strong) id device; -(NSString *)deviceName; -(void)setDeviceName:(NSString *)name; -(BOOL) isDeviceEqualTo:(id)otherDevice; @end

Then we inherit from the BBCommonService depending on the type of connection and override the start- and stop-, clean methods, and then in the right places we call the delegate methods.

Video chat


For video, we used QuickBlox. First you need to register - as a result you will get access to the admin panel. In it, you create your application. Then download the framework itself from the official site . Connection is described in more detail here - http://quickblox.com/developers/IOS-how-to-connect-Quickblox-framework . In short, then:

1) Download Quickblox.framework, add to the project, connect 15 pieces of libraries - they are listed in the tutorial
2) After that, you need to return to the admin panel, select your application and copy three parameters - Application id, Authorization key and Authorization secret to the project settings:

 [QBSettings setApplicationID:APP_ID]; [QBSettings setAuthorizationKey:AUTH_KEY]; [QBSettings setAuthorizationSecret:AUTH_SECRET];

Everything, now it is possible to work.

1. Session

In order to make client-server interactions with QuickBlox, you need to create a session. This is done very simply:

 [QBAuth createSessionWithDelegate:self];

Thus, a request to create a session is sent and the response comes to the delegate method:

 - (void)completedWithResult:(Result *)result { QBAAuthResult *authResult = (QBAAuthResult *)result; if ([authResult isKindOfClass:[QBAAuthResult class]]) { // do something } }


2. Create a user or login.

For further work, we need a user. Without it, nowhere. This is also done quite simply:

 // registration QBUUser *user = [QBUUser new]; user.password = aPass; user.login = aLogin; [QBUsers signUp:user delegate:self];

or

 // login [QBUsers logInWithUserLogin:aLogin password:aPass delegate:self];

For these and all requests, a response from the server comes to the completedWithResult: delegate method completedWithResult:

Accordingly, the username / password can be taken from UITextFields if desired. In our case, in order not to force the user to add anything else, we did a hidden authorization, so we created a login and password based on vendorID.

3. Storing pair information

After synchronization, we decided to create the Pair entity in which to store our id and opponent (second device synchronized with this one). Also, it did not prevent to send it somewhere to the server in order not to do synchronization in the future. The Custom Objects module, which is essentially a database with custom fields, helped us with this. So, it looked like this:

 QBCOCustomObject *customObject = [QBCOCustomObject customObject]; customObject.className = @"Pair"; // Object fields [customObject.fields setObject:@(userID) forKey:@“opponentID”]; customObject.userID = self.currentUser.ID; // permissions QBCOPermissions *permissions = [QBCOPermissions permissions]; permissions.readAccess = QBCOPermissionsAccessOpen; permissions.updateAccess = QBCOPermissionsAccessOpen; customObject.permissions = permissions; [QBCustomObjects createObject:customObject delegate:self]; - (void)completedWithResult:(Result *)result { QBCOCustomObjectResult *coResult = (QBCOCustomObjectResult *)result; if ([authResult isKindOfClass:[QBCOCustomObjectResult class]]) { // do something QBCOCustomObjectResult *customObjectResult = (QBCOCustomObjectResult *)result; BBPair *pair = [BBPair createEntityFromData:customObjectResult.object]; self.currentPair = pair; // .. } }

The only thing - here you need to go to the admin panel and in the Custom Objects tab create the corresponding model with fields. Everything is very simple and intuitive there, so I’ll not give an example (what you need to pay attention to - the supported data types for the fields are integer, float, boolean, string, file).

If you need to get some entities from the database, this is done as follows -

 NSMutableDictionary *getRequest = [NSMutableDictionary dictionary]; [getRequest setObject:@(self.currentUser.ID) forKey:@"user_id[or]"]; [getRequest setObject:@(self.currentUser.ID) forKey:@"opponentID[or]"]; [QBCustomObjects objectsWithClassName:@"Pair" extendedRequest:getRequest delegate:self];

This request searches for all entities where this user is either the current user or opponent.

Delete a custom object is even easier - you just need to know its ID.

 NSString *className = @"Pair"; [QBCustomObjects deleteObjectWithID:self.currentPair.pairID className:className delegate:self];

For us, this is necessary when the user wants to unsynchronize his ipad / ipod / iphone then, for example, to connect him later with another device. In the application, we have provided for this the “Unpair” button in the Settings interface.

4. Video broadcast

Here is a little more complicated. First of all, we must, besides creating the session and login, also log in to the chat, because chat server is used for video signaling. This is done as follows -

 [QBChat instance].delegate = self; [[QBChat instance] loginWithUser:self.currentUser]; // self.currentUser - QBUUser

thus, we take the current user and log in to the chat, after setting the delegate. If everything is good, then almost immediately one of the methods will work:

 -(void)chatDidLogin { self.presenceTimer = [NSTimer scheduledTimerWithTimeInterval:30 target:[QBChat instance] selector:@selector(sendPresence) userInfo:nil repeats:YES]; } -(void)chatDidNotLogin { } -(void)chatDidFailWithError:(NSInteger)code { }

depending on the outcome. We also immediately send a presence to the timer in case of a successful login. Without them, we will automatically go offline in about a minute.

If everything went well, then you can proceed to the hardest part. To work with video calls, we are offered the QBVideoChat class.

The “calling party” first creates an instance using

 self.videoChat = [[QBChat instance] createAndRegisterVideoChatInstance];

To do this, set up a view for yourself and your opponent, if necessary, the sound state (on / off) and additional settings - for example, useBackCamera:

 self.videoChat.viewToRenderOwnVideoStream = myView; self.videoChat.viewToRenderOwnVideoStream = opponentView; self.videoChat.useHeadphone = NO; self.videoChat.useBackCamera = NO; self.videoChat.microphoneEnabled = YES;

and make a call:

 [self.videoChat callUser:currentPair.opponentID conferenceType:QBVideoChatConferenceTypeAudioAndVideo];


The next step is implementing delegate methods according to behavior. If everything is successful, the opponent should work out the following method:

 -(void) chatDidReceiveCallRequestFromUser:(NSUInteger)userID withSessionID:(NSString *)_sessionID conferenceType:(enum QBVideoChatConferenceType)conferenceType { self.videoChat = [[QBChat instance] createAndRegisterVideoChatInstanceWithSessionID:_sessionID]; // video chat setup self.videoChat.viewToRenderOwnVideoStream = nil; self.videoChat.useHeadphone = NO; self.videoChat.useBackCamera = NO; if (self.videoSide == BBVideoParentSide) { self.videoChat.viewToRenderOpponentVideoStream = self.renderView; self.videoChat.viewToRenderOwnVideoStream = nil; self.videoChat.microphoneEnabled = NO; }else if (self.videoSide == BBVideoChildSide) { self.videoChat.viewToRenderOpponentVideoStream = nil; self.videoChat.viewToRenderOwnVideoStream = self.renderView; self.videoChat.microphoneEnabled = NO; } BBPair *currentPair = [QBClient shared].currentPair; [self.videoChat acceptCallWithOpponentID:currentPair.opponentID conferenceType:QBVideoChatConferenceTypeAudioAndVideo]; }

or

 -(void) chatCallUserDidNotAnswer:(NSUInteger)userID { }

We hope for a successful outcome :) In our case, we specifically set the view and sound from our side. Here everything is the same as at the beginning, with the only difference being that at the end we send accept to the initiator of the call.

His method should work.

 -(void) chatCallDidAcceptByUser:(NSUInteger)userID { }

And then on both sides it will work.

 -(void)chatCallDidStartWithUser:(NSUInteger)userID sessionID:(NSString *)sessionID { }

this method is useful for UI - for example, you spin the spinner until the whole thing happens, and then you hide it in this method. From now on, video calling should work.

When you need to end the session and "hang up" - call

 [self.videoChat finishCall];

after which the delegate method is triggered on the opposite side

 -(void)chatCallDidStopByUser:(NSUInteger)userID status:(NSString *)status { }

There is also a version of this method with parameters, in case you need to transfer something else.

In this case, a standard audio-video session is used. Depending on the TK - if you need to record video and audio, for example, and then do something with it, you should use custom audio-video sessions. SDK allows it. This article does not cover this, but you can read more here: http://quickblox.com/developers/SimpleSample-videochat-ios#Use_custom_capture_session

So, the video link is established. Now the last thing to do is to realize the inclusion of a lullaby on the child’s device, change the camera, take a screenshot, etc ...
All this is done quite simply. Remember we logged in addition to the chat? so - this is another module, it is called Chat :)
It can send messages. What we will do is just send different messages, and on the opponent’s side, parse them and, depending on the message, perform some actions - turn on the flash for example or something else.

Sending a message is done simply (we carried out in a separate method) -

 -(void) sendServiceMessageWithText:(NSString *)text parameters:(NSMutableDictionary *)parameters{ BBPair *currentPair = [QBClient shared].currentPair; QBChatMessage *message = [QBChatMessage new]; message.text = text; message.senderID = currentPair.myID; message.recipientID = currentPair.opponentID; if (parameters) message.customParameters = parameters; [[QBChat instance] sendMessage:message]; }

Text is our message type in this case.

The message comes here -

 -(void)chatDidReceiveMessage:(QBChatMessage *)message { if ([message.text isEqualToString:kRouteFlashMessage]) { // .. do something }else if ([message.text isEqualToString:kRouteCameraMessage]) { // .. } }

All the rest is UI and some additional features. In general, everything turned out well. In the end I would like to draw attention to two nuances:

1) the life of the session - 2 hours. It is automatically renewed after each request executed. But if for example the user turned the application for half a day, then it needs to be somehow restored. This is done easily - using extended request:

 QBASessionCreationRequest *extendedRequest = [QBASessionCreationRequest new]; extendedRequest.userLogin = self.currentUser.login; extendedRequest.userPassword = self.currentUser.password; [QBAuth createSessionWithExtendedRequest:extendedRequest delegate:self];

you can run, for example, in the applicationWillEnterForeground .

2) Method - (void)completedWithResult:(Result *)result grows very quickly, which becomes rather inconvenient. Almost every method is in 2 versions - simple and with context. Alternatively, you can use blocks - pass them as context. Here is what it looks like on the example of session creation:

 typedef void (^qbSessionBlock)(BOOL isCreated, NSError *anError); -(void) createSessionWithBlock:(qbSessionBlock)aBlock { void (^block)(Result *) = ^(Result *result) { if (!result.success) { aBlock(NO, [self errorWithQBResult:result]); } else { aBlock(YES, nil); } }; [QBAuth createSessionWithDelegate:self context:(__bridge_retained void *)(block)]; } - (void)completedWithResult:(Result *)result context:(void *)contextInfo { void(^myBlock)(Result *result) = (__bridge void (^)(Result *__strong))(contextInfo); myBlock(result); Block_release(contextInfo); }

So much easier.

On this, in principle, everything. If something is not clear - write in a personal or comments. Here you can add that the application, which was discussed in this article, was recommended by Apple, came out in the US Appstore in the first place and of the three paid in-app purchase, the video monitor function turned out to be the most demanded. We work a lot on applications related to video calls for iOS, Android, Web - usually this is dating / social or security / video surveillance, so I will be happy to help with advice or code examples if you do something like that.

Source: https://habr.com/ru/post/221091/

More articles:


All Articles