My experience using WebRTC in an iOS application

WebRTC (English Real-time communications - real-time communications) is an open source project designed to organize the transfer of streaming data between browsers or other applications supporting it using point-to-point technology.

For a complete understanding of the article I recommend to familiarize yourself with the basic principles of the technology here .

How I came to the need to use WebRTC

Project Objective:
')
Imagine that we need to connect two random users to each other in order to transfer real-time streaming video to each other.
What are the solutions to the problem?

Option 1:

Organize the connection of two users to the server as to the source and receiver of the video stream and wait for it to receive video from the interlocutor.

Minuses:

• Expensive server hardware
• Low scalability due to limited server resources
• Video delay time increased due to intermediary

Pros:

• There are no problems with the choice of the user and connecting to him - this is what the server does for us

Option 2:

Connect to the user directly, forming a data transfer tunnel for the “inter-party fighter”, knowingly receiving information about this user from the server.

Minuses:

• Long operation of connecting two users
• The difficulty of creating peer-to-peer connections
• Difficulties with the NAT protocol for transmitting data directly
• Requires multiple servers to create a connection

Pros:

• Server unloading
• The server may stop working, but the video will not stop.
• The image quality directly depends on the quality of the data transmission channel of users.
• The ability to use end-to-end encryption
• Connection management is in the hands of users.

Also, the task was to make it as budget as possible, without serious server loads.
It was decided to use option number 2.

After diligent searches on the Internet and over9000 lots of time spent, I came across WebRTC technology, which offered to solve the drawbacks that I revealed in the process of analyzing the approaches to solving the problem.

Implementation

Began the second stage of the search. On it, I stumbled upon a lot of different paid libraries that offered me to solve my problem, found a lot of materials about WebRTC, but I couldn’t find the implementation for iOS.

Then, by chance, I went to the git kernel for the browser from Google and noticed there a folder called “WebRTC”, in which there were several files and two folders: “Android” and “iOS”. Going into a folder that I was more sympathetic to, I found out a lot of library source files and a folder called “Example”, which contained a project called “AppRTC”.



I immediately launched this project on my device and on the second iOS device that was on hand, and I managed to connect them and chat with myself on two phones.

The fastest way to integrate video chat into my application, which already had some functions (registration, authorization), I saw by imposing an example on my application “AppRTC”, which I began to do.

At this time, during the project, the stage came when it was necessary to implement the server part for the application. We decided to find an example of servers for “AppRTC”, and also killed a lot of time with the server-side developer to debug the server and “make friends” the server with the client.

In the end, everything in the “service” was divided into two parts:

• Authorization, registration, interlocutor search filter, settings, purchases, and so on;
• Video conference.

The development went, but after the debugging began, I was faced with the fact that the library is absolutely unstable, there are constant connection breaks, poor video quality, device processor utilization by 120%, low frame refresh rates and much more. A lot of time was spent on optimization, but this did not lead to a real result: there were insignificant changes in the speed of work, but still not a cake. Also, a lot of resources were required by the VP8 codec, and the others did not have service at that time.

The thought came to update the library completely. After alteration of 80% of the written before updating the code, it still did not work, and the example was not updated to the current version. The server part also refused to work with the new version of the library.

The solution was not to use your server for WebRTC. That is, the server of the application itself only analyzed the users connected to it and offered them to connect, after this the applications started working with the server of “AppRTC” itself and all interactions with packet transmission, STUN and TURN servers took place there. I also had to rewrite everything to the current version in the application on my own. During this, I noticed that the encryption of data that was transmitted between users was added, and the H264 codec was integrated, which significantly reduced the load on the device and gave an increase in frames per second.

A complete video chat code was refactored and the application worked even faster and became more usable. As a result of code optimization, the library was moved to a separate “Pod” here .

Conclusion

The task was solved, we received an application that connects two random users who downloaded it and launched it among themselves. Almost without material costs, video chat is implemented on new technologies, and the quality of the video directly depends on the bandwidth of the Internet channel among users.

Several solutions for optimizing the work of the “AppRTC” library itself were developed and presented and they were sent to the developers.