Webrtc, Peer Connection - creating a full-fledged video chat in the browser

Introduction

Webrtc on Habré has been repeatedly mentioned, I would like to tell a little about the technical part of the implementation and highlight the creation of a small video chat. I want to immediately state that the implementation of webrtc is constantly changing, including the names of the api functions, their parameters.
Anyone who would just like to see how it all works, here: apprtc.appspot.com demo from Google all you need is to follow the link and send it to someone else with the room number. At the end you need to change the numbers if it turns out that the room is full. Who cares how it all works welcome under the cat

a common part

API webrtc itself consists of three parts:

1. getUserMedia (MediaStream), if simplified, then this is a video stream capture in the browser, for example, just look at yourself;).
   On Habré there is a good article .
2. RTCPeerConnection is used to communicate between browsers directly. Actually, the RTCPeerConnection will be mainly discussed further.
3. RTCDataChannel: necessary for the exchange of various data: text, files and others. At the moment they write that it is available in 25 chrome only in the test version, without the inclusion of flags it will be available only in 27 chrome.

Peer connection

So, let's begin. In fact, in order not to reinvent the wheel, it was decided to take the code from this demo, make it a bit more universal (it is tied to the google app engine) and simplify in a couple of places. Another adapter.js library is connected here - it is needed for some unification of the code, because a lot of things are written with prefixes, and also differ for the main browsers.
')
The RTCPeerConnection itself is called quite simply:

// Stun          ,    NAT,  , , google    . var pc_config = {"iceServers": [{"url": "stun:stun.l.google.com:19302"}]}; var pc_constraints = {"optional": [{"DtlsSrtpKeyAgreement": true}]}; pc = new RTCPeerConnection(pc_config, pc_constraints); pc.onicecandidate = onIceCandidate; pc.onaddstream = onRemoteStreamAdded; 

In the old version, slightly different parameters were passed to RTCPeerConnection ().

Initialize and Run RTCPeerConnection

 function initialize() { //          localVideo = document.getElementById("localVideo"); miniVideo = document.getElementById("miniVideo"); remoteVideo = document.getElementById("remoteVideo"); //                PeerConnection getUserMedia( {'audio':true, 'video':{"mandatory": {}, "optional": []}}, function(localVideo, stream){ //   PeerConnection. localVideo.src = window.URL.createObjectURL(stream); if (initiator) maybeStart(); }, function(error){console.log("Failed to get access to local media. Error code was " + error.code);} ); if (initiator) maybeStart(); sendMessage(); } 

Message exchange

At this stage, browsers exchange different messages to learn how to communicate with each other. In messages of the type of candidate come different options, including those received from the stun server.

 //           . S->C: {"type":"candidate","label":1,"id":"video","candidate":"a=candidate:2437072876 1 udp 2113937151 192.168.1.2 35191 typ host generation 0\r\n"} S->C: {"type":"candidate","label":0,"id":"audio","candidate":"a=candidate:941443129 1 udp 1845501695 111.222.111.222 35191 typ srflx raddr 192.168.1.2 rport 35191 generation 0\r\n"} 

 // CallBack ,    RTCPeerConnection     ,      . ,        -   websokets,  ajax. pc.onicecandidate = onIceCandidate; function onIceCandidate(event) { if (event.candidate) { sendMessage({type: 'candidate', label: event.candidate.sdpMLineIndex, id: event.candidate.sdpMid, candidate: event.candidate.candidate}); } else { console.log("End of candidates."); } } 

Our function of sending a message through the server is quite simple, so it was decided to use the Ajax as a simpler and more accessible option for writing a small test version and for implementing the server part:

 function sendMessage(message) { var msgString = JSON.stringify(message); console.log('C->S: ' + msgString); $.ajax({ type: "POST", url: "/chat/tv", dataType: "json", data: { room:room, user_id:user_id, last:last, mess:msgString, is_new:is_new }, success: function(data){ console.log(['data.msg', data.msg]) if( data.last) last = data.last; for (var res in data.msg){ var msg = data.msg[res]; processSignalingMessage(msg[2]); } } }); is_new = 0; function repeat() { timeout = setTimeout(repeat, 5000); sendMessage(); } if (!timeout) repeat(); } 

If the request is successful, then the response comes from the accumulated messages from another browser:

 function processSignalingMessage(message) { //               . //        peeronnection var msg = JSON.parse(message); if (msg.type === 'offer') { if (!initiator && !started){ if (!started && localStream ) { createPeerConnection(); pc.addStream(localStream); started = true; if (initiator) pc.createOffer(setLocalAndSendMessage, null, {"optional": [], "mandatory": {"MozDontOfferDataChannel": true}}); } pc.setRemoteDescription(new RTCSessionDescription(msg)); pc.createAnswer(setLocalAndSendMessage, null, sdpConstraints); } else if (msg.type === 'answer' && started) { pc.setRemoteDescription(new RTCSessionDescription(msg)); } else if (msg.type === 'candidate' && started) { var candidate = new RTCIceCandidate({sdpMLineIndex:msg.label, candidate:msg.candidate}); pc.addIceCandidate(candidate); } else if (msg.type === 'bye' && started) { pc.close(); } } function setLocalAndSendMessage(sessionDescription) { //  preferOpus  . sessionDescription.sdp = preferOpus(sessionDescription.sdp); pc.setLocalDescription(sessionDescription); sendMessage(sessionDescription); } 

In general, this is practically all, it now remains to assign the video stream to the <video> element and do not forget to call the function initializing the launch of the entire code.

 pc.onaddstream = onRemoteStreamAdded; function onRemoteStreamAdded(event) { remoteVideo.src = window.URL.createObjectURL(event.stream); remoteStream = event.stream; } setTimeout(initialize, 1); 

Server part

Our backend must be fairly simple; the server must coordinate the browsers before they can communicate directly.
And another nuance, the parameter var initiator = {{initiator}} determines which of the browsers will establish a connection, and which one waits.
That is, one should have 0, respectively, the other 1.

The server part is quite simple, for a GET request we create a room in the database, transfer its id to the template, if it is not in the database, create a new one.

 def chat(room): doc = db.chat.find_one({'_id':room}) initiator = 1 if not doc: initiator = 0 doc = {'_id':room, 'mess': []} db.chat.save(doc) return templ('rtc.tpl', initiator = initiator, room=room) 

On a POST request, we receive data from the client and if the client has sent a non-empty message, then we enter its content into the room, then we check that the messages received “from the counter visitor in the chat” and they are new then return them to their browser.

 def chat_post(): lst = 0.0; msg = [] room = get_post('room') user_id= get_post('user_id') last= float(get_post('last', 0)) mess= get_post('mess') doc = db.chat.find_one({'_id':room}) if mess: doc['mess'].append((time.time(), mess, user_id)) db.chat.save(doc) for i_time, i_msg, i_user in doc['mess']: if i_user != user_id and i_time > last: lst = i_time msg.append((i_time, i_user, i_msg)) if not lst: lst = last return json.dumps({'result': 'ok', 'last': lst, 'msg': msg}) 

On this description of the northern part can be completed.

Sources:
Help on webrtc at html5rocks.com
Official site webrtc