Miracast standard - old protocols in a new wrapper
Not so long ago (starting with JellyBean 4.2), Google added support for Miracast technology to Android.
Practical research of this technology using reverse engineering techniques is the subject of the article.
What is Miracast in a nutshell? This is the latest offspring of the Wi-Fi alliance - the standard for transmitting multimedia content over Wi-Fi in peer-to-peer mode. For the user, this means, first of all, that to connect to the TV (for example) he does not need a Wi-Fi router. Two devices on the idea of ​​an alliance must communicate with each other directly. This is ensured by using the Wi-Fi Direct standard for the authorship of the same organization. In other words, the new standard solves problems very similar to Apple's AirPlay, Intel's WiDi, or good old DLNA.
Why was a fuss - you ask. Why not use the existing solution? Then it will be difficult for me to answer. It is clear that licensing solutions from direct competitors or even from Intel is not a kosher variant, which has, in addition, a fatal flaw , but why not take the same DLNA, perhaps, having slightly improved the rasp. Perhaps, I wanted something new, with fashionable nonce words peer-to-peer? I will not guess. Anyway, the technology was implemented in Android, and fresh phones like the Nexus 4 and Samsung Galaxy S3 have it on board.
The situation is worse with TV manufacturers. If DLNA support already exists in almost every modern TV of a high enough level, then things are worse with Miracast. Despite the existence of chips, models of televisions and projectors who can take Miracast can be counted on fingers. However, the situation is likely to change in 2014, but for now - the user can be content with numerous gadgets that receive a signal via Wi-Fi and convert it to HDMI. Such a thing is stuck in the HDMI connector of the TV, and now you have a Miracast-enabled device!
')
One of the engineering samples with the Broadcom chip fell into my tenacious hands:
After making sure that everything works with a bang with an Android-smartphone, I thought about the question - is it possible to start broadcasting via Miracast directly from under Linux? After all, what is Android inside? The same Linux ...
To begin with, I wanted to understand how the Miracast protocol stack looks like? What is behind the beautiful name? Whether the video signal is chasing directly in Ethernet frames or using IP and even higher-level protocols. Unfortunately, the standard itself , although open, was far from free, so we had to find other, more traditional ways of research. In some presentation, I grabbed keywords - MPEG-TS and RTSP, and this made it possible to unwind the tangle further. If I understand at least something, then RTSP is TCP, and TCP is IP. And IP is a suitable protocol that can be listened to by tcpdump! Said and done by running tcpdump on Nexus and turning on the Wireless display in the settings, after 5 minutes I had a package dump acceptable for further analysis.
Temporarily postponing the difficulties with the connection via Wi-Fi, I immediately took up the analysis of the TCP stream. And that's what I saw:
Doesn't it remind the usual RTSP. So, part of the case is done. It remains to be understood how the Miracast implementation of RTSP differs from the standard one. For those who have never come across RTSP (Real Time Streaming Protocol), I remind you that it is used to control multimedia stream from a server to a client. That is to say - allows you to issue commands such as PLAY, PAUSE, TEARDOWN, etc. It is also possible to exchange options and customize settings. It is {GET | SET} _PARAMETER that became my main headache in the analysis. Having no standard at hand, I could not know what all these wfd_video_formats, wfd_audio_codecs, etc. mean. But he could guess!
Since I understood from the analysis of MPEG-TS frames that the standard resolution was 720x480, and the H.264 (AVC) codec, it was a good idea to create a video file with exactly the same parameters, and then fields like wfd_video_formats can be left unchanged! Rummaging through the DVDs, I recoded a small VOB from the television series Cracker into the format I wanted through ffmpeg. Now it was necessary only to feed the file to the server. But for this you need to find a server!
In order not to write the RTSP server on my own (which was not part of my plans), I began to browse the Open Source options, which would be easy to modify to a state compatible with Miracast. If you carefully looked at the logs from tcpdump, you might have noticed a few oddities. The traditional client-server model of RTSP is replaced by a “peer-to-peer” interaction. This means that the activity in the requests can come not only from the client (in this case, a TV or a projector), but also from the "server" (that is, the phone or computer). Why it was necessary to do this is not clear, but the fact remains that both the “client” and “server” can send requests when they feel like it negates their traditional roles. However, the side that sends the video signal I will continue to call the server (in our case, it is Linix-PC), and the side that receives and decodes the video client (in our case, it will be the projector).
So, after several hours of searching, I stopped at live555 . This server is written in C ++, distributed under the LGPL license and supports both RTSP and broadcasting in MPEG-TS. Looking at the RTSP handler, I realized that it is quite possible to process it under the peer-to-peer specifics of Miracast. But, it remained to force the client (i.e., Miracast gadget) to connect to Linux!
This task was more difficult than Goethe's Faust. Before, I have never set up even regular Wi-Fi in Linux, rightly believing that the wires are somehow more reliable. What can we say about Wi-Fi Direct. However, after reading a stack of manual, I realized that I had to dig in the direction of the mysterious WPA supplicant . What is this supplicant for? It provides authentication when connecting via Wi-Fi to an access point or to another node. As I wrote above, Miracast works in p2p mode, i.e. devices communicate directly, bypassing routers. This feature is, fortunately, supported in recent versions of wpa_supplicant. I don’t know exactly from what point p2p support was added, but in version 2.1-devel it already exists.
However, update the supplicant a little! You must also have configuration files for it. With half my sins, I wrote a configuration acceptable for my device (NetGear, WNA1100 Wireless-N 150 [Atheros AR9271]), maybe it will work for you.
So, in the /etc/wpa_p2p.conf file, we write:
Next, you need a shell script to run supplicant:
That's all (I’ll clarify that this configuration works in the Ubuntu-based distribution of Linux Mint 13 Maya, the kernel version is 3.2.0-57-generic).
Next you need to master such a utility as wpa_cli, it allows you to manage the connection "manually".
After running wpa_supplicant through a script, you need to open a separate console and issue something like:
This is the command interface to supplicant. Turning on the gadget, we can find all devices in the district using the p2p_find command, ready to connect to us in p2p mode. Next, using the p2p_connect command, we make the connection itself.
Here is an example log for my device:
In principle, everything is clear from the log, except perhaps the mysterious word 'pbc' in the p2p_connect command after the device address. What does it mean? This is one of the authentication options when connecting via Wi-Fi direct. It means - Push Button Control. This is a simplified authentication that does not require the user to enter a password or even a pin code. Just at the time of connection, you need to press a button on the device, and authentication will be considered successful.
So, from the log, we see that the connection is successful. And now we have the opportunity to get the IP address for the wlan0 interface.
In this case, the DHCP server will be a TV or a projector. We introduce in a separate terminal:
If we then start tcpdump, we will find attempts to send a SYN packet to port 7236. This port is different from the standard port for RTSP (554), but it should not scare us. Most importantly, the gadget wants to negotiate with us! By launching a slightly modified livemedia server on this port (7236), we are able to debug the actual “client-server” interaction.
I will not bore the reader with the details of the debugging of the protocol; I will just say that all the problems were somehow solved. And now, finally, the result is obvious - I was able to watch the video from my PC through the new-fangled Miracast!
Do you need it? I do not know. In any case, to understand the new standard is always interesting (unless of course it is not ASN.1).
For those who were too lazy to delve into the technical details, I will briefly outline the connection procedure for Miracast-based devices:
Of the obvious flaws of Miracast (not mentioned in the Wiki), I would point out the following:
That's all. If you have any questions - ask in the comments.
Practical research of this technology using reverse engineering techniques is the subject of the article.
What is Miracast in a nutshell? This is the latest offspring of the Wi-Fi alliance - the standard for transmitting multimedia content over Wi-Fi in peer-to-peer mode. For the user, this means, first of all, that to connect to the TV (for example) he does not need a Wi-Fi router. Two devices on the idea of ​​an alliance must communicate with each other directly. This is ensured by using the Wi-Fi Direct standard for the authorship of the same organization. In other words, the new standard solves problems very similar to Apple's AirPlay, Intel's WiDi, or good old DLNA.
Why was a fuss - you ask. Why not use the existing solution? Then it will be difficult for me to answer. It is clear that licensing solutions from direct competitors or even from Intel is not a kosher variant, which has, in addition, a fatal flaw , but why not take the same DLNA, perhaps, having slightly improved the rasp. Perhaps, I wanted something new, with fashionable nonce words peer-to-peer? I will not guess. Anyway, the technology was implemented in Android, and fresh phones like the Nexus 4 and Samsung Galaxy S3 have it on board.
The situation is worse with TV manufacturers. If DLNA support already exists in almost every modern TV of a high enough level, then things are worse with Miracast. Despite the existence of chips, models of televisions and projectors who can take Miracast can be counted on fingers. However, the situation is likely to change in 2014, but for now - the user can be content with numerous gadgets that receive a signal via Wi-Fi and convert it to HDMI. Such a thing is stuck in the HDMI connector of the TV, and now you have a Miracast-enabled device!
')
One of the engineering samples with the Broadcom chip fell into my tenacious hands:
After making sure that everything works with a bang with an Android-smartphone, I thought about the question - is it possible to start broadcasting via Miracast directly from under Linux? After all, what is Android inside? The same Linux ...
To begin with, I wanted to understand how the Miracast protocol stack looks like? What is behind the beautiful name? Whether the video signal is chasing directly in Ethernet frames or using IP and even higher-level protocols. Unfortunately, the standard itself , although open, was far from free, so we had to find other, more traditional ways of research. In some presentation, I grabbed keywords - MPEG-TS and RTSP, and this made it possible to unwind the tangle further. If I understand at least something, then RTSP is TCP, and TCP is IP. And IP is a suitable protocol that can be listened to by tcpdump! Said and done by running tcpdump on Nexus and turning on the Wireless display in the settings, after 5 minutes I had a package dump acceptable for further analysis.
Temporarily postponing the difficulties with the connection via Wi-Fi, I immediately took up the analysis of the TCP stream. And that's what I saw:
OPTIONS * RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 1 Require: org.wfa.wfd1.0 RTSP / 1.0 200 OK CSeq: 1 Public: org.wfa.wfd1.0, GET_PARAMETER, SET_PARAMETER OPTIONS * RTSP / 1.0 CSeq: 1 Require: org.wfa.wfd1.0 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 1 Public: org.wfa.wfd1.0, SETUP, TEARDOWN, PLAY, PAUSE, GET_PARAMETER, SET_PARAMETER GET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 2 Content-Type: text / parameters Content-Length: 83 wfd_content_protection wfd_video_formats wfd_audio_codecs wfd_client_rtp_ports RTSP / 1.0 200 OK CSeq: 2 Content-Type: text / parameters Content-Length: 751 wfd_content_protection: none wfd_video_formats: 00 00 02 10 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 02 08 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 02 04 0001bdeb 3ffffffff 00000fff 00 0000 0000 11 none, 02 02 0001bdeb 3fffffff 000001 00deeb 00ff 0101 00001 none none, 02 01 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 10 0001bdeb 3fffffff 00000fff 00 0000 0000 11 11 none none, 01 08 0001 00 deB 3fffffff 00000fff 00 0000 0000 11 none none, 01 04 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none, 01 02 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none, 01 01 0001bdeb 3fffffff 00000fff 00 0000 0000 11 none none wfd_audio_codecs: LPCM 00000003 00 wfd_client_rtp_ports: RTP / AVP / UDP; unicast 6500 0 mode = play SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 3 Content-Type: text / parameters Content-Length: 248 wfd_video_formats: 28 00 02 02 00000020 00000000 00000000 00 0000 0000 00 none none wfd_audio_codecs: LPCM 00000002 00 wfd_presentation_URL: rtsp: //192.168.16.40/wfd1.0/streamid=0 none wfd_client_rtp_ports: RTP / AVP / UDP; unicast 6500 0 mode = play RTSP / 1.0 200 OK CSeq: 3 SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:37:54 +0000 Server: Mine / 1.0 CSeq: 4 Content-Type: text / parameters Content-Length: 27 wfd_trigger_method: SETUP RTSP / 1.0 200 OK CSeq: 4 SETUP rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 2 Transport: RTP / AVP / UDP; unicast; client_port = 6500 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:55 +0000 Server: Mine / 1.0 CSeq: 2 Session: 1219569791; timeout = 30 Transport: RTP / AVP / UDP; unicast; client_port = 6500; server_port = 15550 PLAY rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 3 Session: 1219569791 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:37:55 +0000 Server: Mine / 1.0 CSeq: 3 Session: 1219569791; timeout = 30 Range: npt = now- SET_PARAMETER rtsp: //localhost/wfd1.0 RTSP / 1.0 Date: Fri, 08 Mar 2013 12:38:07 +0000 Server: Mine / 1.0 CSeq: 5 Content-Type: text / parameters Content-Length: 30 wfd_trigger_method: TEARDOWN RTSP / 1.0 200 OK CSeq: 5 TEARDOWN rtsp: //192.168.16.40/wfd1.0/streamid=0 RTSP / 1.0 CSeq: 4 Session: 1219569791 RTSP / 1.0 200 OK Date: Fri, 08 Mar 2013 12:38:09 +0000 Server: Mine / 1.0 CSeq: 4 Session: 1219569791; timeout = 30 Connection: close
Doesn't it remind the usual RTSP. So, part of the case is done. It remains to be understood how the Miracast implementation of RTSP differs from the standard one. For those who have never come across RTSP (Real Time Streaming Protocol), I remind you that it is used to control multimedia stream from a server to a client. That is to say - allows you to issue commands such as PLAY, PAUSE, TEARDOWN, etc. It is also possible to exchange options and customize settings. It is {GET | SET} _PARAMETER that became my main headache in the analysis. Having no standard at hand, I could not know what all these wfd_video_formats, wfd_audio_codecs, etc. mean. But he could guess!
Since I understood from the analysis of MPEG-TS frames that the standard resolution was 720x480, and the H.264 (AVC) codec, it was a good idea to create a video file with exactly the same parameters, and then fields like wfd_video_formats can be left unchanged! Rummaging through the DVDs, I recoded a small VOB from the television series Cracker into the format I wanted through ffmpeg. Now it was necessary only to feed the file to the server. But for this you need to find a server!
In order not to write the RTSP server on my own (which was not part of my plans), I began to browse the Open Source options, which would be easy to modify to a state compatible with Miracast. If you carefully looked at the logs from tcpdump, you might have noticed a few oddities. The traditional client-server model of RTSP is replaced by a “peer-to-peer” interaction. This means that the activity in the requests can come not only from the client (in this case, a TV or a projector), but also from the "server" (that is, the phone or computer). Why it was necessary to do this is not clear, but the fact remains that both the “client” and “server” can send requests when they feel like it negates their traditional roles. However, the side that sends the video signal I will continue to call the server (in our case, it is Linix-PC), and the side that receives and decodes the video client (in our case, it will be the projector).
So, after several hours of searching, I stopped at live555 . This server is written in C ++, distributed under the LGPL license and supports both RTSP and broadcasting in MPEG-TS. Looking at the RTSP handler, I realized that it is quite possible to process it under the peer-to-peer specifics of Miracast. But, it remained to force the client (i.e., Miracast gadget) to connect to Linux!
This task was more difficult than Goethe's Faust. Before, I have never set up even regular Wi-Fi in Linux, rightly believing that the wires are somehow more reliable. What can we say about Wi-Fi Direct. However, after reading a stack of manual, I realized that I had to dig in the direction of the mysterious WPA supplicant . What is this supplicant for? It provides authentication when connecting via Wi-Fi to an access point or to another node. As I wrote above, Miracast works in p2p mode, i.e. devices communicate directly, bypassing routers. This feature is, fortunately, supported in recent versions of wpa_supplicant. I don’t know exactly from what point p2p support was added, but in version 2.1-devel it already exists.
However, update the supplicant a little! You must also have configuration files for it. With half my sins, I wrote a configuration acceptable for my device (NetGear, WNA1100 Wireless-N 150 [Atheros AR9271]), maybe it will work for you.
So, in the /etc/wpa_p2p.conf file, we write:
ctrl_interface=/var/run/wpa_supplicant ap_scan=1 device_name=JellyFish device_type=1-0050F204-1 Next, you need a shell script to run supplicant:
sudo iwconfig wlan0 mode ad-hoc sudo ip link set wlan0 up sudo wpa_supplicant -Dnl80211 -c /etc/wpa_p2p.conf -i wlan0 -dt
That's all (I’ll clarify that this configuration works in the Ubuntu-based distribution of Linux Mint 13 Maya, the kernel version is 3.2.0-57-generic).
Next you need to master such a utility as wpa_cli, it allows you to manage the connection "manually".
After running wpa_supplicant through a script, you need to open a separate console and issue something like:
sudo wpa_cli
This is the command interface to supplicant. Turning on the gadget, we can find all devices in the district using the p2p_find command, ready to connect to us in p2p mode. Next, using the p2p_connect command, we make the connection itself.
Here is an example log for my device:
wpa_cli v2.1-devel Selected interface 'wlan0' Interactive mode > p2p_find Ok <3> P2P-DEVICE-FOUND 02: 90: 4c: 04: 04: 04 p2p_dev_addr = 02: 90: 4c: 04: 04: 04 pri_dev_type = 7-0050F204-1 name = 'MLT-52-2123' config_methods = 0x4688 dev_capab = 0x25 group_capab = 0xa > > p2p_connect 02: 90: 4c: 04: 04: 04 pbc Ok <3> P2P-FIND-STOPPED <--- Here you have to press a button on the device <3> P2P-GO-NEG-SUCCESS <4> Failed to initiate AP scan <4> Failed to initiate AP scan <4> Failed to initiate AP scan <4> Failed to initiate AP scan <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE-PBC <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE-PBC <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Associated with 02: 90: 4c: 04: 84: 04 <3> CTRL-EVENT-EAP-STARTED EAP authentication started <3> CTRL-EVENT-EAP-PROPOSED-METHOD vendor = 14122 method = 1 <3> CTRL-EVENT-EAP-METHOD EAP vendor 14122 method 1 (WSC) selected <3> WPS-CRED-RECEIVED <3> WPS-SUCCESS <3> P2P-GROUP-FORMATION-SUCCESS <3> CTRL-EVENT-EAP-FAILURE EAP authentication failed <3> CTRL-EVENT-DISCONNECTED bssid = 02: 90: 4c: 04: 84: 04 reason = 3 locally_generated = 1 <3> CTRL-EVENT-SCAN-RESULTS <3> WPS-AP-AVAILABLE <3> SME: Trying to authenticate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Trying to associate with 02: 90: 4c: 04: 84: 04 (SSID = 'DIRECT-fCMLT-52-2123' freq = 2412 MHz) <3> Associated with 02: 90: 4c: 04: 84: 04 <3> WPA: Key negotiation completed with 02: 90: 4c: 04: 84: 04 [PTK = CCMP GTK = CCMP] <3> CTRL-EVENT-CONNECTED - Connection to 02: 90: 4c: 04: 84: 04 completed [id = 0 id_str =] <3> P2P-GROUP-STARTED wlan0 client ssid = "DIRECT-fCMLT-52-2123";
In principle, everything is clear from the log, except perhaps the mysterious word 'pbc' in the p2p_connect command after the device address. What does it mean? This is one of the authentication options when connecting via Wi-Fi direct. It means - Push Button Control. This is a simplified authentication that does not require the user to enter a password or even a pin code. Just at the time of connection, you need to press a button on the device, and authentication will be considered successful.
So, from the log, we see that the connection is successful. And now we have the opportunity to get the IP address for the wlan0 interface.
In this case, the DHCP server will be a TV or a projector. We introduce in a separate terminal:
sudo dhclient wlan0
If we then start tcpdump, we will find attempts to send a SYN packet to port 7236. This port is different from the standard port for RTSP (554), but it should not scare us. Most importantly, the gadget wants to negotiate with us! By launching a slightly modified livemedia server on this port (7236), we are able to debug the actual “client-server” interaction.
I will not bore the reader with the details of the debugging of the protocol; I will just say that all the problems were somehow solved. And now, finally, the result is obvious - I was able to watch the video from my PC through the new-fangled Miracast!
Do you need it? I do not know. In any case, to understand the new standard is always interesting (unless of course it is not ASN.1).
For those who were too lazy to delve into the technical details, I will briefly outline the connection procedure for Miracast-based devices:
- Using Wi-Fi direct, devices find each other (usually the source of the video data finds the display device)
- Using some form of authentication (in our case, pbc), the devices are combined into a P2P group
- One of the devices receives an IP address via DHCP (in our case, this is the source of the video data)
- At the data source on port 7236, an RTSP server is started.
- The client connects to the RTSP server and requests a predefined URL (/wfd1.0/streamid=0)
- The RTSP server begins to transmit video (and possibly audio) data in the form of MPEG-TS packed into RTP packets.
- The client unpacks the data and displays it on the output device.
Of the obvious flaws of Miracast (not mentioned in the Wiki), I would point out the following:
- If you connect to a Miracast device, you lose the ability to work via normal (not P2P) Wi-Fi. To use traditional Wi-Fi and Wi-Fi direct at the same time, you need a special two-channel Wi-Fi adapter. It is not available in all phones!
- The picture quality on dynamic scenes suffers even at a resolution of 720x480, 30 FPS. I'm not talking about Full HD. Of course, with the advent of more powerful processors, the picture will change, but so far everything is sad.
That's all. If you have any questions - ask in the comments.
Source: https://habr.com/ru/post/207456/
All Articles