We play sound on DualShock4 from the computer

Choosing a gamepad for my computer, I stopped at DualShock4, because I liked the idea that you can listen to audio through headphones connected to it. But after the purchase, I learned that, it turns out, no one knows how to transfer sound to a gamepad via Bluetooth. Therefore, I decided to deal with this issue. If it is interesting to you to learn how DualShock4 communicates with the game console, I wait under a cat.



Unfortunately, I do not have a PlayStation 4, so I had to be content with only dumped on the Internet, as well as already known fragments of the exchange.

In the process of studying the topic this page helped me a lot. It describes the main points of data transfer between the console and gamepad, as well as a dump of this data. We are interested in the dump file ds4_uart_hci_cap_playroom_needs_sorting.pcap.gz . Open it in Wireshark and start exploring. We sort the packets by time, since, apparently, the dump was recorded separately for reception and transmission. The dump was filmed directly from the UART gamepad, after which it was converted to pcap.



In the beginning is the setting of the Bluetooth module itself. Next, from the No. 49th to the No. 163rd package, the connection is set up and the transmission channel is configured. Very well, this process is described in the article Wireless sound. Part 1. Dissect Bluetooth.

But for our task it is non-trivial.



After all the “preparatory work” the gamepad starts to send the HID Report. The message format is described on the wiki page. The first packet with data from the console is packet No. 70181. Let's analyze it using data from the wiki page .

We are only interested in data that is transmitted through the HID Profile.

Here is its content.

Byte number	bit 7	bit 6	bit 5	bit 4	bit 3	bit 2	bit 1	bit 0
[0]	0x0a - Data Type				0x00 - Reserved		0x02 - Transmission Direction
[one]	0x11 - Operation Code
[2 - 3]	Unknown
[four]	0xf0 Forbids changing the data on the gamepad, 0xf3 Allows changing
[5 - 6]	Unknown
[7]	Rumble (right / weak)
[eight]	Rumble (left / strong)
[9]	RGB color (Red)
[ten]	RGB color (Green)
[eleven]	RGB color (Blue)
[12-24]	Unknown
[25]	Volume of sound in%
[26 - 74]	Unknown
[75 - 78]	CRC-32 from previous data

Although 26 bytes are marked on the page mentioned above as unknown, during my experiments I managed to find out that he is responsible for the volume of the sound and is set as a percentage. Also, although the crc field is present, the gamepad does not check it and you can simply send a zero value.

')

Since we are interested in what data the console transmits, let's filter it on the 0th byte of the HID Profile, which will help us determine the direction of the packet. Data from gempada have the value 0xa1, from console 0xa2. The filter for Wireshark is: bthid [0] == 0xa2.



If you scroll through the packets, then, starting with packet No. 98516, the size of the data has greatly increased. Judging by the data from the wiki page, then the beginning of the packages with the operation code 0x15 and 0x19 is the same as that of 0x11, only without the CRC, which is at the end.

Everything is HID

Here we come to the most interesting - how to transfer the sound to the gamepad. This is what the audio package looks like.





If you look closely at the packets with the operation codes 0x14, 0x15, 0x17, 0x19, then there is a certain constancy, namely the successive bytes 0x9c, 0x75, 0x19. This is very similar to the Bluetooth SBC header ( SBC is one of the standard codecs for transmitting audio over Bluetooth). And although there is an A2DP standard for transmitting SBC via Bluetooth, the creators of PS4 decided to go their own way and transmit sound directly in HID messages. Also, if you look at the packets further, it is clear that two bytes are also changing before the Bluetooth SBC header, this is the frame counter. Let's check our assumption that this is a standard SBC codec. To do this, use the following script in Python.

#!/usr/bin/env python3 from pcapfile import savefile import collections import struct class bluetooth(object): def __init__(self, packet, number): self.direction = packet.raw()[3] self.payload = packet.raw()[4:] self.time = ((packet.timestamp_ms-444738)/1000000)+(packet.timestamp-3) self.number = number pcap = savefile.load_savefile(open('ds4_uart_hci_cap_playroom_needs_sorting.pcap', 'rb')) bluetooth_packet = [] number=1 for pkt in pcap.packets: bluetooth_packet.append(bluetooth(pkt, number)) number+=1 sbc = open('test.sbc', 'wb') bluetooth_packet.sort(key=lambda pkt: pkt.time) count = 0 for bt in bluetooth_packet: count+=1 if(bt.payload[0]==2): l2cap_len = struct.unpack("<H",bt.payload[5:7])[0] if(l2cap_len>5): sony_opcode = bt.payload[10] if(sony_opcode == 0x19): sbc.write(bt.payload[0x5b:-0x12]) if(sony_opcode == 0x17): sbc.write(bt.payload[0x10:-0x8]) if(sony_opcode == 0x15): sbc.write(bt.payload[0x5b:-0x1D]) if(sony_opcode == 0x14): sbc.write(bt.payload[0x10:-0x28]) 

The script works as follows: open the dump, put all the packages in the list, and then sort by time. Then we go through all the packages in order, taking the audio data from messages with the operation code 0x19.0x17.0x15 and 0x14 and writing them to a file.



Now let's try to reproduce the resulting file, for which we use gstreamer'om:



gst-launch-1.0 filesrc location=test.sbc ! sbcparse ! sbcdec ! autoaudiosink



At the beginning of the file will be silence (this can be seen from the stored data). For convenience, we convert the data to wav:



gst-launch-1.0 filesrc location=test.sbc ! sbcparse ! sbcdec ! audioconvert ! wavenc ! filesink location=output.wav



If we rewind the resulting wav for 41 seconds, we will hear a sound.

Thus, we made sure that DualShock4 uses conventional SBC coding for audio transmission.



Now it is interesting to try to generate the data for playback on the gamepad.

We use for this all the same tools. Gstreamer will encode, and Python will transfer data to the DualShock4.

In Linux, you can very easily work with a gamepad due to the fact that everything in it (including devices) are files.

You can find out which file corresponds to a gamepad after pairing the DualShock4 with a computer. As a result of a successful pairing in the dmesg output, the line will appear

sony 0005: 054C: 05C4.0007: input, hidraw5 : BLUETOOTH HID v1.00 Gamepad [Wireless Controller]

This means that our controller is present in the system as a file named / dev / hidraw5, and we can transfer data to the gamepad, simply by writing the necessary data to this file.

Here is the script with which you can do this:

 #!/usr/bin/env python3 import struct from sys import stdin import os from io import FileIO hiddev = os.open("/dev/hidraw5", os.O_RDWR | os.O_NONBLOCK) pf = FileIO(hiddev, "wb+", closefd=False) #pf=open("ds_my.bin", "wb+") rumble_l = 0 rumble_r = 0 r = 0 g = 0 b = 50 crc = 0 volume = 50 flash_bright = 150 flash_dark = 150 def frame_number(inc): res = struct.pack("<H", frame_number.n) frame_number.n += inc if frame_number.n > 0xffff: frame_number.n = 0 return res frame_number.n = 0 def joy_data(): data = [0xf3,0x4,0x00] data.extend([rumble_l,rumble_r,r,g,b,flash_bright,flash_dark]) data.extend([0]*8) data.extend([0x43,0x43,0x00,volume,0x85]) return data def _11_report(): data = joy_data() data.extend([0]*(48)) data.append(crc) return bytearray(data) def _14_report(audo_data): return b'\x14\x40\xA0'+ frame_number(2) + b'\x02'+ audo_data + bytearray(40) def _15_report(audo_data): data = joy_data(); data.extend([0]*(52)) return b'\x15\xC0\xA0' + bytearray(data)+ frame_number(2) + b'\x02' + audo_data + bytearray(29) def _17_report(audo_data): return b'\x17\x40\xA0' + frame_number(4) + b'\x02' + audo_data + bytearray(8) stdin = stdin.detach() data = bytearray() count = 1 while True: # if count % 200: if True: data = _14_report(stdin.read(224)) if count % 3 else _15_report(stdin.read(224)) else: data = _17_report(stdin.read(448)) print('big') count+=1 pf.write(data) 

The script reads audio data encoded in SBC from a standard stream and generates two types of packages 0x14 and 0x15 (commenting / uncommenting lines can also include the formation of a double enlarged package with opcode 0x17) and sends them to the gamepad by writing to the hidraw device.

Let's try to use this script to play a test tone.

This signal will be generated using gstreamer and sent to the standard output stream, where it will be taken from the script.



gst-launch-1.0 -q audiotestsrc is-live=true ! sbcenc ! 'audio/x-sbc,channels=2,rate=32000,channel-mode=dual,blocks=16,subbands=8,bitpool=25' ! queue ! fdsink | ./play.py



And we did it (almost). The sound is coming, but small stuttering is occasionally heard. With what they are connected, I could not understand. Perhaps, I’m not quite working with a hid device in linux - if someone can tell me how to do it correctly, I will be grateful. An attempt to use a Bluetooth socket was also unsuccessful - after half a second of sound playback, everything ended (See UPD).

Conclusion

I would like to thank such projects as DS4Windows and ds4drv .

These projects allow you to use the gamepad on the computer. I hope this article will help add support for sound transmission in these projects.



Thanks for attention.



UPD:

Small addition.

If you add is-live = true to audiotestsrc, then the sound goes almost without stuttering.

Here is a useful pipeline for gstreamer which allows you to capture everything that goes to the audio output and send it to DualShock4.



gst-launch-1.0 -q pulsesrc device="alsa_output.pci-0000_00_1b.0.analog-stereo.monitor" ! queue ! audioresample ! 'audio/x-raw,rate=32000' ! audioconvert ! sbcenc ! 'audio/x-sbc,channels=2,rate=32000,channel-mode=dual,blocks=16,subbands=8,bitpool=25' ! queue ! fdsink | ./play.py



You can get the device name by the following command.

pacmd list-sources | grep -e device.string -e 'name:'