The history of reverse engineering of one fluffy animal

On a quiet morning on the third of January, when Moscow was already dozing after the New Year holidays, the doorbell rang in our apartment. Mail has finally delivered a package with Christmas gifts ordered on Amazon. Among other things, it contained a gift for the son, the electronic pet Furby . Buying it was, in general, impulsive. The toy was in the bestsellers of the New Year season and was relatively inexpensive. I did not understand the Furby varieties, but once upon a time I heard something positive about the toy.

My little son, by virtue of his one-year-old age, was not very impressed with the gift, but I was sorry to allow me to throw a complicated electronic device on the floor and tear off this device, and everything went to removing the gift on the shelf until better times, but my mind fell on one inscription on colorful packaging ...
')
The inscription said that for this toy in the AppStore, you can download an application with which you can feed a cyber howler, give him all sorts of commands, and also translate phrases he pronounced in his language Furbish into English. The application was downloaded, the pet was fed with all sorts of edible and inedible objects, which he either swallowed with appetite or spat out, and the translator from furbish to English worked surprisingly accurately.

Does audio recognition work in our time so reliably and even in a rather noisy environment? Something is wrong here. And how does the app transmit Furby commands? IR disappears (earlier versions of Furby, as it turned out, had infrared to communicate with each other), Bluetooth, too. Only audio remains. This is interesting ... Now, if you could hack the communication protocol with this creature and be able to manage it from your computer ... Find some “Easter eggs”, hidden or service commands! Or…

In general, as you understood, the father of the family made a present for himself on the New Year.

***

To start, synced the iPhone with the computer and looked inside the application file (.ipa). Among other trials, there were several dozens of short WAV files numbered in a special way. All this was very much like ready-made audio teams. The first file began with the number 350. After playing this file in Audacity Furby, he busily chewed something and gave out a joyful “Mmm, yum!”. “Aha!” Thought Stirlitz. “Now that you are my stuff!”

Commands in the application began with the 350th and ended the 900th, with large spaces in the numbering. So potentially, Furby can perceive a much larger number of commands than there are in the hands of these ready-made WAV files. We must look further.

The appearance of the signal in Audacity suggested that some kind of frequency modulation was used, one signal being sent, a small pause, then a visual signal again. The total duration is one and a half seconds. Once the modulation frequency, it would be nice to look at the spectrum. I looked at the chart - it clearly stood out five peaks at equal distances from each other in the region of 16-19KHz:



The tower from Mordor is, of course, beautiful, but how to decipher it? I rummaged in Audacity a bit more and discovered the audio display mode in the form of a spectrogram. This picture was already much more beautiful than the first:



Here two parcels with a pause in the middle are clearly visible, differing from each other in the order of the “notes” (base frequencies). Moreover, the average frequency is a carrier, constantly alternating with the other four "notes".

For convenience of decoding the sequence, I made a mask in the graphic editor, which I put on top of the spectrogram screenshot, assigned each note consecutive numbers from 0 to 3 and began to analyze successive teams (as we remember, iOS application developers helpfully numbered all WAV files). At first it turned out that in neighboring teams, numbers sometimes “jump”, i.e. do not go the way I would like if we increment the numbers sequentially. After some analysis, it became clear that the “notes” should be numbered as in the figure below:



Here, the parcel is decoded as 3233 3012 1032 (for convenience of perception, I divided the sequences into blocks of four digits; in the quaternary system, each such block is one byte).

Further analysis of the commands, translation of their binary form and bitwise comparison revealed the following structure of the premise and commands in general:

1. The first byte ( 3233 in the example), being written in binary form, has the following structure: 11 1 01111 , where the upper two bits are always 11 , the next bit is 0 for the first message in the command and 1 for the second, and 01111 are themselves data (part of the command identifier);
2. The second byte ( 3012 ) is a checksum depending on 6 bits of the command (where 1 bit is the ID of the package and 5 bits is the data itself);
3. The third and final byte of the parcel is always 1032 .

What does it mean? First, the command is divided into two packets of 5 bits of data each. In total, we get a 10-bit number, i.e. the potential number of commands that Furby can send or receive is 1024. However, it was not possible to calculate the checksum calculation method. After analyzing the numbers of the teams, it turned out that I can find 7 of 32 checksums for the first package and 31 of 32 checksums for the second package based on the available WAV files. In total, this gave 217 potential teams instead of the existing 76 (in the form of ready-made WAV files), which is also quite good.

I wrote a script that generated a WAV file, similar to a ready one, according to the desired command number, and began to sort through the ranges of commands available to me. As it turned out, undocumented teams really were - Furby reacted to them in different ways, sang songs, read rap, sneezed, imitated sleep and did other simple things.

This spurred research appetite, but the algorithm for calculating the checksum for reverse engineering stubbornly resisted, which means most of the teams remained inaccessible to me.

Once again, combing the Internet for any clues, I suddenly found a link to the official Furby app for Android (about which there was not a word on the box with the toy). “Android → Java → bytecode → source codes → ... → PROFIT!”. Never before has Stirlitz been so close to a clue ...

***

When I finally found the .apk file I needed on some dump, I climbed inside and did not see a single WAV file with commands, although in general the set of resources was similar to the one in the iOS application. If there are no WAV files, does the application generate commands on the fly? That's what I need! Decompiling and viewing Java code gave some interesting clues, but as it turned out, all the interesting stuff, namely the generation and analysis of audio, is inside the native .so library, which has one method that I needed, namely private static native byte[] GenerateComAirCommand(int paramInt); .

How to reach the native method? Pointing brain, Stirlitz decided to download the Android SDK. As a result, a small project was assembled, which included the native library itself and a minimal binding that provides access to the function I need only. At the very start, the application itself simply created WAV files for the minimum required set of WAV files, where the teams had the same missing upper and lower 5 bits for which I needed checksums. After some smoking of Stack Overflow (I didn’t have any experience writing Android), the application started and generated on my virtual SD-card emulator a set of WAV files I needed, which I dragged via adb pull into a normal file system. Analysis of these files gave me full coverage - all 64 checksums by which you can recreate any of the 1024 teams.

During the analysis of Furby's reactions to the teams, another range of commands was found, to which Furby somehow reacted. However, no atomic commands like “open eyes”, “close eyes”, “ears moved” were found. Neither was the self-destruct or eula reading command found on furbish (this does not mean that there are no specialized commands, they can be activated, for example, by a special sequence or by another set of codes — but it is hardly possible to find out ).

***

However, I decided to go ahead and write a Furby response analyzer, as some commands, although not giving visible results, can trigger Furby's response in the form of response commands, which is also interesting. As a result, a Perl script was written, analyzing the PCM data stream from the microphone, making it on the fly and deciphering these messages. All this was written under Windows, where for Perl, unfortunately, there are no normal ways to record data from a microphone, so I had to make a Delphi console program that reads data from a microphone and outputs them continuously to STDOUT. The data stream is redirected to the script where the analysis is already taking place. Such is the Unix way for Windows.

“Stop, stop, stop,” the tired reader will say, “And what is all this for?”

It was interesting for me to see “what’s inside” without physically breaking the toy (I didn’t buy it myself). Along the way, I gained knowledge about the generation and analysis of sound in Perl, FFT, window functions, and work with Android, which is fascinating in itself.

Perhaps this article will be useful to someone when implementing their own protocol, because there are all sorts of interesting gadgets for the iPhone that transmit data just through the audio jack.

And finally, the ability to control Furby through a computer potentially opens the emotional method of notification of some events. For example, when mail arrives from a specific addressee, you can ask Furby to dance something, when a commit comes to Git from a certain person, to purr, and from another, to make a sound less decent (of which Furby has it in stock). The truth for this is still necessary to solve a couple of tasks of hardware. First, prevent Furby from falling asleep after 10 minutes of inactivity (and physical braking is considered an activity - for this he has a position sensor in space) and power it not from batteries, but from a power supply or USB. Maybe on Habré there are iron experts who will want to finally tame the animal?

The code itself after some combing is laid out on GitHub . Wishes and finds in every way welcome. Of course, all the information and software code presented is provided solely for educational purposes.

Oo-tah-toh-toh. Kah way-loh.