KiQ anecdote or as a talking toy for adults did

In the world there are an incredible number of talking toys for children and this only exacerbates the impression that adults are bored. Our team decided to correct this situation.

Beginning of the story

It all started with the fact that a bright thought came to the inquisitive mind of one of my friends and colleagues: why not just make fun of thematic jokes not only children, but also adults? The benefit of affordable and well-proven iron is now never uncommon, and it can not only flash lights, but also sing songs .

In this regard, he called me and offered to repeat the epic feat to make sure whether the ESP8266 plays well in MP3 format over the network and if the hamster is tearing to pieces does it really have enough performance, which I did quickly in the New Year holidays of 2016:
')

On the video, the ESP8266 plays Internet radio, outputting sound using the 5-bit PWM over I2S hack.

However, playing MP3 radio from the Internet was eating almost the entire ESP8266 resource and sounded “so-so”, while imagination was already drawing us more than just an MP3 radio column. With these creative impulses, we went to our team and beloved chef Vladimir .

After a small brainstorming session, a more detailed picture began to emerge, and even a logical chain of what we not only want, but also can do:

• MP3 replaced with something less demanding of resources and more free, stopped at Speex , which flexxnn rather successfully and quickly ported to ESP8266.
  
  
• Records jokes taken from the server in the cloud and stored locally on the SD card, because The Internet is not everywhere there, and even if you keep it on ESP8266 permanently, you can’t get enough of any battery.
  
  
• To tie the "jokes of humor" to some events, and not just to speak randomly. The smartphone was chosen as an event generator, as almost everyone already has it. Well, the actual events can be easily received from the Apple Notification Center Service (ANCS), which is already on the iPod, iPhone and iPad.
  
  
• Use Bluetooth Low Energy (BLE) to communicate with your smartphone, because charging a toy more than once a week will not seem funny to anyone.
  
  
• So that the toy does not infuriate the user with frequent jokes to all the coming events, they decided to use logic in the script language. We stopped at Pawn language .
  
  
• Since everyone has different tastes for jokes, and it’s already sad to listen to the same joke - it was decided to give the user the opportunity to vote for the jokes by “pat” and “beat” (accelerometer) to form on the content server the next time more relevant playlist.

Iron selection

Initially, I wanted to make a very simple, but fully working prototype on the all-in-one board, so we decided to try MediaTek on LinkIt ONE . And netsnail is almost immediately succeeded. There was a connection with the phone and with the SD card MP3 perfectly played.
It would seem - here it is happiness and the platform is ready! But then we faced the harsh reality of the limitations of their closed SDK and the euphoria passed.

We realized that there was no ready-made platform on one board satisfying all our wishes in nature, which means we had to create it from scratch.

So, we had to choose (February 2016) the most suitable BLE chip, which had to be connected with ESP8266. As a connection between the ESP8266 and the BLE chip (we didn’t even know what kind of chip it would be) they decided to use a standard TWI (I2C).

But with BLE, the choice was not very big then:

• TI CC2541 - somehow didn’t immediately attract due to the architecture of 8051 , and even 8 KB of RAM didn’t look promising easy life, considering the number of BLE services and features that we needed in the toy.
  
  
• BCM20732 - looked very promising, but in reality the SDK turned out to be very raw, and the BLE “keychain” of the Broadcom WICED Smart was so buggy that I wouldn't write anything special here, so as not to swear. Let me know if the situation has improved now.
  
  
• nRF52832 - the first chip, which immediately pleased. But then there was no compact BLE module - rekod began to prototype on the debug board , the SDK really liked, contained many different working examples. An example of BLE ANCS started immediately and we immediately saw notifications from the iPhone in the logs with UART. By the way, on this debug board there is a real programmer and debugger SEGGER J-Link, which greatly facilitated our life in the future.
  
  
• nRF51822 - at that time there were ready-made modules , and programmed them through SEGGER from nRF52832. What surprised me the most was that the code from the older model nRF52832 is perfectly transferred to the younger nRF51822 and back!

As a result, the first iron looked like a tarantula spider like this:



Catching glitches due to falling off dupont cables, especially on the nRF51822 module, where the pin's are a bit smaller, was very sad.

BLE services on nRF51822 and iOS application

Nevertheless, I and the affair started implementing BLE services and features on nRF51822 already on this spider-iron, divided everything roughly in half:

• Apple Notification Center Service ( ANCS ) - to receive notification of events from a smartphone.
• Apple Current Time Service (CTS) - when connected, the toy takes time from the phone.
• HandShake Service (HSS) - this service, by the way, makes the toy “unwilling”, unlike ordinary headsets and similar gadgets. The fact is that in the cloud there is a ClientID and the user does not just make a pair of phone and toys via Bluetooth, but also an additional binding, which is not so easy to reset. The description of this process requires a separate article, so I will limit myself to a brief mention.
• Battery As Service (BAS) - transfer the battery status to the application on the smartphone.
• Content Service (CONTS) - with this you can see a list of previously played jokes with text (text is taken from a server in the cloud).
• Settings Service (SETTS) - these can be used to configure inappropriate content of jokes, the time when the toy should be silent (at night, for example), etc.

MaximKit at the same time did an application for iOS .


The picture on the right shows the main application screen.

Here flexxnn could not bear our suffering because of falling off the wiring and made the first normal prototype at home using the laser iron method:



The photo is not visible, but there is already a slot for microSD cards. On the small square shawl below - ALC5627 , the sound was already much better than the 5-bit PWM via I2S.

For power management chose AXP209 . Allows you to charge and use the battery at the same time, as well as monitor its condition. The chip is very common, but it has a lot of specifics both in programming and in the required external components, the so-called. "Harness".

Magic on ESP8266

In parallel with this, flexxnn ported to ESP8266 Speex , FatFs , made a bootloader for ESP8266 and nRF51822, so that you could not sew the updated firmware over the wires, but put it on the microSD firmware files.

Personally, I think he was helped by aliens, since he did it all in about a month. But he does not confess contact with the alien mind.

And then the second more normal prototype:



But a more normal prototype is no longer crawling into a soft toy. And if he did, then with a sudden move on a potential user from the toy, an “alien” would be useful, which of course was unacceptable for MVP.

Therefore, it was decided to order a fee to professionals and this is what happened in the end:



In a cat, such a design already fits perfectly, although it looks like something inhumane:



At the same time, fears were confirmed that events from the phone could come quite often and eventually the user would get annoyed instead of fun. It required flexible logic for filtering, which can be easily changed at any time without flashing. In this regard, I, having studied the materiel and realizing that for ESP8266 there are no fast and compact scripts (Lua and microPython interpreters), I decided to port the Pawn language , which quickly (only 18 times slower than the native one) executes a compact bytecode.

By the way, the source for Pawn is very similar to C, so modifying scripts is a pleasure - they just need to be compiled into AMX bytecode and copied to the SD card.

Cheating gadgets. Pretending iOS on Android

And then "suddenly" we all remembered that Android phones in the world are not so few, and even more than phones with iOS.

As a result, I had to check in tempo if we could fool gadgets and pretend to be iOS on Android , so that they could see the ANCS service, indistinguishable from Apple.

In the end, everything worked out and after a week the gadgets (and our cat is no exception) sincerely believed that I had an iPhone with ANCS service, connecting to my Nexus 5 and even to the old Samsung Galaxy with Android 4.4.

Go cloud

Since no one likes to listen to the joke several times (fact), and there are preferences and cultural restrictions, it was decided to create a unique playlist for each user. For this affair closely engaged in the server in the cloud.

But how to decide who will like it? That's right - let the user decide! For this, I made a basic gesture detection on the LIS3DH accelerometer from STMicroelectronics. I liked the joke - I tilted the cat, as if “yes”, I didn’t like the joke - I tilted it sideways, as if “no”. He also added “forget-me-not” for the phone - if the connection via BLE disappears and the accelerometer detects movement, then the cat is carried somewhere, and the phone is forgotten.

After the user is “polaykal” and thus outlines his preferences, the information goes to the server, where the “genomes” of the playlists are compared and new sets of jokes are formed to the user. But this is a topic for a separate article.

Results

As a result, the entertainment turned out to be a great platform that can:

• contact the phone via BLE and receive notification of events;
• communicate via WiFi and download content and firmware updates;
• save and read files in FAT32 on microSD;
• play jokes packed by speex codec;
• define user preferences using an accelerometer (gesture detection);
• to form playlists for each user based on his preferences in humor;

Remarkably, “very good hands” can easily connect to the appropriate pins (even the holes for the connectors on the board are left) and “wise up the cat” as they please. This makes the all-in-one platform attractive to geeks.

With this platform we went to kickstarter, calling the finished toy KiQ. On the first day, 40% was collected, and now it is already 57%. But that's another story.

Thanks to all those who read to this line. I am waiting for interesting questions from you.