The first video game for Alice, or voice assistant as a game controller

At work, I create useful skills for Alice , and in my free time I tried to use a voice assistant to play with friends, at home. I have already described my development of a text game for one or two players, and this time I share the experience of creating a video game for a whole company.

Rules of the game

I didn’t want to port an existing game under Alice, I wanted to come up with an original game that uses Alice’s ability to recognize speech.



On the screen there are many images in circles and squares of different colors. The player’s task is to find a pair of objects on the same background and make a phrase like: adjective + noun. Moreover, the adjective must be formed from the object in the circle, and the noun - from the object in the square. Hence the name of the game - “Round Square”.

')

The player who calls the phrase gets points. The winner is the one who scored 10 points or the most points during a round (2 minutes). Gameplay is shown in a short video.





Determined the rules - go to the technique.

Controllers

To display the playing field, I use a large TV on which I open the site of the game. If the TV does not have an Internet connection, you can display an image on it from a computer or play on the monitor.



The visual part of the game is written in JavaScript. All objects: the playing field and HUD - are created using the Canvas 2D API methods for drawing rectangles, text and images. Every 50 ms the screen is updated. All code is concentrated in a single HTML file, including images encoded in Base64 format.



Now you need to implement an interface that accepts phrases of players. Fortunately, the problem of speech recognition is solved by voice assistants. Yandex provides this service free of charge to Alice’s users and developers of voice applications (skills).



Add the skill of the game to voice assistant. Now each player can activate the voice application on his own phone or in the smart column and send cues to the site through the microphone.

Backend

It remains to associate the replica of the user with a specific game party. For this we need a server: for each batch, a number is generated, the HTML page registers it on the server and displays it to the user. The user calls this number to the voice assistant, and he registers a new player with the corresponding game through the server’s web hook. Subsequent cues of the user are transferred to his game via WebSocket.



The client code for creating a web socket and reconnecting when disconnecting looks like this:

function start(websocketServerLocation){ ws = new WebSocket(websocketServerLocation); ws.onopen = function(){ gameN = game.id; }; ws.onclose = function(){ gameN = "--"; start(HOST); }; wsSend({"gameId": game.id}); } start(HOST); 

When connected to the server, the client sends him the game number. Below is the server code that establishes the connection and initiates control of the game with the received number:

 const wss = new SocketServer({ server }); wss.on('connection', (ws) => { ws.on('close', () => { remConn(ws.gameId); }); ws.on('message', msg => { const data = JSON.parse(msg); if (data.gameId) { ws.gameId = data.gameId.toString(); addGame(ws); } }); }); 

With this, the same server can render pages with a browser and respond to Alice's web hooks:

 var server = express().use(bodyParser.json()).use('*', (req, res) => { if (req.body && req.body.request) { //   toAnswer  - res.json({ version: req.body.version, session: req.body.session, response: toAnswer, }); } else { // -   res.sendFile(INDEX); } }).listen(PORT); 

Conclusion

The result was an original video game for the company. It was published in Alice on November 28, 2018 and was the first video game I knew of for this voice assistant.



By the way, even before the appearance of Alice and Google Assistant, people felt a fan from voice control in games . Thanks to the ubiquitous distribution of devices with microphones and speakers, thanks to the development of speech technologies, voice assistants are becoming available game controllers that provide a new user experience.