Ok google get me a car
The future is getting closer. 10 years ago I could not have imagined that I would start the car with the help of a voice command!
In recent years, I have watched with interest the rapid development of voice assistants. After the release of the Google Home Mini, I decided that it was already time for me to try, since the price had become more or less adequate for the “toy”. The first project is the integration of the voice assistant with the StarLine GSM module for autorun, control of coordinates, battery voltage and other parameters given by the car alarm system. So, let's go?
The presence of Google Home is not necessary, everything described below will work with the Google Assistant application on the phone. I have installed a GSM / GPS module StarLine M31, but should work with all GSM alarms from StarLine.
')
The general scheme of the application for Google Assistant
- Google Home / Google Assistant is responsible for converting voice to text and vice versa + interaction with standard Google services. When calling our application, Action in the terminology of Google, requests are transmitted to DialogFlow (API.AI in the diagram).
- DialogFlow is responsible for defining the dialog scheme, processing text requests in natural language, highlighting entities, forming responses and interacting with the outside world by calling WebHook if necessary.
- WebHook - WEB-service for interaction with the outside world. At the input is the dialogue branch (Intent) + parameters extracted from the request (Entities). At the exit - the answer to the user.
1. DialogFlow.com
First we need to create an application (agent) at the dialogflow (former API.AI).
We register with a Google account to which we will have Google Home attached.
Unfortunately, the Russian language is not yet available for Google Assistant, we choose English.
Next we need to create Intents. Intent in the terminology DialogFlow - one of the branches of the dialogue responsible for a particular action. In our case it will be: GetBattery, GetTemperature, StartEngine, StopEngine. There is also a Default Intent that works at the very beginning, usually a greeting and a short story about what you can do with the help of this application.
In each Intent we need to specify examples of voice commands (User says), preferably 5-10 different options.
In all Intents, except the default one, we need to send requests to our script (WebHook), therefore we put Fulfillment - Use webhook.
2. WebHook to interact with the Starline server
We need a script that receives an Intent from a DialogFlow request and jerks the Starline commands. The quickest way I managed to implement it in Python + Flask.
Interaction with StarLine is taken from here + checked for relevance by the sniffer in the browser .
To run on the server, I used gunicorn
gunicorn -b :3333 flask.starline:app + nginx as reverse proxy.
Note HTTPS is required!
starline.py
from flask import Flask, request from flask_restful import reqparse, Resource, Api, abort import requests import logging DEVICE_ID = 1234567 # Use HTTPS sniffer to find your DEVICE_ID in https://starline-online.ru/ traffic LOGIN = 'YOUR_STARLINE_EMAIL' PASS = 'YOUR_STARLINE_PASSWORD' logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s') header = { 'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:28.0) Gecko/20100101 Firefox/28.0', 'Accept': 'application/json, text/javascript, */*; q=0.01', 'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8', 'X-Requested-With': 'XMLHttpRequest'} def start_engine(): with requests.Session() as session: t = session.get('https://starline-online.ru/', headers=header) login = session.post('https://starline-online.ru/user/login', { 'LoginForm[login]': LOGIN, 'LoginForm[pass]': PASS, 'LoginForm[rememberMe]': 'off'}, headers=header) logging.debug(login.content) r0 = session.get('https://starline-online.ru/device', headers=header) logging.debug(r0.content) r = session.post('https://starline-online.ru/device/{0}/executeCommand'.format(DEVICE_ID), { 'value': '1', 'action': 'ign', 'password': ''}, headers=header, timeout=1) logging.debug(r.status_code) logging.debug(r.content) logout = session.post('https://starline-online.ru/user/logout', { '': ''}, ) return ('Engine started!') def stop_engine(): with requests.Session() as session: t = session.get('https://starline-online.ru/', headers=header) login = session.post('https://starline-online.ru/user/login', { 'LoginForm[login]': LOGIN, 'LoginForm[pass]': PASS, 'LoginForm[rememberMe]': 'off'}, headers=header) logging.debug(login.content) r0 = session.get('https://starline-online.ru/device', headers=header) logging.debug(r0.content) r = session.post('https://starline-online.ru/device/{0}/executeCommand'.format(DEVICE_ID), { 'value': '0', 'action': 'ign', 'password': ''}, headers=header) logging.debug(r.status_code) logging.debug(r.content) logout = session.post('https://starline-online.ru/user/logout', { '': ''}, ) return ('Engine stopped!') def get_params(): with requests.Session() as session: t = session.get('https://starline-online.ru/', headers=header) login = session.post('https://starline-online.ru/user/login', { 'LoginForm[login]': LOGIN, 'LoginForm[pass]': PASS, 'LoginForm[rememberMe]': 'off'}, headers=header) logging.debug(login.content) r0 = session.get('https://starline-online.ru/device', headers=header) logging.debug(r0.content) res_dict = r0.json()['answer']['devices'][0] logout = session.post('https://starline-online.ru/user/logout', { '': ''}, ) return {'battery': res_dict['battery'], 'temperature': res_dict['ctemp']} def get_battery_text(): return ("Battery voltage {0} volts.".format(get_params()['battery'])) def get_temperature_text(): return ("Temperature: {0} degrees.".format(get_params()['temperature'])) app = Flask(__name__) app.config['BUNDLE_ERRORS'] = True api = Api(app) class ProccessGoogleRequest(Resource): def get(self): return {"status": "OK"} def post(self): req = request.get_json() logging.debug(request.get_json()) response = '' if req['result']['metadata']['intentName'] == 'GetBattery': response = get_battery_text() if req['result']['metadata']['intentName'] == 'GetTemperature': response = get_temperature_text() if req['result']['metadata']['intentName'] == 'StartEngine': response = start_engine() if req['result']['metadata']['intentName'] == 'StopEngine': response = stop_engine() if response == '': abort(400, message='Intent not detected') return {"speech": response, "displayText": response} api.add_resource(ProccessGoogleRequest, '/starline/') if __name__ == '__main__': app.run(debug=False) Yes, taking this opportunity, I want to contact the StarLine team - guys, why not make a normal API with documentation? Looking and integration with third-party products would be many times more?
3. We test in the simulator and on the real device
For testing in DialogFlow, go to Integrations -> Google Assistant -> INTEGRATION SETTINGS -> Test and get into the simulator Actions on Google
And here is the test result in the real world
The only cant, in this version, it answers “Engine started” before the real engine start, since it does not have time to wait for a response from Starline.
Ideas:
1. Request location from Google Assistant, sounding the distance to the car (Starline can give coordinates). It is not yet clear how for Python WebHook to request the location of Google Home.
2. Simplify Google's <-> Starline integration, then there will be no need to hardcode your password. Without participation from Starline, as I understand it, this is not possible.
Known issues:
1. Google Assistant does not have time to wait for the Starline server to respond to the status of the engine start
2. While testing, you can use only the default application name (Invocation) - Hey Google, talk to my test app .
Useful links:
1. Video from Google
2. An example using Entities
Source: https://habr.com/ru/post/346120/
All Articles