Flask Mega-Tutorial, Part XIV: Ajax

(edition 2018)

Miguel grinberg

There

This is the fourteenth part of the Flask, k Mega-Tutorial, in which I am going to add a real-time text translation function using the Microsoft translation service and some JavaScript.

Under the spoiler is a list of all articles in this 2018 series.

Note 1: If you are looking for old versions of this course, this is here .

Note 2: If suddenly you would like to speak in support of my (Miguel) work, or simply do not have the patience to wait for the article for a week, I (Miguel Greenberg) offer the full version of this manual (in English) in the form of an electronic book or video. For more information, visit learn.miguelgrinberg.com .

In this article, I’m going to go to the “safe zone” of server-side development and work on a function that has equally important server and client components. Have you seen the “Translate” links that some sites show next to user-generated content? These are links that trigger automatic translation of content from a user’s native language in real time. Translated content is usually entered below the original version. Google shows search results in foreign languages. Facebook does this for posts. Twitter does it for tweets. Today I will show you how to add the same feature to Microblog!

GitHub links for this chapter: Browse , Zip , Diff .

Server and client side

In the traditional server model, which I have followed so far, there is a client (a web browser that the user manages) that makes HTTP requests to the application server. A request may simply request an HTML page, for example, when you click the " profile " link, or it may initiate an action, for example, when you click the Submit button after editing the profile information. In both types of requests, the server performs the request by sending a new web page to the client, either directly or by redirection. The client then replaces the current page with a new one. This cycle repeats while the user stays on the application website. In this model, the server does all the work, while the client simply displays web pages and accepts user input.

There is another model in which the client takes a more active role. In this model, the client issues a request to the server, and the server responds with a web page, but unlike in the previous case, not all of these pages are HTML, there are also sections of the page with code usually written in Javascript. As soon as the client receives the page, it displays HTML fragments and executes the code. From this point on, you have an active client that can work independently, without any contact with the server. In a strict client application, the entire application is downloaded to the client with a request for the initial page, and then the application is executed completely on the client, only occasionally contacting the server to receive or store data and make dynamic changes to the appearance of only the first and only web page. This type of application is called Single Page Applications or SPA.

Most applications are a hybrid between the two models and combine the technologies of both. My Microblog application is basically a server application, but today I will add some client-side actions to it. To perform real-time message translations, the client browser will send asynchronous requests to the server, to which the server will respond without causing the page to refresh. The client will then dynamically insert translations into the current page. This method is known as Ajax , which is a shorthand for asynchronous JavaScript and XML (although nowadays, XML is often replaced by JSON).

Real-time translation workflow

The application has good support for foreign languages ​​thanks to Flask-Babel , which will support as many languages ​​as I can find translators. But, of course, one element is missing. Users will write blog posts in their native languages. Therefore, it is possible that the user will encounter posts that are written in languages ​​unknown to him. The quality of automated translations is not always great, but in most cases it is good enough if all you want is to have a basic understanding of what the text in another language means.

This is an ideal opportunity to implement AJAX service. Note that the index or explore pages may display several messages, some of which may be in foreign languages. If I implement the translation using traditional server-side methods, the translation request will replace the original page with a new page. The fact is that the request to translate one of the many blog posts displayed is not large enough to require a full page refresh; this function works much better if the translated text is dynamically inserted under the source text, leaving the rest of the page intact.

Realization of automated real-time translation requires several steps. First, I need a way to determine the source language of the text to translate. I also need to know the preferred language for each user, because I want to show the “translate” link only for messages written in other languages. When the link to the translation is offered and the user clicks on it, I will need to send an AJAX request to the server, and the server will contact the third-party translation API. After the server sends a response with the translated text, the client javascript code will dynamically insert this text into the page. As you probably noticed, there are several non-trivial problems. Consider them one by one.

Language Identification

The first problem is determining which language the entry was written in. This is not an exact science, since it is not always possible to uniquely identify a language, but in most cases automatic detection works quite well. Python has a good language registration library called guess_language . The original version of this package is quite old and has never been ported to Python 3, so I’m going to install a derived version that supports Python 2 and 3:

 (venv) $ pip install guess-language_spirit 

The plan is to feed each blog post to this package to try to determine the language. Since this analysis takes a lot of time, I don’t want to repeat this work every time a message is displayed on a page. What I'm going to do is set the source language for the message at the time it was sent. The detected language will then be stored in the Posts table.

The first step is to add the language field to the Post model:

app / models.py : Add model to Post model.

 class Post(db.Model): # ... language = db.Column(db.String(5)) 

As you remember, every time there are changes in database models, you need to migrate the database:

 (venv) $ flask db migrate -m "add language to posts" INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.autogenerate.compare] Detected added column 'post.language' Generating migrations/versions/2b017edaa91f_add_language_to_posts.py ... done 

Then you need to apply the migration to the database:

 (venv) $ flask db upgrade INFO [alembic.runtime.migration] Context impl SQLiteImpl. INFO [alembic.runtime.migration] Will assume non-transactional DDL. INFO [alembic.runtime.migration] Upgrade ae346256b650 -> 2b017edaa91f, add language to posts 

Now I can detect and save the language when sending a message:

app / routes.py : Save language for new posts.

 from guess_language import guess_language @app.route('/', methods=['GET', 'POST']) @app.route('/index', methods=['GET', 'POST']) @login_required def index(): form = PostForm() if form.validate_on_submit(): language = guess_language(form.post.data) if language == 'UNKNOWN' or len(language) > 5: language = '' post = Post(body=form.post.data, author=current_user, language=language) # ... 

With this change, every time a message is sent, I skip the text through the guess_language function to try to determine the language. If the language returns as unknown or I get an unexpectedly long result, I reinsure myself and save an empty string in the database. I am going to accept an agreement that any message that has a language with an assigned value is an empty string, then it is assumed that it has an unknown language.

Displaying the "Translate" link

The second step is very simple. I will add a "Translate" link next to any messages that differ from the language that is active for the current user.

app/templates/_post.html : Add a translate link to posts.

 {% if post.language and post.language != g.locale %} <br><br> <a href="#">{{ _('Translate') }}</a> {% endif %} 

I do this in the _post.html so that this functionality is displayed on any page that displays blog entries. The link to the translation will be displayed only on messages for which the language has been detected, and this language does not correspond to the language selected by the function decorated by the localeselector Flask-Babel. Recall from Chapter 13 that the selected locale is stored as g.locale . The link text must be added in such a way that it can be translated by Flask-Babel, so I use the _() function to define it.

Please note that I have not yet linked the action to this link. First I want to figure out how to do actual translations.

Using a third-party translation service

The two main translation services are the Google Cloud Translation API and the Microsoft Translator Text API . Both are paid services, but Microsoft’s offer has an entry-level option for a small amount of translations, which is free. Google offered a free translation service in the past, but today, even the lowest level of service is paid. Since I want to be able to experiment with translations without incurring costs, I am going to implement a Microsoft solution.

Note Translator: There is still Yandex about which Miguel does not seem to know. Not a bad API! There is a free version of up to 1 M characters per day and 10 M per month versus 2 M per month for the microsoftware.

Before you can use the Microsoft Translator API, you must get an account in Azure, the Microsoft cloud service. You can choose the Free level, during the offer provide the credit card number during the registration process. Your card will not be charged as long as you remain at this level of service.

After you have an Azure account, go to the Azure portal and click the "New" button in the upper left corner, then enter or select "Translator Text API". When you click the "Create" button, you will be presented with a form in which you will define a new translator resource that will be added to your account. Below you can see how I filled out the form:

When you click the "Create" button again, the API translator resource will be added to your account. If you wait a few seconds, you will receive a notification in the row at the top of the panel that the translator has been deployed. Click the "Go to resource" button in the notification, and then on the "Keys" option on the left sidebar. You will now see two keys labeled "Key 1" and "Key 2". Copy one of the keys to the clipboard, and then enter it into the environment variable in your terminal (if you are using Microsoft Windows, replace export with set ):

 (venv) $ export MS_TRANSLATOR_KEY=<paste-your-key-here> 

This key is used for authentication using the translation service, so you need to add it to the application configuration:

config.py : Add Microsoft Translator API key to the configuration.

 class Config(object): # ... MS_TRANSLATOR_KEY = os.environ.get('MS_TRANSLATOR_KEY') 

As always with configuration values, I prefer to set them into environment variables and import them into the Flask configuration from there. This is especially important when there is confidential information, such as keys or passwords, that provide access to third-party services. You definitely do not want to write them explicitly in code.

The Microsoft Translator API is a web service that accepts HTTP requests. Python has several HTTP clients, but the most popular and easiest to use is requests . Let's install it in a virtual environment:

 (venv) $ pip install requests 

Below you can see the function that I encoded to translate text using the Microsoft Translator API. I am adding a new app / translate.py module:

app / translate.py : Text translation function.

 import json import requests from flask_babel import _ from app import app def translate(text, source_language, dest_language): if 'MS_TRANSLATOR_KEY' not in app.config or \ not app.config['MS_TRANSLATOR_KEY']: return _('Error: the translation service is not configured.') auth = {'Ocp-Apim-Subscription-Key': app.config['MS_TRANSLATOR_KEY']} r = requests.get('https://api.microsofttranslator.com/v2/Ajax.svc' '/Translate?text={}&from={}&to={}'.format( text, source_language, dest_language), headers=auth) if r.status_code != 200: return _('Error: the translation service failed.') return json.loads(r.content.decode('utf-8-sig')) 

The function accepts the text to be translated, and the source and target languages ​​are encoded as arguments, and returns a string with the translated text. The action starts with checking the presence of a key for the translation service in the configuration, and if it does not exist, it returns an error. The error is also a string, so from the outside it will look like translated text. This ensures that in case of an error the user will see an error message.

The get() method from the requests packet sends an HTTP request with a GET method to the URL specified as the first argument. I use the URL /v2/Ajax.svc/Translate , which is the endpoint of the translation service that returns translations as JSON data. Text, source and target languages ​​should be specified as arguments of the query string in a URL named text , from and to respectively. To authenticate with the service, I need to pass a key that I added to the configuration. This key must be specified in a custom HTTP header named Ocp-Apim-Subscription-Key . I created an auth dictionary with this header, and then passed it to the queries in the headers argument.

The requests.get() method returns a response object containing all the information provided by the service. First you need to check that the status code is 200, which is the code of the successful request. If I get other codes, it means an error has occurred, so in this case I return an error string. If the status code is 200, then the response body has a JSON encoded string with the translation, so all I need to do is use the json.loads() function from the standard Python library to decode the JSON into a Python string that I could use. The response object attribute content contains the raw response text as a byte string, which is converted to a utf-8 string and sent to json.loads() .

Below you can see a Python console session in which I use the new translate() function:

 >>> from app.translate import translate >>> translate('Hi, how are you today?', 'en', 'es') # English to Spanish 'Hola, ¿cómo estás hoy?' >>> translate('Hi, how are you today?', 'en', 'de') # English to German 'Are Hallo, how you heute?' >>> translate('Hi, how are you today?', 'en', 'it') # English to Italian 'Ciao, come stai oggi?' >>> translate('Hi, how are you today?', 'en', 'fr') # English to French "Salut, comment allez-vous aujourd'hui ?" 

Pretty cool, right? Now it's time to integrate this functionality with the application.

Ajax From The Server

I'll start with the implementation of the server side. When the user clicks the "Translate" link that appears under the message, an asynchronous HTTP request is issued to the server. I will show you how to do this in the next session, so for the moment I will focus on the server’s handling of this request.

An asynchronous (or Ajax) request is similar to routes and browsing functions that I created in an application, with the only difference that instead of returning HTML or redirecting, it simply returns data formatted as XML or more often JSON . Below you can see the translation preview function, which calls the Microsoft Translator API, and then returns the translated text in JSON format:

app / routes.py : Text translation view function.

 from flask import jsonify from app.translate import translate @app.route('/translate', methods=['POST']) @login_required def translate_text(): return jsonify({'text': translate(request.form['text'], request.form['source_language'], request.form['dest_language'])}) 

As you can see, it is simple. I ran this route as a POST request. There is no absolute rule regarding when to use GET or POST (or other query methods that you have not yet seen). Since the client will send the data, I decided to use a POST request, since this is similar to the requests that the form data represents. The request.form attribute is a dictionary that Flask provides with all the data included in the view. When I worked with web forms, I didn’t need to look for request.form , because Flask-WTF does everything to work as it should, but in this case there is actually no web form, so I have to directly access the data .

So, let's look at what I am doing in this function. The translate() function from the previous section is called and with it three arguments are passed directly from the data that was sent with the request. The result is included in the dictionary with one single key under the name text , and the dictionary is passed as an argument to the jsonify() function Flask, which converts the dictionary into a formatted JSON payload. The return value from jsonify() is the HTTP response that will be sent back to the client.

For example, if a customer wants to translate the string Hello, World! In Spanish, the response from this request will have the following useful data:

 { "text": "Hola, Mundo!" } 

Ajax from the client

So now that the server can provide translations via the URL / translate , I need to call this URL when the user clicks on the “Translate” link that I added above, passing the text to be translated, as well as the source and target languages. If you are not familiar with working with JavaScript in the browser, it will be a good learning experience.

When working with JavaScript in the browser, the currently displayed page is internally represented as a document object model or only DOM. This is a hierarchical structure that refers to all elements that exist on the page. JavaScript code running in this context can make changes to the DOM to trigger changes on the page.

Let's first discuss how my JavaScript code running in a browser can get three arguments that I need to send to the translation function that runs on the server. To get the text, I need to find a node in the DOM that contains the body of the blog post and read its contents. To make it easier to identify DOM nodes containing blog entries, I'm going to add a unique identifier to them. If you look at the _post.html template, the line that displays the message body simply reads {{post.body}} . I'm going to wrap this content in a <span> element. This will not change anything visually, but it gives me a place where I can insert an identifier:

app/templates/_post.html : Add an ID to each blog post.

  <span id="post{{ post.id }}">{{ post.body }}</span> 

This will assign a unique identifier to each blog post in post1 , post2 format, etc., where the number corresponds to the database identifier for each post. Now that each blog post has a unique identifier, given the ID value, I can use jQuery to search for the <span> element for this message and extract the text in it. For example, if I wanted to get the text for a message with ID 123, here's what I would do:

 $('#post123').text() 

Here, the $ sign is the name of the function provided by the jQuery library. This library is used by Bootstrap, so it was already included in Flask-Bootstrap. # is part of the selector syntax used by jQuery, which means that the element ID is next.

I also want to have a place where I will insert the translated text after receiving it from the server. Since I'm going to replace the "Translate" link with the translated text, I also need to have a unique identifier for this node:

app/templates/_post.html : I app/templates/_post.html ID to the link to translate.

  <span id="translation{{ post.id }}"> <a href="#">{{ _('Translate') }}</a> </span> 

So now I have a post<ID> for the blog post and the corresponding translation<ID> node where I’ll need to replace the “Translation” link with the translated text as soon as I receive it.

The next step is to write a function that can do all the translation work. This function will accept input and output nodes of the DOM, as well as the source and destination languages, issue an asynchronous request to the server with three necessary arguments and, finally, replace the “Translation” link with the translated text after the server responds. This sounds like a lot of work, but the implementation is pretty simple:

app / templates / base.html : Client-side translation feature.

 {% block scripts %} ... <script> function translate(sourceElem, destElem, sourceLang, destLang) { $(destElem).html('<img src="{{ url_for('static', filename='loading.gif') }}">'); $.post('/translate', { text: $(sourceElem).text(), source_language: sourceLang, dest_language: destLang }).done(function(response) { $(destElem).text(response['text']) }).fail(function() { $(destElem).text("{{ _('Error: Could not contact server.') }}"); }); } </script> {% endblock %} 

The first two arguments are unique IDs for messages and translation nodes. The last two arguments are the source and target language codes.

The function begins with a nice touch: it adds a spinner , replacing the “Transfer” link, so that the user knows that the transfer is in progress. This is done using jQuery using the $(destElem).html() function to replace the original HTML, which identified a link to the translation with new HTML content based on the <img> link. For a spinner, I'm going to use a small animated GIF tick, which I added to the app / static / loading.gif directory , which Flask reserves for static files. To generate a URL that references this image, I use the url_for() function, passing the special name of the static route and giving the file name as an argument. You can find the load.gif image in the download package for this chapter.

So now I have a decent timer spinner, which replaced the "Translate" link and the user now knows that he needs to wait until the translation appears.

The next step is to send a POST request to the URL / translate , which I defined in the previous section. For this, I'm also going to use jQuery, in this case the $.post() function. This function sends data to the server in a format similar to how the browser submits the web form, which is convenient because it allows Flask to include this data in the request.form dictionary. The two $.post() arguments are, first, the URL to send the request, and then the dictionary (or object, as they are called in JavaScript) with the three data elements that the server expects.

You probably know that JavaScript works a lot with callback functions or a more advanced form of callback called promises . What I want to do now is bash what I will do when this request is completed and the browser gets an answer. In JavaScript, there is no such thing as waiting for anything, everything is asynchronous. Instead, you must provide a callback function that the browser will call when the response is received. And also as a way to make everything as reliable as possible, I want to specify what to do if an error suddenly occurred, so the second callback function will be error handling. There are several ways to specify these callbacks, but in this case the use of promises makes the code quite clear. The syntax is as follows:

 $.post(<url>, <data>).done(function(response) { // success callback }).fail(function() { // error callback }) 

The promise syntax allows you to basically hook up callbacks to the return value of the $.post() call. On a successful callback, all I need to do is call $(destElem).text() with the translated text that comes into the turnkey text dictionary. In the case of an error, I do the same thing, but the text I show will be a general error message, which I am sure is entered into the basic template as text that can be translated.

So, it now remains only to call the translate() function with the correct arguments as a result of the user clicking the "Translate" link. There are also several ways to do this. What I'm going to do is simply embed a function call into the href attribute of the link:

app/templates/_post.html : Translate link handler.

  <span id="translation{{ post.id }}"> <a href="javascript:translate( '#post{{ post.id }}', '#translation{{ post.id }}', '{{ post.language }}', '{{ g.locale }}');">{{ _('Translate') }}</a> </span> 

The href link element can accept any JavaScript code if it has the javascript: prefix, so this is a convenient way to make a call to the translation function. , , {{ }} . . # , post<ID> translation<ID> , , ID .

Now the live translation feature is complete! If you have set a valid Microsoft Translator API key in your environment, you should now be able to trigger translations. Assuming you have your browser set to prefer English, you will need to write a post in another language to see the "Translate" link. Below you can see an example:

! Microsoft Translator API , . , , "". :

, , , :

 (venv) $ flask translate update 

messages.po , , GitHub.

, :

 (venv) $ flask translate compile 

There