Flask Mega-Tutorial, Part XIII: I18n and L10n (Edition 2018)

Miguel grinberg

There

This is the thirteenth part of the Flask Mega-Tutorial series, in which I will tell you how to implement multi-language support for your application. As part of this work, you will also learn about creating your own CLI extensions for flask.

For reference, below is a list of articles in this series.

Note 1: If you are looking for old versions of this course, this is here .

Note 2: If suddenly you would like to speak in support of my (Miguel) work, or simply do not have the patience to wait for the article for a week, I (Miguel Greenberg) offer the full version of this manual (in English) in the form of an electronic book or video. For more information, visit learn.miguelgrinberg.com .

This chapter focuses on internationalization and localization, abbreviated as I18n and L10n. In order to make the application accessible to people who do not speak English, a translation process will be implemented, which, with the help of language translation services, will allow me to offer users a language-application to choose from.

GitHub links for this chapter: Browse , Zip , Diff .

Introduction to Flask-Babel

As you probably already guessed, there is an extension Flask, which simplifies the work with translations. The extension is called Flask-Babel and is installed using pip:

(venv) $ pip install flask-babel 

Flask-Babel is initialized, like most other Flask extensions:

app/__init__.py : Initializing Flask-Babel.

 # ... from flask_babel import Babel app = Flask(__name__) # ... babel = Babel(app) 

As an example, I will tell you how to translate the application into Spanish, as I happen to speak this language. I could also work with translators who know other languages ​​and support them. To keep track of the list of supported languages, add a configuration variable:

config.py : List of supported languages.

 class Config(object): # ... LANGUAGES = ['en', 'es'] 

I use two-letter language codes for this application, but if you need to be more specific, you can add a country code. For example, you can use en-US , en-GB and en-CA to support English with different dialects of the United States, United Kingdom, or Canada.

The Babel instance is provided by the localeselector decorator. The decorated function is called for each request to select the translation of the language to use:

app/__init__.py : Select your preferred language.

 from flask import request # ... @babel.localeselector def get_locale(): return request.accept_languages.best_match(app.config['LANGUAGES']) 

Here I use the attribute of the Flask request object, called accept_languages . This object provides a high-level interface for working with the Accept-Language header sent by clients with the request. This header indicates the client’s language and language preferences as a weighted average. The contents of this header can be configured on the browser settings page, while the default is usually imported from the language settings in the computer's operating system. Most people do not even know that such a setting exists, but this is useful because users can provide a list of preferred languages, each of which has weight. If you're interested, here's an example of the Accept-Languages complex header:

 Accept-Language: da, en-gb;q=0.8, en;q=0.7 

We see that Danish ( da ) is the preferred language (the default weight value is 1.0), followed by British English ( en-gb ) with a weight of 0.8, and as a last option General English ( en ) with a weight of 0.7 .

To choose the best language, you need to compare the list of languages ​​requested by the client with the languages ​​that the application supports, and using the weights provided by the client, find the best language. Perhaps this logic seems too complicated to you, but all this is encapsulated into the best_match() method, which takes the list of languages ​​offered by the application as an argument and returns the best choice.

Text marking for translation in Python source code

Early rejoiced. Now about the sad. A typical workflow when creating an application in several languages ​​is to markup all texts that need translation in the source code. After the texts are marked, Flask-Babel will scan all the files and extract these texts into a separate translation file using the gettext tool. Unfortunately, this is a tedious task that needs to be completed for translation.

I'm going to show you some examples of this labeling, but you can get a complete set of changes from the package for this chapter or the GitHub repository.

The way in which texts are marked for translation is to wrap them in a function call, which is called as a _() convention, just an underscore. The simplest cases are those where literal strings appear in the source code. The following is an example of a flash () `statement:

 from flask_babel import _ # ... flash(_('Your post is now live!')) 

The idea is that the _() function transfers the text to the base language (in this case, English). She will use the best language in her opinion, chosen by the get_locale function, decorated with the localeselector function, to find the right translation for this client. Then the _() function will return the translated text, which in this case will be an argument for flash() .

Unfortunately, not all cases are so simple. Consider this other flash() call from the application:

 flash('User {} not found.'.format(username)) 

This text has a dynamic component that is inserted in the middle of the static text. The _() function has a syntax that supports this type of text, but is based on the old string substitution syntax:

 flash(_('User %(username)s not found.', username=username)) 

There is an even more difficult case. Some string literals are assigned outside the request, usually when the application is launched, so when these texts are evaluated, there is no way to know which language to use. Examples of this are labels associated with form fields. The only solution for processing these texts is to find a way to postpone the evaluation of the string until it is used, which will be under the actual query. Flask-Babel provides a lazy evaluation version of the _() deferred calculation, called lazy_gettext() :

 from flask_babel import lazy_gettext as _l class LoginForm(FlaskForm): username = StringField(_l('Username'), validators=[DataRequired()]) # ... 

Here I import the alternative translation function and rename it to _l () , so that it was similar in name to the original _() . This new function transfers the text to a special object containing the translation method, which will take place later, when the string is used.

The Flask-Login extension displays a message every time a user is redirected to the login page. This message is written in English and is formed in the defaults of the extension itself. To make sure that this message is also translated, I am going to override the default message and provide another option decorated by the _l() function for a deferred call:

 login = LoginManager(app) login.login_view = 'login' login.login_message = _l('Please log in to access this page.') 

Markup of texts for translation in templates

In the previous section, you saw how to mark up translatable texts in the source code of Python modules, but this is only part of the process, since the template files also contain text. The _() function is also available in templates, so the process is very similar. For example, consider this HTML fragment from 404.html :

 <h1>File Not Found</h1> 

Version with translation support:

 <h1>{{ _('File Not Found') }}</h1> 

Note that here, besides wrapping text with _() , you need to add {{...}} to make _() compute instead of being considered a literal in the template.

For more complex phrases containing dynamic components, you can use arguments:

 <h1>{{ _('Hi, %(username)s!', username=current_user.username) }}</h1> 

There is a particularly difficult case in _post.html , which made me understand:

  {% set user_link %} <a href="{{ url_for('user', username=post.author.username) }}"> {{ post.author.username }} </a> {% endset %} {{ _('%(username)s said %(when)s', username=user_link, when=moment(post.timestamp).fromNow()) }} 

The problem here is that I wanted the username to be a link pointing to the user profile page, not just a name, so I had to create an intermediate variable called user_link using the set and endset template directives and then pass this as translation function argument.

As I mentioned above, you can download a version of the application with all the translated texts in the Python source code and templates.

Extract text for translation

After you have an application with all _() and _l() in place, you can use the pybabel command to extract them into the a.pot file, which means portable object template . This is a text file containing all texts that have been marked as needing translation. The purpose of this file is to serve as a template for creating translation files into any other language.

The extraction process requires a small configuration file that tells pybabel which files to scan for translatable texts. Below you can see the babel.cfg that I created for this application:

babel.cfg : PyBabel configuration file.

 [python: app/**.py] [jinja2: app/templates/**.html] extensions=jinja2.ext.autoescape,jinja2.ext.with_ 

The first two lines define the names of the Python and Jinja2 template files, respectively. The third line defines two extensions provided by the Jinja2 template engine, which help Flask-Babel correctly analyze template files.

To extract all texts in a .pot file, you can use the following command:

 (venv) $ pybabel extract -F babel.cfg -k _l -o messages.pot . 

The pybabel extract command reads the configuration file specified in the -F parameter, and then scans all the py and html files in directories corresponding to the configured sources, starting from the directory specified in the command (current directory or . In this case.) By default, pybabel will be look for _() as a text marker, but I also used the lazy option, which I imported as _l() , so I need to tell the search tool with the -k _l option. The -o parameter specifies the name of the output file.

I should note that messages.pot is not a file that should be included in the project. This is a file that can be easily regenerated at any time simply by running the command above again. Thus, there is no need to transfer this file to the version control system.

Creating a Language Catalog

The next step in the process is to create a translation for each language, which will be supported in addition to the basic one, which in this case is English. I said that I was going to start by adding Spanish ( es language code), so the command that does this is:

 (venv) $ pybabel init -i messages.pot -d app/translations -l es creating catalog app/translations/es/LC_MESSAGES/messages.po based on messages.pot 

The pybabel init command takes the messages.pot file as input and creates a new directory for a specific language specified in the -l parameter to the directory specified in the -d parameter. I will save all translations in the app / translations directory, because there Flask-Babel will look for the default translation files. The command will create an es subdirectory inside this directory for Spanish data. In particular, there will be a new file called app / translations / es / LC_MESSAGES / messages.po . That is where the translations should be made.

If you want to support other languages, then repeat the above command with each of the language codes. So that each language gets its own repository with the file messages.po .

This messages.po file created in each language repository uses a format that is the de facto standard for language translations, the format used by the gettext utility. Here are a few lines of the beginning of Spanish messages.po:

 # Spanish translations for PROJECT. # Copyright (C) 2017 ORGANIZATION # This file is distributed under the same license as the PROJECT project. # FIRST AUTHOR <EMAIL@ADDRESS>, 2017. # msgid "" msgstr "" "Project-Id-Version: PROJECT VERSION\n" "Report-Msgid-Bugs-To: EMAIL@ADDRESS\n" "POT-Creation-Date: 2017-09-29 23:23-0700\n" "PO-Revision-Date: 2017-09-29 23:25-0700\n" "Last-Translator: FULL NAME <EMAIL@ADDRESS>\n" "Language: es\n" "Language-Team: es <LL@li.org>\n" "Plural-Forms: nplurals=2; plural=(n != 1)\n" "MIME-Version: 1.0\n" "Content-Type: text/plain; charset=utf-8\n" "Content-Transfer-Encoding: 8bit\n" "Generated-By: Babel 2.5.1\n" #: app/email.py:21 msgid "[Microblog] Reset Your Password" msgstr "" #: app/forms.py:12 app/forms.py:19 app/forms.py:50 msgid "Username" msgstr "" #: app/forms.py:13 app/forms.py:21 app/forms.py:43 msgid "Password" msgstr "" 

If you skip the header, you can see that below is a list of strings that were extracted from the _() and _l() calls. For each text you get a link to the location of the text in the application. Then the string msgid contains the text in the base language, and the next line msgstr contains the empty string. These blank lines must be edited to have the text in the target language.

There are many applications that work with the translation of .po files. If you feel comfortable when editing a text file, then this is sufficient, but if you are working with a large project, then it may be recommended to work with a specialized editor. The most popular translation application is open source poedit , which is available for all major operating systems. If you are familiar with VIM , then the po.vim plugin provides some key mappings that make working with these files easier.

Below you can see part of the Spanish version of messages.po after I added the translation:

 #: app/email.py:21 msgid "[Microblog] Reset Your Password" msgstr "[Microblog] Nueva Contraseña" #: app/forms.py:12 app/forms.py:19 app/forms.py:50 msgid "Username" msgstr "Nombre de usuario" #: app/forms.py:13 app/forms.py:21 app/forms.py:43 msgid "Password" msgstr "Contraseña" 

The download package for this chapter also contains this file, so you do not have to worry about this part of the application.

The file messages.po is a kind of source file for translations. If you want to start using these translated texts, the file must be compiled into a format that is effective for the application to use at run time. To collect all translations for an application, you can use the pybabel compile as follows:

 (venv) $ pybabel compile -d app/translations compiling catalog app/translations/es/LC_MESSAGES/messages.po to app/translations/es/LC_MESSAGES/messages.mo 

This operation adds the messages.mo file next to the messages.po in each language repository. The .mo file is the file that Flask-Babel will use to download translations to the application.

After creating messages.mo for Spanish or any other languages ​​added to the project, these languages ​​are ready for use in the application. If you want to see how the application looks in Spanish, you can change the language configuration in a web browser so that Spanish is the preferred language. For Chrome, this is the extended part in the settings:

If you prefer not to change your browser settings, another alternative is to force the use of a language, forcing the localeselector function localeselector always return the same. For Spanish, it looks like this:

app/__init__.py : Spanish selection (directive).

 @babel.localeselector def get_locale(): # return request.accept_languages.best_match(app.config['LANGUAGES']) return 'es' 

Running the application in a browser configured in Spanish, or in the case of forcing the es value of the localeselector , will cause all texts to appear in Spanish in the application.

Update translations

One of the common situations when working with translations is that you can start using the translation file, even if it is incomplete. This is perfectly normal; you can compile incomplete files messages.po . In this case, po-files and any available translations will be used, and the missing ones will use the base language. Then you can continue working on the translations and compile to update messages.mo .

Another common occurrence is if you skipped some texts when adding _() wrappers. In this case, you will see that the texts you have missed will remain in English, because Flask-Babel does not know anything about them. In this case, you need to add _() or _l() wrappers when detecting texts that do not have them, and then perform an update procedure that includes two steps:

 (venv) $ pybabel extract -F babel.cfg -k _l -o messages.pot . (venv) $ pybabel update -i messages.pot -d app/translations 

The extract command is identical to the one I described earlier, but now it will generate a new version of messages.pot with all the previous texts, plus something new that you have recently wrapped with _() or _l() . An update call accepts a new messages.pot file and merges it into all the messages.po files associated with the project. It will be an intelligent merge, in which any existing texts will be left alone, while only entries that have been added or deleted in messages.pot will be affected.

After updating messages.po, you can continue and translate all new tests, and then compile the messages again to make them available to the application.

Date and time translation

Now I have a complete Spanish translation for all the texts in the Python code and templates. But if you run the application in Spanish and are a good observer, then you will notice that there are a few more places that have remained in English. I mean timestamps created by Flask-Moment and moment.js, which obviously were not included in the translation, because none of the texts created by these packages is part of the source code or application template.

moment.js supports localization and internationalization, so all I need to do is configure the correct language. Flask-Babel returns the selected language and locale for this case using the get_locale() function, so I'm going to add a locale to the g object in order to access it from the base template:

app/routes.py : Save the selected language in flask.g.

 # ... from flask import g from flask_babel import get_locale # ... @app.before_request def before_request(): # ... g.locale = str(get_locale()) 

The get_locale() function from Flask-Babel returns an object, but I just want to have a language code that can be obtained by converting an object to a string. Now that I have g.locale , I can access it from the base template to set up moment.js with the correct language:

app / templates / base.html : Set the locale for moment.js.

 ... {% block scripts %} {{ super() }} {{ moment.include_moment() }} {{ moment.lang(g.locale) }} {% endblock %} 

And now all dates and times should appear in the same language as the text. Below you can see how the app looks in Spanish:

At this stage, all texts, except those that were provided by the user in blog posts or profile descriptions, should be translated into other languages.

Command line enhancements

You will probably agree with me that the pybabel commands are long and hard to remember. I'm going to use this feature to show you how you can create custom commands integrated with the flask command. So far you have seen the use of flask run , flask shell , and several flask db sub-commands in Flask-Migrate. In fact, it is easy to add application-specific commands to flask. So now I'm going to create some simple commands that run the pybabel with all the arguments that are specific to this application. The commands I'm going to add are:

• flask translate init LANG add new language
• flask translate update update all language repositories
• flask translate compile to compile all language repositories

babel export will not be a command, because the generation of the messages.pot file is always a prerequisite for the execution of init or update commands. Therefore, the implementation of these commands will generate the translation template file as a temporary file.

Flask relies on Click for all its command-line operations. Commands, such as translate , which are the root for several subcommands, are created using the app.cli.group() decorator. I'm going to put these commands in a new module called app / cli.py :

app / cli.py : Translate a group of commands.

 from app import app @app.cli.group() def translate(): """Translation and localization commands.""" pass 

, docstring. , , .

Update - compile - , :

app/cli.py : .

 import os # ... @translate.command() def update(): """Update all languages.""" if os.system('pybabel extract -F babel.cfg -k _l -o messages.pot .'): raise RuntimeError('extract command failed') if os.system('pybabel update -i messages.pot -d app/translations'): raise RuntimeError('update command failed') os.remove('messages.pot') @translate.command() def compile(): """Compile all languages.""" if os.system('pybabel compile -d app/translations'): raise RuntimeError('compile command failed') 

, translate . , translate() — , , Click . , translate() , docstrings - --help .

, , . , . , RuntimeError , . update() , , messages.pot , .

init . :

app/cli.py : Init — sub- .

 import click @translate.command() @click.argument('lang') def init(lang): """Initialize a new language.""" if os.system('pybabel extract -F babel.cfg -k _l -o messages.pot .'): raise RuntimeError('extract command failed') if os.system( 'pybabel init -i messages.pot -d app/translations -l ' + lang): raise RuntimeError('init command failed') os.remove('messages.pot') 

@click.argument . Click , , init .

, . microblog.py :

microblog.py : .

 from app import cli 

, , cli.py , - , .

, flask --help translate . flask translate --help -, :

 (venv) $ flask translate --help Usage: flask translate [OPTIONS] COMMAND [ARGS]... Translation and localization commands. Options: --help Show this message and exit. Commands: compile Compile all languages. init Initialize a new language. update Update all languages. 

, . , :

 (venv) $ flask translate init <language-code> 

_() _l() :

 (venv) $ flask translate update 

:

 (venv) $ flask translate compile 

There