Localization of products or another post with the title "Toggle to start"
Almost from its very birth, Lingualeo has been conceived as a global service through which students all over the world will learn English. Brazil was chosen as the testing ground for the process of entering new markets.
In the summer of 2012, the team was faced with the task of localizing the Lingualeo service into Portuguese. At the same time it was necessary to create a process that is easily scalable into other languages.
The process of localization of the product was born in Montenegro, where the Lingualeo team traveled to work intensively on new features. According to the initial localization assessment, it should have taken only a couple of weeks of development before the system can be given to translators. The estimates turned out to be too optimistic, the team “stuck in” for 1.5 months. Lingualeo went through a difficult path and faced a number of problems that others can avoid by reading this post to the end. We asked Igor Lyubimov , Head of localization, honorary veteran of localization Lingualeo, to retell the story in detail.
System selection
')
In order to choose a suitable tool, it was necessary to determine which tasks he would solve. First of all, it was important for us that the product supports localization on several platforms: several file formats for the website, an Android application, an iOS application, a Windows Phone application, and we were also interested in the API - for tight integration of the CI system and translation platforms.
Initially, we looked at the good old Pootle, since our backend was written in PHP, and we used gettext with .po files for localization. We quickly realized that this was not our option, because for translating JSON and other files, we had to write a bunch of converters - Pootle did not support all the “zoo” formats we needed. Plus, I didn’t want to host the translation platform at home, so we started looking for some SaaS solution.
Our criteria for selection are: support of all possible formats out of the box, the ability to add new lines through the API, convenient user rights management, high uptime, short integration time, and the ability to launch crowdsource-localization in the future.
As a result, we stopped at a simple but functional service WebTranslateIt.com (hereinafter - “WTI”), as it fit all our requirements.
I note that if we chose the system now, then we would stop at Crowdin . It is more convenient and more interesting than the one we use, but not so much as to invest in refactoring many parts of the project.
So, WebTranslateIt.com. This is a small ready-made SaaS platform, created by the Frenchman Édouard Brière. 3-4 years ago, it became a real find for Lingualeo. At that time, we liked her very much. It had a good API and it met the basic requirement - the ability to localize various formats and platforms. Plus, the friendly creator answered all questions very promptly and made improvements specifically for us.
By the way, we wrote and published in Open Source a library under composer (PHP) for working with WebTranslateIt.com: https://packagist.org/packages/lingualeo/wti-api . We are waiting for your pull requests!
Integration process
Web, browser extensions and mobile versions
The main task that we faced was not to burden developers with unnecessary localization maintenance tasks, but to do everything possible so that they were busy directly creating the product. Localization issues must be removed from the main workflow. We imagined it this way: the developer writes new lines, sends them to the system (either with one team or automatically), and then they are translated into several languages ​​and returned to the project, without any obvious efforts by the developer. The mantra “What can be automated, it should be automated” often jumps in the conversations of our developers.
Our thoughts led us to create a special utility that was designed to make life easier for developers. For the website, we wrote a couple of console utilities.
The first adds a new line to the selected file.
Sometimes it is convenient, but the second utility is the most useful, which parses our templates, pulls out strings from them for localization, adds them to WTI and does other magic. In order for the utility to correctly understand which strings should be translated and which should not, developers simply need to frame the corresponding text with the @ symbol.
An example for our client templates:
After that we execute:
And in WTI, a new line is created in the file someTemplate.jst with the hi_i_am_leo key and the Russian text Hello! I am Leo !, and our template takes a different look:
The developer needs only to commit changes and not to sweat about translations.
Similar parsers were made for other template formats, including Twig and simple PHP scripts with business logic. Depending on the file extension, different formats for replacing placeholders are used.
In our process of working with git-branches, we follow the rules of git-flow. This allows us to associate the CI system with tasks from Jira, helps to maintain order in the repository, and indeed, it's cool. Once we have a branch defined by ticket number, why not use it to define strings for localization? The question “What lines were added as part of the task DEV-1940?” Will be removed by itself. We have upgraded the wti-parse utility - it takes the current branch number and adds all the lines with the additional task label. As a result, in the WTI interface it looks like this:
Lines are added, the programmer goes on to make the product, nothing more is needed from the part of the translations.
Further, with each build on our CI server, the latest translations are downloaded from WTI, and it works like a clock :)
Courses
In the same system, we have integrated translation courses. Courses are not interface lines, they are created by content managers. Initially, we tried to do translations manually, through sending Word-files, but we quickly realized that this was a bad idea: this is a possible, but absolutely non-scalable process. Soon the course builder was integrated with WTI. And now, as soon as the Lingualeo content managers make any changes in the courses (add exercises, change the wording, etc.), they are immediately reflected in the WTI. All translators receive notifications, and the above described algorithm runs. Thus, we do not need to manually manage the translation process: everything works automatically.
Email newsletter
Like any service that builds a close dialogue with its users, Lingualeo does various emails . About 13 000 000 users, tens of millions of emails sent every month - in our case there was no point in implementing the entire infrastructure inside, so we use an external email provider. But the trouble is that our provider does not have built-in mechanisms for localizing letter templates, and therefore we invented the next “bicycle” by crossing two APIs - Emarsys (namely, they send all our letters) and WTI.
Every 6 hours, several scripts retrieve Russian letter templates from Emarsys, extract the strings from them for localization, create files for WTI from them, and from WTI take ready translations and create templates in the necessary languages ​​in Emarsys. And already in the settings of mailings one or another templates are used, depending on the user's native language.
The process of working with translators
For simultaneous translation into several languages ​​we follow the following scheme of work. First we create lines in Russian, then we translate into English, and from English into other languages ​​(Turkish, Spanish, Portuguese, a couple of Spanish dialects, etc.).
We agreed that a translator into English enters the system on Tuesday and Thursday in the morning. Picks up lines, and within a few hours prepares all necessary translations. Translators to other languages ​​come in the same days, but in the evening. Thus, within 24 hours we receive all transfers. The schedule is tailored to our release cycle so that we have new lines on time. Of course, sometimes there are still problems, but this happens only when the developer added lines at the very last moment, she was quickly tested and the next day went into release.
This system is suitable not only for work with translation agencies, free-lancers or regular translators, but also for using crowd sourcing in translation. When we launched the “Translate Lingualeo” project and invited our users to translate at least a few lines into their familiar languages, we created a landing page and integrated a page with statistics from WTI with the site.
This is the form for translating phrases:
Internationalization
As a result, the Lingualeo technology platform and localization processes were developed. It works for both the web and mobile applications, without requiring the constant participation of team members. Localization is transparent to developers: they simply write code, develop a product for target markets. Translators do their work regardless of them. Today, adding ready-made translations takes only 1-2 days.
Of course, product translation is only part of the process of preparing a product for launch in new language markets. Local currencies, payment systems, dictionaries for new languages, the creation of special courses, the establishment of groups in social networks in new languages ​​- all these activities are also simplified and automated to the maximum. The entire configuration process is reduced to editing several YML configuration files, and it takes literally several hours to configure the entire environment. But this is the material for the next article.
In the summer of 2012, the team was faced with the task of localizing the Lingualeo service into Portuguese. At the same time it was necessary to create a process that is easily scalable into other languages.
The process of localization of the product was born in Montenegro, where the Lingualeo team traveled to work intensively on new features. According to the initial localization assessment, it should have taken only a couple of weeks of development before the system can be given to translators. The estimates turned out to be too optimistic, the team “stuck in” for 1.5 months. Lingualeo went through a difficult path and faced a number of problems that others can avoid by reading this post to the end. We asked Igor Lyubimov , Head of localization, honorary veteran of localization Lingualeo, to retell the story in detail.
System selection
')
In order to choose a suitable tool, it was necessary to determine which tasks he would solve. First of all, it was important for us that the product supports localization on several platforms: several file formats for the website, an Android application, an iOS application, a Windows Phone application, and we were also interested in the API - for tight integration of the CI system and translation platforms.
Initially, we looked at the good old Pootle, since our backend was written in PHP, and we used gettext with .po files for localization. We quickly realized that this was not our option, because for translating JSON and other files, we had to write a bunch of converters - Pootle did not support all the “zoo” formats we needed. Plus, I didn’t want to host the translation platform at home, so we started looking for some SaaS solution.
Our criteria for selection are: support of all possible formats out of the box, the ability to add new lines through the API, convenient user rights management, high uptime, short integration time, and the ability to launch crowdsource-localization in the future.
As a result, we stopped at a simple but functional service WebTranslateIt.com (hereinafter - “WTI”), as it fit all our requirements.
I note that if we chose the system now, then we would stop at Crowdin . It is more convenient and more interesting than the one we use, but not so much as to invest in refactoring many parts of the project.
So, WebTranslateIt.com. This is a small ready-made SaaS platform, created by the Frenchman Édouard Brière. 3-4 years ago, it became a real find for Lingualeo. At that time, we liked her very much. It had a good API and it met the basic requirement - the ability to localize various formats and platforms. Plus, the friendly creator answered all questions very promptly and made improvements specifically for us.
By the way, we wrote and published in Open Source a library under composer (PHP) for working with WebTranslateIt.com: https://packagist.org/packages/lingualeo/wti-api . We are waiting for your pull requests!
Integration process
Web, browser extensions and mobile versions
The main task that we faced was not to burden developers with unnecessary localization maintenance tasks, but to do everything possible so that they were busy directly creating the product. Localization issues must be removed from the main workflow. We imagined it this way: the developer writes new lines, sends them to the system (either with one team or automatically), and then they are translated into several languages ​​and returned to the project, without any obvious efforts by the developer. The mantra “What can be automated, it should be automated” often jumps in the conversations of our developers.
Our thoughts led us to create a special utility that was designed to make life easier for developers. For the website, we wrote a couple of console utilities.
The first adds a new line to the selected file.
$ wti-add localizationFilename.json uniqueStringKey Please provide russian text > , ! String is added Sometimes it is convenient, but the second utility is the most useful, which parses our templates, pulls out strings from them for localization, adds them to WTI and does other magic. In order for the utility to correctly understand which strings should be translated and which should not, developers simply need to frame the corresponding text with the @ symbol.
An example for our client templates:
<!-- someTemplate.jst --> <h1>! !</h1> <!-- someTemplate.jst --> <h1>@! !@</h1> After that we execute:
$ wti-parse someTemplate.jst localizationFilename.json 1 string to add found Please provide key for “! !” > hi_i_am_leo 1 string parsed and added And in WTI, a new line is created in the file someTemplate.jst with the hi_i_am_leo key and the Russian text Hello! I am Leo !, and our template takes a different look:
<!-- someTemplate.jst --> <h1><%=i18n.localizationFilename.hi_i_am_leo%></h1> The developer needs only to commit changes and not to sweat about translations.
Similar parsers were made for other template formats, including Twig and simple PHP scripts with business logic. Depending on the file extension, different formats for replacing placeholders are used.
In our process of working with git-branches, we follow the rules of git-flow. This allows us to associate the CI system with tasks from Jira, helps to maintain order in the repository, and indeed, it's cool. Once we have a branch defined by ticket number, why not use it to define strings for localization? The question “What lines were added as part of the task DEV-1940?” Will be removed by itself. We have upgraded the wti-parse utility - it takes the current branch number and adds all the lines with the additional task label. As a result, in the WTI interface it looks like this:
Lines are added, the programmer goes on to make the product, nothing more is needed from the part of the translations.
Further, with each build on our CI server, the latest translations are downloaded from WTI, and it works like a clock :)
Courses
In the same system, we have integrated translation courses. Courses are not interface lines, they are created by content managers. Initially, we tried to do translations manually, through sending Word-files, but we quickly realized that this was a bad idea: this is a possible, but absolutely non-scalable process. Soon the course builder was integrated with WTI. And now, as soon as the Lingualeo content managers make any changes in the courses (add exercises, change the wording, etc.), they are immediately reflected in the WTI. All translators receive notifications, and the above described algorithm runs. Thus, we do not need to manually manage the translation process: everything works automatically.
Email newsletter
Like any service that builds a close dialogue with its users, Lingualeo does various emails . About 13 000 000 users, tens of millions of emails sent every month - in our case there was no point in implementing the entire infrastructure inside, so we use an external email provider. But the trouble is that our provider does not have built-in mechanisms for localizing letter templates, and therefore we invented the next “bicycle” by crossing two APIs - Emarsys (namely, they send all our letters) and WTI.
Every 6 hours, several scripts retrieve Russian letter templates from Emarsys, extract the strings from them for localization, create files for WTI from them, and from WTI take ready translations and create templates in the necessary languages ​​in Emarsys. And already in the settings of mailings one or another templates are used, depending on the user's native language.
The process of working with translators
For simultaneous translation into several languages ​​we follow the following scheme of work. First we create lines in Russian, then we translate into English, and from English into other languages ​​(Turkish, Spanish, Portuguese, a couple of Spanish dialects, etc.).
We agreed that a translator into English enters the system on Tuesday and Thursday in the morning. Picks up lines, and within a few hours prepares all necessary translations. Translators to other languages ​​come in the same days, but in the evening. Thus, within 24 hours we receive all transfers. The schedule is tailored to our release cycle so that we have new lines on time. Of course, sometimes there are still problems, but this happens only when the developer added lines at the very last moment, she was quickly tested and the next day went into release.
This system is suitable not only for work with translation agencies, free-lancers or regular translators, but also for using crowd sourcing in translation. When we launched the “Translate Lingualeo” project and invited our users to translate at least a few lines into their familiar languages, we created a landing page and integrated a page with statistics from WTI with the site.
This is the form for translating phrases:
Internationalization
As a result, the Lingualeo technology platform and localization processes were developed. It works for both the web and mobile applications, without requiring the constant participation of team members. Localization is transparent to developers: they simply write code, develop a product for target markets. Translators do their work regardless of them. Today, adding ready-made translations takes only 1-2 days.
Of course, product translation is only part of the process of preparing a product for launch in new language markets. Local currencies, payment systems, dictionaries for new languages, the creation of special courses, the establishment of groups in social networks in new languages ​​- all these activities are also simplified and automated to the maximum. The entire configuration process is reduced to editing several YML configuration files, and it takes literally several hours to configure the entire environment. But this is the material for the next article.
Source: https://habr.com/ru/post/273223/
All Articles