History of one internationalization

Initial data

Within the framework of one company, several ideologically linked web services with Russian-language content live and develop. At one point, they were destined to set foot on the beautiful but thorny path of internationalization. Moreover, the list of supported languages ​​was planned to expand with enviable intensity.

We had to go through the following stages:

1. Preparation of projects for internationalization;
2. Integration into the workflow development of the process of translating texts;
3. Selection and configuration of tools for working with a translation service;
4. The development of interaction between developers and translators.

I will tell you about the problems and solutions that took place in the framework of this task (and also a little about dancing gnomes) below.

First step

He is the hardest. Not being supporters of premature optimizations (as well as not having time for it), we decided to take the simple “trodden” path.

What do we have in the arsenal? Projects are based on the PHP framework Yii , which has a built -in internationalization mechanism . Well, great, it would be foolish not to use it.
')
Let's go: each project has got a “language” directory, where at the beginning there was only the source and everyone's favorite RU. Then all the texts in the templates, error messages and other internationalized content moved there in the form of dictionaries in the key-value format. Here the first lesson “Kapitansky” was learned: think about the internalization of projects in advance . If today there is no requirement to support another language, then it may appear tomorrow. The first step should not be the hardest.

Since the services had a common ideological foundation, we immediately encountered a large number of duplicate keys scattered across dictionaries in different projects. The logical solution would be the elimination of duplicates and “sharing” common dictionaries for all projects. This will reduce the cost of making changes to existing translations, as well as contribute to the development of new projects in this group.

As a result, the general dictionaries were moved to the framework level and we came up with the following scheme:


Fuf. It seems our projects are ready to be “translated”. Let's see what happens.

Internationalize it

Oh, I forgot to mention the fact that we had our own team of translators. True, they had not previously interacted with the development of web services, were engaged in their tasks, with their tools and in their processes. But still it was wonderful, because working with external contractors would complicate our lives even more.

The first time we worked with translators in the black box mode. The developers packed the dictionaries into the archive (having previously collected them from all the projects), sent them by letter to the translators and, having received a preliminary forecast on the terms, waited for a similar archive with the translated dictionaries.

And now, the long-awaited moment came when our services acquired a second language version.


Hooray, a rainbow began to play in the sky and the dwarfs began to dance. But I didn’t say anything, didn’t I? Between the packing of the dictionaries in the archive and the dancing of the dwarfs, life did not stop and the first alarm bells were already heard.

First, translations are translations, but there are other tasks. Every project that fell under internationalization developed more or less actively. New features were made, bugfixes were added, and often this affected content: new texts were added, existing ones were edited. The first full translation lasted about three weeks and during this time the dictionaries in the developer repositories have undergone quite large changes.

Here we learned lesson number two: consider synchronizing development and translation processes . In our case, it would be more reasonable to send dictionaries for translation in small parts so that in a couple of days you could get them back and add them to the developers repository. At this time it is necessary to freeze all work affecting these dictionaries. This would help to avoid such a painful manual merging of two versions of dictionaries, highlighting untranslated or edited keys and re-sending them for translation.

The next noticeable problem was the similar questions of translators, repeated with frightening regularity: “You have the phrase“ It’s impossible to open an account ”here. Does this mean a financial account or an account in a sports competition? ”. Yes, we do not have a football simulator, but it was necessary to make sure that the translator had access to the subject area in advance. So, the third lesson: high-quality translation is impossible without context .

With some delay, we provided the translation team with access to our test benches and gave them the opportunity to reproduce some user stories that are important for correct translations. In addition, a final review of the services was conducted with the new language version already connected.

After the long-awaited release, it is time to reflect on the changes in the usual development process, taking into account the new “international” status of services.

Translation difficulties

Made a feature - translate it! This is the motto that entitled the new era in the life of our team. The question of exactly how to translate lay on the shoulders of the developer. In some cases, you could show off your knowledge of the language and independently add something like “Hello, {userName}!” To the dictionaries. But in most situations, you had to pull translators again and again.

This increased the amount of manual work, development time for features and, of course, testing time. Quite often, cases began to occur when a carelessly added translation, hidden in the depths of some service, was already detected after the fact and forced the team to force a full release cycle that included only revisions of translations.

The result of a stormy discussion of this problem was the removal of dictionaries from all projects into a separate library:


This allowed us to conduct “content” releases separately, without affecting the functional part. Plus, the noise associated with the work on translations was removed from the project repositories. Of course, this change did not cost us free. The internationalization modules at the framework level have been expanded, the build system and the build server configuration have been improved. But, as a result, we considered these steps justified.

In anticipation of the emergence of another language version, the protocol of interaction between developers and translators caused legitimate concerns.
The following problems were voiced by translators:

• The lack of a uniform format of incoming transfers. The developers sent dictionaries, pieces of dictionaries, sentences, and even individual words;
• Ineffective at this scheme using its own linguistic tools;
• Lack of access to the current repository with translations for reviews, discussions, viewing the change history.

The developers mostly complained about the large number of manual operations, as well as the need to check the sent dictionaries on the observance of syntax and formatting rules.

We needed a centralized service for working with translations, and we started searching.

Automate it

First of all, a list of requirements was compiled, which the service of our dreams should satisfy:

1. Accessibility for developers, translators and other interested parties;
2. The ability to distinguish between rights within accounts, groups, projects;
3. Support for the required dictionary formats;
4. Full external API;
5. Additional functionality for making transfers:
   
   • Filtering not translated keys;
   • Tracking translation history;
   • Ability to discuss specific cases;
   • Search for duplicate and similar phrases;
6. Flexible notification model:
   
   • Translators - about the appearance of new keys and about editing existing ones;
   • Developers - on the status of translation and on issues that have appeared with translators;
7. Sane interface that does not cause epilepsy attacks.

A quick assessment of the time and resources that we would have to spend on developing and maintaining our own service limited the search direction to ready-made solutions.

It was pleasantly surprised not only the availability of such services, but also their functional diversity. A review of this market segment will be a good topic for another article, therefore, having scrolled time ahead, I will just say that this one is the most suitable for us. It is only necessary to note the fourth lesson: when choosing an external internationalization service, first check it out on a test project . Only by collecting feedback from developers and translators, assessing the reliability of the service, the level of support and the cost of using it, you can make a sensible decision, which later will not be regretted.

By the time we gave a presentation of the new service, the development team had already implemented its API and we had a customized mirror of our repository with translations.


Of course, our own repository with dictionaries remained primary, the external service became just its “web snout”. If in one unfortunate day it dissolves in the air, we, of course, will be upset, but we will quickly roll back to the old scheme, without blocking the development process.

The rules of work on internationalization took the following form:

1. There are new text.
   
   1. The developer adds them to the dictionaries of the “main” language (in our case, Russian);
   2. Using the API, the developer sends dictionaries to the translation service;
   3. Translators receive a notification about the appearance of untranslated texts and translate them;
   4. The developer receives a notification about the receipt of translations and again, using the API, synchronizes dictionaries and updates the repository;
2. Changed existing text:
   
   1. We try to translate the situation to claim 1, i.e. without editing existing keys in dictionaries, add new ones and use them;
   2. If the previous version is not suitable, then the developer makes changes directly to the keys of the dictionary of the “main” language and sends them to the translation service;
   3. A smart service understands that the text of existing keys has changed and marks them in other languages ​​as requiring re-translation;
   4. The translator has the ability to either leave the old translation (for example, if the grammatical error was simply corrected), and specify a new one;
   5. The developer receives notifications and synchronizes dictionaries;
3. Error found in translation:
   
   1. The one who discovered (this may be the developer) either independently or with the help of a translator edits it on the translation service;
   2. During the next synchronization of dictionaries, this change falls into the repository (if necessary, synchronization is forced by a direct appeal to the developers);
4. Added support for new language:
   
   1. A new language is added to the translation service. All his dictionaries are initially marked as not translated;
   2. Depending on the presence in the staff of translators who speak the given language, the translation is carried out either on their own or by using third-party translators by providing them with limited access to the service;
   3. After the translations are completed, the developers make changes to the configuration of the projects so that when synchronizing dictionaries a new language is added to the repository and then participates in the overall process.

These simple rules made our life a little bit better, and also partially eliminated a nervous tic at the mention of the word internationalization. A significant reduction in the number of manual operations, as well as streamlining the interaction protocol of teams, reduced the overhead costs of internationalization in the development of features, and also eliminated most of the errors associated with human factors.

Lesson five and final: like most technological processes, the internationalization process can be automated . The extent of this automation should be determined based on the specific situation and needs of the projects.

Summary

For those who have scrolled and for those who have read to the end, I forgot what was said at the beginning:

1. think about the internalization of projects in advance;
2. consider synchronizing development and translation processes;
3. high-quality translation is impossible without context;
4. When choosing an external internationalization service, pre-test it on a test project;
5. like most technological processes, the internationalization process can be automated.

Outside of this article, there was an equally interesting aspect of the internationalization of web services - test automation in the world of multilingualism. But more about that another time.

I hope that the experience I described will be useful for future internationalizers.

Thank.