How LinkedIn makes localization in 19 languages for 1 night
“I want that after the programmer added a new line to the interface, she herself translated into 19 languages and put herself in SVN and was ready for release in the morning” - this is the dream of any developer who has tasted the forbidden fruit of localizing the product into foreign languages. In Alconost, we help if we do not fulfill this dream, then at least come closer to it. Yes, a solution similar to the one described in the article exists not only for LinkedIn developers, but also for mere mortals.
How the process is built in LinkedIn - in this article (note - Java).
Internationalization has a critical impact on the operations and development of the LinkedIn network, which is currently available in 19 different languages. To speed up work with localized texts, an international engineering team has developed a system for dynamically introducing translated content lines into working services.
')
The new system allows us to quickly modify the content: make quick edits and changes without involving developers, rebuilding and restarting the service.
All content is originally written in English by our developers and product managers. Usually the text that needs translation is contained in the properties file:
Then our full-time localization team translates the content into different languages. Here, for example, the same properties-file, translated into Italian:
In order for the text to be displayed on the page, we use the internationalization functions in the templates, which refer to the properties file for the user locale using the specified key.
Prior to the introduction of dynamic language loading, the system collected all the properties files into one artifact together with the application code (WAR). This led to certain problems:
The new system collects and uploads the properties-files separately from the application code. We introduced the concept of a language pack - this is a JAR file containing all the translated content for a specific language. Updated versions of such language packs can be uploaded to the web server at any time. They can also be rolled back at any time if errors are detected.
We added a new resource loading library that determines the availability of new language packs and starts using updated translations — all without restarting the service. If the library does not find the translation, it uses the original English strings.
The introduction of new translations is only part of the big picture: we also needed to find a way to quickly find new and freshly modified strings, deliver them to translators and include the result of the translation into the language pack. This is how our process looks like:
Due to the fact that the system of dynamic loading of languages separates the localized texts from the code, it has become possible to spread new lines of content into the working service before the code of the application that uses them. Therefore, now all our internationalization resources must be backward compatible with previous versions in order to ensure that nothing breaks when new lines are introduced.
In this context, backward compatibility means that adding new lines is always safe, but if you change an existing one, you must keep the number and type of variables unchanged. For example, initially we had this line:
We can easily change some words:
But deleting, adding, or changing the type of variables will break backward compatibility, since the application code will only provide values for the old variables:
We achieve backward compatibility with a pre-commit code that prevents the deletion of resources and at the same time checks that updates to existing text resources are compatible with a set of variables. The code snippet below demonstrates part of our validation logic:
The new system has accelerated the translation process in LinkedIn: getting new lines to a working service has become much faster and easier, and we were able to gradually roll out the translations and roll them back if necessary. And most importantly, our internationalization system can now scale to accommodate the growing number of applications, languages, and participants.
About the translator
The article is translated in Alconost.
Alconost is engaged in the localization of applications, games and websites in 60 languages. Language translators, linguistic testing, cloud platform with API, continuous localization, 24/7 project managers, any formats of string resources.
We also make advertising and training videos - for websites selling, image, advertising, training, teasers, expliners, trailers for Google Play and the App Store.
Read more: https://alconost.com
How the process is built in LinkedIn - in this article (note - Java).
Internationalization has a critical impact on the operations and development of the LinkedIn network, which is currently available in 19 different languages. To speed up work with localized texts, an international engineering team has developed a system for dynamically introducing translated content lines into working services.
')
The new system allows us to quickly modify the content: make quick edits and changes without involving developers, rebuilding and restarting the service.
Introduction
All content is originally written in English by our developers and product managers. Usually the text that needs translation is contained in the properties file:
add_to_network__send_invitation=Send invitation add_to_network__cancel=Cancel add_to_network__add_message=Add a personal message add_to_network__user_message=I'd like to add you to my professional network.\n\n- {0} Then our full-time localization team translates the content into different languages. Here, for example, the same properties-file, translated into Italian:
add_to_network__send_invitation=Invia un invito add_to_network__cancel=Annulla add_to_network__add_message=Aggiungi un messaggio personale add_to_network__user_message=Vorrei aggiungerti alla mia rete professionale.\n\n\n\n-{0} In order for the text to be displayed on the page, we use the internationalization functions in the templates, which refer to the properties file for the user locale using the specified key.
<form> <label for="message">${i18n('add_to_network__add_message')}</label> <textarea id="message">${i18n('add_to_network__user_message', fullname)}</textarea> <input type="submit" value="${i18n('add_to_network__send_invitation')}"/> <input type="button" value="${i18n('add_to_network__cancel')}"/> </form> The old way
Prior to the introduction of dynamic language loading, the system collected all the properties files into one artifact together with the application code (WAR). This led to certain problems:
- Adding translations meant rebuilding and restarting the entire service.
- If there was an error in the translation, it was impossible to simply return to the old version of the text: it was necessary to roll back the application code to the previous version.
- Translations can be used in many services - and all of them had to be recompiled and restarted in case of anything.
In a new way
The new system collects and uploads the properties-files separately from the application code. We introduced the concept of a language pack - this is a JAR file containing all the translated content for a specific language. Updated versions of such language packs can be uploaded to the web server at any time. They can also be rolled back at any time if errors are detected.
We added a new resource loading library that determines the availability of new language packs and starts using updated translations — all without restarting the service. If the library does not find the translation, it uses the original English strings.
Process
The introduction of new translations is only part of the big picture: we also needed to find a way to quickly find new and freshly modified strings, deliver them to translators and include the result of the translation into the language pack. This is how our process looks like:
- The engineer sends new or updated English language strings to the version control system.
- The localization server scans the version control system for changes once a day and issues a request to translate all new and changed strings.
- Once an hour, the localization server collects ready-made translations. It checks the new content and then publishes a full language pack with all translations for a specific language in the Vault.
- The deployment system sends the updated language packs for testing once an hour and loads them into the running service twice a day.
- In case the localization team needs to change the translation urgently, there is an opportunity to load the translations manually at any time with one click.
backward compatibility
Due to the fact that the system of dynamic loading of languages separates the localized texts from the code, it has become possible to spread new lines of content into the working service before the code of the application that uses them. Therefore, now all our internationalization resources must be backward compatible with previous versions in order to ensure that nothing breaks when new lines are introduced.
In this context, backward compatibility means that adding new lines is always safe, but if you change an existing one, you must keep the number and type of variables unchanged. For example, initially we had this line:
accept_invite__hello_connect=Hello {0}, would you like to connect with {1}? We can easily change some words:
# This change is backwards compatible accept_invite__hello_connect=Hi {0}, would you like to add {1} to your professional network? But deleting, adding, or changing the type of variables will break backward compatibility, since the application code will only provide values for the old variables:
# This change is NOT backwards compatible! accept_invite__hello_connect=Hello {0}, would you like to connect with {1}, a coworker at {2}? We achieve backward compatibility with a pre-commit code that prevents the deletion of resources and at the same time checks that updates to existing text resources are compatible with a set of variables. The code snippet below demonstrates part of our validation logic:
/** * Verify that translation isn't missing indexes used with the * original as well as does not specify additional indexes not * present in the original. */ private boolean verifyPlaceholderIndexMatch(String originalMessageTemplate, PlaceholderInfo originalPlaceholderInfo, PlaceholderInfo placeholderInfo) { if(originalPlaceholderInfo.keySet().equals(placeholderInfo.keySet()) == false) { // remove all index numbers in the translation from the index // set in the original to determine what's missing in the // translation Set<String> missingIndexSet = new HashSet<String>(originalPlaceholderInfo.keySet()); missingIndexSet.removeAll(placeholderInfo.keySet()); // remove all index numbers in the original from the set of // indexes in the translation to determine what extra index // we have in the translation Set<String> extraIndexSet = new HashSet<String>(placeholderInfo.keySet()); extraIndexSet.removeAll(originalPlaceholderInfo.keySet()); return false; } return true; } 19 languages are implemented, all the rest are ahead!
The new system has accelerated the translation process in LinkedIn: getting new lines to a working service has become much faster and easier, and we were able to gradually roll out the translations and roll them back if necessary. And most importantly, our internationalization system can now scale to accommodate the growing number of applications, languages, and participants.
About the translator
The article is translated in Alconost.
Alconost is engaged in the localization of applications, games and websites in 60 languages. Language translators, linguistic testing, cloud platform with API, continuous localization, 24/7 project managers, any formats of string resources.
We also make advertising and training videos - for websites selling, image, advertising, training, teasers, expliners, trailers for Google Play and the App Store.
Read more: https://alconost.com
Source: https://habr.com/ru/post/173467/
All Articles