Serge - the solution for continuous localization from Evernote
Today I want to tell you about the project that I worked on (and continue to work) at Evernote since 2008, and which became Free Software a few days ago.
For many developers, localization is associated with an additional layer of problems: how to keep localized resources up to date? And what if languages are not 2-3, but 20-30? How to send new lines in time for translation? But what if during the translation the development went ahead, and there were no lines anymore, but there were new ones? How to merge resource files sent by translators? It is no secret that because of this, many people simply hammer to localization or try to postpone it for later.
')
Evernote now has more than 150 million users worldwide, more than 70% of these users are outside the US, every month we translate 15,000 new words in more than 40 projects into more than 26 languages, and release new releases of our products simultaneously to all languages. In this case, the technical support of the entire system requires one person, and then only occasionally.
When the Evernote service was just beginning to develop, we had a great desire to make the company international from the very beginning. This required the operational localization of Evernote applications across all platforms into a large list of languages. And at the same time we wanted to make this process as transparent as possible for developers. Therefore, in 2008, we began to create a platform that would simplify and automate the localization process, make this process continuous.
And seven years later, this platform, which we called Serge ( S tring E xtraction and R esource G eneration E ngine), became accessible to everyone .
Serge allows you to set up a process in which strings, when committed to a version control system, are automatically extracted and sent for translation, and translations are automatically integrated into localized resource files and automatically get into the version control system. Serge relieves developers of the need to work with localized files in general (their concern is to edit only source resources, for example, English), it saves translators (or your localization agency) from having to mess around with different file formats. Serge can be integrated into any localization process - with the help of paid translators or volunteers, online or offline.
Serge is a command line application that accepts a command and one or more configuration files as an input parameter. Each configuration file describes one localization project (synchronize with which remote repository, how to communicate with the translation server, what local directory and which mask to scan files, what parser to process, how to name the files sent for translation, where and in what encoding to create localized files, which additional plugins to use, etc.).
This is what happens when Serge runs the sync command for each configuration file:
Continuous localization is achieved by the fact that as soon as Serge passes this cycle for each configuration file, the cycle can be started again. In the case of Evernote, the entire cycle for all projects takes about 15 minutes. This means that as soon as the engineer zakommitil some change in the resource file, within 15 minutes comes a commit from the localization robot, symmetrically changing all localized copies of this resource; when the translator adds a translation or changes an existing one, within 15 minutes its change is committed to the repository. These changes cause the launch of the CI-build and the subsequent automatic testing, which allows you to quickly identify errors associated with internationalization and localization, to translate in parallel with the development of the product.
Serge architecture is built on plugins. Serge supports several types of version control systems, more than 20 file formats, as well as plug-ins that change the behavior of the system (for example, pre- and post-processing of localized files, the ability to specify a list of languages for translation directly in the file, etc.). Serge's website has product documentation, plugins, installation and configuration.
We very much hope that the opening of the Serge source will allow both individual developers and large companies to take a fresh look at the localization process and closely integrate localization into the development cycle. Well and, of course, we will be glad to receive comments, suggestions and error messages.
Localization is easy!
For many developers, localization is associated with an additional layer of problems: how to keep localized resources up to date? And what if languages are not 2-3, but 20-30? How to send new lines in time for translation? But what if during the translation the development went ahead, and there were no lines anymore, but there were new ones? How to merge resource files sent by translators? It is no secret that because of this, many people simply hammer to localization or try to postpone it for later.
')
Evernote now has more than 150 million users worldwide, more than 70% of these users are outside the US, every month we translate 15,000 new words in more than 40 projects into more than 26 languages, and release new releases of our products simultaneously to all languages. In this case, the technical support of the entire system requires one person, and then only occasionally.
How do we do it?
When the Evernote service was just beginning to develop, we had a great desire to make the company international from the very beginning. This required the operational localization of Evernote applications across all platforms into a large list of languages. And at the same time we wanted to make this process as transparent as possible for developers. Therefore, in 2008, we began to create a platform that would simplify and automate the localization process, make this process continuous.
And seven years later, this platform, which we called Serge ( S tring E xtraction and R esource G eneration E ngine), became accessible to everyone .
Serge allows you to set up a process in which strings, when committed to a version control system, are automatically extracted and sent for translation, and translations are automatically integrated into localized resource files and automatically get into the version control system. Serge relieves developers of the need to work with localized files in general (their concern is to edit only source resources, for example, English), it saves translators (or your localization agency) from having to mess around with different file formats. Serge can be integrated into any localization process - with the help of paid translators or volunteers, online or offline.
How does this thing work?
Serge is a command line application that accepts a command and one or more configuration files as an input parameter. Each configuration file describes one localization project (synchronize with which remote repository, how to communicate with the translation server, what local directory and which mask to scan files, what parser to process, how to name the files sent for translation, where and in what encoding to create localized files, which additional plugins to use, etc.).
This is what happens when Serge runs the sync command for each configuration file:
- ( pull ) downloads changes from a remote repository with resource files;
- ( pull-ts ) download translations from the translation server (local files are updated);
- ( localize ) localization actually occurs:
- resource files parsing; information about files and lines fall into the internal database;
- files with translations parsitsya; transfers go to the internal database;
- Translation files are updated in accordance with the changed structure of the source files;
- resource files are updated according to changes in translations;
- ( push-ts ) updated files with translations are uploaded to the translation server;
- ( push ) updated localized resource files are uploaded to the external repository.
Continuous localization is achieved by the fact that as soon as Serge passes this cycle for each configuration file, the cycle can be started again. In the case of Evernote, the entire cycle for all projects takes about 15 minutes. This means that as soon as the engineer zakommitil some change in the resource file, within 15 minutes comes a commit from the localization robot, symmetrically changing all localized copies of this resource; when the translator adds a translation or changes an existing one, within 15 minutes its change is committed to the repository. These changes cause the launch of the CI-build and the subsequent automatic testing, which allows you to quickly identify errors associated with internationalization and localization, to translate in parallel with the development of the product.
Serge architecture is built on plugins. Serge supports several types of version control systems, more than 20 file formats, as well as plug-ins that change the behavior of the system (for example, pre- and post-processing of localized files, the ability to specify a list of languages for translation directly in the file, etc.). Serge's website has product documentation, plugins, installation and configuration.
We very much hope that the opening of the Serge source will allow both individual developers and large companies to take a fresh look at the localization process and closely integrate localization into the development cycle. Well and, of course, we will be glad to receive comments, suggestions and error messages.
Localization is easy!
Source: https://habr.com/ru/post/269049/
All Articles