Localization of the application and support for RTL. Report Yandex.Taxi
When localizing the service, it is important to carefully consider the coordination of transfers between each other. The leader of the Android client development team at Yandex.Taxi, Alexander Bonell, explained what practices and tools simplify localization. In the second part of the report, Sasha shared his experience of supporting the RTL language in the application: what is good and what does not quite work for Android out of the box, what problems arise because of support for RTL and how to minimize them in the future.
- In my report, I want to tell you what basic ideas and practices we use in mobile application development teams Taxi to solve issues related to the localization and translation actualization in our applications. Then I will tell you how we implemented support for working in the drawing mode from right to left in the application.
')
I'll start with localization. First of all, I would like to clarify the terminology. According to my observations, a large number of people believe that localization is limited to only translations, although in fact it also solves problems related to the formatting of numbers, dates, times, working with text and symbols, legal aspects, serving content to the consumer in such a way that he had a semantic value for him and was not perceived ambiguously. And much more. (Communication with the audience about who has how many languages ​​are supported in applications - approx. Ed.)
I want to tell you a story that happened in 2014. We then made a major redesign of the Taxi application. In one of the regular releases, we gave users the opportunity to indicate the access number, as well as look at this information on the trip information screen and, most importantly, on the order evaluation screen, which we showed to all users, because we definitely wanted to get an estimate for the trip .
At that time, we were rather frivolous about the localization process and translations. For the most part, the whole process was manual. Updating texts in the necessary XML files, adding new translations was done manually. And the human factor played its role. The developer did not define the next translation in the default locale. This led to the fact that for users with a system locale other than Russian, if they asked for an entrance, the application crashed after the trip. And people could not make another order.
At that time, we had two languages ​​supported in the application: Russian and English. And we worked only in Russia in three cities: Moscow, Yekaterinburg, St. Petersburg.
We are currently working in 16 countries and translated into 18 languages. I think you understand all the delicacy of the problem, if we were in such a state at that time. We already realized that we need to change something.
In Yandex, most mobile application developers use the service developed internally to solve localization problems. It has a very convenient UI-interface, it is possible to define your project, set a set of keys, a set of languages ​​into which the project should be translated. And after the translations appear, they can be unloaded in any convenient format. It also has very good integration with our internal services. There is a versioned version. But personally for me as a developer, the most important thing is that he has an API, because it already suggests that all manual work related to translations can be automated. What we did. We developed a plugin for Gradle. In fact, translation actualization in the application now comes down to the fact that the developer performs a single task: updateTranslations.
In the first approximation, he goes to the localization service. Checks the availability of current translations, uploads them to the project, decomposes them into the necessary files. But besides this, he also verifies the status of transfers in the localization service in relation to what is now in the project.
When you next run git status, the developer sees in his working directory which files have changed.
He can also look at the generated HTML report about which key has changed to which value in which language.
And also he can see what problems are currently present in the translations of his project.
Plugin configurable. We can determine the number of languages ​​that our application supports. It is very important if localizers get a new language, and it inadvertently seeps into the project, being not passed through proofreading and reading.
There is also a specific ability to remap. I will talk about it a little later in more detail when I talk about RTL.
What problems is this plugin analyzing now? The problem that we faced in 2014 is banal, the lack of translation. Now, in order to add a new line, a new translation to the project, it is not enough for the developer to simply define it in the XML file.
First, if he does it in a default locale, then his newly entered key will simply be lost during the next synchronization, and he will know about it at the compilation stage.
If he determines the key in the default file, but forgets to do it in the localization service, then the next time the updatetranslations task is executed, he will receive an error from the plugin that will tell him: “This key is in your project, but it is not in the localization service, it’s there need to start. "
The second typical problem is the lack of translation. We get a list of links to those keys for which there is no translation, and, as a rule, we assign them to our manager so that they can later set up communication with translators. The third point is often found when they decided to revise the use of format arguments in some key, and, for example, they began to use a string format instead of an integer format.
But at the same time, in one of the translations, they forgot to change the number to a string.
And the fourth problem is also due to the human factor, mainly during translation. If Unicode characters are used in the translation, it is very easy to get confused in the positioning of characters and, instead of an inseparable space, get a break to another line — something that we are trying to get rid of.
It's all about localization.
Now I want to tell you about the experience of introducing support for rendering from right to left in the application using the example of Israel. We launched in 2018 under the Yango brand in Israel. As it turned out, people in Israel do not read in the way we do. They read from right to left.
Where do you want to start? That in Android, in principle, right-to-left rendering support is implemented quite well from the box.
It begins with the fact that in the application manifest, in the Application element, you need to declare the supportRtl = ”true” attribute. But I want to please you: if you integrate the Facebook SDK for social authorization in your applications, the caring Facebook developers have already done it for you. And if you do not validate the manifest during merging, you will have this attribute with a value of true in your application, and it will already be able to be in Rtl. This is something to think about.
I think that most of those sitting in the audience probably minimally support the version of Android 4.4. Someone more lucky - this is 5.0 and higher. Someone might have less luck, and they support 4.0, or, God forbid, 2.3. In this case, you will have big problems, because the full support for Rtl in Android appeared since version 4.2. Therefore, with minSdk 17th, you will simplify your life by an order of magnitude.
Transferring a project to work with Rtl begins with the refactoring tool in Android Studio - Add RTL Support Where Possible. We must pay tribute, works quite well. It traverses your markup XML files, looks for attributes with left and right values ​​for gravity, paddings, margins, and replaces them with start and end. If you want to see how your application works by default when drawing in right-to-left mode, you do not need to have content with so-called RTL strong characters in your hand — in Hebrew or Arabic. You just need to enable the Force RTL Layout Direction option in the developer settings.
What possibilities for customizing UI and text rendering Android provides for RTL?
The first point is the layoutDirection attribute, which uses the Force RTL Layout Direction option described. She has only four possible values. That is, it is inherited from the parent by default. It can be said that the layout is rendered based on the selected locale. You can clearly say that it is drawn from right to left or left to right.
The second element, which many probably out of habit, bypass when they align text is textAlignment. Many still continue to use gravity. For text alignment, do not use gravity, but use textAlignment.
The third attribute is the text rendering direction, textDirection. He has a fairly flexible setting. It allows you to determine how the text is rendered, depending on which first strong character was caught in the text, whether it is from the Latin character set strong LTR or from Hebrew strong RTL. But if you fully support RTL in the application, you do not need to get round it, because when you ignore, there is such a funny artifact.
Whether it is a Easter egg, or whether it is such a side effect in the editText implementation: your hint and editText starts to split. Therefore, no value should be given to him in any case.
If you need to observe a certain rendering of some graphic resources or markup in RTL mode, then Android provides the opportunity to set qualifier ldrtl in your daddy, in which you can put any resource, and it will be drawn in RTL drawing mode as you specified.
Sometimes there is a need for a runtime that the text coming from the backend is drawn as it is, how it came, ignoring the mode in which your application is currently running. In fact, in the text renderer, this is due to the fact that the text is framed by the so-called Bidi Unicode control characters. The logic for working with these characters in itself contains the BidiFormatter class, providing only one single unicodeWrap method. It takes the SharSequence at the input, and returns you a string framed by these characters. There are an exhaustive number of them, and, in principle, it should solve most of your problems.
But if there is something so specific, w3.org has a good article on how to use these symbols.
What problems did we acquire when we started RTL support in our application? First of all, it is a conflict of localization standards.
BCP47 is a newer standard for localization, which in particular has changed the codes of some locales. And the fact is that there is a problem with the Java locale class. It defaults to backward-compatibility mode, and the locale that comes with it from BCP converts to ISO. That is, we saw this when the iw code went to our backend in the Accept-Language header instead of the he code.
If you remember the slide with the configuration of our plugin, remapping was needed just for this - in order to transfer content from he to iw.
If you need to use the BCP format, this is fully implemented in the Apache Cordova project. They have a Globalization class and an implementation of the localeToBcp47Language.
Another point that "delivers" with the support of RTL in the application is animation. Here, the benefit, in fact, affects only animations on the X axis, and perhaps the only solution or approach that will help you avoid more problems is the ability to revise absolute animations in the direction of relative animations.
One problem that can be solved, perhaps, is only reflection, because it is rigidly implemented in the component itself - this is the hint alignment of TextInputLayout. The fact is that if hint is specified in RTL strong Hebrew characters, it will be clearly aligned to the right, even though the content you enter in edit text may be from Latin characters. And vice versa. If you hint in Latin, it will be clearly aligned to the left. This can be solved only with the help of reflection.
In the end, after you have implemented RTL in your application, it makes sense to review the severities for the RTL issues that Lint analyzes. There are only four of them. If you do not support RTL in your application, but obviously use RTL specific attributes somewhere, then this is RtlEnabled (Lint will tell you about it).
If you support RTL in the application, but your application’s minSdkVersion is below 17, then you must also continue to use the left and right attributes, you cannot use only the start and end attributes in your markup. This is what RtlCompat analyzes. RtlSymmetry swears when it sees asymmetric paddings. In general, my opinion is that aapt when processing resources should crash if it stumbles upon symmetrical paddings. If you need any asymmetric access, use margins as a full layout parameter.
Finally, the fourth problem is hardcoded RTL. This is precisely the fact that only the left and right values ​​are specified in the attributes. In effect, this negates the refactoring tool Add RTL Support Where Possible. This is all out of Lint over the RTL entry. Also, the implementation of the analysis is available in the AOSP in the RtlDetector class. That is, if fantasy allows you, you can come up with something of your own by looking at how the analysis is taking place now.
My report is coming to an end. I want to cite several theses on building the correct localization in our application, which we eventually came to.
First of all, at the start of the project, think about internationalization. I did not really mention it. This is not the same as localization. In fact, internationalization makes localization possible in an application, in a product. It works out of the box for Java due to the fact that there is full Unicode support, and for Android - due to a rich system of resources. Perhaps the only thing that can happen here is our fixation as developers on the subject of “I write the application only for Russian, which means I will check it only with the Russian language”. And when you start using Armenian, Georgian, Hebrew or Arabic in the application, the picture looks very different and breaks about your usual paradigm.
Second moment. Think about embedding localization in your CI. It is a pity to learn about the problems associated with translations at the stage of testing a release candidate for whom you could prepare for several weeks. That is, we now run this task to every pool-request, so that we are sure that everything is good with the translations and we can freeze the new code into the main branch of the project.
Third moment. Respect the work of translators and your budget. First, remove unused resources, because people can continue to spend time and money on translating strings that you don’t use in your project. Secondly, follow the translations that are in your repository in the localization service, but which are not in your code.
Ideally, if the API of your localization service provides information about when a particular key was entered, and you can use heuristics like “If the key is entered more than a month ago,” check its presence in the latest revision of the master branch. If he is not there, most likely, this is an obvious candidate for removing it and for localizers not to waste time on it.
A more or less universal candidate for a silver bullet is to store lines on the backend, not on the client. At the very least, this will give you the opportunity to always keep them relevant to the application.
We have a very diverse audience of users, each perceives the world and the content of the application in their own way. Our job as developers is to help keep that perception. My report is over. Thank you very much!
- In my report, I want to tell you what basic ideas and practices we use in mobile application development teams Taxi to solve issues related to the localization and translation actualization in our applications. Then I will tell you how we implemented support for working in the drawing mode from right to left in the application.
')
I'll start with localization. First of all, I would like to clarify the terminology. According to my observations, a large number of people believe that localization is limited to only translations, although in fact it also solves problems related to the formatting of numbers, dates, times, working with text and symbols, legal aspects, serving content to the consumer in such a way that he had a semantic value for him and was not perceived ambiguously. And much more. (Communication with the audience about who has how many languages ​​are supported in applications - approx. Ed.)
I want to tell you a story that happened in 2014. We then made a major redesign of the Taxi application. In one of the regular releases, we gave users the opportunity to indicate the access number, as well as look at this information on the trip information screen and, most importantly, on the order evaluation screen, which we showed to all users, because we definitely wanted to get an estimate for the trip .
At that time, we were rather frivolous about the localization process and translations. For the most part, the whole process was manual. Updating texts in the necessary XML files, adding new translations was done manually. And the human factor played its role. The developer did not define the next translation in the default locale. This led to the fact that for users with a system locale other than Russian, if they asked for an entrance, the application crashed after the trip. And people could not make another order.
At that time, we had two languages ​​supported in the application: Russian and English. And we worked only in Russia in three cities: Moscow, Yekaterinburg, St. Petersburg.
We are currently working in 16 countries and translated into 18 languages. I think you understand all the delicacy of the problem, if we were in such a state at that time. We already realized that we need to change something.
In Yandex, most mobile application developers use the service developed internally to solve localization problems. It has a very convenient UI-interface, it is possible to define your project, set a set of keys, a set of languages ​​into which the project should be translated. And after the translations appear, they can be unloaded in any convenient format. It also has very good integration with our internal services. There is a versioned version. But personally for me as a developer, the most important thing is that he has an API, because it already suggests that all manual work related to translations can be automated. What we did. We developed a plugin for Gradle. In fact, translation actualization in the application now comes down to the fact that the developer performs a single task: updateTranslations.
In the first approximation, he goes to the localization service. Checks the availability of current translations, uploads them to the project, decomposes them into the necessary files. But besides this, he also verifies the status of transfers in the localization service in relation to what is now in the project.
When you next run git status, the developer sees in his working directory which files have changed.
He can also look at the generated HTML report about which key has changed to which value in which language.
And also he can see what problems are currently present in the translations of his project.
Plugin configurable. We can determine the number of languages ​​that our application supports. It is very important if localizers get a new language, and it inadvertently seeps into the project, being not passed through proofreading and reading.
There is also a specific ability to remap. I will talk about it a little later in more detail when I talk about RTL.
What problems is this plugin analyzing now? The problem that we faced in 2014 is banal, the lack of translation. Now, in order to add a new line, a new translation to the project, it is not enough for the developer to simply define it in the XML file.
First, if he does it in a default locale, then his newly entered key will simply be lost during the next synchronization, and he will know about it at the compilation stage.
If he determines the key in the default file, but forgets to do it in the localization service, then the next time the updatetranslations task is executed, he will receive an error from the plugin that will tell him: “This key is in your project, but it is not in the localization service, it’s there need to start. "
The second typical problem is the lack of translation. We get a list of links to those keys for which there is no translation, and, as a rule, we assign them to our manager so that they can later set up communication with translators. The third point is often found when they decided to revise the use of format arguments in some key, and, for example, they began to use a string format instead of an integer format.
But at the same time, in one of the translations, they forgot to change the number to a string.
And the fourth problem is also due to the human factor, mainly during translation. If Unicode characters are used in the translation, it is very easy to get confused in the positioning of characters and, instead of an inseparable space, get a break to another line — something that we are trying to get rid of.
It's all about localization.
Now I want to tell you about the experience of introducing support for rendering from right to left in the application using the example of Israel. We launched in 2018 under the Yango brand in Israel. As it turned out, people in Israel do not read in the way we do. They read from right to left.
Where do you want to start? That in Android, in principle, right-to-left rendering support is implemented quite well from the box.
It begins with the fact that in the application manifest, in the Application element, you need to declare the supportRtl = ”true” attribute. But I want to please you: if you integrate the Facebook SDK for social authorization in your applications, the caring Facebook developers have already done it for you. And if you do not validate the manifest during merging, you will have this attribute with a value of true in your application, and it will already be able to be in Rtl. This is something to think about.
I think that most of those sitting in the audience probably minimally support the version of Android 4.4. Someone more lucky - this is 5.0 and higher. Someone might have less luck, and they support 4.0, or, God forbid, 2.3. In this case, you will have big problems, because the full support for Rtl in Android appeared since version 4.2. Therefore, with minSdk 17th, you will simplify your life by an order of magnitude.
Transferring a project to work with Rtl begins with the refactoring tool in Android Studio - Add RTL Support Where Possible. We must pay tribute, works quite well. It traverses your markup XML files, looks for attributes with left and right values ​​for gravity, paddings, margins, and replaces them with start and end. If you want to see how your application works by default when drawing in right-to-left mode, you do not need to have content with so-called RTL strong characters in your hand — in Hebrew or Arabic. You just need to enable the Force RTL Layout Direction option in the developer settings.
What possibilities for customizing UI and text rendering Android provides for RTL?
The first point is the layoutDirection attribute, which uses the Force RTL Layout Direction option described. She has only four possible values. That is, it is inherited from the parent by default. It can be said that the layout is rendered based on the selected locale. You can clearly say that it is drawn from right to left or left to right.
The second element, which many probably out of habit, bypass when they align text is textAlignment. Many still continue to use gravity. For text alignment, do not use gravity, but use textAlignment.
The third attribute is the text rendering direction, textDirection. He has a fairly flexible setting. It allows you to determine how the text is rendered, depending on which first strong character was caught in the text, whether it is from the Latin character set strong LTR or from Hebrew strong RTL. But if you fully support RTL in the application, you do not need to get round it, because when you ignore, there is such a funny artifact.
Whether it is a Easter egg, or whether it is such a side effect in the editText implementation: your hint and editText starts to split. Therefore, no value should be given to him in any case.
Link from the slide
If you need to observe a certain rendering of some graphic resources or markup in RTL mode, then Android provides the opportunity to set qualifier ldrtl in your daddy, in which you can put any resource, and it will be drawn in RTL drawing mode as you specified.
Sometimes there is a need for a runtime that the text coming from the backend is drawn as it is, how it came, ignoring the mode in which your application is currently running. In fact, in the text renderer, this is due to the fact that the text is framed by the so-called Bidi Unicode control characters. The logic for working with these characters in itself contains the BidiFormatter class, providing only one single unicodeWrap method. It takes the SharSequence at the input, and returns you a string framed by these characters. There are an exhaustive number of them, and, in principle, it should solve most of your problems.
But if there is something so specific, w3.org has a good article on how to use these symbols.
What problems did we acquire when we started RTL support in our application? First of all, it is a conflict of localization standards.
BCP47 is a newer standard for localization, which in particular has changed the codes of some locales. And the fact is that there is a problem with the Java locale class. It defaults to backward-compatibility mode, and the locale that comes with it from BCP converts to ISO. That is, we saw this when the iw code went to our backend in the Accept-Language header instead of the he code.
Link from the slide
If you remember the slide with the configuration of our plugin, remapping was needed just for this - in order to transfer content from he to iw.
If you need to use the BCP format, this is fully implemented in the Apache Cordova project. They have a Globalization class and an implementation of the localeToBcp47Language.
Another point that "delivers" with the support of RTL in the application is animation. Here, the benefit, in fact, affects only animations on the X axis, and perhaps the only solution or approach that will help you avoid more problems is the ability to revise absolute animations in the direction of relative animations.
One problem that can be solved, perhaps, is only reflection, because it is rigidly implemented in the component itself - this is the hint alignment of TextInputLayout. The fact is that if hint is specified in RTL strong Hebrew characters, it will be clearly aligned to the right, even though the content you enter in edit text may be from Latin characters. And vice versa. If you hint in Latin, it will be clearly aligned to the left. This can be solved only with the help of reflection.
In the end, after you have implemented RTL in your application, it makes sense to review the severities for the RTL issues that Lint analyzes. There are only four of them. If you do not support RTL in your application, but obviously use RTL specific attributes somewhere, then this is RtlEnabled (Lint will tell you about it).
Link from the slide
If you support RTL in the application, but your application’s minSdkVersion is below 17, then you must also continue to use the left and right attributes, you cannot use only the start and end attributes in your markup. This is what RtlCompat analyzes. RtlSymmetry swears when it sees asymmetric paddings. In general, my opinion is that aapt when processing resources should crash if it stumbles upon symmetrical paddings. If you need any asymmetric access, use margins as a full layout parameter.
Finally, the fourth problem is hardcoded RTL. This is precisely the fact that only the left and right values ​​are specified in the attributes. In effect, this negates the refactoring tool Add RTL Support Where Possible. This is all out of Lint over the RTL entry. Also, the implementation of the analysis is available in the AOSP in the RtlDetector class. That is, if fantasy allows you, you can come up with something of your own by looking at how the analysis is taking place now.
My report is coming to an end. I want to cite several theses on building the correct localization in our application, which we eventually came to.
First of all, at the start of the project, think about internationalization. I did not really mention it. This is not the same as localization. In fact, internationalization makes localization possible in an application, in a product. It works out of the box for Java due to the fact that there is full Unicode support, and for Android - due to a rich system of resources. Perhaps the only thing that can happen here is our fixation as developers on the subject of “I write the application only for Russian, which means I will check it only with the Russian language”. And when you start using Armenian, Georgian, Hebrew or Arabic in the application, the picture looks very different and breaks about your usual paradigm.
Second moment. Think about embedding localization in your CI. It is a pity to learn about the problems associated with translations at the stage of testing a release candidate for whom you could prepare for several weeks. That is, we now run this task to every pool-request, so that we are sure that everything is good with the translations and we can freeze the new code into the main branch of the project.
Third moment. Respect the work of translators and your budget. First, remove unused resources, because people can continue to spend time and money on translating strings that you don’t use in your project. Secondly, follow the translations that are in your repository in the localization service, but which are not in your code.
Ideally, if the API of your localization service provides information about when a particular key was entered, and you can use heuristics like “If the key is entered more than a month ago,” check its presence in the latest revision of the master branch. If he is not there, most likely, this is an obvious candidate for removing it and for localizers not to waste time on it.
A more or less universal candidate for a silver bullet is to store lines on the backend, not on the client. At the very least, this will give you the opportunity to always keep them relevant to the application.
We have a very diverse audience of users, each perceives the world and the content of the application in their own way. Our job as developers is to help keep that perception. My report is over. Thank you very much!
Source: https://habr.com/ru/post/456250/
All Articles