Improved Android NDK (Native Development Kit)
For a long time, while developing Android, I gradually came to the conclusion that I, as a developer, lack a lot of things. Then, in early 2010, only C ++ exceptions and RTTI were missing. Without them, any non-trivial C ++ code simply could not be ported to Android and required almost complete rewriting. It was a very significant limitation that did not give me rest. Fortunately, Android is an open source, therefore, armed with a maxim "if you need something, do it yourself," I sat down to work. To my surprise, it was pretty easy to support full-fledged C ++ with exceptions and RTTI. It took only about a week of work. Next, a website was made, on which the resulting packages for Windows, Linux and Mac OS X were laid out, as well as a patch and assembly instructions.
The problem was relevant, so the project turned out to be very popular and during the first month I received a huge number of letters with questions, requests for additional functionality and just thanks. One of the most frequently asked questions was “Will Google include your changes in the mainline?”. I honestly answered that I did not know, but there is hope, because David Turner (system architect of Android) was very interested in my patches and promised to pay attention to them.
Soon the tale is affected, but it is not done soon. The corporation of good found many more urgent tasks, so my patches were pulled into the mainline only about a year later. During this time, there were two releases from Google (r3 and r4) and I adapted them the same way as r2. I continued to post the modified packages along with the patches and build instructions on my website. Thus, during this time, a rather large audience of users who needed these opportunities was recruited. Using my NDK, many open source and commercial projects have been able to port their code to Android. Open Source is probably the most famous, perhaps, OpenCV and Ogre3D .
')
Finally, with the release of the NDK r5, Google integrated my patches (almost unchanged) into the mainline. It would seem that there is an end to the project, but by that time there were quite a lot of other complaints about NDK. It lacked a lot, and so I decided to continue the project, reorienting it from “Android NDK + full C ++” to “Improved Android NDK”, because I learned from experience that this is the best way to make these changes in the mainline. The specific list of improvements is constantly updated, it can be seen on my website on the page of a specific release.
So, here I will describe in sequence the main improvements in the CrystaX NDK, which distinguish it from Google NDK.
I do not know why, but a sabotage was committed in Google NDK - wchar_t was made 1 byte in size. The explanations given by David Turner did not satisfy me. He stressed that the code that uses wchar_t is usually non-portable, so you need to use UTF-8, stored in regular char strings. Not at all protesting against the use of UTF-8, I, however, pointed out to him that exactly such solutions as in Google NDK make unchartable code with wchar_t. There was no particular discussion, everyone was left to his own opinion, but I understood that support for wide characters / lines / streams is necessary.
Based on code from FreeBSD, I started developing support for wchar_t in the Standard C library, and at the same time Std C locales. The task was difficult and it took several months before I got a stable implementation. Unfortunately, it is still not complete (there is still no full support for locales, only UTF-8 is supported), but in practice it is already more than enough for many projects. I myself participated in projects where wide characters / string / streams were actively used, and also I know from the letters that used my implementation.
With the release of the new international standard C ++, I wanted to use the new features of the language and the standard library. Unfortunately, Google NDK is based on GCC 4.4.3, which, although it includes some of the features of C ++ 0x, is nevertheless somewhat outdated. Therefore, it was decided to add a new GCC 4.6.3 based toolchain in the CrystaX NDK. No sooner said than done. At the moment, CrystaX NDK contains two versions of the compiler - 4.4.3 (as in Google) and 4.6.3 (new). Switching between them is very easy - in Application.mk of your project it’s enough to write:
Please note that this is not enough to use the features of C ++ 0x. You must enable C ++ 0x explicitly (also in Application.mk):
The CrystaX NDK GCC 4.6.3 is built with the support of the Graphite optimization framework. There’s nothing to talk about for a long time, so I’ll just give a link to the wiki page.
Google NDK only supports C and C ++ for native development. This, of course, is not bad, but at some point it turned out that there is quite a lot of code originally written for iOS on Objective-C, which now needs to be ported to Android. You can, of course, go the traditional way and rewrite it from scratch, or you can add support for Objective-C in Android. Upon reflection, I came to the conclusion that having an opportunity is better than its absence and proceeded to implement this idea. The Objective-C compiler and the base GNU Objective-C library are currently ported. The plans include the GNUStep port on Android (then it will be possible to transfer Cocoa code with minimal changes), but this task is rather big and will obviously take some time. Anyway, you can start using Objective-C on Android right now - just add sources with the extensions .m (Objective-C) or .mm (Objective-C ++) to LOCAL_SRC_FILES.
I, like many developers, are a bit paranoid. I am never sure for one hundred percent in my code - too often I came across stupid mistakes, both my own and others'. Therefore, I follow the main rule - “There are no too many checks of the code”. There are quite a few different code analyzers, but for Android I decided to start with the Clang static code analyzer. This project is actively developing and is already quite well analyzing C and Objective-C code (C ++, unfortunately, not yet). To take advantage of this opportunity, you first need to install Clang , make sure that the 'scan-build' script from the Clang package is available from the command line, and run the Android project build:
This team will assemble the Android project as usual, but also analyze it in parallel using the Clang analyzer and, in the case of errors found, report about them at the end.
I plan to add support for other analyzers, but so far there is no specifics on this score.
This feature is not yet ready for full use and is in active development, but I still decided to mention it - who knows, maybe it will save someone weeks or months of work.
In an application compiled using the CrystaX NDK, all calls to the IO API (such as open, read, write, etc.) are intercepted and analyzed. This is done to embed the so-called "drivers" - filters that can change the writeable / readable data in a transparent way for the application. For example, to compress or encrypt data. Please note that changing the application code is not required! Just need to insert into the code at the start of something like:
and voila - all data recorded in / target will be automatically compressed with gzip, and read from / target will be automatically expanded.
I remind once again - this opportunity is not ready yet, but is at the final stage of development. It is possible that over the next month it will stabilize. However, it is possible that it will seem useful to someone right now - well, I do not mind.
The more I work on the Android platform, the more unmet needs arise. So far I have enough time and energy to realize everything myself, but it is obvious that this cannot last forever. Therefore, I urge enthusiasts to participate in the project - let's make Android such a platform, as we like! In the end, this is the main advantage of Android over iOS and Blackberry - we can make such an OS and such tools for it as we need, and not the vendor. Forward!
On this joyous note, let me leave.
The problem was relevant, so the project turned out to be very popular and during the first month I received a huge number of letters with questions, requests for additional functionality and just thanks. One of the most frequently asked questions was “Will Google include your changes in the mainline?”. I honestly answered that I did not know, but there is hope, because David Turner (system architect of Android) was very interested in my patches and promised to pay attention to them.
Soon the tale is affected, but it is not done soon. The corporation of good found many more urgent tasks, so my patches were pulled into the mainline only about a year later. During this time, there were two releases from Google (r3 and r4) and I adapted them the same way as r2. I continued to post the modified packages along with the patches and build instructions on my website. Thus, during this time, a rather large audience of users who needed these opportunities was recruited. Using my NDK, many open source and commercial projects have been able to port their code to Android. Open Source is probably the most famous, perhaps, OpenCV and Ogre3D .
')
Finally, with the release of the NDK r5, Google integrated my patches (almost unchanged) into the mainline. It would seem that there is an end to the project, but by that time there were quite a lot of other complaints about NDK. It lacked a lot, and so I decided to continue the project, reorienting it from “Android NDK + full C ++” to “Improved Android NDK”, because I learned from experience that this is the best way to make these changes in the mainline. The specific list of improvements is constantly updated, it can be seen on my website on the page of a specific release.
So, here I will describe in sequence the main improvements in the CrystaX NDK, which distinguish it from Google NDK.
Wide characters support
I do not know why, but a sabotage was committed in Google NDK - wchar_t was made 1 byte in size. The explanations given by David Turner did not satisfy me. He stressed that the code that uses wchar_t is usually non-portable, so you need to use UTF-8, stored in regular char strings. Not at all protesting against the use of UTF-8, I, however, pointed out to him that exactly such solutions as in Google NDK make unchartable code with wchar_t. There was no particular discussion, everyone was left to his own opinion, but I understood that support for wide characters / lines / streams is necessary.
Based on code from FreeBSD, I started developing support for wchar_t in the Standard C library, and at the same time Std C locales. The task was difficult and it took several months before I got a stable implementation. Unfortunately, it is still not complete (there is still no full support for locales, only UTF-8 is supported), but in practice it is already more than enough for many projects. I myself participated in projects where wide characters / string / streams were actively used, and also I know from the letters that used my implementation.
C ++ 11
With the release of the new international standard C ++, I wanted to use the new features of the language and the standard library. Unfortunately, Google NDK is based on GCC 4.4.3, which, although it includes some of the features of C ++ 0x, is nevertheless somewhat outdated. Therefore, it was decided to add a new GCC 4.6.3 based toolchain in the CrystaX NDK. No sooner said than done. At the moment, CrystaX NDK contains two versions of the compiler - 4.4.3 (as in Google) and 4.6.3 (new). Switching between them is very easy - in Application.mk of your project it’s enough to write:
APP_TOOLCHAIN_VERSION := 4.6.3Please note that this is not enough to use the features of C ++ 0x. You must enable C ++ 0x explicitly (also in Application.mk):
APP_USE_CPP0X := trueGraphite optimization framework
The CrystaX NDK GCC 4.6.3 is built with the support of the Graphite optimization framework. There’s nothing to talk about for a long time, so I’ll just give a link to the wiki page.
Objective-c
Google NDK only supports C and C ++ for native development. This, of course, is not bad, but at some point it turned out that there is quite a lot of code originally written for iOS on Objective-C, which now needs to be ported to Android. You can, of course, go the traditional way and rewrite it from scratch, or you can add support for Objective-C in Android. Upon reflection, I came to the conclusion that having an opportunity is better than its absence and proceeded to implement this idea. The Objective-C compiler and the base GNU Objective-C library are currently ported. The plans include the GNUStep port on Android (then it will be possible to transfer Cocoa code with minimal changes), but this task is rather big and will obviously take some time. Anyway, you can start using Objective-C on Android right now - just add sources with the extensions .m (Objective-C) or .mm (Objective-C ++) to LOCAL_SRC_FILES.
Static code analysis
I, like many developers, are a bit paranoid. I am never sure for one hundred percent in my code - too often I came across stupid mistakes, both my own and others'. Therefore, I follow the main rule - “There are no too many checks of the code”. There are quite a few different code analyzers, but for Android I decided to start with the Clang static code analyzer. This project is actively developing and is already quite well analyzing C and Objective-C code (C ++, unfortunately, not yet). To take advantage of this opportunity, you first need to install Clang , make sure that the 'scan-build' script from the Clang package is available from the command line, and run the Android project build:
ndk-build ANALYZE=1This team will assemble the Android project as usual, but also analyze it in parallel using the Clang analyzer and, in the case of errors found, report about them at the end.
I plan to add support for other analyzers, but so far there is no specifics on this score.
Application-level file system driver
This feature is not yet ready for full use and is in active development, but I still decided to mention it - who knows, maybe it will save someone weeks or months of work.
In an application compiled using the CrystaX NDK, all calls to the IO API (such as open, read, write, etc.) are intercepted and analyzed. This is done to embed the so-called "drivers" - filters that can change the writeable / readable data in a transparent way for the application. For example, to compress or encrypt data. Please note that changing the application code is not required! Just need to insert into the code at the start of something like:
mount("/path/to/storage", "/target", "compressed", 0, "GZIP")and voila - all data recorded in / target will be automatically compressed with gzip, and read from / target will be automatically expanded.
I remind once again - this opportunity is not ready yet, but is at the final stage of development. It is possible that over the next month it will stabilize. However, it is possible that it will seem useful to someone right now - well, I do not mind.
Conclusion
The more I work on the Android platform, the more unmet needs arise. So far I have enough time and energy to realize everything myself, but it is obvious that this cannot last forever. Therefore, I urge enthusiasts to participate in the project - let's make Android such a platform, as we like! In the end, this is the main advantage of Android over iOS and Blackberry - we can make such an OS and such tools for it as we need, and not the vendor. Forward!
On this joyous note, let me leave.
Source: https://habr.com/ru/post/135856/
All Articles