3DO and Android NDK and how not to get into anything ...

There are probably quite a few applications that are almost impossible to do in Java, due to the large C ++ source code base or performance requirements. And it turned out that I was developing one of these applications, namely the 3DO game console emulator - Real3DOPlayer . In my case, the role played as a code base, and performance requirements. The code was based on my desktop project “Phoenix” , and it slowed even on medium desktops, not like on embedded processors. I don’t remember how many damnations were addressed to Gooogle Corporation, but I gained invaluable experience, which I want to share here.

Beginner, beating his forehead against the wall ...

')
Despite the fact that I am an experienced programmer (more than 15 years), I felt myself in the shoes of a beginner as soon as I took on the NDK.

The cone is the first. Having installed Android Studio under Windows, I decided to jot down a simple “Hello, world!” With a call to JNI. Everything is very simple:

JNIEXPORT jstring JNICALL Java_ru_arts_union_real3doplayer_NativeCore_stringFromJNI(JNIEnv* env, jclass clazz) { return env->NewStringUTF("Hello, World!"); } 

Compilation error. Just a compilation error. Understand, the code is not compiled and a point. The day I tried to compile the code, actively google the problem, but after the day the code could not be collected. The next day, I took a laptop with Linux and decided to continue tormenting these two lines, to my surprise, everything gathered right away. Began to read the forums, why under Windows you can not collect the code? It turned out - one C / CPP file is not enough for building a project, at least 2 is required!

The second bump. I launch application, and it falls, informing that the stringFromJNI method was not found. How so not found? Here he is, right here and in the disassembler is visible! Do not panic, I watch all the code from and to, thank God, it is not so much. I look at the prefix in the function name ru_arts_union_real3doplayer, and something tells me that something is wrong here ... Indeed, the name of my package looked like this: ru.arts-union.real3doplayer, and the name of the method encodes the period and the dash equally, this is not good , and it is strange that the medium itself allowed such a name for the package. We change the name of the package and method, removing the dash. Hooray! The method is seen, and two days later, “Hello, world!” Works.

The bump is the third. It does not matter, there will be a lot of files, the name is corrected, we go on, thinking that the worst is over. We add our source codes, which were tested under Linux / Windows for various processors (including ARM). Everything compiled perfectly, without a single error. The application crashes on startup without any warnings. Continuing to midter over the code, I observe a floating line with a long double, I remember that NEON holds a double maximum, but this code is not even called, it just exists, and I deleted real80 in the type header. And then, the code was compiled without errors! Hmm .., than knocking on a tambourine, better drink this code ... Run, and bingo! The code works, the games run, albeit with wild brakes!

Here it is worth mentioning the work with seemingly trivial things - standard C libraries, be careful, it often happens that some methods that are present in one version are missing in another, and your application crashes due to the inability to load the desired method. Only extensive testing or rejection in favor of STL with static linking will help here. For example, TinyXML falls on a sufficiently large number of devices due to the lack of functions to convert numbers into a string and back.

The less experience you have, the more bumps you will have - be mentally prepared, for sure I did not remember everything ...

In pursuit of performance or plunge times ...

How to increase the performance of the emulator, which sometimes strained even the i7? Very simply, you need to limit yourself to a computing resource, which is what happened with the attempt to port it to Android.

In such a difficult situation, we had to implement texture caching with a graphics processor or partly or fully pixel-processed GPUs; triangulation of square polygons (3DO nonlinear texture mapping); static recompilation of code, frequently used libraries of OS console; parallelization of emulation of console subsystems. All this has reduced hardware requirements by more than three times. Where else can one save cycles, apart from classical techniques, optimization in assembler and prolonged sitting in the profiler? Obviously, in what most users will not notice the difference. So, for example, I saved well on a 16-bit raster, instead of a 24-bit, when filling a frame.

The optimization process itself is very entertaining and pulls into a separate article, but all of the above does not apply to Android as such. And what can the platform and its features give us? Obviously direct recording to the texture memory, since the CPU and GPU on mobile platforms in the vast majority of devices share it. But alas, this mechanism, known as GraphicBuffer, is not intended for general use! Nothing, you can get it through the dlopen. But this should be done optionally, since this hack may not work everywhere. Implementation can be found here (it is rather difficult to find through a search): android.googlesource.com/platform/external/deqp/+/deqp-dev/framework/platform/android/tcuAndroidInternals.cpp .

And what we really do not want to give Android, but really need? Of course control over the application cycle! But, if sooo it is necessary, then NativeActivity will help or this one here, which impresses with its simplicity and convenience: github.com/tsaarni/android-native-egl-example . Unlike NativeActivity, which is not clear how to work from a normal Activity, this approach allows you to capture the window context and draw in a separate thread, when we need to, and not when Java decides. And it was here that I plunged, without knowing it.

The fact is that this mechanism does not work on all versions of Android (at least according to my tests, version 4.1-4.3 does not support it), and this is about 40% of target devices for my application, judging by Google statistics, i.e. 40% profit off. And you can find out about it very late, because usually the user having installed the application and seeing that it does not work, immediately demolishes it and does not write a review, it is understandable, because you need to quickly return the money. Correcting the wrong application cycle is not a pleasant thing, because it is guaranteed to determine on which device it works, and on which not - it’s impossible, you can set a slower solution by default, but more compatible - it’s possible, but you spoil your karma, because not everyone will climb check in the settings: “Is it possible to twist something so that it becomes as before and does not slow down?” It is better to immediately make a more oak version, and then add an option, but in my case it's too late.

Cutting down the essence or plunging two ...

Quite often, the SDK already has functionality that you don’t want to duplicate in native code, for example, to work with fonts or images, and a lot of things, because Java has a huge library. And there is a natural desire to make a call to the Java method from C ++. I just had such a desire, and I did a cool thing - rendering fonts to texture on demand from native code, I tested everything on my devices and a bunch of virtual devices. Everything worked fine, and once again he fell in love - he made a release ... About 1% of users (who have already purchased the application) started to crash. It was, to put it mildly - very bad, negative reviews, and the situation itself - the person paid and then bang - he does not work. What is the reason, it is difficult to say, I suspect some manufacturers use a modified axis or something else. Of course, I don’t rule out a mistake, but looking at such a report, I doubt it very much (of course this was the method, in black and white):

 java.lang.NoSuchMethodError: no method with name='loadConfig' signature='(Ljava/lang/String;)Ljava/lang/String;' in class Lru/vastness/altmer/real3doplayer/MainActivity; at dalvik.system.NativeStart.run(Native Method) 

I had to very quickly cut out this solution and replace it with another. Cost a little blood. In itself, the solution is very good (1% incompatibility can be endured, just pass by and not bought), but only before launching the application, then, when many copies have already been sold - I would strongly not advise.

Updating arsenal or plunged three ...

Having rearranged the system, I set the SDK to a newer one and, accordingly, targetSdkVersion also changed. Immediately there was a nuisance. SD-drives stopped seeing, which is caused by Google’s strange machinations with access rights. At the same time, downloading an APK with an older version of the SDK was possible! It is impossible! But I got off lightly, the problem was solved by requesting permissions. And if the problem would not be solved so easily? What to do then? The question is very interesting ...

The most unpleasant thing that came with updates, which I cannot get rid of to this day, is the unwillingness of the environment to update the APK with changes in the native library, I observe the problem both under Linux and under Windows. This is annoying, so that the latest freshly assembled APK appears on the device during the debugging process, you need to do the following (Android Studio 2.1.1):

1. Rebuild the project.
2. Run the project (in this case, it will be reevaluated again, if you skip the first step, it will not be redefined).
3. Stop the application.
4. Build an APK.
5. Run the application.

And you thought, if you made a rebuild, then by running the application you will get the current freshly assembled APK on the device? Not here it was, check, but rather make yourself a pop-up window or a message in the log with the version of the changes. Otherwise, you risk losing a lot of time checking your workable code. And do not expect that the second or third run will.

Those who have similar problems, as compensation for time, if this has not already been done, I advise you to specify jobs N for the parallel assembly of native code.

The bottom line ...

In general, I haven't seen such a curvature for ten years in a draft from the development tools on this platform, but I have something to compare with. Of course, I understand that Java is a priority, but still ... With this all, only the bug found in the Intel compiler that I found many years ago when shifting to zero in a variable, since no, no, yes I will write:

  if(shift)return x>>shift; return x; 

Nevertheless, it is worth paying tribute to Google, they did everything to ensure that the salaries of native Android developers were high!

For me, as a C ++ developer, the situation with Android applications looks paradoxical, on the one hand, there are necessary mechanisms, but the cumulative amount of problems in the platform is such that when I try to make any improvements, I have to choose between loss of reputation and development applications. No platform has put me before such a choice.

Unwittingly you catch yourself thinking that it is better not to improve anything under this platform, otherwise how could something happen ...

PS Do not think - the emulator will continue to evolve, just boiling over.