Detective story with CMake 3.10 and Android Studio
Disclaimer: Everything described below is not a good practice. You should not read this text as a guide to action - its role is rather entertaining. For the same reason, it does not make sense to advise the author (me) to change the language, tools, operating system, hardware, gender and country of residence.
I have one project. To build it uses CMake, as well as a package manager for C ++ called Hunter , well with CMake integrated. The project needs to be built for several platforms, one of which is Android. Hunter builds dependencies for Android without problems - but he needs a version of CMake> = 3.7, since it was in 3.7 that improved support for this platform was added. This is a very important detail.
CMake supports project generation for Android under Windows / Visual Studio using the NVIDIA Nsight Tegra plugin. Unfortunately, this plugin is not installed on Visual Studio 2017, and I didn’t want to install 2015 for the sake of one project. On the other hand, VS2017 has build support for Android — but there is no generation of such projects from CMake. More precisely, it is, but it requires a self-made version of CMake from Microsoft , the development of which was abandoned in the area of ​​version 3.4, that is, it does not suit me.
Well, if not Visual Studio, then Android Studio! CMake, of course, does not support it, but it supports it, as much as from version 2.x. You specify the path to CMakeLists.txt in build.gradle, start synchronization, and everything should work. But it does not work, but says "You have not installed CMake." Why - but because it also works with its own CMake fork, which needs to be downloaded from the Android repository, and which is frozen on version 3.6.
')
It would seem a dead end. But in fact, no, because since version 3.0, Android Studio is able to work with external versions of CMake older than 3.6, due to the very "improved support for Android." It is enough to set the cmake.dir field in local.properties, and everything should work. But it does not work, but it says ... Hmm, to be honest, I already forgot what he said there, but the bottom line is that the error text, scored on Google, led me to a conversation in a bug tracker, where it was mentioned that in versions 3.7 and 3.8 CMake with Android Studio somehow communicate incorrectly, and this should all be fixed in version 3.9.
And I just have CMake 3.8. Well, it doesn't matter, we go to the site and download the freshest CMake, which has version 3.10 (well, the truth is 3.10 should be better than 3.9, or at least not worse?). Install and everything should work. But it does not work, but says “Error communicating with CMake Server - see logs”. Logs also concisely declare:
As you can see, absolutely no mistakes. By the way, yes, but what about CMake Server? The first time I've heard. We go to the documentation and read that this is a new mode of operation for CMake, which allows external programs to receive information about the CMake project in a machine-readable form, which simplifies integration with various IDEs and additional tools.
This is where the detective begins. The first evidence is the “major” and “minor” fields in the server's response. Once there is a protocol version, someone can check it and not work with the “wrong” versions. The assumption is very probable, but only an assumption - it would be good to get its confirmation.
In fact, at this moment I decided to pause andtransfer the case to the police to get a bug in the Android Studio bug tracker, as they suggest in their documentation. Unfortunately, there was no reaction to the bug, and after a few days it disappeared from the tracker - it was not closed, even if it was from WONTFIX, but it disappeared without a trace. Being angry, I decided to bring my own investigation to the end.
Perhaps among the readers there are people who are actively working with the Android Open Source Project, or at least just familiar with its device. Let them laugh at my further adventures.
I had to find the source code of the Android plugin for Gradle, which is responsible for integrating with CMake. As an ordinary decent citizen, I scored the phrase “android gradle plugin source” on Google, and received in response a repository page . However, a brief search on it using the web interface showed that the last commit to master was 2 years ago. And where are the fresh samples? Unclear!
The second link from the same search led me to the instruction “how to download the AOSP source code” . It’s not easy to do this: Google, as always, isn’t like people, and you can’t just extort a specific project from Git. You need to take their own tool called repo, and use it.
Edit: in the comments suggest that it is possible, and the projects still lie in a separate Git each - and I just blindly did not see the big inscription "Clone this repo:" at the top of the page. I apologize for the misinformation!
We take repo, make sure that this is a regular Python script, run it, and get an error - no fcntl import was found. Yes, this script does not work under Windows. Hooray, I love you, google. We launch a virtual machine with Ubuntu, randomly lying in a corner, download the repo there, launch it, and, seeing that something is happening, we are leaving for lunch.
Upon my return, I was greeted by the error "no space left on device". Well, yes, there it was 5Gb, but the project I need cannot weigh in so much, even with all the dependencies! However, a short investigation revealed that repo is pumping out the entire, or almost all, of the AOSP repository, and there is absolutely no possibility to point out a separate project to it. Partly, as I understand it, this is Git's fault, or rather, how Google uses it. Instead of sticking each project into their own repository, they put everything in one. And Git, unlike SVN, does not know how to extort the sub-directories of the repository from the word “absolutely”.
“No,” I decided, “to pump out the entire Android with circus and horses for one or two files that I need to watch, I don’t want to.”
Intuition suggested that somewhere there should be the source code of the plugin version I needed. But the mind is at a dead end. So I decided to look for help on official resources for Android developers. We reject the group on Google+ right away - this is not a platform for serious conversations. But the good old Mailing List of Google Groups is what we need! There are two of them - “just Android (android-developers)” and “for those who are NDK (android-ndk)”. My problem with the NDK relationship seems to be small, so we open the first link ... And we get the message - “This group is sending malware, so we closed it nafig. Signature, your Google. ".
Scene of a little surprise.
Ok, open the second group for NDK developers. She is working. In the search we drive in the word CMake, hoping to hook on something interesting - and we find! The topic is “How to tell Android Studio external CMake?” . Her brief review shows that this is exactly the place where the change was discussed, which led to the release of plug-in version 3.0.0. this opportunity. A person who understands something in the plugin was called to the topic. He gave the author of the topic a link to the repository (all the same useless where it is written to use repo), but most importantly he could not restrain himself, and copied a part of the code from a file into one of the letters, indicating his name!
That was enough for me. I scored the name in Google search, and immediately found it in their repository. True, it was a file from the studio-2.2-preview3 branch, that is, an earlier one that does not have integration with external versions of CMake.
Then I realized what was supposed to go much earlier - two years ago, the plug-in developers stopped committing to master, and began to lead the whole development in the branches, never pouring the results into the main one. The only thing that justifies me is that I never worked with Git in this style, and in general, ALL the projects I saw usually did change the master at the end. However, even if I knew that I had to dig in the list of branches, it’s not a fact that I would then find the right one — there are hundreds of them, I don’t have a clue about the name of the correct one.
We change the name of the branch in studio-3.0 right in the URL, and ... We find, but not quite that: there is no mention in this file of the version of the CMake Server protocol. But this is no longer scary, since through the web interface, unlike repo, you can download a separate repository folder in tgz, and search for it in the necessary folder.
Review of the code showed that the case was really in the protocol version. The plugin compared it to a clear match 1.0 (in the getSupportedVersion function), and the new CMake used 1.1 (in the next version, by the way, it will already be 1.2). From the study of CMake code, I knew that version 1.1 does not add anything fundamentally breaking, so there were no obstacles to the fact that Android Studio could work with it, except for this stupid check. Therefore, I found the file gradle-core-3.0.0.jar on my disk, got the necessary class out of it, and using the Java Bytecode Editor, I changed the check so that everything worked ( disclaimer: do not repeat at home (and there more on workplace); the trick is done by a desperate and very pissed programmer ).
And it all worked.
PS In fact, not quite: for some reason, when building for Android, CMake uses incorrect relative paths to the source files, which then go to the object files, and as a result, to debug information inside the library. And then the debugger cannot find anything along these paths, because the library is not there at all. But this is another, and a much shorter story.
I have one project. To build it uses CMake, as well as a package manager for C ++ called Hunter , well with CMake integrated. The project needs to be built for several platforms, one of which is Android. Hunter builds dependencies for Android without problems - but he needs a version of CMake> = 3.7, since it was in 3.7 that improved support for this platform was added. This is a very important detail.
CMake supports project generation for Android under Windows / Visual Studio using the NVIDIA Nsight Tegra plugin. Unfortunately, this plugin is not installed on Visual Studio 2017, and I didn’t want to install 2015 for the sake of one project. On the other hand, VS2017 has build support for Android — but there is no generation of such projects from CMake. More precisely, it is, but it requires a self-made version of CMake from Microsoft , the development of which was abandoned in the area of ​​version 3.4, that is, it does not suit me.
Well, if not Visual Studio, then Android Studio! CMake, of course, does not support it, but it supports it, as much as from version 2.x. You specify the path to CMakeLists.txt in build.gradle, start synchronization, and everything should work. But it does not work, but says "You have not installed CMake." Why - but because it also works with its own CMake fork, which needs to be downloaded from the Android repository, and which is frozen on version 3.6.
')
It would seem a dead end. But in fact, no, because since version 3.0, Android Studio is able to work with external versions of CMake older than 3.6, due to the very "improved support for Android." It is enough to set the cmake.dir field in local.properties, and everything should work. But it does not work, but it says ... Hmm, to be honest, I already forgot what he said there, but the bottom line is that the error text, scored on Google, led me to a conversation in a bug tracker, where it was mentioned that in versions 3.7 and 3.8 CMake with Android Studio somehow communicate incorrectly, and this should all be fixed in version 3.9.
And I just have CMake 3.8. Well, it doesn't matter, we go to the site and download the freshest CMake, which has version 3.10 (well, the truth is 3.10 should be better than 3.9, or at least not worse?). Install and everything should work. But it does not work, but says “Error communicating with CMake Server - see logs”. Logs also concisely declare:
CMAKE SERVER: [== "CMake Server" ==[ CMAKE SERVER: {"supportedProtocolVersions":[{"isExperimental":true,"major":1,"minor":1}],"type":"hello"} CMAKE SERVER: ]== "CMake Server" ==] As you can see, absolutely no mistakes. By the way, yes, but what about CMake Server? The first time I've heard. We go to the documentation and read that this is a new mode of operation for CMake, which allows external programs to receive information about the CMake project in a machine-readable form, which simplifies integration with various IDEs and additional tools.
This is where the detective begins. The first evidence is the “major” and “minor” fields in the server's response. Once there is a protocol version, someone can check it and not work with the “wrong” versions. The assumption is very probable, but only an assumption - it would be good to get its confirmation.
In fact, at this moment I decided to pause and
Perhaps among the readers there are people who are actively working with the Android Open Source Project, or at least just familiar with its device. Let them laugh at my further adventures.
I had to find the source code of the Android plugin for Gradle, which is responsible for integrating with CMake. As an ordinary decent citizen, I scored the phrase “android gradle plugin source” on Google, and received in response a repository page . However, a brief search on it using the web interface showed that the last commit to master was 2 years ago. And where are the fresh samples? Unclear!
The second link from the same search led me to the instruction “how to download the AOSP source code” . It’s not easy to do this: Google, as always, isn’t like people, and you can’t just extort a specific project from Git. You need to take their own tool called repo, and use it.
Edit: in the comments suggest that it is possible, and the projects still lie in a separate Git each - and I just blindly did not see the big inscription "Clone this repo:" at the top of the page. I apologize for the misinformation!
We take repo, make sure that this is a regular Python script, run it, and get an error - no fcntl import was found. Yes, this script does not work under Windows. Hooray, I love you, google. We launch a virtual machine with Ubuntu, randomly lying in a corner, download the repo there, launch it, and, seeing that something is happening, we are leaving for lunch.
Upon my return, I was greeted by the error "no space left on device". Well, yes, there it was 5Gb, but the project I need cannot weigh in so much, even with all the dependencies! However, a short investigation revealed that repo is pumping out the entire, or almost all, of the AOSP repository, and there is absolutely no possibility to point out a separate project to it. Partly, as I understand it, this is Git's fault, or rather, how Google uses it. Instead of sticking each project into their own repository, they put everything in one. And Git, unlike SVN, does not know how to extort the sub-directories of the repository from the word “absolutely”.
“No,” I decided, “to pump out the entire Android with circus and horses for one or two files that I need to watch, I don’t want to.”
Intuition suggested that somewhere there should be the source code of the plugin version I needed. But the mind is at a dead end. So I decided to look for help on official resources for Android developers. We reject the group on Google+ right away - this is not a platform for serious conversations. But the good old Mailing List of Google Groups is what we need! There are two of them - “just Android (android-developers)” and “for those who are NDK (android-ndk)”. My problem with the NDK relationship seems to be small, so we open the first link ... And we get the message - “This group is sending malware, so we closed it nafig. Signature, your Google. ".
Scene of a little surprise.
Ok, open the second group for NDK developers. She is working. In the search we drive in the word CMake, hoping to hook on something interesting - and we find! The topic is “How to tell Android Studio external CMake?” . Her brief review shows that this is exactly the place where the change was discussed, which led to the release of plug-in version 3.0.0. this opportunity. A person who understands something in the plugin was called to the topic. He gave the author of the topic a link to the repository (all the same useless where it is written to use repo), but most importantly he could not restrain himself, and copied a part of the code from a file into one of the letters, indicating his name!
That was enough for me. I scored the name in Google search, and immediately found it in their repository. True, it was a file from the studio-2.2-preview3 branch, that is, an earlier one that does not have integration with external versions of CMake.
Then I realized what was supposed to go much earlier - two years ago, the plug-in developers stopped committing to master, and began to lead the whole development in the branches, never pouring the results into the main one. The only thing that justifies me is that I never worked with Git in this style, and in general, ALL the projects I saw usually did change the master at the end. However, even if I knew that I had to dig in the list of branches, it’s not a fact that I would then find the right one — there are hundreds of them, I don’t have a clue about the name of the correct one.
We change the name of the branch in studio-3.0 right in the URL, and ... We find, but not quite that: there is no mention in this file of the version of the CMake Server protocol. But this is no longer scary, since through the web interface, unlike repo, you can download a separate repository folder in tgz, and search for it in the necessary folder.
Review of the code showed that the case was really in the protocol version. The plugin compared it to a clear match 1.0 (in the getSupportedVersion function), and the new CMake used 1.1 (in the next version, by the way, it will already be 1.2). From the study of CMake code, I knew that version 1.1 does not add anything fundamentally breaking, so there were no obstacles to the fact that Android Studio could work with it, except for this stupid check. Therefore, I found the file gradle-core-3.0.0.jar on my disk, got the necessary class out of it, and using the Java Bytecode Editor, I changed the check so that everything worked ( disclaimer: do not repeat at home (and there more on workplace); the trick is done by a desperate and very pissed programmer ).
And it all worked.
PS In fact, not quite: for some reason, when building for Android, CMake uses incorrect relative paths to the source files, which then go to the object files, and as a result, to debug information inside the library. And then the debugger cannot find anything along these paths, because the library is not there at all. But this is another, and a much shorter story.
Source: https://habr.com/ru/post/357712/
All Articles