Cocos2d-x for Android: speeding up file reading
One of my recent projects is porting a game from iOS to Android. The game is written using Cocos2d-x , a fairly popular cross-platform game engine.
I want to tell you how you can easily increase the speed of loading a large game repeatedly, and also improve the overall smoothness of the reaction to user actions.
The game on iOS uses version 0.13, so for Android they took the same one (including the latest fixes from the official repository). Everything went in general normally, without major disappointments and found significant bugs in the engine, the version of the toy for weak graphics (the old iPhone 480x320) worked fine. It's time to add “HD” graphics mode (960x640).
On iOS, in the first cocos2d branch, support for other modes is performed by automatically using a file with the current mode suffix, if such a file exists, otherwise, with a file without such a suffix. Suffixes are usually ' -hd ' for iPhone HD and iPad SD modes, and also ' -ipadhd ' for iPad HD. So, when you try to load the file 'image.png', the file 'image-hd.png' can actually load. For the version for Android, this was not added independently. The graphics got better.
The toy began to load longer, and it is much worse to respond to user actions (transitions in the menu, for example).
')
At first I decided that since every time a file with a suffix is checked for existence, a simple cache in the form of std :: unordered_map <std :: typed requestedFile, std :: string receivedFile> is enough to check the file for existence only 1 time - partially helped but the situation remained very difficult. It was surprising that for the port of the same game on Win8 such a problem was not observed (and the version for Win8 supported everything, including the iPad HD mode - with textures under the screen 2048x1536).
Began to understand further.
CCFileUtils :: getFileData - the method responsible for reading files in Cocos2d-x (on github - and further links there) - for downloading from the APK, calls CCFileUtils :: getFileDataFromZip :
That is, every time for any file from the APK - there is a separate opening / closing of the archive. The simplest action is to open at least once, I decide to check if it is fixed in 2.0.x versions of Cocos2d-x - I find that it is not only not fixed, but also worsened - due to the changed logic of files for different graphics (already added for Android version) - double attempt to read from the APK file is possible.
All right, I look further - call unzLocateFile. What does this feature do?
That is, every time there is a linear search in all files in the archive. Everytime. There deeper in the code - with saving memory, so now and then - moving through the file with the archive and reading. In the APK with a toy - more than 1300 files with resources.
And for gles20, the version of cocos2d-x is generally the worst case when the file is not read, and you need to look for the second one again.
Well - I decided to somehow cache at least a list of files, and preferably its position in the archive so that unzLocateFile would work faster. I start digging further - and with surprise I discover that everything is already there, you just need to start using:
So, in general, it is elementary to generate a list of files with positions 1 time, and further on - calmly read directly what is required.
The verification of the idea showed an acceleration of loading 5 times (and the version with heavy graphics, compared with the old version with simplified graphics), and the overall improvement of the game (as there are a lot of graphics, it is impossible to keep everything in memory, and you always have to load something) .
Later it turned out that it was not necessary to call unzGetCurrentFileInfo separately after unzGoToFirstFile / unzGoToNextFile, they still read the same information.
So in the end, a helper class was created that reads the entire list of files from the ZIP in about the same time as searching for one existing file of the original unzLocateFile.
Pull request for the latest version of Cocos2d-x, for Android - it works exactly. The version for 0.13 is obtained by minor editing.
Use on health.
Why is this possible? Everything in the source is actually ready, the search for the solution took just a couple of hours. Nevertheless, the problem existed and was solved by various methods, mainly - “fewer files in the APK”.
Then I already found an offer 3 months ago to use Asset Manager - I’ll have to look at what is there and how, and whether there will be any performance increase (or already a drop).
I think the same thing was repeatedly solved in different companies, just no one returned the fix back.
The used parts of the MiniZip in Cocos2d-x - in fact, simply demonstrate how to do it, and were never intended to constantly read random files.
So despite two years, in the public version there are still many places where it is elementary to improve something. Productivity - yes, the same classes from CCProfiling.h / .cpp, intended for profiling the engine, where something is calculated by adding the old average and the new value, and then dividing it in half . So the results for, for example, the same values of 10 4 1 and 1 4 10 are different (4 and 6.25, respectively). Exactly the same problem in the source code of cocos2d-iphone, so I will do a general correction a bit later - if no one is interested in it.
I hope there will be time, and other corrections will also be added to the official version, with a translation from 0.13 to the current 2.x.
Learn more about Cocos2d-x for Android
Cocos2d-x for Android has actually been two years old, quite a considerable age. Open source, MIT license (does not require return of changes).
The latest stable release for OpenGL ES 1.x is 0.13.0, released in March of this year.
The first release for OpenGL ES 2.0 - 2.0.2, appeared in late August.
The latest stable release for OpenGL ES 1.x is 0.13.0, released in March of this year.
The first release for OpenGL ES 2.0 - 2.0.2, appeared in late August.
I want to tell you how you can easily increase the speed of loading a large game repeatedly, and also improve the overall smoothness of the reaction to user actions.
Quiet work
The game on iOS uses version 0.13, so for Android they took the same one (including the latest fixes from the official repository). Everything went in general normally, without major disappointments and found significant bugs in the engine, the version of the toy for weak graphics (the old iPhone 480x320) worked fine. It's time to add “HD” graphics mode (960x640).
On iOS, in the first cocos2d branch, support for other modes is performed by automatically using a file with the current mode suffix, if such a file exists, otherwise, with a file without such a suffix. Suffixes are usually ' -hd ' for iPhone HD and iPad SD modes, and also ' -ipadhd ' for iPad HD. So, when you try to load the file 'image.png', the file 'image-hd.png' can actually load. For the version for Android, this was not added independently. The graphics got better.
Problem
The toy began to load longer, and it is much worse to respond to user actions (transitions in the menu, for example).
')
At first I decided that since every time a file with a suffix is checked for existence, a simple cache in the form of std :: unordered_map <std :: typed requestedFile, std :: string receivedFile> is enough to check the file for existence only 1 time - partially helped but the situation remained very difficult. It was surprising that for the port of the same game on Win8 such a problem was not observed (and the version for Win8 supported everything, including the iPad HD mode - with textures under the screen 2048x1536).
Began to understand further.
CCFileUtils :: getFileData - the method responsible for reading files in Cocos2d-x (on github - and further links there) - for downloading from the APK, calls CCFileUtils :: getFileDataFromZip :
Source code
unsigned char* CCFileUtils::getFileDataFromZip(const char* pszZipFilePath, const char* pszFileName, unsigned long * pSize) { unsigned char * pBuffer = NULL; unzFile pFile = NULL; *pSize = 0; do { CC_BREAK_IF(!pszZipFilePath || !pszFileName); CC_BREAK_IF(strlen(pszZipFilePath) == 0); pFile = unzOpen(pszZipFilePath); CC_BREAK_IF(!pFile); int nRet = unzLocateFile(pFile, pszFileName, 1); CC_BREAK_IF(UNZ_OK != nRet); char szFilePathA[260]; unz_file_info FileInfo; nRet = unzGetCurrentFileInfo(pFile, &FileInfo, szFilePathA, sizeof(szFilePathA), NULL, 0, NULL, 0); CC_BREAK_IF(UNZ_OK != nRet); nRet = unzOpenCurrentFile(pFile); CC_BREAK_IF(UNZ_OK != nRet); pBuffer = new unsigned char[FileInfo.uncompressed_size]; int nSize = 0; nSize = unzReadCurrentFile(pFile, pBuffer, FileInfo.uncompressed_size); CCAssert(nSize == 0 || nSize == (int)FileInfo.uncompressed_size, "the file size is wrong"); *pSize = FileInfo.uncompressed_size; unzCloseCurrentFile(pFile); } while (0); if (pFile) { unzClose(pFile); } return pBuffer; } That is, every time for any file from the APK - there is a separate opening / closing of the archive. The simplest action is to open at least once, I decide to check if it is fixed in 2.0.x versions of Cocos2d-x - I find that it is not only not fixed, but also worsened - due to the changed logic of files for different graphics (already added for Android version) - double attempt to read from the APK file is possible.
All right, I look further - call unzLocateFile. What does this feature do?
err = unzGoToFirstFile(file); while (err == UNZ_OK) { char szCurrentFileName[UNZ_MAXFILENAMEINZIP+1]; err = unzGetCurrentFileInfo64(file,NULL, szCurrentFileName,sizeof(szCurrentFileName)-1,NULL,0,NULL,0); if (err == UNZ_OK) { if (unzStringFileNameCompare(szCurrentFileName, szFileName,iCaseSensitivity)==0) return UNZ_OK; err = unzGoToNextFile(file); } } That is, every time there is a linear search in all files in the archive. Everytime. There deeper in the code - with saving memory, so now and then - moving through the file with the archive and reading. In the APK with a toy - more than 1300 files with resources.
And for gles20, the version of cocos2d-x is generally the worst case when the file is not read, and you need to look for the second one again.
Decision
Well - I decided to somehow cache at least a list of files, and preferably its position in the archive so that unzLocateFile would work faster. I start digging further - and with surprise I discover that everything is already there, you just need to start using:
/* unz_file_info contain information about a file in the zipfile */ typedef struct unz_file_pos_s { uLong pos_in_zip_directory; /* offset in zip file directory */ uLong num_of_file; /* # of file */ } unz_file_pos; int ZEXPORT unzGetFilePos(unzFile file, unz_file_pos* file_pos); int ZEXPORT unzGoToFilePos(unzFile file, unz_file_pos* file_pos); So, in general, it is elementary to generate a list of files with positions 1 time, and further on - calmly read directly what is required.
The verification of the idea showed an acceleration of loading 5 times (and the version with heavy graphics, compared with the old version with simplified graphics), and the overall improvement of the game (as there are a lot of graphics, it is impossible to keep everything in memory, and you always have to load something) .
Later it turned out that it was not necessary to call unzGetCurrentFileInfo separately after unzGoToFirstFile / unzGoToNextFile, they still read the same information.
So in the end, a helper class was created that reads the entire list of files from the ZIP in about the same time as searching for one existing file of the original unzLocateFile.
Pull request for the latest version of Cocos2d-x, for Android - it works exactly. The version for 0.13 is obtained by minor editing.
Use on health.
findings
Why is this possible? Everything in the source is actually ready, the search for the solution took just a couple of hours. Nevertheless, the problem existed and was solved by various methods, mainly - “fewer files in the APK”.
Then I already found an offer 3 months ago to use Asset Manager - I’ll have to look at what is there and how, and whether there will be any performance increase (or already a drop).
I think the same thing was repeatedly solved in different companies, just no one returned the fix back.
The used parts of the MiniZip in Cocos2d-x - in fact, simply demonstrate how to do it, and were never intended to constantly read random files.
So despite two years, in the public version there are still many places where it is elementary to improve something. Productivity - yes, the same classes from CCProfiling.h / .cpp, intended for profiling the engine, where something is calculated by adding the old average and the new value, and then dividing it in half . So the results for, for example, the same values of 10 4 1 and 1 4 10 are different (4 and 6.25, respectively). Exactly the same problem in the source code of cocos2d-iphone, so I will do a general correction a bit later - if no one is interested in it.
I hope there will be time, and other corrections will also be added to the official version, with a translation from 0.13 to the current 2.x.
Source: https://habr.com/ru/post/158439/
All Articles