OpenGL ES 3.0 in Android 4.3 - ETC2 texture compression

Just recently a new version of Android has been released - 4.3. Already long before its release, there were leaks first for the Galaxy S4, and then the Nexus 4. In these firmware, I immediately found libraries for working with OpenGL ES 3.0, which was overjoyed - rumors that the demos of OpenGL ES 3.0 shown back in March HTC One is working on native Android libraries, confirmed (as well as rumors of support for Bluetooth Low Energy).

And on Friday evening, the OTA updates came to our two devices at the same time - the Nexus 4 and the Nexus 10. On the Nexus 7, the update has not yet arrived at 4.3, but this does not upset us at all (why - I will explain later). Of course, my hands itched to try this good.

Introduction - What's new in OpenGL ES 3.0

The new version of OpenGL ES 3.0 has a number of new features that I will not list, you can learn about them from the documentation here: www.khronos.org/opengles/3_X
and in a brief press release here: www.khronos.org/news/press/khronos-releases-opengl-es-3.0-specification
In this article, we will touch upon the simplest and fastest-to-use feature of OpenGL ES 3.0 — the new standard ETC2 texture compression format.
')

ETC2

The ETC2 compression format was developed based on ETC1 and its principle of operation is based on sequences of bits not used in ETC1. In order to understand the genius of how we managed to expand ETC1 to ETC2, it is worth reading these documents:
Description of the compression algorithm: www.jacobstrom.com/publications/StromPetterssonGH07.pdf
and a description of the ETC2 format: www.graphicshardware.org/previous/www_2007/presentations/strom-etc2-gh07.pdf

ETC2, as well as ETC1, works with blocks of 4x4 pixels, but for each block a certain compression algorithm is selected - the usual ETC1 or one of three additional algorithms.

As a result, ETC2 has turned out to be a very balanced compression algorithm, which allows both to preserve the sharpness of the boundaries in some blocks and not distort smooth gradients in others.

Here is an example of compressing various images from documents:

Initializing OpenGL ES 3.0

To initialize OpenGL ES 3.0 in Android 4.3, no additional manipulations are required. You should create the usual OpenGL ES 2.0 context. If the GPU supports OpenGL ES 3.0, then Android automatically creates the OpenGL ES 3.0 context, which is fully backward compatible with OpenGL ES 2.0. That is, on Android 4.3, all applications using OpenGL ES 2.0 actually work with OpenGL ES 3.0. In order to make sure that the received context is 3.0, you should check the string GL_VERSION. Sample code from Android 4.3 source: android.googlesource.com/platform/frameworks/base/+/android-4.3_r0.9/libs/hwui/Extensions.cpp
Thanks to Romain Guy for this and other explanations about using OpenGL ES 3.0 on release day 4.3: plus.google.com/u/0/+RomainGuy/posts/iJmTjpUfR5E

Java code example:

protected int mOpenGLVersionMajor; protected int mOpenGLVersionMinor; String strGLVersion = GLES20.glGetString(GLES20.GL_VERSION); if (strGLVersion != null) { Scanner scanner = new Scanner(strGLVersion); scanner.useDelimiter("[^\\w']+"); int i = 0; while (scanner.hasNext()) { if (scanner.hasNextInt()) { if (i == 0) { mOpenGLVersionMajor = scanner.nextInt(); i++; } if (i == 1) { mOpenGLVersionMinor = scanner.nextInt(); i++; } } if (scanner.hasNext()) { scanner.next(); } } } protected Boolean isES2() { return mOpenGLVersionMajor == 2; } protected Boolean isES3() { return mOpenGLVersionMajor == 3; } 

nVidia

Then perhaps it makes sense to explain why we don’t care about the delay in updating Nexus 7 to Android 4.3. The fact is that nVidia Tegra 2/3 chips do not support OpenGL ES 3.0. Unfortunately, even Tegra 4 does not support it. Nvidia simply continues to push its desktop solutions into mobile chips, and their marketing department pushes this outdated solution successfully. That there is a very ridiculous justification for this in the Tegra 4 Whitepaper, page 11: www.nvidia.com/docs/IO/116757/Tegra_4_GPU_Whitepaper_FINALv2.pdf They admit that the chip does not support the full ES 3.0 specification, and openly say that it’s “we We do not expect that soon applications / games will use ES 3.0 ”. Surprise - Android 4.3 itself uses OpenGL ES 3.0.

Although despite the fact that nVidia is not going to expand the current chips to support ES 3.0, Tegra 5 will still support it: www.ubergizmo.com/2013/07/nvidia-tegra-5-release-date-specs-news

Creating compressed textures

To create an ETC2 texture, use the Mali Texture Compression Tool: malideveloper.arm.com/develop-for-mali/mali-gpu-texture-compression-tool . To obtain the best quality texur, the compression method “Slow” and Error Metric “Perceptual” were used.

Below are images for comparing the compression quality of ETC1 and ETC2, as well as the difference between the original and the compressed image, enhanced 4 times for clarity.

When compressing the texture in ETC1, artifacts are visible in the form of horizontal (especially well visible) and vertical stripes. When compressing ETC2, these artifacts are virtually absent.
In our live wallpapers, on areas of scenes for which the texture quality is critical, we use uncompressed textures. As seen in the comparative image, ETC1 introduces the most noticeable distortion in textures with smooth gradients — compression artifacts are clearly visible, caused by the compression feature of squares measuring 4x4 pixels. Therefore, we apply textures without compression to the sky, and they take up quite a lot of space - because their size is 2048x512. Compression in the PVRTC format also gives fairly good texture quality, but is only available on PowerVR chips. The use of the standard for ES 3.0 format ETC2 allowed us to achieve an acceptable texture quality while reducing the amount of video memory allocated for the texture by 4 times:
For texture 2048x512:
Uncompressed (16-bit color 565 - 2 bytes per pixel): 2 * 2048 * 512 = 2097152 // 2 MB of data
Compressed (16 bytes - PKM header): 524304-16 = 524288 // 512 kB data.

Loading ETC2 textures

The texture is loaded from a .pkm file. The same format is used to store ETC1 textures. The header format is described here: forums.arm.com/index.php?/topic/15835-pkm-header-format

I decided not to try to load ETC2 textures using ETC1Util, since it has header validation.
Texture Download Code:

 protected int loadETC2Texture(String filename, int compression, Boolean bClamp, Boolean bHighQuality) { int[] textures = new int[1]; GLES20.glGenTextures(1, textures, 0); int textureID = textures[0]; GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, textureID); if (bHighQuality) { GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_LINEAR); GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); } else { GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MIN_FILTER, GLES20.GL_NEAREST); GLES20.glTexParameterf(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_MAG_FILTER, GLES20.GL_LINEAR); } if (bClamp) { GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_CLAMP_TO_EDGE); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_CLAMP_TO_EDGE); } else { GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_S, GLES20.GL_REPEAT); GLES20.glTexParameteri(GLES20.GL_TEXTURE_2D, GLES20.GL_TEXTURE_WRAP_T, GLES20.GL_REPEAT); } InputStream is = null; try { is = mWallpaper.getContext().getAssets().open(filename); } catch (IOException e1) { e1.printStackTrace(); } try { byte[] data = readFile(is); ByteBuffer buffer = ByteBuffer.allocateDirect(data.length).order(ByteOrder.LITTLE_ENDIAN); buffer.put(data).position(PKM_HEADER_SIZE); ByteBuffer header = ByteBuffer.allocateDirect(PKM_HEADER_SIZE).order(ByteOrder.BIG_ENDIAN); header.put(data, 0, PKM_HEADER_SIZE).position(0); int width = header.getShort(PKM_HEADER_WIDTH_OFFSET); int height = header.getShort(PKM_HEADER_HEIGHT_OFFSET); GLES20.glCompressedTexImage2D(GLES20.GL_TEXTURE_2D, 0, compression, width, height, 0, data.length - PKM_HEADER_SIZE, buffer); checkGlError("Loading of ETC2 texture; call to glCompressedTexImage2D()"); } catch (Exception e) { Log.w(TAG, "Could not load ETC2 texture: " + e); } finally { try { is.close(); } catch (IOException e) { // ignore exception thrown from close. } } return textureID; } ... if (isES3()) { textureID = loadETC2Texture("textures/etc2/sky1.pkm", GLES30.GL_COMPRESSED_RGB8_ETC2, false, false); } else { textureID = loadTexture("textures/sky1.png"); } ... 

Thus, if there is an ES 3.0 context, the ETC2 texture will load, and in ES 2.0 mode, the normal uncompressed texture.

Of course, to access the GLES30 class, you need to set android: targetSdkVersion = "18" in the application manifest and target = android-18 in the project.properties.

Result

In the application, the difference between the uncompressed texture (without distortion) and ETC2 is not noticeable:


The application is available here: play.google.com/store/apps/details?id=org.androidworks.livewallpapercarfree

Conclusion

We always try to use the new features of each new version of Android to optimize performance and enhance capabilities. In particular, wallpaper support and work in the screen saver mode - daydream. Although the article was not large enough, but I hope that our modest experience in using ETC2 can help someone to optimize their applications.

PS

If someone has set themselves the leaked 4.3 firmware for Samsung Galaxy S4, please check the operation of this application on this device.