If you use sprites with transparency in your game (and usually it happens, at least for UI), then you probably should pay attention to fully transparent texture pixels (or “texels”).

Even if the alpha channel value is 0, the color value is still associated with the pixel. This color does not affect anything, right? In the end, the pixel is completely transparent, who cares about its color ...

So, in fact, this color is important, if you do not understand this, you can get artifacts that are noticeable in many games. Most often, the distortions are very small and not noticeable, but sometimes they are really striking.

Distortion example

It's time to give an example from real life! Here is my PS3 XMB (main menu), with demo versions of several games.
')
Limbo is selected first, then I just press "up" to move to The Unfinished Sawn
(by the way, both games are great).


Start.


I press down. Limbo goes down.


The background turns white.


Artifacts.

See what happened to the Limbo logo area?

The background changed to the white background of The Unfinished Swan and as a result the “absolutely white” Limbo logo was drawn over the background, which is also completely white. This area should be completely white, then where did these strange gray pixels come from?

Most likely, the distortion is due to the fact that the texture Limbo uses the wrong RGB colors for completely transparent pixels.

Texture filtering

Artifacts actually arise from the way the video processor filters the texture when rendering the sprite on the screen. Let's look at his work on a simple example.

Here is a small 12x12 pixel texture with a red cross:



But its enlarged image, a checkerboard, simply shows that it is a completely transparent area with an alpha value of 0.

You can use this sprite as an icon for displaying health in the UI or as a texture for the first-aid kit game model. ( Although not! Actually, this is not worth doing! )

Let's create three versions of this sprite by simply changing the pixel color value with zero alpha transparency.


Transparent area: green. How the image looks like:


Transparent area: blue. How the image looks like:


Transparent area: red. How the image looks like:

(You can download files and check the RGB values ​​of transparent pixels)

These three sprites look exactly the same on the screen, right? This is logical: we just changed the color value of transparent pixels, which will still be invisible.

But let's see what happens when these sprites are in motion. Here is an enlarged image to better see screen pixels:



We see here the distortion! Brown shade at the first sprite and purple at the second. The third is all right, this is how it should look.

Let's look at the blue version:



As we can see, the problem occurs when the position of the texture does not correspond pixel by pixel to screen pixels. This can be explained by bilinear filtering , which the video processor performs when rendering the sprite on the screen: when sampling the texture, the video processor averages the color values ​​of the nearest neighboring pixels with the requested coordinates in the vertical and horizontal directions.

Consider the case when the position of the sprite does not match exactly half a pixel:



Each screen pixel samples the sprite texture exactly between two texels. This is exactly what happens with the pixel seen in the image: it selects a sprite texture in the middle between a solid red texel and a transparent blue texel. The average color is:

$$$$ display $$ 0.5 \ ast \ begin {bmatrix} \ color {# ff2c2c} 1 \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} 0 \\ 1 \\ \ end {bmatrix} + 0.5 \ ast \ begin {bmatrix} \ color {# ff2c2c} 0 \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} 1 \\ 0 \\ \ end {bmatrix} = \ begin {bmatrix } \ color {# ff2c2c} {0,5} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0,5} \\ 0.5 \\ \ end {bmatrix} $$ display $ $

This is partially transparent purple, like this: █

This color, returned by the texture sampler, will now be added to the alpha channel of the rendering result (solid white).

The mixing equation is:

$$

$$\ alpha_ {background} \ ast RGB_ {sprite} + (1 - \ alpha_ {sprite}) \ ast RGB_ {background}$$

$$$$ display $$ = 0.5 \ ast \ begin {bmatrix} \ color {# ff2c2c} {0.5} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0.5} \ \ \ end {bmatrix} + (1 - 0.5) \ ast \ begin {bmatrix} \ color {# ff2c2c} 1 \\ \ color {# 00c300} 1 \\ \ color {# 2f9fff} 1 \\ \ end {bmatrix} = \ begin {bmatrix} \ color {# ff2c2c} {0.75} \\ \ color {# 00c300} {0.5} \\ \ color {# 2f9fff} {0.75} \\ \ end {bmatrix} $$ display $$

Therefore, the final color of the pixel on the screen will be something like this: █

It does not suit us. The correct result (which we obtained when the transparent pixels were red) would be:

$$$$ display $$ \ begin {bmatrix} \ color {# ff2c2c} {1} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0} \\ 0.5 \\ \ end {bmatrix } $$ display $$

- this is a bilinear interpolated value, which is then mixed and obtained

$$$$ display $$ \ begin {bmatrix} \ color {# ff2c2c} {1} \\ \ color {# 00c300} {0.5} \\ \ color {# 2f9fff} {0.5} \\ \ end { bmatrix} $$ display $$

The pixel on the screen looks like this: █

How can we avoid these unpleasant artifacts?

How to avoid this problem

If you are an artist: let everything leak out!

If you are responsible for creating graphic resources, then secure your work and do not trust the programmers and the engine.

It is likely that at some stage of the conveyor the color of transparent pixels will “leak out” to the surrounding graphics. We have already seen how this happens with bilinear filtering of textures, but it can happen this way when generating MIP textures ...

You can fight such a seepage of flowers ... an additional seepage !


Source sprite


RGB only

By this, I mean that before exporting graphics to a disk, you must first make sure that all opaque pixels “leak” into the RGB values ​​of neighboring transparent pixels (this is also called flood-filling or edge-padding ). In this case, when transparent pixels in the process of playing the game leak to their opaque neighbors, they will at least leak with the correct color.

The images above show an example from the real world: an atlas of vegetation sprites, extracted from GTA V , with and without alpha channel.

Note the border around pixels with non-zero transparency: transparent pixels borrow the colors of their nearest visible neighbors. Rockstar did not accidentally do all this work.

Tools can help us in this process: Photoshop has a Solidify plugin ,
there is a plugin for Gimp too ...

Also, be careful when exporting RGB values ​​of transparent pixels, for example, when saving PNGs: many programs by default discard RGB data of transparent pixels and replace them when exporting with a solid color (white or black) to improve compression.

If you are a programmer: use Premultiplied Alpha!

If you are a programmer, you already know that you should not blindly trust graphic resources created by artists. Fortunately, programmers have more opportunities to deal with this problem.

You can use a tool to automate color leakage that we talked about earlier. It should be used when importing resources. But we have a much better and more reliable solution: premultiplied alpha.

I will not talk about it in detail, because other people have written good descriptions, for example, here and here .

I also highly recommend Tom Forsyth's posts: 1 and 2 on this topic.

The idea is very simple: instead of storing textures like

$$$$ display $$ \ begin {bmatrix} \ color {# ff2c2c} {R} \\ \ color {# 00c300} G \\ \ color {# 2f9fff} {B} \\ \ alpha \\ \ end {bmatrix} $$ display $$

you need to keep it as

$$$$ display $$ \ begin {bmatrix} \ alpha \ ast \ color {# ff2c2c} {R} \\ \ alpha \ ast \ color {# 00c300} {G} \\ \ alpha \ ast \ color {# 2f9fff} {B} \\ \ alpha \\ \ end {bmatrix} $$ display $$

The RGB components are simply multiplied by the pixel alpha transparency value. The original color can still be easily obtained by dividing the value into alpha transparency.

So you can turn the sprite:


Original


Premultiplied Alpha

You also need to change the blending equation, because our texture now contains the result of the first multiplication, and it does not need to be multiplied by the alpha transparency value again:

$$

$$RGB_ {sprite} + (1 - \ alpha_ {sprite}) \ ast RGB_ {background}$$

In OpenGL, this is expressed in the following changes to the blending function:

glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) ⇒ glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA)

Let's return to our sprite with a red cross shifted by half a pixel. In the case of the Premultiplied Alpha mode, the bilinear interpolator will look for the average between

$$$$ display $$ \ begin {bmatrix} \ color {# ff2c2c} {1} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0} \\ 1 \\ \ end {bmatrix} and \ begin {bmatrix} \ color {# ff2c2c} {0} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0} \\ 0 \\ \ end {bmatrix}, \; returning \ begin {bmatrix} \ color {# ff2c2c} {0,5} \\ \ color {# 00c300} 0 \\ \ color {# 2f9fff} {0} \\ 0.5 \\ \ end {bmatrix} = \ color {# 7f0000} █ \; with \; mixing, \; but \; then \; \ begin {bmatrix} \ color {# ff2c2c} {1} \\ \ color {# 00c300} 0.5 \\ \ color {# 2f9fff} {0.5} \\ \ end {bmatrix} = \ color {# ff7f7f} █ $$ display $$

This correctly and fairly well solves all our problems! The end result was exactly the same as what we expected with “traditional blending”, except that we got rid of all the artifacts. You will notice that when working with premultiplied alpha, a fully transparent pixel is always RGB black, so we don’t need to worry about what is actually contained in the transparent areas of the sprite. Premultiplied alpha also avoids a headache when generating chains of MIP-textures and layering several translucent sprites one above the other.

Summarize

So, back to the original topic: was the problem with the Limbo logo really caused by “garbage” in RGB transparent pixels?

There is only one way to find out, so I extracted the PNG file from the demo package /PS3_GAME/ICON0.PNG .

At first glance, the image looks great, but let's remove the alpha channel to render the full RGB values:


Original


RGB only

So it is: instead of a solid white color, the letters “B” and “O” contain incorrect RGB values ​​that “leak” and cause a graphical bug we have seen before.

The problem with these artifacts is that they are difficult to detect. I did not notice anything strange with the Limbo logo until it was rendered on a white background. Not everyone knows about this issue, so familiarization with the topic will be useful.

If you are an artist, then you are the first line of defense, pay attention to the colors that are inside the transparent pixels. If you are a programmer, consider using premultiplied alpha.