Tamed floating point numbers. Is it possible to debug shaders for mobile devices on a PC?
In the gaming industry, work on the visual quality of effects is being done at the intersection of art and technology. This is especially true of mobile applications.
In this article we will talk about the use of floating point numbers when debugging shaders for mobile devices on a PC. We are sure that the experience of the Plarium studio in Krasnodar will be useful for you regardless of which engine you use.
How we identified the problem
')
In the process of creating effects, we are constantly faced with the fact that some of them eventually scatter into pixels.
We investigated this problem and identified the causes of its occurrence: it turned out, not all gadgets are subject to artifacts. This mainly applies to old mobile devices or models with weak characteristics. Moreover, some seemingly identical effects then worked as they should, then they lost their smooth movements and acquired pixelation.
In the analysis of the technical documentation of video processors at risk, we found out that not all gadgets have full support for 32-bit floating point numbers. This is what causes problems with artifacts and images.
And what about the binary system?
Recall how floating point numbers look in binary.
You can imagine such a number as a container for storing a fixed number of significant digits and a comma anywhere in this number. Obviously, larger numbers have less accuracy.
For example, we have a significant number - 123456789. If we put a comma after the first character (1.23456789), then we can change this number to 0.00000001. But if you put a comma before the last sign (12345678.9), then the accuracy will be 0.1.
If we use 32-bit numbers, then any such number accurately conveys 7 characters. This allows the application to work about 70 hours without visible problems. We are talking about the time when the user sees exactly the game that was started, and not on a pause or in a collapsed state. Enough stock, isn't it?
But the fact is that many mobile devices support only 16-bit floating point numbers in a pixel (fragmentary) shader. There are only 4 exact signs in such numbers, which leads to a noticeable loss of accuracy during texture animation after 10–15 minutes.
In search of a solution:
Solving the problem on each specific device is unrealistic, so we focused on visualizing the error on the PC when creating the effect and taking steps to fix it in advance.
On a PC, it is not possible to directly emulate the operation mode with numbers of 16 bits. The video card driver automatically converts all 16-bit instructions to 32 bits, and you can only visually track the loss of accuracy after a long time. Therefore, to visualize artifacts from loss of accuracy, we wrote our own time manager and extensions for shaders.
Time manager
Our “debugging” time starts several hours ahead. Thus, the artist can immediately see the problems with the effect. Also, the manager automatically resets the time each time the application is paused or minimized. It is at this point that the leap of effects is least noticeable to the player.
With the introduction of the system of frame rendering (Scriptable Rendering Pipeline) in the new version of Unity, we were able to abandon a separate time management script and do everything directly when forming a frame.
Shader Extensions
The idea was to overload the 32-bit floating point number in the same way that the 16-bit one is overloaded. It was necessary to shift the significant part to a certain number of bits. But bit operations are not available for Shader Model 2.0, and in addition, using a higher shader model results in the engine no longer warning artists of the effects of overly complex calculations for mobile devices and the effect budget is exceeded.
We decided to make the most accurate simulation algorithm that will allow us to obtain the desired result.
Thus, slightly changing the shader, we get a visually identical image for weak devices. This allows our artists to control the process of developing effects and reduces the number of errors in the work.
Hint: frac to help
When working on effects, try to ensure that all the numbers that are responsible for the smoothness of the effect and that require accuracy are in the range from 0 to 1. Use the whole-part clipping operation - frac (x).
Essentially, frac is the casting of a floating-point number to an integer followed by subtracting that integer from the original. That is, frac = x - floor (x).
Since this operation uses the original number in its original form, a loss of accuracy can occur during the transfer of this number inside the shader. Therefore, we recommend using a time that does not increase to large values. A good practice would be to force a zero time at the moments of loading or when the game interface overlaps the game level.
The work done has significantly reduced the number of visual errors. Our artists were able to control the process of creating effects without long and complex testing on mobile devices.
In this article we will talk about the use of floating point numbers when debugging shaders for mobile devices on a PC. We are sure that the experience of the Plarium studio in Krasnodar will be useful for you regardless of which engine you use.
How we identified the problem
')
In the process of creating effects, we are constantly faced with the fact that some of them eventually scatter into pixels.
We investigated this problem and identified the causes of its occurrence: it turned out, not all gadgets are subject to artifacts. This mainly applies to old mobile devices or models with weak characteristics. Moreover, some seemingly identical effects then worked as they should, then they lost their smooth movements and acquired pixelation.
In the analysis of the technical documentation of video processors at risk, we found out that not all gadgets have full support for 32-bit floating point numbers. This is what causes problems with artifacts and images.
And what about the binary system?
Recall how floating point numbers look in binary.
You can imagine such a number as a container for storing a fixed number of significant digits and a comma anywhere in this number. Obviously, larger numbers have less accuracy.
For example, we have a significant number - 123456789. If we put a comma after the first character (1.23456789), then we can change this number to 0.00000001. But if you put a comma before the last sign (12345678.9), then the accuracy will be 0.1.
If we use 32-bit numbers, then any such number accurately conveys 7 characters. This allows the application to work about 70 hours without visible problems. We are talking about the time when the user sees exactly the game that was started, and not on a pause or in a collapsed state. Enough stock, isn't it?
But the fact is that many mobile devices support only 16-bit floating point numbers in a pixel (fragmentary) shader. There are only 4 exact signs in such numbers, which leads to a noticeable loss of accuracy during texture animation after 10–15 minutes.
In search of a solution:
- We divided the causes of loss of accuracy into three groups:
- When transmitted, the value is converted from 32 bits to 16 bits.
- When calculating - attempt operations with numbers of different order.
- At overflow - any floating point number has its maximum and minimum value.
Solving the problem on each specific device is unrealistic, so we focused on visualizing the error on the PC when creating the effect and taking steps to fix it in advance.
On a PC, it is not possible to directly emulate the operation mode with numbers of 16 bits. The video card driver automatically converts all 16-bit instructions to 32 bits, and you can only visually track the loss of accuracy after a long time. Therefore, to visualize artifacts from loss of accuracy, we wrote our own time manager and extensions for shaders.
Time manager
Our “debugging” time starts several hours ahead. Thus, the artist can immediately see the problems with the effect. Also, the manager automatically resets the time each time the application is paused or minimized. It is at this point that the leap of effects is least noticeable to the player.
With the introduction of the system of frame rendering (Scriptable Rendering Pipeline) in the new version of Unity, we were able to abandon a separate time management script and do everything directly when forming a frame.
Shader Extensions
The idea was to overload the 32-bit floating point number in the same way that the 16-bit one is overloaded. It was necessary to shift the significant part to a certain number of bits. But bit operations are not available for Shader Model 2.0, and in addition, using a higher shader model results in the engine no longer warning artists of the effects of overly complex calculations for mobile devices and the effect budget is exceeded.
We decided to make the most accurate simulation algorithm that will allow us to obtain the desired result.
- Add some big number to the original one. This large number itself is determined by the number of digits of the main number.
- Subtract it. It is necessary to deceive the compiler, which will automatically reduce operations. To do this, we introduce a slight error in the number after the increase.
Thus, slightly changing the shader, we get a visually identical image for weak devices. This allows our artists to control the process of developing effects and reduces the number of errors in the work.
Hint: frac to help
When working on effects, try to ensure that all the numbers that are responsible for the smoothness of the effect and that require accuracy are in the range from 0 to 1. Use the whole-part clipping operation - frac (x).
Essentially, frac is the casting of a floating-point number to an integer followed by subtracting that integer from the original. That is, frac = x - floor (x).
Since this operation uses the original number in its original form, a loss of accuracy can occur during the transfer of this number inside the shader. Therefore, we recommend using a time that does not increase to large values. A good practice would be to force a zero time at the moments of loading or when the game interface overlaps the game level.
The work done has significantly reduced the number of visual errors. Our artists were able to control the process of creating effects without long and complex testing on mobile devices.
Source: https://habr.com/ru/post/352906/
All Articles