UI performance comparison in WPF, Qt, WinForms and FLTK
By a UI performance measure, we mean the number of responses to user actions per unit of time. And under the response - the requested user response of the application.
A short response time can explain a number of user preferences:
1. Preference for analog interfaces to digital (when there is a delay in processing digital input);
2. At the dawn of Windows, user preferences to work with DOS programs in “text mode”, and not with GUI counterparts in Windows (the response time in text mode was then noticeably shorter on a similar platform);
3. Preference of real game consoles to their emulators (emulators often have a response time different from the response time of the original consoles);
4. Preference of iOS and Android users regarding WinCE and Symbian (among other things, for example, iOS had the goal of fast response and support of 60 FPS, while Android did not set such goals noticeably more responsive than WinCE and Symbian);
5. In cars - an ambiguous attitude of users to automatic transmissions, an electronic gas pedal and some other systems that introduce a delay between the control action and the response to it (this applies to the least advanced versions of these solutions).
')
A great response time is essentially a “feedback with a delay”, about which you can read in more detail here: “Feedback with a delay in a hot water faucet, rover and demographic pyramid” .
Delayed response to user input in digital systems is natural and inevitable. However, in the UI for perception only those delays that the user is able to sense are of importance. For example, in the perception of visual information, the average user is not able to distinguish delays of less than 1/60 of a second, so a further decrease in the visual response time is hardly justified.
Consider the responsiveness of the user interface of the WPF , Qt , WinForms, and FLTK libraries. It is difficult to measure the responsiveness of all controls of these libraries, and it is difficult in each case to measure the interval between user input and the response of the control to this input. Therefore, we slightly simplify the assessment task. We will test one, but complex, and generally indicative control, which is present in all libraries - DataGrid . Responsiveness will be measured by FPS in response to scrolling the contents of this control. In order to avoid counting incomplete frames, we will use double buffering.
While scrolling the grid, one way or another we will check the main library mechanisms responsible for the response to input, rendering of controls, text, processing of the visual tree of controls, and partly the composition. This will give us the opportunity to assess the potential of the library to work with the loaded UI.
For each library, I prepared a test application that fills a grid with data of the same type. This data is 5,000 rows with 20 columns with some arbitrary information (I consider such a number of rows and columns as close to the real maximum needs of displaying complex objects). I have not optimized the filling of the grid, so you should not attach importance to the fact that for some libraries it is filled slowly. Performance will be measured after the grid is filled.
I checked performance only under Windows (although the libraries, except for WPF cross-platform), for checking, an FPS counter (for Windows and 32-bit color depth) was written to determine frame changes along the top of the main screen. The error of the counter can be about 1 frame.
Performance measurement technique:
1. Run the FPSCounter.exe counter.
2. Run one of the test applications FltkGrid.exe, FormsGrid.exe, QtGrid.exe or WpfDatagridTest.exe and expand it to the whole main screen (this is necessary because only the change of the upper part of the frame on the main screen is detected)
3. Move the vertical grid scroller up and down until it stops, while moving the slider, look at the FPS value in the upper left corner of the screen or in the counter window. (to get maximum FPS values, the slider must be moved quickly, otherwise we will rest on our own “performance”, and not on the performance of UI)
I made FPS measurements on several platforms that were at hand, almost all platforms on Windows 7 x64, closed other applications for the duration of the measurements.
Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid.exe, QtGrid.exe and WpfDatagridTest.exe, as well as Qt libraries needed for a test application on Qt (16 Mb)
Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid.exe (i.e., without Qt, but very small)
Applications compiled under x64 platform. In addition, the run may require msvs runtime 2010
Below is a table with a list of platforms on which measurements were launched and their results. The table also indicates the assessment of single thread performance of the platform (Single Thread Performance) in accordance with www.cpubenchmark.net .
In addition to the table, I want to add that the Office applications and, for example, the Chrome browser (on “good” pages) show FPS approximately equal to FLTK or slightly less.
For a more visual illustration, add a graph of FPS versus Single Thread performance scores. The graph is based on the data given in the table.
Schedule worth a little comment. On a platform with a rating of 1000, a low vertical resolution of the screen was used, which greatly reduced the number of visible cells, thereby significantly increasing the vertical scrolling FPS. I also want to add that a number of optimizations were disabled for compiling FLTK examples, so if you turn them on, you can slightly increase the FLTK sample estimates on all platforms (for the same reason, the console window appears during the start of the FLTK example).
In general, the dependence of FPS on single-threaded processor performance is fairly linear. There is an assumption that multi-threaded performance is still little applicable to UI libraries (for example, the superiority of i7 in multi-threaded performance has little effect on FPS). The same measurements showed a weak dependence of FPS in our test on the video card (there is certainly a dependence, but the video card does not seem to be a bottleneck in these tests) Another interesting detail was the limitation of 30 FPS on a number of platforms. Not sure if this is related to the video card driver or some of its settings, but in some cases it was not possible to get more than 30 FPS ...
Based solely on measurement data, I cannot recommend FLTK as a universal means of increasing the responsiveness of your UI, especially since writing UI on FLTK is very laborious (it took me about 20 minutes to write an FLTK example, although other examples were written much faster). However, the measurement results make us think that many libraries do not fully reveal the potential of iron on which they are performed, making applications less attractive for the user (in terms of responsiveness at least).
I would be happy if someone similarly tests other UI libraries, offers a way to optimize the performance of test examples (for the scrolling case in question), or just talk about FPS in test examples on its configuration.
I enclose the sources of the counter and tests .
As can be seen from the test, stable 60 FPS, in the case of a heavy UI, we can only get on expensive hardware and the most expensive UI library in development (that is, development of both iron and consumption of iron) will probably be worth it, but for now it is rather an exception. However, if you do not go to extremes and set yourself a goal to get at least 20 FPS in the interfaces loaded with information. What will it cost us?
For the considered libraries, there are probably not so many options:
1. FLTK + is almost the cheapest iron. We will spend much more time on developing the UI, but at current prices we can save ~ $ 100– $ 200 on hardware for the user's workplace.
2. Qt + average iron. On iron, saving is not particularly possible, but the development of UI will be cheaper than in the case of FLTK. Probably in some cases the option will be optimal.
3. WPF + expensive iron i. additional 200-300 $ per workplace. After all, if on i7-3770 we get only 12 frames, then we need at least one and a half times more powerful hardware. Probably i7-5930K or possibly i7-4790K paired with a good video card will cope with the task of 20 FPS. However, it is unlikely to be an effective solution, and whether they can cope ... unfortunately there is no such iron on hand to check, but if you extrapolate single-threaded performance, then its estimate must be over 3000, to get 20 FPS at 1280x1024 ... such iron simply does not exist, at least here ...
4. Facilitate the UI, until we meet the 20 FPS. For example, if using WPF, instead of 20 columns in the grid, leave only 6, then on i7-3770 we will get stable 20 FPS, if we leave only 3-4 columns, then we will get 20 FPS on the budget hardware. Reducing the size of the grid itself should also have a positive effect (although it is different in different libraries, and oddly enough for the WPF case, the effect is least pronounced). Are such solutions acceptable? Not applicable everywhere, but still cover a number of tasks that are not focused on the presentation of data.
PS: The idea to compare the performance of grids came after I was faced with the low performance of the WPF grid. In the comments to my previous article, “The Choice Between C ++ and C #,” I, in particular, dealt with this problem.
Then I wondered how alternative libraries cope with the task of mapping a grid, so test applications appeared and the results presented in this article.
A short response time can explain a number of user preferences:
1. Preference for analog interfaces to digital (when there is a delay in processing digital input);
2. At the dawn of Windows, user preferences to work with DOS programs in “text mode”, and not with GUI counterparts in Windows (the response time in text mode was then noticeably shorter on a similar platform);
3. Preference of real game consoles to their emulators (emulators often have a response time different from the response time of the original consoles);
4. Preference of iOS and Android users regarding WinCE and Symbian (among other things, for example, iOS had the goal of fast response and support of 60 FPS, while Android did not set such goals noticeably more responsive than WinCE and Symbian);
5. In cars - an ambiguous attitude of users to automatic transmissions, an electronic gas pedal and some other systems that introduce a delay between the control action and the response to it (this applies to the least advanced versions of these solutions).
')
A great response time is essentially a “feedback with a delay”, about which you can read in more detail here: “Feedback with a delay in a hot water faucet, rover and demographic pyramid” .
Performance comparison
Delayed response to user input in digital systems is natural and inevitable. However, in the UI for perception only those delays that the user is able to sense are of importance. For example, in the perception of visual information, the average user is not able to distinguish delays of less than 1/60 of a second, so a further decrease in the visual response time is hardly justified.
Consider the responsiveness of the user interface of the WPF , Qt , WinForms, and FLTK libraries. It is difficult to measure the responsiveness of all controls of these libraries, and it is difficult in each case to measure the interval between user input and the response of the control to this input. Therefore, we slightly simplify the assessment task. We will test one, but complex, and generally indicative control, which is present in all libraries - DataGrid . Responsiveness will be measured by FPS in response to scrolling the contents of this control. In order to avoid counting incomplete frames, we will use double buffering.
While scrolling the grid, one way or another we will check the main library mechanisms responsible for the response to input, rendering of controls, text, processing of the visual tree of controls, and partly the composition. This will give us the opportunity to assess the potential of the library to work with the loaded UI.
For each library, I prepared a test application that fills a grid with data of the same type. This data is 5,000 rows with 20 columns with some arbitrary information (I consider such a number of rows and columns as close to the real maximum needs of displaying complex objects). I have not optimized the filling of the grid, so you should not attach importance to the fact that for some libraries it is filled slowly. Performance will be measured after the grid is filled.
I checked performance only under Windows (although the libraries, except for WPF cross-platform), for checking, an FPS counter (for Windows and 32-bit color depth) was written to determine frame changes along the top of the main screen. The error of the counter can be about 1 frame.
Performance measurement technique:
1. Run the FPSCounter.exe counter.
2. Run one of the test applications FltkGrid.exe, FormsGrid.exe, QtGrid.exe or WpfDatagridTest.exe and expand it to the whole main screen (this is necessary because only the change of the upper part of the frame on the main screen is detected)
3. Move the vertical grid scroller up and down until it stops, while moving the slider, look at the FPS value in the upper left corner of the screen or in the counter window. (to get maximum FPS values, the slider must be moved quickly, otherwise we will rest on our own “performance”, and not on the performance of UI)
I made FPS measurements on several platforms that were at hand, almost all platforms on Windows 7 x64, closed other applications for the duration of the measurements.
Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid.exe, QtGrid.exe and WpfDatagridTest.exe, as well as Qt libraries needed for a test application on Qt (16 Mb)
Archive with FPSCounter.exe, FltkGrid.exe, FormsGrid.exe (i.e., without Qt, but very small)
Applications compiled under x64 platform. In addition, the run may require msvs runtime 2010
results
Below is a table with a list of platforms on which measurements were launched and their results. The table also indicates the assessment of single thread performance of the platform (Single Thread Performance) in accordance with www.cpubenchmark.net .
In addition to the table, I want to add that the Office applications and, for example, the Chrome browser (on “good” pages) show FPS approximately equal to FLTK or slightly less.
For a more visual illustration, add a graph of FPS versus Single Thread performance scores. The graph is based on the data given in the table.
Schedule worth a little comment. On a platform with a rating of 1000, a low vertical resolution of the screen was used, which greatly reduced the number of visible cells, thereby significantly increasing the vertical scrolling FPS. I also want to add that a number of optimizations were disabled for compiling FLTK examples, so if you turn them on, you can slightly increase the FLTK sample estimates on all platforms (for the same reason, the console window appears during the start of the FLTK example).
In general, the dependence of FPS on single-threaded processor performance is fairly linear. There is an assumption that multi-threaded performance is still little applicable to UI libraries (for example, the superiority of i7 in multi-threaded performance has little effect on FPS). The same measurements showed a weak dependence of FPS in our test on the video card (there is certainly a dependence, but the video card does not seem to be a bottleneck in these tests) Another interesting detail was the limitation of 30 FPS on a number of platforms. Not sure if this is related to the video card driver or some of its settings, but in some cases it was not possible to get more than 30 FPS ...
Based solely on measurement data, I cannot recommend FLTK as a universal means of increasing the responsiveness of your UI, especially since writing UI on FLTK is very laborious (it took me about 20 minutes to write an FLTK example, although other examples were written much faster). However, the measurement results make us think that many libraries do not fully reveal the potential of iron on which they are performed, making applications less attractive for the user (in terms of responsiveness at least).
I would be happy if someone similarly tests other UI libraries, offers a way to optimize the performance of test examples (for the scrolling case in question), or just talk about FPS in test examples on its configuration.
I enclose the sources of the counter and tests .
How to get high FPS?
As can be seen from the test, stable 60 FPS, in the case of a heavy UI, we can only get on expensive hardware and the most expensive UI library in development (that is, development of both iron and consumption of iron) will probably be worth it, but for now it is rather an exception. However, if you do not go to extremes and set yourself a goal to get at least 20 FPS in the interfaces loaded with information. What will it cost us?
For the considered libraries, there are probably not so many options:
1. FLTK + is almost the cheapest iron. We will spend much more time on developing the UI, but at current prices we can save ~ $ 100– $ 200 on hardware for the user's workplace.
2. Qt + average iron. On iron, saving is not particularly possible, but the development of UI will be cheaper than in the case of FLTK. Probably in some cases the option will be optimal.
3. WPF + expensive iron i. additional 200-300 $ per workplace. After all, if on i7-3770 we get only 12 frames, then we need at least one and a half times more powerful hardware. Probably i7-5930K or possibly i7-4790K paired with a good video card will cope with the task of 20 FPS. However, it is unlikely to be an effective solution, and whether they can cope ... unfortunately there is no such iron on hand to check, but if you extrapolate single-threaded performance, then its estimate must be over 3000, to get 20 FPS at 1280x1024 ... such iron simply does not exist, at least here ...
4. Facilitate the UI, until we meet the 20 FPS. For example, if using WPF, instead of 20 columns in the grid, leave only 6, then on i7-3770 we will get stable 20 FPS, if we leave only 3-4 columns, then we will get 20 FPS on the budget hardware. Reducing the size of the grid itself should also have a positive effect (although it is different in different libraries, and oddly enough for the WPF case, the effect is least pronounced). Are such solutions acceptable? Not applicable everywhere, but still cover a number of tasks that are not focused on the presentation of data.
PS: The idea to compare the performance of grids came after I was faced with the low performance of the WPF grid. In the comments to my previous article, “The Choice Between C ++ and C #,” I, in particular, dealt with this problem.
Then I wondered how alternative libraries cope with the task of mapping a grid, so test applications appeared and the results presented in this article.
Source: https://habr.com/ru/post/263897/
All Articles