Windows Performance Station or how I taught the computer to work effectively

Hello, dear readers! So I got to Habr and want to tell here the story of my project. Many aspects of our own experience and work have to be described in fairly general terms in order not to hurt the commercial secrets of the organizations in question.

The essence of this story is to share the found method of solving the problem of computer performance on Windows and tell about the approach that made this decision completely universal and individual for each user. It took about 2 years of daily work to write the code and select the most effective mechanisms, as well as advice and assistance from a large number of my comrades from different IT areas, for which I thank them so much.

So, this story began in the 2013th year, at that time I worked in a large international organization producing powders and shampoos. I had a position as a sysadmin, but, as is often the case, in fact I was a “programmer”. The essence of the work was to maintain some systems of EDS, writing various macros on VBA, utilities for automating or integrating different simple systems, etc.
')
And just at the same time, I solved one of the urgent problems of the organization, namely, the unsatisfactory performance of various software on new laptops of many employees. In fact, I had to face the problem that many sysadmins of large companies face - on staff computers, besides their favorite MS Office, SAP, 1C, etc., there was at least one favorite by all IT specialists “mcafee”, “punto switcher "," Mail satellite "and that just is not. It is worthwhile to clarify that the company then promoted a liberal attitude towards the computers of its employees, and, in fact, gave rise to BYOD.

Since performance complaints have been received, incl. and to me, I began to look for applications for performance tuning like Process Lasso. My choice then fell on "Bill2`s Process Manager". At that time, I found many alternatives, but they all had a terribly overloaded interface and ate up more resources for my own work than could be freed up with optimization.


Found at that time, "Bill2` s Process Manager "had a lot of advantages over system tools, because was pretty easy to manage and set up. In fact, he allowed to do the same thing that the task manager is able to, namely, set the base priority and the kernel mask for applications, but, unlike the dispatcher, made it possible to save these settings after rebooting and distinguished the applications that are displayed in the windows on the desktop desk or work in the background.

Thanks to this application, I managed to solve one of the most pressing problems, explain to the computer with which applications I am working now, and what is more important for me - EXCEL or background installation of updates.

But, despite the fact that for me this application solved the main problem, it had a lot of flaws:

1. The lack of continuity of the config, it was impossible to simply copy the application from one computer to another with a similar configuration, it was necessary to re-configure. Partially, this problem was solved by packing a customized application into a Portable file. (Maybe somehow it was possible to pull the parameters out of the registry, but at that time I didn’t figure it out).
   
   
2. Lack of control of RAM, namely the compression of RAM. Many tweakers have a so-called “memory defragmentation,” which is essentially nonsense, since defragmenting RAM is completely meaningless in terms of performance. I needed to clean the data that hangs in the RAM and does not fall into SWAP, so that with a heavy load on the RAM, it was already free and the OS did not try to guess what you now need in RAM and what not. By the way, a year later this chip appeared regularly in Windows 10, but there it’s archiving inside the RAM, which, in fact, doesn’t completely solve the problem either.
   
   
3. Lack of priority I / O control on the hard disk (or SSD). Normally, Windows has this function even with whists, but it was actively used only in Win10.
   
   
4. Lack of definition of a “focal” application, since Windows is not everything, if I work with a dozen applications, in one of which something compiles, and in the other I watch YouTube, then I don’t want YouTube lagging if the focus of the window is on it, and not on other windows.
   
   
5. The inability to specify a folder with applications to apply the rule to newly installed programs at once.

In fact, studying the architecture of the Windows operating system, I constantly found many excellent mechanisms that allow you to build performance in the system ideally, provided one “BUT”. BUT, if the programmers who write their software will tell the operating system what their software does, what it is intended for (games, movies, etc.) and with what priority it should apply to the CPU / RAM / HDD.



In life, it turns out that only a small part of software is written taking into account the software architecture of Windows, most manufacturers, incl. very serious and large, they write software without specifying these parameters and the OS uses everything by default.

And this is not to mention the services that completely shitlessly eat off any available resources as soon as the event comes to action (such as downloading and installing updates). Those who want to check the situation on their PC can open the task manager and display the column “basic priority” in the tab with processes.

MMCSS corrects this situation a little, but it is too gentle on the processes, almost without affecting their actual performance.


The most embarrassing thing about multitasking Windows that constantly haunted me was listening to music during demanding operations. Since during the work I listened to music through Chrome and started calculating large tables in Excel, the sound had a peculiarity just at the moment of full CPU load or constantly hoarse at any jump of this load. This situation was almost completely solved by using “Bill2`s Process Manager”, but it had to be manually set to Chrome so that it wouldn’t grunt and it turned out that chrome was always at a higher priority even when the music wasn’t needed, which was no longer buzzing.

Immediately it should be clarified that the computers and then used quite powerful by the standards of laptops. At that time, it was a Core i5 2430m / 8GB Ram / 256GB SSD.

At the same time, the then-emerging BYOD company assumed that employees could work with their working laptops everywhere, incl. from home and, at the same time, use a working PC as a home to watch movies, games, etc.


All these factors ultimately prompted me to write my own application, which will incorporate all the best from existing alternatives and eliminate their shortcomings, while not just an interface for setting up as a task manager, but a fully automatic system for adapting performance to the task. In fact, I was going to create an ideal structure for BYOD on my PC, in which I could work without reducing the performance of Steam, iTunes or the like. and play, watch movies without the influence of corporate software, antiviruses, etc.

To solve this problem, I needed to attract several of my friends, who also worked in IT. The initial idea was to make an application for our own convenience and to expand our own knowledge on the operation of the OS and various applications.


It was a small matter, at first we wrote a simple alternative to “Bill2`s Process Manager”, but without configuration interfaces, so that the application works with a pre-configured config. The main feature was to determine which windows are deployed, which processes work in the background and which application is in focus. Just immediately added the option of assigning priority not only to the process, but to all processes from a specific folder. It was quite easy to write this application, and thanks to the experience we had immediately managed to prevent memory leaks, handlers, crashes, and other bugs. the application had to work in the background for a long time without rebooting. To make the application easy and efficient, we used only standard OS and WinAPI functions. At the very beginning, we did without an interface.


The next step was to write a local learning algorithm, which, according to the input parameters, would determine the purpose of the application. The basis was a mathematical model of a neural network with neurons based on a threshold transfer function, but at that time weights were not taken into account, which we added later. At first, this neural network studied only on the basic parameters of working with the application, its name, rule in the configuration and location on the hard disk (for example, if many applications are constantly used and do not work in the background, they are located in the same root folder of the type “C: / Games / ", Then most likely all subsequent applications launched from this folder will be used actively and you do not need to lower their priority in the background).

It took almost half a year to bring this algorithm to perfection. Subsequently, the number of input parameters for analysis increased and the optimal rules applied to third-party applications were selected.

It is worth clarifying here that, unlike the classic task dispatchers, we decided not to manage priorities, cores, RAM and I / O separately, but combined these parameters into rules that combined in different proportions. For example, for the main application, the rule = FOREGROUND (all kernels are available, priority is above average, I / O priority is high, we do not reset RAM to SWAP).


Later, the window positioning parameters were added to the learning algorithm (whether the application has a displayed window, is minimized or maximized. If it is maximized and focused, then you need to pay more attention to this process than background, etc.).

The definition of services was added because equally-called svchost.exe service processes perform completely different tasks.

The most difficult, perhaps, was to teach the application to distinguish between the same processes for different purposes. For example, many browsers launch a separate process for each individual tab (Chrome, Firefox, Yandex, Opera, etc.), and the sound and video reproduce the host processes, hence from 20 chrome processes, when it is not in focus, you need to enhance those processes that reproduce the sound. The same applies to any other processes that can work in the background.



In order not to prescribe in the config all processes known to the world, eventually the application learned how to determine not only how often the user works with the process, but also the actual purpose of the process and its importance for the work.

Thus, the application has learned to distribute the load across the key nodes of the computer, depending on what the user is currently working with. Moreover, the load is distributed individually for each PC depending on the usage scenario of an application, i.e. whether it is used in the window, whether it plays video / sound / 3d, how much the user accesses this application and in which folder this application is located.

The application turned out to be simple and universal, the installation takes place in a couple of clicks and no settings are required for operation, everything adapts automatically.



All this logic could not exist without tight integration with the system, so for the implementation of all the mechanisms we had to use the maximum of the Windows API functions. This made it possible to reach deep core functions and to maintain maximum compatibility with any PC on an x86-64 processor at the minimum cost of the calculated power.

It was also important to take into account the type of computer the user is working with. For example, when working from a battery, several different algorithms for working with CPU cores are used to reduce the load and save the charge. Power management options have been added so that you can not turn off your laptop or tablet, and, having turned off the screen, continue to listen to music or download / copy / archive files with power consumption during sleep mode.

It was also important to learn how to determine how the processor works with multithreading and how it distributes the load on the cores.

All these algorithms together helped to create a truly universal solution that is not a tweaker or overclocking and does not limit the functionality of the OS and other software. In fact, the application teaches the computer to understand user requests and manage the load on key nodes in the way an experienced admin would do for each specific task.


Over time, other my comrades joined the work on this project and helped to design, site, VK group, etc. questions.

Anticipating various tips from the guru, I want to immediately mention that my total experience in IT at the moment is 8 years. While working on this application, I have encountered many conflicting information on the features of the OS and some PC components, so I would not be surprised that someone might disagree with some of the described approaches, but the final testing showed the effectiveness of these solutions.

Even in the professional field of my colleagues, many do not fully understand the subtleties of the CPU, multithreading, multitasking and multi-core work. Information about some of the system functions of Windows had to be found in the most remote parts of the Internet. The most difficult, perhaps, was to find a working (!) Way of setting high I / O priority for third-party applications.

The described mechanism is not a spherical algorithm in vacuum and successfully works on real machines. In cases where multitasking is actually used - the performance increase can be up to 50%.


The final touch for the work on the application was the signing of the “code signing” certificate for Windows.

Many thanks to all who mastered this text to the end.

PS I apologize for some confusion, I tried to put only the most important info so that the text would not grow much.

We distribute the application according to the Freemium scheme so that any user can use the most basic functions, but we will be very pleased if the Habra community supports us so that our project can develop further.

We still have a lot of ideas to increase the functionality and performance of our application to make it even better and more efficient.

I sincerely hope that our efforts will help ease the work with the computer to many users who do not have the special skills to configure their PC, and the burned specialists will be able to get rid of the constant choice between speed and functionality.

I will be happy to tell in more detail about the various aspects of the application work separately, if this article is of interest.