Lack of RAM in Linux on a working PC: optimization and actions when it hangs

On any operating system there is often not enough RAM. Consider, as well as save on the increase in hardware resources of the machine with Linux, and continue to more or less comfortable to use a computer with Linux in conditions of insufficient memory.

This situation is typical: there is a swap (swap, swap partition), which begins to be used when there is a shortage of RAM, and it is placed on the HDD, that is, a hard disk with a low speed of reading information. In such situations, the operating system starts to slow down, the mouse cursor hangs, it is difficult to switch to the next tty, etc. Why? Because the Linux kernel scheduler cannot execute a request for an action in a running program until it has access to its RAM, it cannot perform the following action either, it creates a queue of read requests from the disk, and the system “hangs” precisely because That queue processing is much slower than the user wants.

If at such a moment you run htop or uptime , the Load Average (LA) will be very high, despite the low load on the processor cores. The combination of a high load average and a low processor load indicate a clogged processor queue.

Often on the Internet, it is advised to change the Linux kernel parameter vm.swappiness . You can find out its current value on your system as follows:
')
sysctl vm.swappiness

The answer will be 60 almost certainly. This means that the Linux kernel starts swapping rarely used pages of RAM when the use of free RAM reaches 100% - 60 % = 40%. Often there are recommendations to put, for example, vm.swappiness = 10, so that the swap does not begin to be used until the OZ load reaches 90%. In fact , you do not need to touch vm.swappiness, you are no smarter than the Linux kernel developers, who did not just set 60 by default . Why?

Imagine that you have only 4 GB of RAM, of which 3 GB is now occupied, vm.swappiness = 10, the hard disk swap (HDD) is 0% busy, and you open a heavy website in the browser, which requires more than available free 1 GB, for example, 2 GB. The operating system begins on an emergency basis to send to the swap at least 0.5 GB (and in fact more), so that you can allocate the required amount of RAM to the browser. This procedure becomes the highest priority, and you will have to sacrifice even the movements of the mouse cursor in order to complete it as quickly as possible. Are you waiting. It takes 5 minutes, and the system breaks down, because it has completed the procedure of 100% loading of the access queue to the slow hard disk on which the RAM (swap) is located. With default vm.swappiness = 60, rarely used memory pages are dumped into a swap in advance , and there is no sudden hang for 5-10 minutes.

zram and swap priorities

I recommend to enable zram - transparent compression of the contents of RAM . This is automated in Ubuntu, just install the package:

sudo apt install zram-config

Hereinafter, for Rosa and Fedora distributions everything is the same, but instead of zram-config -

zram-start .

The systemd zram-config on Ubuntu will be automatically added to startup when the package is installed and started when the system is rebooted. To start manually:

sudo systemctl start zram-config

stops:

sudo systemctl stop zram-config

Remove from autorun:

sudo systemctl disable zram-config

Adding to autorun:

sudo systemctl enable zram-config

When running, zram-config takes a number equal to 50% of the total RAM, then makes one virtual device / dev / zramN, where N starts at 0, for each processor core, and the volume of each / dev / zramN is 50% of the entire operative memory divided by the number of processor cores. This is done to parallelize the compression of the contents of RAM by the processor cores; as far as I know, on modern Linux kernels, a single device / dev / zramN is enough, and it will parallelize itself, but I am completely satisfied with the sparking work of the zram-config, and I prefer not to go into it with my hands.

The swapon -s will list all involved swaps with their priority. The first is the swap whose priority is higher. If you already have a disk swap and zram is enabled, then in the case of the above-described auto-configurator package, the priorities out of the box will be correct. For example, a disk swap will have -1, and all / dev / zramN will be 5. Thus, zram is first used, and only then the disk.

By the way, zram is often used on smartphones, it does not create any noticeable load on the processor with the default compression method lz4.

Also, the priority of the swap can be specified in /etc/fstab . I will show with an example how it was done on my work computer with 6 GB of RAM.

 $ cat /etc/fstab | grep swap # swap  SSD UUID=844fc9fb-509d-4dab-9ea5-a3d5142f76d8 none swap sw,pri=2 0 0 # swap  HDD   UUID=b643c42a-0abd-4f35-8865-7a51be5769e8 none swap sw,pri=1 0 0 

The pri=X mount option sets swap priorities. If you also enable zram, the picture will be like this:

 $ swapon -s Filename Type Size Used Priority /dev/sdb3 partition 1005564 0 2 /dev/sda2 partition 6655996 0 1 /dev/zram0 partition 690764 0 5 /dev/zram1 partition 690764 0 5 /dev/zram2 partition 690764 0 5 /dev/zram3 partition 690764 0 5 

First of all, it will swap into a zram, that is, it will be compressed inside the RAM without using an external device for swap, the second will use a small swap on the SSD. Almost never will be used 6 GB swap on the HDD, but they will be needed if I want to send the computer to sleep in conditions of a large load of RAM. (Actually, my zram is disabled).

On office PCs with 4 GB of RAM (Xubuntu 16.04, 17.10) I always put the zram-config package. Chromium, according to observations, by the eye, is very well compressed in RAM, with the result that zram allows you to make the work much more comfortable without upgrading the iron.

Quickly cut down the program, overloading the RAM. RAM capacity for SSH

It happens that even with vm.swappiness = 60, some feature, as a rule, the browser, requires a lot of RAM, and the system hangs. It is solved very simply: the Alt + SysRq (PrintScreen) + F key combination forces oom_killer to forcibly turn on and cut down the process that at the time of the call takes up the most memory. Strictly 1 process for 1 challenge, and strictly something will be killed. If you press it many times in a row, the graphical session will most likely restart. The kill event is reflected in dmesg red.

However, this thing, called Magic SysRq, is disabled in most distributions out of the box, because an unprivileged user can kill absolutely any process. For this, the kernel.sysrq kernel parameter kernel.sysrq ; you can find out its current value as follows:

sysctl kernel.sysrq .

Alt + SysRq + F requires kernel.sysrq = 1. To do this, we will edit the kernel parameters located in the /etc/sysctl.conf files (usually the symlink on /etc/sysctl.d/99-sysctl.conf) and /etc/sysctl.d/*.conf. It is best to create a separate file:

sudo nano /etc/sysctl.d/99-dumalogiya.conf

We write in it:

 #    Alt+SysRq,  .. Alt+SysRq+F    OOM Killer kernel.sysrq = 1 #  8    ,       ,      SSH  . vm.admin_reserve_kbytes = 60192 

Press Ctrl + O, Enter to save.

In the case of the browser Chromium Alt + SysRq (PrintScreen) + F will cut down one tab, without closing the browser itself, which is very convenient.



Magic SysRq keystrokes are intercepted directly by the Linux kernel, so they work even when the X server hangs because of the processor queue.

vm.admin_reserve_kbytes is the size of RAM in kilobytes, which will be kept guaranteed free for administrative purposes, for example, SSH. The default is about 8 MB. It is advisable to increase the number 60192 almost from the bald.

I uploaded /etc/sysctl.d/99-dumalogiya.conf to the deb-package dumalogiya-sysctl, which I put in my repository and put it on all computers, very conveniently, you can update the config in a centralized way, and you do not need to manually configure each machine. As an instruction for building simple deb-packs on my knees, I recommend https://debian.pro/1390 . The repository was created using aptly, which simply creates the structure of files and folders within the web server directory.