What is a DBMS in RAM and how it effectively stores data

Salvador Dali, Disintegration of the constancy of memory . 1952-1954. Canvas, oil.

Hello. Some of you may already be familiar with the DBMS for in-memory data, but just in case, click the link for a general description. In short, such DBMS stores data entirely in RAM. What does this mean? Every time you send a request to search or update data, you only access RAM bypassing the hard disk — no operations are performed on it. And this is good, because the RAM runs much faster than any disk. An example of such a DBMS is Memcached .

Wait a second, you say, but how to restore the data after a reboot or breakdown of a machine with such a DBMS? If the machine has a DBMS for storing data only in RAM, you can forget about them: when you turn off the power, the data will disappear.
')
Is it possible to combine the advantages of in-memory data storage with the reliability of time-tested DBMS like MySQL or Postgres ? Of course! Will it affect performance? You'd be surprised, but no!

Meet the DBMS for in-memory data, ensuring their safety: Redis , Aerospike , Tarantool .

You may ask, how do these DBMSs ensure data integrity? The trick is that all data is still stored in RAM, but each operation is also written to the transaction log on disk. Look at the image below:



The first thing that you may take out of this scheme is that the speed of execution of queries does not suffer at all, despite the fact that now the DBMS constantly creates log entries. Performance does not sag at the same time, because all requests still refer directly to the RAM. Great news! :-) And what about the data update? Each update (let's call it a transaction) not only occurs in memory, but is written to disk. On a slow disk. Is this a problem? Let's look at the following image:



Transactions are always recorded at the very end of the log. What is the plus of this approach? The disc runs pretty fast. If we talk about the classic hard disk drive (HDD), it can write data to the end of the file at speeds up to 100 MB / s. Do not believe? Run this test from the command line on Unix / Linux / macOS:

cat /dev/zero >some_file

, some_file. , , : 100 .

, , , 100 /. , - 100 /! (100 /) (100 000 000 /) , , ; .

(SSD), - . (1—10 ) (200—300 /) : - . .

, , , 100 /. ? . , 100 , ! , . , , :

→ https://gist.github.com/danikin/a5ddc6fe0cedc6257853.

:

1. , .
2. - , , .

? -, , ( , ). , , , , , , .

-, , , , , , . , .

— B/B+-, . , - . B/B+-, InnoDB MySQL Postgres.

, , — LSM-. , , , . , , RocksDB, LevelDB Vinyl. :



: , , , , — , , : , .

P. S.

— , — , , . . — . . , , , .

? , . , ( , , ).

, , . , , , . , ? , .

: ? .