Dd command and everything related to it

On UNIX systems, there is one very ancient command called dd. It is designed to copy something somewhere byte-by-byte. At first glance, nothing outstanding, but if you consider all the features of this universal tool, you can perform quite complex operations without involving additional software, for example: perform a backup of the MBR, create data dumps from various drives, mirror media, restore data from a backup on media and more, and, when combining the capabilities of dd and supporting the cryptographic algorithms of the Linux kernel, you can even create encrypted files that contain whole files yy system.
Again, in the note I will describe the most frequently used examples of using the command, which greatly facilitate the work on UNIX systems.

I will start with a small example, clearly illustrating the main parameters of the command:

# dd if = / dev / urandom of = / dev / null bs = 100M count = 5

Options:

• if: indicates the source, i.e. to where we copy from. Specifies a file that can be either a regular file or a device file.
• of: indicates the destination file. The same thing, we can write both in a regular file and directly in the device.
• bs: the number of bytes to be written at a time. You can think of this argument as the size of the piece of data that will be written or read, and the number of pieces is regulated by the next parameter.
• count: just the number that indicates: how many pieces will be copied.

Thus, the described command reads 5 * 100 megabytes from the / dev / urandom device to the / dev / null device. Giving this command a sense load, it turns out that the system will generate 500 megabytes of random values ​​and write them into a null device. Of course, the only thing this command will do is load the processor for a few seconds. Consider examples from practice:
')
Creating a disk image:

# dd if = / dev / cdrom of = image.iso

The command will read data from the device and write to the file until it reaches the end of the device. If the disk is broken, you can try to read it, ignoring read errors:

# dd if = / dev / cdrom of = image.iso conv = noerror

The “conv” parameter allows you to connect several filters applicable to the data stream. The “noerror” filter just turns off the program shutdown when it encounters a read error. Thus, some data from the disk can still be read. In exactly the same way, I saved the data from my Corsair flash drive, which was bent: I found the right position when there was a contact, and made a file system dump.
By the way, such images can be connected using the mount command with the "-o loop" key:

# mount -o loop image.iso / mnt / image

If something does not work, the process is divided into 2 levels:

# losetup -e / dev / loop0 image.iso
# mount / dev / loop0 / mnt / image

If this does not work, then the image file system has flown.

Work with media

A very simple, though not optimal solution for cloning a hard disk:

# dd if = / dev / sda of = / dev / sdb bs = 4096

All the same byte copying with a buffer size of 4 KB. The disadvantage of this method is that when any sections are full, all bits will be copied, which is not beneficial when copying sections with a small fullness. To reduce copying time when manipulating large amounts of data, you can simply transfer the MBR to a new medium (I will describe how), re-read the kernel partition table (using the same fdisk), create file systems, and simply copy the files (remembering to save access rights to files).

Alternatively, you can even make a backup of the partition on the network. Destroying ssh keys will work such a scheme:

# dd if = / dev / DEVICE | ssh user @ host "dd of = / home / user / DEVICE.img".

I once read a study, according to which a very large proportion of hard drives in the flea market undergo data recovery without the involvement of something specialized, and contains confidential information. So that nothing could be restored on the carrier - you can score it with zeros:

# dd if = / dev / zero of = / dev / DEVICE

I think it is clear what you need to replace DEVICE. After giving lectures on Linux, I began to follow very carefully what I was writing.
You can check the same dd, but converting the data to hex:

# dd if = / dev / sda | hexdump -C

Must be sprinkled with zeros.

Operations with MBR

MBR is located in the first 512 bytes of the hard disk, and consists of a partition table, bootloader and a pair of additional. byte. Sometimes, it has to be backed up, restored, etc. Backup runs like this:

# dd if = / dev / sda of = mbr.img bs = 512 count = 1

You can restore easier:

# dd if = mbr.img of = / dev / sda

The reasons for these frauds with the MBR may be different, but I want to tell one feature taken from experience: after restoring a long-standing copy of the MBR, where one of the partitions was ext3, and later became FAT and was used by Windows, the partition stopped seeing Windows. The reason is the partition ID, which is stored in the MBR. If UNIX mounts file systems according to the superblock, then Windows is guided by partition IDs from the MBR. Therefore, you should always check partition IDs with fdisk, especially if there are Windows on the computer.

File generation

With dd, you can generate files and then use them as containers of other file systems, even in encrypted form. The technology is as follows:
With dd, a file is created that is clogged with zeros (it is not rational to score with random numbers: long and pointless):

# dd if = / dev / zero of = image.crypted bs = 1M count = 1000

Created a file size of almost gigabytes. Now you need to make this file a block device and, at the same time, pass it through the linux kernel encryption mechanism. I will choose the blowfish algorithm. Module loading:

# modprobe cryptoloop
# modprobe blowfish

Associate an image with a block device with encryption enabled:

# losetup -e blowfish / dev / loop0 image.crypted

The team will ask you to enter a password, which will be the key to the image. If the key is entered incorrectly, the system will not be mounted. You can recreate the data in the image using the new key, but you will not have access to the old data.
Create a file system and mount:

# mkfs.ext2 / dev / loop0
# mount / dev / loop0 / mnt / image

The image is ready to write data. After you finish working with it, you need to remember to unmount it and disconnect it from the block loop device:

# umount / dev / loop0
# losetup -d / dev / loop0

Now the encrypted image is ready.

I wrote out the main ideas, but the set of tasks that can be solved with the help of a small program whose name consists of two letters is much wider. The “dd” program is a vivid example of what IT's called the “UNIX way”: one program is part of the mechanism, performs only its task, and performs it well. In the hands of a person who knows his business, who has a non-standard approach to solving a problem, such small programs will help to quickly and effectively solve complex tasks that, at first glance, should be solved by large specialized packages.