How ZFS Works - Part 1: vdev
Vdev, or Virtual Device is the basic unit on which the ZFS dataset (zpool) is built. ZFS requires at least one vdev - a virtual device that allows random access to information at the block level.
Usually, whole disks or iScsi / FC LUNs (raw-disk vdev) are used as such blocks, but you can also use disk partitions or files. Entire disks are preferred because ZFS uses onboard write cache for them, as a result of which write performance can be greatly improved. I will not explain the difference between logical vdev and leaf vdev yet - this is a topic for a separate article about how RaidzN and zfs mirror work.
In the description of the structure of the virtual device, you can see what parameters are used for what, but I'm not going to give an explanation to everyone - the purpose of the article is the description of how ZFS stores and writes data.
Each vdev is divided into logical partitions (metaslab), that are created automatically when you add a disk in ZFS Pool. The number of such partitions depends on the size of vdev — about 200 partitions, but it can actually be <128 and> 256.
Further, inside each metaslab there is a so-called space map. It, like most structures in ZFS, is made on the basis of the AVL tree , and allows you to work effectively with blocks of occupied and free space, regardless of the block size and its place inside the metaslab.
')
Here is a small example of how ZFS sees this structure using
Usually, whole disks or iScsi / FC LUNs (raw-disk vdev) are used as such blocks, but you can also use disk partitions or files. Entire disks are preferred because ZFS uses onboard write cache for them, as a result of which write performance can be greatly improved. I will not explain the difference between logical vdev and leaf vdev yet - this is a topic for a separate article about how RaidzN and zfs mirror work.
src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/vdev_disk.c#314if (vd->vdev_wholedisk == 1) { int wce = 1; /* * If we own the whole disk, try to enable disk write caching. * We ignore errors because it's OK if we can't do it. */ (void) ldi_ioctl(dvd->vd_lh, DKIOCSETWCE, (intptr_t)&wce, FKIOCTL, kcred, NULL); } In the description of the structure of the virtual device, you can see what parameters are used for what, but I'm not going to give an explanation to everyone - the purpose of the article is the description of how ZFS stores and writes data.
Each vdev is divided into logical partitions (metaslab), that are created automatically when you add a disk in ZFS Pool. The number of such partitions depends on the size of vdev — about 200 partitions, but it can actually be <128 and> 256.
src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/vdev.c#1552 void vdev_metaslab_set_size(vdev_t *vd) { /* * Aim for roughly 200 metaslabs per vdev. */ vd->vdev_ms_shift = highbit(vd->vdev_asize / 200); vd->vdev_ms_shift = MAX(vd->vdev_ms_shift, SPA_MAXBLOCKSHIFT); } Further, inside each metaslab there is a so-called space map. It, like most structures in ZFS, is made on the basis of the AVL tree , and allows you to work effectively with blocks of occupied and free space, regardless of the block size and its place inside the metaslab.
')
Here is a small example of how ZFS sees this structure using
zdb -mmm - 20 metaslab'a vdev'a :
Metaslabs: vdev 0 metaslabs 323 offset spacemap free --------------- ------------------- --------------- ------------- metaslab 0 offset 0 spacemap 34 free 84.8M segments 72 maxsize 24.9M freepct 16% .... [ 73] A range: 001f00ba00-001ffffa00 size: ff4000 [ 74] A range: 001eff8200-001f00ae00 size: 012c00 [ 75] A range: 001b091200-001b130200 size: 09f000 [ 76] A range: 0012a84e00-0013a84e00 size: 1000000 [ 77] A range: 0013a84e00-00141bb800 size: 736a00 [ 78] A range: 0006b88000-0007707200 size: b7f200 [ 79] FREE: txg 9153385, pass 1 [ 80] F range: 0000282000-0000283200 size: 001200 [ 81] F range: 0000b47200-0000b48600 size: 001400 [ 82] ALLOC: txg 9252698, pass 1 [ 83] A range: 0000282000-0000283a00 size: 001a00 [ 84] A range: 0000b47200-0000b48e00 size: 001c00 [ 85] A range: 00008e6e00-00008e8a00 size: 001c00 [ 86] FREE: txg 9279297, pass 2 [ 87] F range: 00008e9000-00008e9c00 size: 000c00 [ 88] FREE: txg 9328122, pass 2 [ 89] F range: 0010a7be00-0010a7c800 size: 000a00 [ 90] FREE: txg 9444756, pass 1 [ 91] F range: 0009b14200-000ab14200 size: 1000000 [ 92] F range: 000ab14200-000b402000 size: 8ede00 ....
, , , . , ZFS - - Copy-on-write . ZFS - , .
space map - 512 (0200), - 16 (01000000). space map , , - , .
- metaslab "" , free map, , , disk seek. , 84 "free" , - 24.9 "maxfree".
ZFS write pipeline - , , () metaslab allocator.
, - , , . .zdb -mmm - 20 metaslab'a vdev'a :
Metaslabs: vdev 0 metaslabs 323 offset spacemap free --------------- ------------------- --------------- ------------- metaslab 0 offset 0 spacemap 34 free 84.8M segments 72 maxsize 24.9M freepct 16% .... [ 73] A range: 001f00ba00-001ffffa00 size: ff4000 [ 74] A range: 001eff8200-001f00ae00 size: 012c00 [ 75] A range: 001b091200-001b130200 size: 09f000 [ 76] A range: 0012a84e00-0013a84e00 size: 1000000 [ 77] A range: 0013a84e00-00141bb800 size: 736a00 [ 78] A range: 0006b88000-0007707200 size: b7f200 [ 79] FREE: txg 9153385, pass 1 [ 80] F range: 0000282000-0000283200 size: 001200 [ 81] F range: 0000b47200-0000b48600 size: 001400 [ 82] ALLOC: txg 9252698, pass 1 [ 83] A range: 0000282000-0000283a00 size: 001a00 [ 84] A range: 0000b47200-0000b48e00 size: 001c00 [ 85] A range: 00008e6e00-00008e8a00 size: 001c00 [ 86] FREE: txg 9279297, pass 2 [ 87] F range: 00008e9000-00008e9c00 size: 000c00 [ 88] FREE: txg 9328122, pass 2 [ 89] F range: 0010a7be00-0010a7c800 size: 000a00 [ 90] FREE: txg 9444756, pass 1 [ 91] F range: 0009b14200-000ab14200 size: 1000000 [ 92] F range: 000ab14200-000b402000 size: 8ede00 ....
, , , . , ZFS - - Copy-on-write . ZFS - , .
space map - 512 (0200), - 16 (01000000). space map , , - , .
- metaslab "" , free map, , , disk seek. , 84 "free" , - 24.9 "maxfree".
ZFS write pipeline - , , () metaslab allocator.
, - , , . .Source: https://habr.com/ru/post/160943/
All Articles