Introduction to Veritas Volume Manager

The static partitioning of disks is a thing of the past; various volume managers and their hybrids with file systems have long since replaced it. On AIX and HP-UX, even quite serious crutches are needed to not use the volume manager. Linux distributions also now do a default breakdown with LVM tools, indicating that this mechanism is sufficiently stable for use in a corporate environment.
But the product of Symantec (which bought Veritas), which is called the Veritas Storage Foundation (in the latest version of the Storage Foundation High Availability), has been present on the market for a long time. It includes quite a lot of disk space management tools: starting from the Multipath setting, ending with its own extremely flexible file system, and is available for all modern server operating systems.

In practice, engineers most often encounter Veritas products, when they come to a new job, log in to a server, Veritas is there, and they shout “WTF ??” to the whole room. I first met him on HP-UX, the acquaintance was very strained, I had to figure it out for a long time.
In this article, I will give some basic understanding of what Veritas Volume Manager (VxVM) is and how it works.

Service advantages over statics, some over LVM:
- Recycling space. Sections can be expanded and reduced without stopping the service, the allocation of space for new services also passes without affecting the service. Supported by Thin Provisioning.
- Dedublication and compression. In a number of tasks helps save space.
- Management paths. The package contains a means to control the paths to the device (multipath), which allows quite flexible configuration of policies for balancing, availability, etc.
- Tiring (tiering). Optimization of data placement depending on their classification
- Data migration. It is useful when you need to transfer disks to another system from any OS. Only one command on the source and one on the receiver, and the group along with the data is available on another server. Of course, the disks should be presented on both servers.

The main disadvantage is the comparative complexity of administration and payment as an additional product (only with HP-UX it comes in the distribution kit, you can choose between LVM and VxVM). If in the same LVM the administrator works with only three components of the manager, then two more are added to Veritas. For basic tasks, these components are not needed, but without understanding how they work, it will be impossible to achieve maximum performance, flexibility, as well as cope with emergency situations.
')
Installation does not cause issues, the installation script does everything you need. After installation, you may need to call “vxinstall” to initialize the services and start them.
As input parameters there are several blank disks, from which, ultimately, you need to get file systems. In Veritas terminology, this is “Physical disks”:

# vxdisk list DEVICE TYPE DISK GROUP STATUS sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:none - - online invalid sdd auto:none - - online invalid sde auto:none - - online invalid 

From the output of the command, we see that the system has 5 disks available, but they are not initialized to work with VxVM. I have a system on “sda”, so I won’t touch it.
To add a device to the VxVM, you need to write a service area to it using the “vxdisksetup” command with the “-i” key (Initialize the private region):

 # /opt/VRTS/bin/vxdisksetup -i sdb # /opt/VRTS/bin/vxdisksetup -i sdc # /opt/VRTS/bin/vxdisksetup -i sdd # /opt/VRTS/bin/vxdisksetup -i sde 

It is important to note that Veritas does not prompt the user for the names of the device files, using the entries from the “DEVICE” field.

Now we have another output of the “vxdisk list” command:

 # vxdisk list DEVICE TYPE DISK GROUP STATUS sda auto:none - - online invalid sdb auto:cdsdisk - - online sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online sde auto:cdsdisk - - online 

The “cdsdisk” type is used by default for data partitions, and does not support booting. If you want to boot from a VxVM encapsulated disk, you will need to initialize the disk as “sliced”.

Now, on the basis of two initialized disks, create the group dg_first, assigning disk names to the devices:

 # vxdg init dg_first disk1=sdb disk2=sdc # vxdisk list DEVICE TYPE DISK GROUP STATUS sda auto:none - - online invalid sdb auto:cdsdisk disk1 dg_first online sdc auto:cdsdisk disk2 dg_first online sdd auto:cdsdisk - - online sde auto:cdsdisk - - online # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - 

The last team showed more detailed information about the group. The rather important LENGTH field indicates the length in sectors. Inconvenient, yes ... The sector in this case is 512 bytes, we get 2GB per disk. When there are many disks in a group, you have to resort to the help of available tools, and you get something like:

 # vxprint -g dg_first|awk 'BEGIN{DGSIZE=0} {DGSIZE=DGSIZE+$5} END{print "DG size:" DGSIZE*512/1024/1024}' DG size:4023.75 

I'll warn you right away that there are plenty of such jokes in Veritas, but over time a habit develops and everything is perceived as it should be.

Now we will create a vol1 volume in the dg_first group of 2 GB in size, on which it will be possible to write the file system so that it can be mounted:

 # vxassist -g dg_first make vol1 2G # vxprint -g dg_first -vt vol1 V NAME RVG/VSET/CO KSTATE STATE LENGTH READPOL PREFPLEX UTYPE v vol1 - ENABLED ACTIVE 4194304 SELECT - fsgen 

In principle, the described process is similar to the process of creating a partition in LVM, the two layers described above were created automatically: plex and subdisk. They will be seen in the output of the vxprint command:

 # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - v vol1 fsgen ENABLED 4194304 - ACTIVE - - pl vol1-01 vol1 ENABLED 4194304 - ACTIVE - - sd disk1-01 vol1-01 ENABLED 4120320 0 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - 

The output of the command can be interpreted as follows: the dg_first disk group consists of one plex vol1-01, which, in turn, consists of two disk1-01 and disk2-01 subdisks (PLOFFS - plex association offset). Add another 1 GB volume and see where it will be located:

 # vxassist -g dg_first make vol2 1G # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - v vol1 fsgen ENABLED 4194304 - ACTIVE - - pl vol1-01 vol1 ENABLED 4194304 - ACTIVE - - sd disk1-01 vol1-01 ENABLED 4120320 0 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - v vol2 fsgen ENABLED 2097152 - ACTIVE - - pl vol2-01 vol2 ENABLED 2097152 - ACTIVE - - sd disk2-02 vol2-01 ENABLED 2097152 0 - - - 

The second volume received its plex, which contains one subdisk, and this subdisk fits on the second disk (the first subdisk took a little second disk, apparently due to the presence of service areas). Graphically, it seems like this:



Detailed information on the location of the plexes and subdisks is given by the vxprint command with the appropriate keys:

 # vxprint -g dg_first -s -t Disk group: dg_first SD NAME PLEX DISK DISKOFFS LENGTH [COL/]OFF DEVICE MODE SV NAME PLEX VOLNAME NVOLLAYR LENGTH [COL/]OFF AM/NM MODE SC NAME PLEX CACHE DISKOFFS LENGTH [COL/]OFF DEVICE MODE sd disk1-01 vol1-01 disk1 0 4120320 0 sdb ENA sd disk2-01 vol1-01 disk2 0 73984 4120320 sdc ENA sd disk2-02 vol2-01 disk2 73984 2097152 0 sdc ENA 

During operation, you can operate with plexes and subdisks: create, delete, associate with each other, etc. This introduces some complexity and possibly confusion, but adds flexibility in data placement.

Now you can create a file system on existing volumes vol1 and vol2. I have already mentioned that the package contains a very flexible and powerful Veritas File System (vxfs) file system. Create it on the volumes and mount:

 # mkfs.vxfs /dev/vx/dsk/dg_first/vol1 version 9 layout 4194304 sectors, 2097152 blocks of size 1024, log size 16384 blocks rcq size 1024 blocks largefiles supported # mkfs.vxfs /dev/vx/dsk/dg_first/vol2 version 9 layout 2097152 sectors, 1048576 blocks of size 1024, log size 16384 blocks rcq size 1024 blocks largefiles supported # mount.vxfs /dev/vx/dsk/dg_first/vol1 /mnt/vol1 # mount.vxfs /dev/vx/dsk/dg_first/vol2 /mnt/vol2 

Note the path to the block device file: the constant path to the VxVM device files is “/ dev / vx”, then dsk (block devices) or rdsk (character devices), the group name and the volume names. Everything is logical. Check the file system sizes:

 # df -h Filesystem Size Used Avail Use% Mounted on /dev/vx/dsk/dg_first/vol1 2.0G 19M 1.9G 1% /mnt/vol1 /dev/vx/dsk/dg_first/vol2 1.0G 19M 943M 2% /mnt/vol2 

All perfectly. But we have 2 more disks: sdd and sde, you can try to add them and expand existing file systems by 2 times:

 # vxdg -g dg_first adddisk disk3=sdd disk4=sde # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - dm disk3 sdd - 4120320 - - - - dm disk4 sde - 4120320 - - - - v vol1 fsgen ENABLED 4194304 - ACTIVE - - pl vol1-01 vol1 ENABLED 4194304 - ACTIVE - - sd disk1-01 vol1-01 ENABLED 4120320 0 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - v vol2 fsgen ENABLED 2097152 - ACTIVE - - pl vol2-01 vol2 ENABLED 2097152 - ACTIVE - - sd disk2-02 vol2-01 ENABLED 2097152 0 - - - 

Added only discs. Resize volumes with file systems:

 # /usr/lib/vxvm/bin/vxresize -F vxfs -g dg_first vol1 4G # /usr/lib/vxvm/bin/vxresize -F vxfs -g dg_first vol2 2G # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - dm disk3 sdd - 4120320 - - - - dm disk4 sde - 4120320 - - - - v vol1 fsgen ENABLED 8388608 - ACTIVE - - pl vol1-01 vol1 ENABLED 8388608 - ACTIVE - - sd disk1-01 vol1-01 ENABLED 4120320 0 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - sd disk2-03 vol1-01 ENABLED 73984 4194304 - - - sd disk3-01 vol1-01 ENABLED 4120320 4268288 - - - v vol2 fsgen ENABLED 4194304 - ACTIVE - - pl vol2-01 vol2 ENABLED 4194304 - ACTIVE - - sd disk2-02 vol2-01 ENABLED 2097152 0 - - - sd disk4-01 vol2-01 ENABLED 2097152 2097152 - - - # df -h /dev/vx/dsk/dg_first/vol1 4.0G 20M 3.8G 1% /mnt/vol1 /dev/vx/dsk/dg_first/vol2 2.0G 19M 1.9G 1% /mnt/vol2 

Expanded volume and file system. Notice how the subdisks are distributed.
Suppose now we have the task to give sdb, which contains data on a mounted file system. Attempting to disable it will not work, because There is a subdisk there, and there would be no place to migrate all the data:

 # vxdg -g dg_first rmdisk disk1 VxVM vxdg ERROR V-5-1-552 Disk disk1 is used by one or more subdisks. Use -k to remove device assignment. 

Only 1 GB is occupied on disk4, so first we will take a gigabyte from the file system, then free up disk1 (on which the disk1-01 subdisk belonging to the vol1-01 plex of volume vol1 rests) and remove it from VxVM:

 # /usr/lib/vxvm/bin/vxresize -F vxfs -g dg_first vol1 -1G # vxprint Disk group: dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk1 sdb - 4120320 - - - - dm disk2 sdc - 4120320 - - - - dm disk3 sdd - 4120320 - - - - dm disk4 sde - 4120320 - - - - v vol1 fsgen ENABLED 6291456 - ACTIVE - - pl vol1-01 vol1 ENABLED 6291456 - ACTIVE - - sd disk1-01 vol1-01 ENABLED 4120320 0 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - sd disk2-03 vol1-01 ENABLED 73984 4194304 - - - sd disk3-01 vol1-01 ENABLED 2023168 4268288 - - - v vol2 fsgen ENABLED 4194304 - ACTIVE - - pl vol2-01 vol2 ENABLED 4194304 - ACTIVE - - sd disk2-02 vol2-01 ENABLED 2097152 0 - - - sd disk4-01 vol2-01 ENABLED 2097152 2097152 - - - # vxassist -g dg_first move vol1 '!disk1' # vxdg -g dg_first rmdisk disk1 # vxprint -g dg_first TY NAME ASSOC KSTATE LENGTH PLOFFS STATE TUTIL0 PUTIL0 dg dg_first dg_first - - - - - - dm disk2 sdc - 4120320 - - - - dm disk3 sdd - 4120320 - - - - dm disk4 sde - 4120320 - - - - v vol1 fsgen ENABLED 6291456 - ACTIVE - - pl vol1-01 vol1 ENABLED 6291456 - ACTIVE - - sd disk3-02 vol1-01 ENABLED 2097152 0 - - - sd disk4-02 vol1-01 ENABLED 2023168 2097152 - - - sd disk2-01 vol1-01 ENABLED 73984 4120320 - - - sd disk2-03 vol1-01 ENABLED 73984 4194304 - - - sd disk3-01 vol1-01 ENABLED 2023168 4268288 - - - v vol2 fsgen ENABLED 4194304 - ACTIVE - - pl vol2-01 vol2 ENABLED 4194304 - ACTIVE - - sd disk2-02 vol2-01 ENABLED 2097152 0 - - - sd disk4-01 vol2-01 ENABLED 2097152 2097152 - - - 

We see that subdisk disk3-02 and disk4-02 were added to vol1, i.e. the system used all free disk3 disk space, then took the required amount of disk4 disk space.

For the first article is enough. I think in general terms how this product works. I did not describe the commands and keys in detail, in order not to inflate the article any more, I recommend to google “vxvm cheat sheet”, select a label and print it.
If there is interest, in the following articles I will discuss such topics as moon migration, space optimization, the vxfs file system, and others.