Hot-swap disk replacement in gmirror under FreeBSD

Situation: There is a RAID array of two SATA drives in a mirror created using gmirror under FreeBSD.

It is necessary to replace one disk, without stopping the server.

Drop theory

There are many articles on the subject of creating a mirror on GEOM.
')
When creating a mirror, gmirror synchronizes all data, including the MBR, says http://people.freebsd.org/~rse/mirror/ (section Summary -> GEOM mirror on whole disk). If the server died during a disk failure, you can boot from the remaining disk, regardless of which disk is out of order. It is important only when loading to choose the right disk from which to boot.

When the array is working, the command

  # gmirror list 

almost always shows Flags: DIRTY. This is normal : the flag is set when information is written to the disk, and at this moment the state of the data on the disks of the array does not match. If the disk is constantly being recorded, the DIRTY flag will be permanently set.

Replacement procedure

Suppose there are two disks in the gm0 array: da0 and da1. Replace need da0.

In this situation, camcontrol was used. Depending on the controller through which the screws are connected, atacontrol may be needed instead.

1. We deduced a disk from a mirror:
   
   

    # gmirror remove gm0 da0
   

   
   
2. We determine which disk to physically need to be replaced - the activity indicator will not work for the disk that is removed from the mirror.
3. Without shutting down the server, pull out the disk .
4. Here the following is possible: the server does not understand anything for a few seconds, and then falls into a coma for half a minute. Need to wait a bit.
5. After this command
   
   

     # geom disk list
    # gmirror list 

   
   help you find that there is no single disk.
6. If point 1 has not been fulfilled, we ask gmirror to forget about all the disks that are currently inactive in the mirror:
   
   

     # gmirror forget gm0 

   
   gmirror detects that da0 is not and forgets about it.
   
7. We insert a new disk (preferably identical to the one with which it will work in pairs, up to the model).
8. Scan the bus so that the system detects a new disk attached:
   
   

    # camcontrol devlist # what we have now?
    # camcontrol rescan all # scan
    # camcontrol devlist # what happened?
   

   
   
9. Add a new da0 to the array :
   
   

     # gmirror insert gm0 da0 

   
   
10. gmirror will detect the new disk and start data synchronization . We look at the composition of the array:
    
    

      # gmirror list 

    
    
11. Do not get scared if it seems to us that the synchronization is going in the wrong direction :) If it was replaced by da0, then now it is in the list AFTER da1, and not before, as it was before.
12. We are experiencing happiness.