From Secure Computing Wiki
Revision as of 16:08, 11 September 2007 by Ecrist (Talk | contribs) (New page: === Basic Command List === ;gmirror status * Lists the current status of all gmirrors on the system. ;gmirror list * Prints detailed information about the devices within a mirror. ;gmirr...)

(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

Basic Command List

gmirror status
  • Lists the current status of all gmirrors on the system.
gmirror list
  • Prints detailed information about the devices within a mirror.
gmirror remove <mirror> <device>
  • Removes <device> from <mirror>. This is the clean and preferred method.
gmirror forget <mirror>
  • Causes missing devices to be dropped from a mirror.
  • This is useful when you need to replace a drive that has failed.
gmirror insert <mirror> <device>
  • Inserts <device> into <mirror>.


  • This method assumes you already have FreeBSD installed on the first drive. Many of these steps are not necessary if your mirror is not going to be your boot device.
  1. Boot system from a FreeBSD 5.4+ Install CD (full disc, not boot-only)
  2. After the installer starts, select Fixit, and From CD/DVD
  3. When the Fixit prompt appears, run the following commands:
    Fixit# chroot /dist
    Fixit# mount_devfs devfs /dev
  4. Run the following command for each of the disks in your new array, where <device_name> is the name of the device as listed in /dev:
    Fixit# gmirror clear /dev/<device_name>
  5. Load the gmirror kernel module:
    Fixit# gmirror load
  6. Create and Initialize the mirror:
    Fixit# gmirror label -v -b load gm0 /dev/<device_name>
    Fixit# mount /dev/mirror/gm0s1a /mnt
  7. Edit /boot/loader.conf to load the geom_mirror kernel module at boot:
    Fixit# echo geom_mirror_load=\"YES\" >> /mnt/boot/loader.conf
  8. Edit /etc/rc.conf to turn off system swap. Swap will be handled automatically by geom:
    Fixit# echo swapoff=\"YES\" >> /mnt/etc/rc.conf
  9. Edit fstab to point to /dev/mirror/gm0xxx either manually, or with sed command here:
    Fixit# sed "s%ad0%mirror/gm0%g" /mnt/etc/fstab > /mnt/etc/
    Fixit# mv /mnt/etc/fstab /mnt/etc/fstab.old
    Fixit# mv /mnt/etc/ /mnt/etc/fstab
    • Note that the sed command above assumes your current boot device is ad0, change this value accordingly for your system.
  10. Reboot your system!
    Fixit# shutdown -r now
  11. Once the system has rebooted, insert the other device into your mirror with the following command, where <device_2_name> is the name in /dev of your second device:
    Fixit# gmirror insert gm0 /dev/<device_2_name>

Disaster Recovery

Inevitably, there will come a time when we need to rebuild an array due to hard disk failure or some other mean nasty. Follow the steps below, and you'll be on your way to a COMPLETE status once again.

  1. First off, remove the offending device if you haven't already done so. Upon removal, you should see some messages indicating the loss of a device. Something like:
    (da1:ahc0:0:1:0): lost device
    (da1:ahc0:0:1:0): Invalidating pack
    GEOM_MIRROR: Cannot write metadata on da1 (device=gm0, error=6).
    GEOM_MIRROR: Cannot update metadata on disk da1 (error=6).
    GEOM_MIRROR: Device gm0: provider da1 disconnected.
    (da1:ahc0:0:1:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0
    (da1:ahc0:0:1:0): removing device entry

    You'll also notice that the device entry has disappeared from /dev.

  2. Once the disk has been removed, replace it with a shiny new(ish) drive.
  3. Reread the SCSI or ATA bus with the appropriate command:
    • SCSI (where 0 is the bus_id):
      # camcontrol rescan 0
    • ATA (where 0 is the bus_id):
      # atacontrol attach 0
    You should see something like this in dmesg output:
    da1 at ahc0 bus 0 target 1 lun 0
    da1: <SEAGATE ST318305LC 2202> Fixed Direct Access SCSI-3 device
    da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing Enabled
    da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C)
  4. Tell gmirror to forget about the missing drive, so we can insert the new one.
    # gmirror forget gm0
  5. Insert the new disk into the mirror where <new_device> is the full path to the new device (ex /dev/da1):
    # gmirror insert gm0 <new_device>

    If the above command is successful, you'll see a message similar to:

    GEOM_MIRROR: Device gm0: provider da1 detected.
    GEOM_MIRROR: Device gm0: rebuilding provider da1.
  6. Check the status of the rebuild with gmirror status command:
    gmirror status
          Name    Status  Components
    mirror/gm0  DEGRADED  da0
                          da1 (48%)