From Secure Computing Wiki
Revision as of 17:45, 23 November 2010 by Esubiguxoc (Talk | contribs)

Jump to: navigation, search

Page Is Unavailable Due To Site Maintenance, Please Visit Reserve Copy Page


Basic Command List

gmirror status
  • Lists the current status of all gmirrors on the system.
gmirror list
  • Prints detailed information about the devices within a mirror.
gmirror remove <mirror> <device>
  • Removes <device> from <mirror>. This is the clean and preferred method.
gmirror forget <mirror>
  • Causes missing devices to be dropped from a mirror.
  • This is useful when you need to replace a drive that has failed.
gmirror insert <mirror> <device>
  • Inserts <device> into <mirror>.


<blockquote> <p>There is a problem with the FreeBSD 6.3-RELEASE and 7.0-BETA/RELEASE CDs and this method of gmirror installation on your boot device. I'm currently working on an updated process until the dev team can fix the fixit CD libraries for gmirror. The super-quick fix is to use a 6.2-RELEASE disk 1 when you start up the fixit CD.</p> <p>-- Eric Crist 30-Jan-2008</p> <p>UPDATE: There is now a solution to the above problem. This will be fixed for 6.4 and 7.1, but you must follow the extra step below for 6.3 and 7.0 releases.</p><p>Ecrist 07:56, 28 February 2008 (CST)</p> </blockquote>

  • This method assumes you already have FreeBSD installed on the first drive. Many of these steps are not necessary if your mirror is not going to be your boot device. However, why wouldn't you want your boot device to be mirrored too?


 <li>Boot system from a FreeBSD 5.4+ Install CD (full disc, not boot-only)
 <li>After the installer starts, select Fixit, and From CD/DVD
 <li>When the Fixit prompt appears, run the following commands:
 <pre>Fixit# mount -t devfs devfs /dist/dev

Fixit# chroot /dist</pre> IF YOU HAVE A 6.3 OR 7.0 ISO, you must run the following command as well: <pre>Fixit# unset GEOM_LIBRARY_PATH</pre>

 <li>Run the following command for each of the disks in your new array, where <device_name> is the name of the device as listed in /dev:
 <pre>Fixit# gmirror clear /dev/<device_name></pre>
 <li>Load the gmirror kernel module:
 <pre>Fixit# gmirror load</pre>
 <li>Create and Initialize the mirror:
 <pre>Fixit# gmirror label -v -b load gm0 /dev/<device_name>
 Fixit# mount /dev/mirror/gm0s1a /mnt</pre>

<li>Edit /boot/loader.conf to load the geom_mirror kernel module at boot: <pre>Fixit# echo 'geom_mirror_load="YES" ' >> /mnt/boot/loader.conf</pre> <li>Edit /etc/rc.conf to turn off system swap. Swap will be handled automatically by geom: <pre>Fixit# echo 'swapoff="YES" ' >> /mnt/etc/rc.conf</pre> <li>Edit fstab to point to /dev/mirror/gm0xxx either manually, or with sed command here: <pre>Fixit# sed -i ".bak" "s%ad0%mirror/gm0%g" /mnt/etc/fstab</pre>

  • Note that the sed command above assumes your current boot device is ad0, change this value accordingly for your system.
  • Thanks to Miroslav Lachman for the refined command.
 <li>Reboot your system!
 <pre>Fixit# shutdown -r now</pre>
 <li>Once the system has rebooted, insert the other device into your mirror with the following command, where <device_2_name> is the name in  /dev of your second device:
<pre>Fixit# gmirror insert gm0 /dev/<device_2_name></pre>


Disaster Recovery

Inevitably, there will come a time when we need to rebuild an array due to hard disk failure or some other mean nasty. Follow the steps below, and you'll be on your way to a COMPLETE status once again.

<ol> <li>First off, remove the offending device if you haven't already done so. Upon removal, you should see some messages indicating the loss of a device. Something like: <pre>(da1:ahc0:0:1:0): lost device (da1:ahc0:0:1:0): Invalidating pack GEOM_MIRROR: Cannot write metadata on da1 (device=gm0, error=6). GEOM_MIRROR: Cannot update metadata on disk da1 (error=6). GEOM_MIRROR: Device gm0: provider da1 disconnected. (da1:ahc0:0:1:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0 (da1:ahc0:0:1:0): removing device entry</pre> You'll also notice that the device entry has disappeared from /dev. <li>Once the disk has been removed, replace it with a shiny new(ish) drive. <li>Reread the SCSI or ATA bus with the appropriate command:

  • SCSI (where 0 is the bus_id):<pre># camcontrol rescan 0</pre>
  • ATA (where 0 is the bus_id):<pre># atacontrol attach 0</pre>

You should see something like this in dmesg output: <pre>da1 at ahc0 bus 0 target 1 lun 0 da1: <SEAGATE ST318305LC 2202> Fixed Direct Access SCSI-3 device da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing Enabled da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C)</pre> <li>Tell gmirror to forget about the missing drive, so we can insert the new one. <pre># gmirror forget gm0</pre> <li>Insert the new disk into the mirror where <new_device> is the full path to the new device (ex /dev/da1): <pre># gmirror insert gm0 <new_device></pre> If the above command is successful, you'll see a message similar to: <pre>GEOM_MIRROR: Device gm0: provider da1 detected. GEOM_MIRROR: Device gm0: rebuilding provider da1.</pre> <li>Check the status of the rebuild with gmirror status command: <pre>gmirror status

     Name    Status  Components

mirror/gm0 DEGRADED da0

                     da1 (48%)</pre>


Monitoring your gmirror

I've written a small perl script that will allow you to monitor the health of your gmirror array with NRPE and Nagios. You may download it here, or the text is below: <pre>

  1. !/usr/bin/perl

use strict; use warnings;

use lib "/usr/local/libexec/nagios"; use utils qw(%ERRORS);

my $result = `/sbin/gmirror status`; $result =~ m/(DEGRADED|COMPLETE)/g; my $status = $1; my $rebuild = ($result =~ m/(\w\w\d \(\d\d?\%\))/); $result =~ m/(\w\w\d \(\d\d?\%\))/; my $progress = $1;

if ($status =~ /COMPLETE/){

       print "Raid Optimal\n";
       exit $ERRORS{'OK'};

} elsif (($status =~ /DEGRADED/) and ($rebuild == 1)) {

       print "Raid Reconstructing: $progress\n";
       exit $ERRORS{'WARNING'};

} else {

       print "Raid Critical!\n";
       exit $ERRORS{'CRITICAL'};

} </pre>

Other Links