Megacli

From Secure Computing Wiki
Revision as of 21:01, 25 March 2010 by Ecrist (Talk | contribs)

Jump to: navigation, search
      • EDIT: As of FreeBSD 8.0, many LSI cards use the mpt driver, for which there is now an mptutil to monitor and manage RAID arrays. I will have an associated Nagios plugin very soon!


FreeBSD has great support for LSI MegaRAID adapters, until you try to monitor them, or get their status. LSI has opted to keep all their drivers and software proprietary. There is a Linux binary to get the functionality we need. In order to use it, we've got to do a couple things to our system.

Hardware Setup

For our setup, we're using a Dell PowerEdge R300 server with a Dell PERC 6/E, connected to a Dell PowerVault MD1000 via SAS.

Linux Compat

To get megacli installed, we need to enable linux binary compatibility. In addition, we need to enable Fedora Core 6, which is in dev. To do this, please do the following:

  1. Add 'compat.linux.osrelease=2.6.12' to /etc/sysctl.conf
  2. Install /usr/ports/emulators/linux_base-fc6
  3. Add the following two lines to /etc/fstab:
    linproc		/compat/linux/proc	linprocfs	rw	0	0
    linsysfs	/compat/linux/sys	linsysfs	rw	0	0

megacli

  1. In our setup, the only RAID adapter we had showed up as host1, rather than host0, in the linux proc device. To fix this, we need to edit some system source. Edit /usr/src/sys/compat/linsysfs/linsysfs.c, and change line 180:

    Original:
    if (dinfo->cfg.baseclass == PCIC_STORAGE) {

    New:

    if (dinfo->cfg.baseclass == PCIC_STORAGE && dinfo->cfg.subclass == PCIS_STORAGE_RAID) {
  2. Now, cd /usr/src and make buildkernel && make installkernel. Don't reboot quite yet.
  3. As of 11/5/2008, the FreeBSD port for linux-megacli was listed as broken due to an unfetchable file. I don't know the backstory to this, but I've found the file and you can download it here. Place this file in /usr/ports/distfiles.
  4. Install the /usr/ports/sysutils/linux-megacli port. Feel free to ignore the install notes that come up, as they're all covered here.
  5. Add the following line to /boot/loader.conf:
    mfi_linux_load="YES"
  6. Reboot.

Testing

After you've rebooted your system, you should be able to run the following command and get output describing your disk layout:

# megacli -LDInfo -LALL -aALL

Here's our output from the above command with a 12 disk RAID 50:

Adapter 0 -- Virtual Drive Information:
Virtual Disk: 0 (Target Id: 0)
Name:
RAID Level: Primary-5, Secondary-3, RAID Level Qualifier-3
Size:4764160MB
State: Optimal
Stripe Size: 64kB
Number Of Drives:6
Span Depth:2
Default Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Current Cache Policy: WriteBack, ReadAheadNone, Direct, No Write Cache if Bad BBU
Access Policy: Read/Write
Disk Cache Policy: Disk's Default

Nagios Monitoring

The following script, coupled with Nagios and NRPE will allow you to monitor the status of your MegaRAID adapter:

#!/usr/bin/perl -w
use strict;
use warnings;

use lib "/usr/local/libexec/nagios";
use utils qw(%ERRORS);

my $usage = "Usage: check_mega_raid\nEvaluates whether volume is in optimal state\n";
my $output = ""; my $errs = 0; my $index = 0; my $count = 0;
my (@disk_status_fmt);
my (@part, $enclosure_id, $slot_number, $state, $progress_hosts);

my $status = `/usr/local/sbin/megacli -ldinfo -L0 -aall| /usr/bin/grep State`;
if ($status =~ m/^State: (.*)$/){
	$output .= "RAID Volume $1. ";
	if ($1 =~  m/Optimal/){
	} elsif ($1 =~ m/Degraded/) {
		## Pull data from all disks.  Grab current enclosure ID and slot number.
		my @disk_status = `/usr/local/sbin/megacli -ldpdinfo -aall`;
		while (my $line = shift @disk_status){
			if ($line =~ m/^\w/){
				push @disk_status_fmt, $line;
			}
		}
		splice @disk_status_fmt, 0, 16; 
		while (@disk_status_fmt){
			@part = splice @disk_status_fmt, 0, 16;
			while (my $line = shift @part){
				if ($line =~ m/Slot Number: (\d+)$/){
					$slot_number = $1;
				} elsif ($line =~ m/Enclosure Device ID: (\d+)$/){
					$enclosure_id = $1;
				} elsif ($line =~ m/Firmware\sstate:\s(\w+)$/){
					$state = $1;
				}
			}
			if ($state =~ m/Online/){
			} elsif ($state =~ m/Rebuild/){
				$errs++;
				my $rebuild_status = `/usr/local/sbin/megacli -pdrbld -showprog physdrv\[$enclosure_id:$slot_number\] -a0 | /usr/bin/grep Reb`;
				if ($rebuild_status =~ m/^.*Completed (\d+)%/){
					$output = $output . "$enclosure_id:$slot_number $state ($1%);";
				}
			} else {
				$errs++;
				$output = $output . "$enclosure_id:$slot_number $state; ";
			}
		}
		## We have a drive rebuilding, let's get it's progress.
		# megacli -pdrbld -showprog physdrv\[12:0\] -a0
		# Rebuild Progress on Device at Enclosure 12, Slot 0 Completed 1% in 2 Minutes.

	}
}
print "$output\n";
if ($errs gt 0){
	exit $ERRORS{'CRITICAL'};
} else {
	exit $ERRORS{'OK'};
}