[prev in list] [next in list] [prev in thread] [next in thread] 

List:       evms-devel
Subject:    Re: [Evms-devel] Raid5 array with out of date members
From:       Mike Tran <mhtran () us ! ibm ! com>
Date:       2006-03-31 16:54:32
Message-ID: 442D5EC8.5010200 () us ! ibm ! com
[Download RAW message or body]

Hi Nate,

Nate Delage wrote:

>When I arrived to check the system after some complaints I rebooted and
>noticed I was missing two drives. I expected sdb to be dead, but sdd_bbr
>was "out of date" and would not be activated and they md0 array wouldn't
>start either (because now I was missing two of the five drives)
>
>Mar 30 12:41:29 charlie _3_ MDRaid5RegMgr: raid5_create_region: About to
>create region md/md0 in degraded mode.
>Mar 30 12:41:29 charlie _3_ Engine: engine_ioctl_object: ioctl to object
>md/md0 failed with error code 19: No such device
>Mar 30 12:41:29 charlie _0_ Engine: plugin_user_message: Message is:
>MDRaid5RegMgr: RAID5 array md/md0 is missing the member  with RAID index
>1.  The array is running in degrade mode.
>
>Mar 30 12:41:30 charlie _3_ MDRaid5RegMgr: md_analyze_volume: Object
>sdd_bbr is out of date.
>Mar 30 12:41:30 charlie _3_ MDRaid5RegMgr: md_analyze_volume: Found 1
>stale objects in region md/md0.
>Mar 30 12:41:30 charlie _0_ MDRaid5RegMgr: sb1_analyze_sb: MD region
>md/md0 is corrupt
>Mar 30 12:41:30 charlie _3_ MDRaid5RegMgr: sb1_analyze_sb: MD region
>md/md0 is degraded
>Mar 30 12:41:30 charlie _3_ MDRaid5RegMgr: md_fix_dev_major_minor: MD
>region md/md0 is corrupt.
>Mar 30 12:41:30 charlie _0_ Engine: plugin_user_message: Message is:
>MDRaid5RegMgr: Region md/md0 is currently in degraded mode.  To bring it
>back to normal state, add 2 new spare devices to replace the faulty or
>missing devices.
>
>Mar 30 12:41:30 charlie _0_ Engine: plugin_user_message: Message is:
>MDRaid5RegMgr: Region md/md0 : MD superblocks found in object(s)
>[sdd_bbr ] are not valid.  [sdd_bbr ] will not be activated and should
>be removed from the region.
>
>Mar 30 12:41:30 charlie _0_ Engine: plugin_user_message: Message is:
>MDRaid5RegMgr: RAID5 region md/md0 is corrupt.  The number of raid disks
>for a full functional array is 5.  The number of active disks is 3.
>Mar 30 12:41:30 charlie _2_ MDRaid5RegMgr: raid5_read: MD Object md/md0
>is corrupt, data is suspect
>
>
>I don't really think sdd_bbr is damaged as I don't have any kernel I/O
>errors like I do for sdb_bbr. It seems the superblock  is just out of
>date so evms doesn't trust its integrity.
>
>I thought of using mdadm to start the array manually
>using /dev/evms/.nodes/sda_bbr etc as members of the array but mdadm
>can't seem to find any superblock info. I thought mdadm would see
>something but none of the drives have a superblock supposedly:
>
>charlie:/var/log# mdadm -E /dev/evms/.nodes/sda_bbr
>mdadm: No super block found on /dev/evms/.nodes/sda_bbr (Expected magic
>a92b4efc, got 00000000)
>
>I thought the above would work, but it's likely I don't understand
>something as evms is able to find 3 of the 5 drives on its own...
>
>I figure if I can just force evms or mdadm to use the out of date drive
>I can at least run the array in degraded mode and copy the data off.
>
>Oddly enough this all might be related due to controller/memory/cosmic
>interference....I was able to do a straight dd of both sda and sdb to
>other drives without any problems. So maybe the drives aren't dead but
>something else brought this on...
>
>I'd be happy to provide output of any commands or submit contents of any
>drive..
>
>Thanks for any help!!!
>
>-Nate
>
>  
>
Reading your message above I suspect that out of 5 disks, 3 are good, 1 
is out of date and 1 is missing/dead..  Please run evmsn -d debug, and 
send me the /var/log/evms-engine.log file.  I will find out what's 
happening.

--
Regards,
Mike T.



-------------------------------------------------------
This SF.Net email is sponsored by xPML, a groundbreaking scripting language
that extends applications into web and mobile media. Attend the live webcast
and join the prime developer group breaking into this new coding territory!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=110944&bid=241720&dat=121642
_______________________________________________
Evms-devel mailing list
Evms-devel@lists.sourceforge.net
To subscribe/unsubscribe, please visit:
https://lists.sourceforge.net/lists/listinfo/evms-devel
[prev in list] [next in list] [prev in thread] [next in thread]