Unsolved
This post is more than 5 years old
2 Posts
1
3901
November 6th, 2018 15:00
Help! Removed predicted failure drive without taking offline
Hi All,
I have an Exchange server with a RAID1 (for the OS) and a RAID 10 for Exchange.
One of the 4 drives in the RAID 10 set was marked as 'Predicted Failure - YES.' I obtained a replacement drive and went to replace the drive. I saw the option to take the drive offline in Dell OpenManage Server Admin, but it gave me a warning about potential data loss... so I just pulled the drive out without offlining the drive.
I replaced the drive and noted that OMSA did not update that a new drive had been inserted--it still shows the old drive as being online and failure predicted. I wasn't worried at first, thinking it was just an inventory update issue... so I logged into the iDRAC and confirmed that the new drive is rebuilding... I thought all was okay... but then I saw PERC errors in the Windows system log and Exchange stopped responding. ACK!
At this point Exchange is not responding, I cannot browse the contents of the virtual disk in Windows.
Should I reinsert the old disk and hope that it comes back online properly so that I can force the drive offline and attempt the replacement again?
Thanks!
EpsilonSrvAdmin
2 Posts
0
November 6th, 2018 16:00
Well, I crossed my fingers and stood on my head... and restarted the server (had to hold the power button cause Windows wouldn't shutdown). I booted to the PERC BIOS and checked the drives and virtual disks... all looked healthy. I booted into Windows and things were a bit slow to get up and running... checking the event logs I saw the PERC controller basically freaking out about the drive that I replaced. The controller marked it as failed, brought it online, rebuilt it, and then marked the VD as online and healthy (all within the span of a 30 seconds or so!).
So far everything has recovered... Exchange has recovered the database that dismounted and has replayed the log files. The drives are all listed as online and healthy.
I'm running another full system backup... but I think I dodged a bullet this time. Lesson learned: When a drive is listed as "Predicted Failure" but is still online, always force it OFFLINE before removing from the server.
Nestico27
15 Posts
0
November 8th, 2018 07:00
For future reference, the data loss warning will occur due to the fact the drive is not there and the RAID array is not to spec. So theoretically, yes data loss can occur, doesn't mean it will. But as you said, in the future take it offline and swap out, better to be safe than sorry. I hope this little bit of info goes a long way for you :)