Unsolved

Closed

9 Posts

856

April 18th, 2023 14:00

R730/H730 - issue with HGST HUC101212CSS600 SAS Drives

Hello -

* R730 w/ 8 Drive Bays (numbered 0..7). BIOS is 2.16.0.

* H730 Mini with 25.3.0.0016 firmware (the behavior is similar, but different when you update to 25.5.9.001)

* Drives are Dell Enterprise Class 0B28470 - HGST HUC101212CSS600 1.2TB 10K RPM 6 GB/S SAS SFF 2.5" (date codes are from 2015)
I believe these are certified". In the RAID configuration utility - advanced, non-certified drives show "Certified ... NO". These Dell drives do not have the "Certified" status shown on this config page.

I've been running with a total of (4) of these drives for more than a year in four different servers - all configured the same.

The issue is when more than (4) are installed, the additional (4) drives are reported as missing (if part of a RAID) and aren't in the physical drive list. On the drive, you see (I believe this means "identifying drive") ...
Disk LED blinks at 2Hz
Health LED blinks at 1Hz

I have other Dell SAS 300GB drives. HP 507119-004 and Dell ST9300603SS. These show up as non-certified, but I can install 8 without issue. All are detected. If I install (4) of the 1.2TB drives and (4) of the 300GB drives, this works. Any 1.2TB drive that is added over and above four will not be detected.

In my testing, all drives are blank with no configuration. Below,

* means drive 1 is detected & working properly
! means drive 1 is NOT detected the LEDs are blinking
- means drive not installed

Scenario #1.
If I install (8) drives and power on (the EVEN# drives have failed).

0 1 2 3 4 5 6 7
! * ! * ! * ! *

Scenario #2.
If I install only (4) on drives where they had failed above, they all work fine after a reboot.

0 1 2 3 4 5 6 7
* - * - * - * -

Scenario #3.
After Scenario #2, with 0,2,4,6 working, installing drives 1,3,5,7 while powered on, the new drives are not detected.

Scenario #4.
After Scenario #3, reboot. Gives the same result as Scenario #1. The ODD# drives work and the EVEN# drives have failed.

Scenario #5
Powered on. No drives. Install one drive at a time (anywhere) and they will work until you get to the 5th one. Reboot and you see Scenario #1 - EVEN#'s failed.


With updated RAID BIOS to 25.5.9.001, the operation is different. I typically see 5 working drives instead or 4, and they are in a different order.

Scenario #6. All 8 in and boot. 2,4,& 6 aren't working.

0 1 2 3 4 5 6 7
* * ! * ! * ! *

Scenario #7. Swap 1-2, 3-4, and 5-6. Power on. Same as above. 2,4,& 6 aren't working.

Scenario #8. If you unplug a non-working (blinking drive) and plug it back in, it will be identified. You can get all 8 to be identified by doing this, but if you reboot, 2,4, & 6 are not working again.

The HGST drive firmware as-received was U5E2. Updating the drives to the (latest?) U850 offers no improvement.

Any advice would be appreciated.

Thanks

Moderator

 • 

3.7K Posts

April 18th, 2023 20:00

Hi @CmpUcom,

 

Tough issue to solve. I'd wonder if the drives you have are DPN# T6TWN? I've heard, T6TWN are trouble drives back then. You might need to deeply investigate into PERC log to try to identify the root cause if there are any hints. 

 

So are all the drives in U850? After scenario #5; the issue now changed from 4 drives not detected to 3 drives? After RAID controller updated to 25.5.9.001, all scenario #1 to #4 no longer there? Do you have iDRAC Enterprise? If yes, has it been updated? 

 

Do you have the 8X2.5" or 16X2.5" backplane? How many SAS cables are connected to the backplane? What is your power supply wattage? How many power supplies is installed on the server?

9 Posts

April 19th, 2023 08:00

Hi Joey -

Thanks for looking at this.

* My drives are not the T6TWN (Seagate Savvio), but are Hitachi HGST with the same specs.

* The HUC101212CSS600 drives are available on Amazon. Someone complained in the comments that this lot was manufactured in 2015. I didn't get them from Amazon, but they are new and physically work fine - except for this issue. Mine seem to be from a similar lot and made in various months of 2015.

* These are 6GB/s drives. The H730 can do 12GB/s. I can see in the RAID setup that these drives have negotiated to 6GB/s. There is a setting to set them to 6GB/s (Manage Link Speed). This doesn't help. I also tried setting all drives to 3GB/s without luck. I've left them at Auto.

* All of the drives now have U850 firmware. The PERC H730 Mini has rev 25.5.9.001.

* I have dual 750W supplies. I too thought it might be a loading issue, but the other drives that work in a full set of 8 are higher current.
300GB SAVVIO (works) : 1.0A @ 5V / 0.3A @ 12V
1.2TB HGST (don't work) : 0.7A @ 5V / 0.4A @ 12V


* I have the 8 X 2.5" backplane. There are (2) SAS cables each connected to a SAS A and SAS B board.

* I did use iDRAC to update the firmware from Dell's HTTPs site. It is Enterprise 2.84.84.84 Build 2.

* Yes, the issue now changed from 4 drives not detected to 3 drives with the change of the H730 firmware to 25.5.9.001. Updating the Drive BIOS to U880 made no difference. This issue is (repeatably) scenario #6.

Scenario #6. All 8 in and boot. 2,4,& 6 aren't working.
0 1 2 3 4 5 6 7
* * ! * ! * ! *

* If you boot into either the GUI Raid Config (Device Settings, Integrated RAID Controller 1: Dell PREC H730 Mini Configuration Utility) or the text based Avago RAID config (CTRL-R), you see 5 physical drives. If you unplug and then plug the 3 undetected drives back in, they will all appear. Rebooting, drive 2,4 & 6 are undetected again.

* I did do a Factory Default in the Avago Raid setup. All drives have been cleared of configuration and have no foreign configs.

* I tried to save the controller log and debug events. "File system is not available". I have a bootable OS drive and also an external USB drive.

* With a bootable SATA drive in slot 0, and 7 of these 1.2TB HGST drives, drives 2 & 6 are not detected. These are 3rd drives (out of 4) on SAS A and SAS B boards.

* The way this issue was originally found was I had working RAID1 using drives in slots 2&3, and 4&5. After adding new drives in 0&1, member drives of my previously configured & working RAID showed as "MISSING" in the RAID setup screen even though the drive was physically there.

Moderator

 • 

3.7K Posts

April 20th, 2023 02:00

Hi @CmpUcom,

 

Can you provide me the drive DPN# for HUC101212CSS600. We will need it to check if it's supported. I am assuming they are Dell drive, since you mentioned it and you are able to update it to U850. So, let us know the DPN#. Do you have any other H730 to swap? 

 

Have you tried reseating the cables within the storage chain? You mentioned you have 4 servers, are they all R730 and all had the same issue? Or you have no spare server to check the issue?

9 Posts

April 20th, 2023 09:00

Hi Joey -

The Dell P/N is 0B28470.  This issue is common across our (7) R730 Servers.  Issues were originally seen with PERC 25.3.0.0016.  I have two machines offline that I've been incrementally updating and re-testing.  The latest PERC 25.5.9.001 version has changed, but not solved the basic issue. 

Please see a photo of the typical HGST drive (inline) below.

I have ordered (8) of the  Dell WXPCX 1.2TB 10K 12GB/s SAS 2.5" Hard Drive Dell R630 R730 R730XD ().  These are advertised as drives specifically for the R730.  I'll report back if these operate correctly.

IMG_5267.JPG

9 Posts

April 21st, 2023 13:00

"BAD" Drives:
DPN 0B28470 HUC101212CSS600 / Hitachi HGST
6Gbps / 1.2TB / Built in 2015

"NEW" Drives:
DPN 0WXPCX PN 1FF200-151 ST1200MM0088 / Seagate
12Gbps / 1.2TB / Built in 2020 & 2021.

* An installation of (8) "NEW" drives works properly. All are detected. I can see in the controller status that each is negotiating to 12Gbps.

* I can verify the same poor behavior with the BAD drives. The magic number of BAD drives that will function reliably in any slot configuration is (4) at a time.

* Test: (2) NEW drives (in slots 0&1) and (6) BAD drives, Drive #6 is not detected. Unplugging and plugging it back in allows it to be detected. The drive #6 is missing again on reboot.

* Test: (1) NEW drive (slot 0) and (7) BAD drives, Drives #2,4,6 are not detected.
* Test: (1) NEW drive (slot 6) and (7) BAD drives, Drives #2,4 are not detected.




* Bottom line. I will use a MAXIMUM of (4) of the BAD drives. These can be placed in any order (but in pairs) - in slots 0..7. I tried all of the combinations I could think of. With (4) BAD and (4) NEW drives, all drives are detected after boot and operate properly.

* On another note, the Dell plastic caddies that came with XX drives : DPN 08FKXC are doo-doo (forum censors made me change my benign word). The spring fingers are too large and don't compress well. You really cannot populate all 8 drives and remove and replace a single drive. My first instinct regarding this issue was that the drives were not seating properly. I actually reinstalled these drives into some old-style caddies I had.

* The new drives came with the same plastic DPN 08FKXC caddies. The spring fingers have been redesigned and now work properly. The better caddies do not have a Dell logo embossed in the externally facing sides.

"NEW" Drive Pictured below.IMG_5270.JPG

9 Posts

April 24th, 2023 07:00

I was wrong when I said the drives with issues were not T6TWN.

I thought that the printed P/N 0B28470 was the DPN. 

The drives at issue are DPN T6TWN made by Hitachi HGST.  HUC101212CSS600 

 

No Events found!

Top