PS4210 won't boot

Many thanks for the reply Don, very timely and useful.

Rebooted because our software has the ability to reboot and shut down the entire system (this array, other servers, network switch, etc) and was testing that capability.

Performed your suggested actions and it booted off the other controller (haven't reinserted the bad one yet).  Array functions fine currently but I won't do anything else until another (current) set of backups is completed and taken offline.

So will the controller with the missing cache (once reinserted) then get that data from the one that's working now, and it will be ok, and no expected problems in the future?   Or should I expect that the controller that had the missing data will do that again someday and therefore it should be replaced?  (support stuff being handled by our contracts people)

Thanks again!

Hello, 

 It sounds like you didn't do a proper shutdown proper restart when you tested your software?   If a proper restart or shutdown was done it should flush cache automatically and that prevents what happened. Where the active CM was out of sync with the passive but the passive held pending IO 

 Re: Cache. Doesn't quite work that way.  The active CM always syncs cache with the passive.  So when you insert the controller it will become the passive and sync with the active automatically.  

 Glad that it came up OK. 

 Regards, 

Don 

This procedure isn't new, we've run this procedure dozens of times on at least 10+ different suites without issues.  We stop all IO first, then use the API to initiate the reboot properly via the EqualLogic itself.  What I was trying to get to was if we have an actual hardware problem with this particular 4210 or was it just a fluke.  Sounds like it was just a fluke.   Thanks again for your help, I'll note this fix in our database so that if anyone else runs into it, there's at least somewhere to start.

Hello, 

 You are very welcome!  

 Which API are you using?   How long from when you stop IO to when you restart or shutdown? 

 I would not call this a "fix".   it's the first step in a triage process.  It can mean that there is a problem in the transaction database the array uses.  

 Regards, 

Don 

We're using the Powershell api, via the eqlpstools.dll file.   There's at least a 10 second delay between both ESXI servers being down (actually down, not just started shutdown) and when the EqualLogic is told to shut down or reboot.

So I've reinserted the 2nd CM, and it's got green and amber lights so it's assumed the secondary role.  I can log in but the 'raidview' command returns 'Initialization has not started' and has been that way for an hour.   Since the volumes are up, and the primary controller reports the Luns ok, is there anything else to check?

Hello, 

 Re: API.  Ok thanks.  

 Re: RAIDTOOL.  Sounds like you are on the seconday CM?  At the CLI>  prompt?   Not the GrpName> prompt 

Only that active controller has access to the disks and RAID 

 On the active CM if you run GrpName>member select MEMBERNAME show  That will give you the current status of the active and passive CMs 

  If the volumes are up then the RAIDset has to be up as well. 

 Regards, 

Don

From the active CM:  member select show    

Returns: status online, no critical or warning conditions, 2 controllers, healthstatus normal, but no specific info about primary vs secondary controllers that I see.  I can get into the PS Group Manager gui (java) and everything looks good there, both CMs up, both batteries good.

Yep, everything looks good here, though the secondary CM doesn't seem to know its temps:

member select xxxxeql show controllers
___________________________ Controller Information ____________________________
SlotID: 0                         Status: active
Model: 70-0485(TYPE 19)           BatteryStatus: ok
ProcessorTemperature: 46          ChipsetTemperature: 32
LastBootTime: 2021-04-20:13:21:46 SerialNumber:
Manufactured: 057K                   CN-04V1MG-XXXXX-XXX-XXXX
ECOLevel: C00                     CM Rev.: A02
FW Rev.: Storage Array Firmware   BootRomVersion: 4.4.6
V10.0.1 (R457114)                 BootRomBuilDate: Mon Jul 28 13:57:41
                                     EDT 2014
_______________________________________________________________________________
SlotID: 1                         Status: secondary
Model: 70-0485(TYPE 19)           BatteryStatus: ok
ProcessorTemperature: 0           ChipsetTemperature: 0
LastBootTime: 2021-04-21:12:04:07 SerialNumber:
Manufactured: 057K                  CN-04V1MG-XXXXX-XXX-XXXX
ECOLevel: C00                     CM Rev.: A02
FW Rev.: Storage Array Firmware   BootRomVersion: 4.4.6
V10.0.1 (R457114)                 BootRomBuilDate: Mon Jul 28 13:57:41
                                     EDT 2014
______________________________ Cache Information ______________________________
CacheMode: write-back Controller-Safe: disabled
Low-Battery-Safe: enabled
_______________________________________________________________________________

Hello, 

 Re: Temp. I suspect that has to do with being booted up on single CM.   If you reboot it again in the future or remove wait 1-2 minutes and re-insert might resolve that.  You can also connect to the secondary CM via serial and reboot it to see if that restores the temp info. 

 Regards, 

Don 

Hello, 

Sorry.  add "controllers" 

member select MEMBERNAME show controllers    You will see 'active' and 'secondary' as their status 

mem sel MEMNAME show controllers
___________________________ Controller Information ____________________________
SlotID: 0

mem sel aer11array-m1 show controllers
___________________________ Controller Information ____________________________
SlotID: 0
Status: active
Model: 70-0477(TYPE 14)
BatteryStatus: ok
ProcessorTemperature: 68
ChipsetTemperature: 27
LastBootTime: 2021-02-21:09:50:38
SerialNumber: XXXXXXXXXXXXXXXX
Manufactured: 02CC
ECOLevel: C00
CM Rev.: A03
FW Rev.: Storage Array Firmware V9.1.2 (R440008)
BootRomVersion: 3.9.1
BootRomBuilDate: Wed Nov 16 14:22:40 EST 2011
_______________________________________________________________________________
_______________________________________________________________________________
SlotID: 1
Status: secondary
Model: 70-0477(TYPE 14)
BatteryStatus: failed
ProcessorTemperature: 0
ChipsetTemperature: 0
LastBootTime: 2021-02-21:09:52:53
SerialNumber: XXXXXXXXXXXXXXX
Manufactured: 02CC
ECOLevel: C00
CM Rev.: A03
FW Rev.: Storage Array Firmware V9.1.2 (R440008)
BootRomVersion: 3.9.1
BootRomBuilDate: Wed Nov 16 14:22:40 EST 2011
_______________________________________________________________________________

______________________________ Cache Information ______________________________
CacheMode: write-back
Controller-Safe: disabled
Low-Battery-Safe: enabled

Hello, 

 The transaction DB is on the RAID not controllers.  If that were bad you would not be able to boot.  It has a self checking mechanism at boot up.  

 Regards, 

Don 

Ok, looks like I'm good then.  Glad there's no lasting worries.  We'll check on the temps thing and look again into the reboot procedure and see if we missed something.

Thanks for all your help, Don!

Glenn

Ok, so other than the temps, anything else to look at to determine if there is a problem in the transaction database ?

Hello, 

 Yup, looks good.   I am glad I could help you. 

  Take care. 

Regards,

Don