PowerStore: How to reinitialize a PowerStore system to factory state using the svc_factory_reset command

Before you begin

• This is a destructive procedure. All data and all configuration on the system is permanently deleted and cannot be recovered. The system is returned to a factory state (same state that it was when initially acquired).
• This procedure reinitializes the system to the same PowerStoreOS version installed on the PowerStore.
• For PowerStore X, the process is the same but the vCenter side must be manually cleaned up from all previous PowerStore X configuration before the system can be initialized again (Unregister VASA, remove the vDS, remove the PowerStore Cluster). See PowerStore: How to perform vCenter cleanup if cluster creation failed. ATTENTION: If the VMware license was acquired from Dell and not VMware, contact technical support before starting the procedure.

Procedure

1. Log in to both nodes over ssh with the service account. See PowerStore: How to connect to Service Console over SSH
   The node IPs can be retrieved from the PowerStore Manager (UI) under Settings > Networking > Network IPs
   If the management IPs are not available, see PowerStore: SSH and PowerStore Manager access using the Service LAN Ports

2. Ensure that both nodes are healthy and that there are no hardware issues.
   Run the following command on both nodes and look for errors or faults. If any failure is detected, it must be addressed before proceeding:

   [SVC:service@1234567-X user]$ svc_diag list --hardware --sub_options fault_status Hardware: ========== Fault Status register  =========== Memory       dimm00: OK | dimm01: OK | dimm02: OK | dimm03: OK | dimm04: OK | dimm05: OK | dimm06: OK | dimm07: OK |              dimm08: OK | dimm09: OK | dimm10: OK | dimm11: OK | dimm12: OK | dimm13: OK | dimm14: OK | dimm15: OK |              dimm16: OK | dimm17: OK | dimm18: OK | dimm19: OK | dimm20: OK | dimm21: OK | dimm22: OK | dimm23: OK | EmbeddedDrve Drive00: OK | Drive01: OK | BackEndDrive Drive00: OK | Drive01: OK | Drive02: OK | Drive03: OK | Drive04: OK | Drive05: OK | Drive06: OK | Drive07: OK |              Drive08: OK | Drive09: OK | Drive10: OK | Drive11: OK | Drive12: OK | Drive13: OK | Drive14: OK | Drive15: OK |              Drive16: OK | Drive17: OK | Drive18: OK | Drive19: OK | Drive20: OK | Drive21: OK | Drive22: OK | Drive23: OK |              Drive24: OK | I/O Module   iom00: OK  | iom01: OK  | Mezz         mez00: OK  | mez01: OK  | PSU          psu00: OK  | psu01: OK  | BBU          bbu00: OK  | bbu01: OK  | FAN          fan00: OK  | fan01: OK  | fan02: OK  | fan03: OK  | fan04: OK  | fan05: OK  | fan06: OK  | fan07: OK  |              fan08: OK  | fan09: OK  | fan10: OK  | fan11: OK  | fan12: OK  | fan13: OK  | Root I2C     i2c00: OK  | i2c01: OK  | i2c02: OK  | i2c03: OK  | i2c04: OK  | i2c05: OK  | i2c06: OK  | i2c07: OK  | IO I2C       i2c00: OK  | i2c01: OK  | i2c02: OK  | i2c03: OK  | i2c04: OK  | i2c05: OK  | i2c06: OK  | i2c07: OK  | Other I2C    i2c00: OK  | i2c01: OK  | i2c02: OK  | i2c03: OK  | i2c04: OK  | i2c05: OK  | i2c06: OK  | i2c07: OK  | Mezz I2C     i2c00: OK  | i2c01: OK  | i2c02: OK  | i2c03: OK  | i2c04: OK  | i2c05: OK  | i2c06: OK  | i2c07: OK  | Drive I2C    i2c00: OK  | i2c01: OK  | i2c02: OK  | i2c03: OK  | i2c04: OK  | i2c05: OK  | i2c06: OK  | i2c07: OK  |              i2c08: OK  | i2c09: OK  | i2c10: OK  | i2c11: OK  | i2c12: OK  | i2c13: OK  | i2c14: OK  | i2c15: OK  |              i2c16: OK  | i2c17: OK  | i2c18: OK  | i2c19: OK  | i2c20: OK  | i2c21: OK  | i2c22: OK  | i2c23: OK  |              i2c24: OK  | System       Bit  0  CPU Module: OK              Bit  1  Management Module: OK              Bit  2  Drive I/O Card 0: OK              Bit  3  eFlash : OK              Bit  4  Expansion Bay 0: OK              Bit  5  Enclosure: OK              Bit  6  CMI: OK              Bit  7  All Frus: OK              Bit  8  External: OK              Bit  9  Expansion Bay 1: OK              Bit 10  Drive I/O Card 1: OK

3. Both nodes must be placed in service mode before the system can be reinitialized.
   This requires two commands. The first sets the service mode flag on the node and the second reboots the node into service mode.
   
   On both nodes, run:
   svc_rescue_state set
   svc_node reboot local
    
   If errors are observed, the -f (force) flag may be used.
   The nodes take around 10 to 15 minutes to reboot.
   
   After the reboot, you can confirm that each node is in service mode as it displays this warning:
   [service@1234567-X user]$ 
   *** WARNING *** This Node is currently in Service Mode *** WARNING ***
   
   Note: In PowerStoreOS v3, this warning is not always displayed. Run this command on both the nodes to confirm the service mode status:
   svc_rescue_state list
   
   Example:

   [SVC:service@1234567-X user]$ svc_rescue_state list Current Base OS status is Service Mode, rc = 0. Last backup rescue reason entry: #### Thu Dec 29 14:44:37 UTC 2023 #### : Servicemode Reason: Initialize history file at reinit, start in normal mode. Bootcount indicates the number of boots since the last successful boot. current bootcount = 4, threshold is 3 UI requested Service Mode

4. Once the nodes have rebooted into service mode, login to node A and reinitialize the system.
   In order to start the reinitialization process, run the factory reset command on node A only and follow the command prompt guidance:
   
   svc_factory_reset
   
   If errors are observed, the --force flag may be used.
   
   Example:

   [service@1234567-A user]$ svc_factory_reset    Current CoreOS status is Service Mode, rc = 0. current bootcount = 4 UI requested Service Mode Current CoreOS status is Service Mode, rc = 0. current bootcount = 4 UI requested Service Mode This command will destructively reimage your appliance and return it to its factory settings. Are you sure you want to continue? [y/N]

What to expect:

• All data and all configuration on the system is permanently deleted and cannot be recovered once the svc_factory_reset command is issued and accepted.
• The nodes are rebooted and returned to a blank factory state.
• This process takes around:
  • 1 hour to complete for PowerStore T, more for a downgrade between major versions
  • 1.5 hours for PowerStore X
• Do not attempt to do anything before the minimum expected duration times listed above.
• Once the reinitialization is complete, the system may be initialized again as if brand new. For info on how to initialize a PowerStore system, consult the documentation available in the support pages.