XtremIO: How to Resolve and Manage Cluster Physical Capacity alerts (XTR0200302-4, XTR0203792 and XTR0203892) (Customer Correctable)

The following alerts are created when the used physical capacity on an XtremIO array exceeds a predefined threshold or completely runs out:
 

Alert Name	Symptom Code	Description
sys_ud_ssd_space_limited	XTR0200302	Cluster free physical capacity is low. Threshold: more than 85 percent is used
sys_ud_ssd_space_very_limited	XTR0200303	Cluster free physical capacity is critically low. Threshold: more than 90 percent is used
sys_ud_ssd_space_no_free	XTR0200304	The cluster has no free physical capacity
user_physical_capacity_high	XTR0203792	Exceeded SSD high utilization threshold. There are 70%(free_ud_ssd_space)s KB remaining. Note: This can be changed to a user-defined percentage
user_physical_capacity_very_high	XTR0203892	Exceeded SSD very high utilization threshold. There are 80%(free_ud_ssd_space)s KB remaining. Note: This can be changed to a user-defined percentage

There is no current impact on the cluster if any of the above symptom codes are raised, with the exception of symptom code XTR0200304. 

Symptom code XTR0200304 indicates that cluster physical space is fully consumed, in which case the cluster will not accept any more writes and cluster data will become read-only/write-protected from a host perspective. This may cause some hosts to disconnect and/or have read-only access to data.

Note: With deduplication and compression, the array is over-provisioned. Due to these features, the amount of physical data that can be stored on the XtremIO array depends on the cluster's compression and deduplication rates, which can be displayed using the show-clusters-savings XMCLI command or the dashboard information on the XMS GUI

Refer to the Issue section of this article for the cause of each of the listed alerts.

If the listed alerts are being reported for a Disaster Recovery (DR) array utilizing RecoverPoint replication, then refer to KB 494416 - XtremIO running out of free capacity (RecoverPoint DR site) for potential known issues and their resolution.

If the above is not applicable, then free-up physical capacity on the affected XtremIO cluster by doing one or more of the following:

1. Reclaim space
Deleted space should be reclaimed on the host side. For ESX 5.x and Windows hosts, refer to KB# 334268 - Handling deleted space (free block reclamation). For all other host OS types, follow the instructions in the relevant Space Reclamation section of the XtremIO Storage Array Host Configuration Guide

2. Delete / remove unused volumes
Perform an assessment of the volumes actively in use within your XtremIO cluster. Consider deleting / removing any volumes that are no longer needed by external applications. Instructions detailing how to delete / remove unused volumes can be found documented within the Deleting Volumes section of the XtremIO Storage Array User Guide corresponding to the software version of the affected cluster. You can also refer to the Resolution section of KB 468164 - XtremIO: Managing Cluster Capacity in Response to Product Alerts for detailed instructions
Note: Although physical capacity will automatically be reclaimed by XtremIO following volume deletion / removal, deleted space should also be reclaimed on the host side by following the instructions in step 1

3. Offload under-utilized volumes to another platform

4. Perform Online Cluster Expansion (OCE) to add more storage to the array. This can be scheduled by contacting your local account team (SAM, DSM, and ASR) to discuss an action plan to scale out the XtremIO array

Note: To raise an alert with a predefined capacity threshold (ranging from 0 to 100), connect to the XMCLI in the GUI or PuTTy and execute the following command:

xmcli (admin)> modify-alert-definition alert-type="user_physical_capacity_very_high" activity-mode=enabled clearance-mode="ack_required" threshold=<threshold_value>
Modified Alert Definition user_physical_capacity_high

• activity-mode=enabled will raise an alert when the defined threshold is reached. The default for user_physical_capacity alerts is activity-mode=disabled
• clearance-mode=ack_required will cause the alert to disappear when the capacity percentage is below the threshold AND the alert is acknowledged
• threshold=<threshold_value> allows to change the percentage of space to be reached before triggering the alert

To check how the alerts are set up, execute the show-alert-definitions command via XMCLI

Note: The above links can be accessed by registered customers.

XtremIO Family, XtremIO HW Gen2 400GB, XtremIO HW Gen2 400GB Encrypt Capbl, XtremIO HW Gen2 400GB Encrypt Disable, XtremIO HW Gen2 400GB Exp Encrypt Disable, XtremIO HW Gen2 400GB Expandable, XtremIO HW Gen2 800GB Encrypt Capbl , XtremIO HW Gen2 800GB Encrypt Disable, XtremIO HW Gen3 40TB, XtremIO HW Gen3 40TB Encrypt Disable, XtremIO HW X2-R, XtremIO HW X2-R Encrypt Disable, XtremIO HW X2-S, XtremIO HW X2-S Encrypt Disable, XtremIO HW X2-T, XtremIO HW X2-T Encrypt Disable, XtremIO X1, XtremIO X2 ... View More View Less