0x100001
|
floor/31
|
0x8a18901f
|
CRITICAL
|
True
|
High Memory Usage
|
Memory usage on this director exceeds the threshold.
|
Either a memory leak has occurred or the VPLEX has exceeded its configuration limits.
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0x100002
|
floor/32
|
0x8a186020
|
ERROR
|
True
|
Env File Update Failure
|
An update to an internal environment file failed.
|
An update to an internal environment file failed.
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0x10001
|
--
|
--
|
CRITICAL
|
True
|
Director Scope Test Alarm
|
TEST: This is an example director scope alarm message.
|
This is an example RCA for the director-scope test alarm.
|
This is an example external remedy for the director-scopetest alarm.
|
DIRECTOR
|
Alarm
|
0x10006
|
--
|
--
|
CRITICAL
|
True
|
Director level Test Operational Alert
|
TEST: This is an example director scope alarm message.
|
This is an example RCA for the director-scope operational test event.
|
This is an example external remedy for the director-scope operational test event.
|
DIRECTOR
|
Operational
|
0x110001
|
ipc/18, ipc/19, ip/2, ip/3
|
0x8a450012, 0x8a450013
|
WARNING
|
True
|
IP Interface State Change
|
The IP port state has changed.
|
Link went down on a port. Depending on the port role, aphysical path to local or remote cluster has been lost.
|
Link went down on a port. Depending on the port role, aphysical path to the local or remote cluster has been lost. Performthe following steps:1. Check the state of the port, and ensure thatit is enabled.2. Check the cable and the SFP, and ensure they areproperly plugged in.3. Check the switch if applicable, and ensureit is operational and the corresponding port is enabled.4. If the link remains down, contact Dell EMC Customer Support.
|
IPPORT
|
Alarm
|
0x20001
|
scsi/156, scsi/157
|
0x8a2d609c, 0x8a2d009d
|
CRITICAL
|
False
|
Array No Access
|
Storage Array is not seen by this director.
|
Storage Array is not reachable by this director.
|
Check for faulty hardware: verify the health of the cables,backend switches and array. Contact Dell EMC Customer Support if theproblem persists
|
ARRAY
|
Alarm
|
0x20002
|
scsi/72, scsi/73
|
0x8a2d0048, 0x8a2d0049
|
ERROR
|
False
|
Unreliable ITNexus Banished
|
The IT nexus has been automatically taken out of service (banished)due to poor reliability in order to prevent performance impact.
|
The Initiator-Target connection is failing and is out ofservice. Once the reliability improves the VPLEX willautomatically resume using it.
|
Check for faulty hardware: verify the health of the cables,backend switches and array. Contact Dell EMC Customer Support if theproblem persists
|
ITNEXUS
|
Alarm
|
0x20003
|
--
|
--
|
CRITICAL
|
False
|
Array Cluster Wide Redundant Access Loss
|
The storage array is not accessible either from n-1 (or) n directors.
|
The storage array is not accessible from all directors in thecluster.
|
Check for faulty hardware: verify the health of the cables,backend switches and array. Contact Dell EMC Customer Support if theproblem persists
|
ARRAY
|
Alarm
|
0x20004
|
scsi/154, scsi/166
|
0x8a2d309A, 0x8a2d00a6
|
WARNING
|
False
|
Unit Busy Condition
|
The logical unit is busy more often than is normal and may impact performance.
|
The array has returned the SCSI BUSY status to VPLEX IO requestsfor this storage-volume more often than what is consideredacceptable.
|
The cause of busy condition on the storage array should beinvestigated. Investigate the health of the storage array, backendfabric and VPLEX backend port to determine the source of the issue.Contact Dell EMC Customer Support if there are performance problems.
|
LOGICALUNIT
|
Alarm
|
0x20005
|
scsi/126
|
0x8a2d307e
|
WARNING
|
False
|
Unit Alua Support Inconsistent
|
ALUA support is incorrectly configured on LU.
|
An inconsistent ALUA support level has been detected for alogical unit.
|
Check the ALUA configuration on the array. All paths to alogical unit should be configured with the same failover mode.
|
LOGICALUNIT
|
Alarm
|
0x20006
|
scsi/71
|
0x8a2d6047
|
ERROR
|
False
|
Unit Path Type Inconsistent
|
A Logical Unit reported an inconsistent path type for the array.Recovery attempts failed and the unit has been banished.
|
A logical unit has inconsistent path types. Exhausted attempts tocorrect this via refresh and have banished the unit.
|
Engage the array vendor or Dell EMC Customer Support toinvestigate why the logical unit has inconsistent path types.
|
LOGICALUNIT
|
Alarm
|
0x20007
|
scsi/79
|
0x8a2d304f
|
WARNING
|
False
|
Array Not Supported SPC Version
|
Array supports an SPC version NOT matching 2, 3 or 4.
|
Target device advertised a behavior which is not supported by theVPLEX initiator.
|
Switch the target device into a supported mode following VPLEXbest practices for this device, do not use this target device, or thedevice may be less highly available than is recommended for normaloperation.
|
ARRAY
|
Alarm
|
0x20008
|
scsi/91
|
0x8a2d605b
|
ERROR
|
False
|
Array Exceeds Max Controller
|
Array exceeds maximum number of controllers allowed.
|
Logical Unit already has maximum controllers allowed on this array.Extra controller is being attempted to be added.
|
Check Array configuration. If problem persists, contact Dell EMC Customer Support.
|
ARRAY
|
Alarm
|
0x2000a
|
scsi/158
|
0x8a2d309e
|
WARNING
|
False
|
Array Serial Number Unexpected
|
Storage Array returned an unexpected serial number.
|
The storage array reported an unexpected serial number.
|
Before using this array, check the storage-arrays context toensure that multiple entries do not have the same identifier,and that this array does not present multiple identifiers.If either of these conditions occur, do not use this array.The array name and version number should be reported to Dell EMCCustomer support. Continuing to use the array could lead to DataUnavailability and Data Loss conditions and unreliable arraybehavior.
|
ARRAY
|
Alarm
|
0x2000b
|
scsi/147
|
0x8a2d6093
|
ERROR
|
False
|
Logical Unit Changed
|
Logical Unit mapping change detected. For this ITL, a differentLogical Unit is reported than what was reported earlier.
|
LU mapping on the storage array had changed. Depending onthe state of the system (host IOs running or not), either DataUnavailable, Data Loss or Data Corruption might have alreadyhappened or might happen in future.
|
The system will automatically run a refresh to destroy the staleITL to the old LUID and re-discover the ITL to the new LUID.Always follow the 'Remove Disk or Array' procedure in theVPLEX SolVe Desktop when de-provisioning storage-volumes and/orre-provisioning storage-volumes to VPLEX to ensure an'array re-discover' is run in between changes in order to preventLUN swapping from occurring.
|
ITLNEXUS
|
Alarm
|
0x2000c
|
apf/15
|
0x8a04900f
|
ERROR
|
False
|
Array No Failover Executor Director
|
No suitable executor director to perform the failover of a groupof Logical Units to a specific array controller.
|
This is likely indicative of severely degraded hardwareand/or fabric condition. Restore connectivity to the arraycontrollers for all directors.
|
Check the backend switches, and engage the array vendor to investigate why the failover to the array controller could not succeed. Contact Dell EMC Customer Support if problem persists.
|
ARRAY
|
Alarm
|
0x2000d
|
scsi/138
|
0x8a2d608A
|
ERROR
|
False
|
Unit Reservation Conflict
|
Reservation conflict response for command sent to the logical uniton a specific IT.
|
The logical unit is reserved by another initiator.
|
Contact your storage-administrator to remove the reservation of the logical unit from either the initiator(s) or storage array.
|
LOGICALUNIT
|
Alarm
|
0x2000e
|
scsi/167
|
0x8a2d60a7
|
ERROR
|
False
|
Scsi Deferred IO Error
|
Received deferred error from a specific target for a previous command.
|
VPLEX received deferred error from the specified target whichindicates exception condition occured on the target duringprocessing of some previous command for which status was alreadyreturned. This can lead to Data Unavailability/Data Loss.
|
Determine which backend storage array the target corresponds to, and then engage the storage array vendor's Customer Support todetermine the cause of the deferred error and to correct theexception condition on the target.
|
TARGET
|
Alarm
|
0x2000f
|
scsi/161
|
0x8a2d30a1
|
ERROR
|
False
|
Inquiry Not Ready
|
INQUIRY data not yet ready. Skipping discovery on itlNexus.
|
An array returned empty INQUIRY data. Device not ready for discovery.
|
Run array re-discover for this array. New LUNs will now be listed.
|
LOGICALUNIT
|
Operational
|
0x20010
|
scsi/162
|
0x8a2d30a2
|
WARNING
|
False
|
VPD Parameters Outside Scope
|
The array did not return a VPD peripheral qualifier status ofConnected. Device may be not ready for discovery.
|
The array did not return a VPD peripheral qualifier status of 'Connected'. Device may be not ready for discovery.
|
Run array re-discover for this array. New LUNs will now be listed. If this event is persistent, engage the array support team to investigate why it did not return a VPD peripheral qualifier status of 'Connected' in the Inquiry Response.
|
LOGICALUNIT
|
Operational
|
0x20011
|
scsi/122
|
0x8a2d607a
|
ERROR
|
False
|
New Device Type Reported
|
New device type reported for LU type was oldPdType,now reported as newPdType.
|
Unexpected change in device type for a back end Logical Unit.
|
Verify if the associated storage-volume is operational on theVP LEX, and investigate the cause of the unexpected change in type from the storage array. If the issue persists and unable to determine the cause engage Dell EMC Customer Service.
|
LOGICALUNIT
|
Operational
|
0x20012
|
scsi/70
|
0x8a2d6046
|
ERROR
|
True
|
Scsi Memory Alloc Failure
|
Memory allocation failure of object in function.
|
Firmware memory allocation failure. Running out of memory resources.
|
Run the collect-diagnostics utility to collect system information to determine why an internal firmware memory allocation failure occurred. Please contact Dell EMC Customer Support.
|
TARGET
|
Operational
|
0x20013
|
scsi/169
|
0x8a2d90a9
|
CRITICAL
|
True
|
Scsi Thin Volume No Space On Write
|
Space allocation failed on scsi vol.
|
Allocation has failed on the thin-enabled storage-volume. There are no more available storage blocks on the BE array to map to the address to which the host has issued a write command.
|
The host administrator can try reclaiming storage using the scsi UNMAP command from the host. If reclaimed storage is notsufficient, the storage administrator must add free block storage to the BE storage array to increase the space available to the thin-enabled storage-volume. Once additional space is available host administrators can re-start the hosts that are suspended/stopped.
|
TARGET
|
Operational
|
0x20014
|
scsi/36
|
0x8a2d0024
|
INFO
|
False
|
ITNexus Ready
|
IT Nexus is ready.
|
The scsi layer found an IT Nexus connection indicated by the tuple.
|
Contact Dell EMC Customer Support.
|
ARRAY
|
Operational
|
0x20015
|
scsi/68
|
0x8a2d3044
|
WARNING
|
False
|
Renew Retry Exceeded
|
Report luns command failing to get logical unit inventory onan IT Nexus.
|
The repeated failure could be due to any one of the following:1. The director was unable to allocate any more memory,2. Attempts to get logical unit inventory were unable to complete.3. The REPORTLUNS command failed as there is no LUNID 0 provisioned to the VPLEX on the path from a Hitachi, HP, SUN or HDS array.
|
Check if the array is a Hitachi, HP, SUN or HDS array, and if it is verify that each path from the array has a LUNID 0 exported to VPLEX. Reference the 'Configure Arrays' procedure in the VPLEX SolVe Desktop. Investigate the health of the BE array. If the issue persists contact Dell EMC Customer Service.
|
ARRAY
|
Operational
|
0x20016
|
scsi/146
|
0x8a203092
|
WARNING
|
False
|
Path Count Above Limit
|
The number of active paths on this director for the specified LogicalUnit is above the recommended limit.
|
The number of active paths on this director for the specified Logical Unit is above the recommended limit.
|
Reduce the number of paths to the recommended limit. Reference the VPLEX Best Practices document, available via the VPLEX SolVe Desktop.
|
LOGICALUNIT
|
Operational
|
0x20017
|
scsi/43, scsi/44
|
0x8a2d302b, 0x8a2d302c
|
WARNING
|
False
|
Port link is down
|
Port link is down.
|
The link is down between the specified VPLEX BE port and the switch.
|
Check the SFP, cable, and switch attached to this back-end port, especially anything that has been changed recently. Reseat/clean/replace the hardware as needed to resolve the problem.If the problem persists and unable to determine the cause contact Dell EMC Customer Support.
|
INTERFACE
|
Alarm
|
0x20018
|
scsi/93
|
0x8a2d305d
|
WARNING
|
False
|
Single Path For LU
|
Path to a backend storage volume of an array is lost. VPLEX directorhas only one path to the LUN on the backend storage device.
|
VPLEX lost a path to a LUN from one director to an array. There is one path remaining. There might be a faulty hardware (cable,backend switch, array).
|
Verify reported array's BE disk health, LUN masking, array configuration and physical connection. If problem persists, contact Dell EMC Customer Support.
|
ARRAY
|
Operational
|
0x20019
|
scsi/80
|
0x8a2d9050
|
CRITICAL
|
False
|
Array Dev Type Not Interoperable
|
Storage Array has registered a Peripheral Device Type which is notinteroperable with VPLEX.
|
The target device advertised a behavior which doesn't work with the VPLEX initiator.
|
Follow the 'Configure Arrays' procedure in the VPLEX SolVe Desktop to ensure the storage is provisioned in a manner supported by VPLEX.
|
ARRAY
|
Operational
|
0x2001a
|
scsi/164
|
0x8a2d30a4
|
WARNING
|
False
|
Inquiry Peripheral Invalid
|
The array did not return a STD INQ peripheral qualifier of Connected.Device may be not ready for discovery.
|
The SCSI target device may be in the Unavailable state or not be capable of accessing the addressed logical unit from the addressed SCSI target port.
|
Run array re-discover for this array. New LUNs will now be listed. If this event is persistent, engage the array support team toinvestigate why the array did not return a STD INQ peripheral qualifier of 'Connected'.
|
ARRAY
|
Operational
|
0x2001b
|
scsi/170
|
0x8a2d30aa
|
WARNING
|
True
|
Thin Threshold Reached
|
A storage volume reported a Thin Provisioning Soft Threshold Reachederror on a VPLEX write.
|
An array returned Unit Attention 6/38/07h THIN_PROVISIONING_SOFT_THRESHOLD_REACHED for a storage-volume on a VPLEX write. The thin pool on the array is running out of space.
|
Add additional block resources to the thin pool on the array from which the storage-volume is provisioned.
|
LOGICALUNIT
|
Operational
|
0x2001c
|
scsi/174
|
0x8a2d30ae
|
WARNING
|
False
|
Report Luns Retry Exhausted
|
Retry limit on successfully processing REPORT LUNS SCSI command responseexceeded on an IT Nexus. VPLEX proceeds with processing as many LUs as it can.
|
Failure to successfully process REPORT LUNS SCSI command response could be be due to one of the following: 1. The array is reporting that it has a greater number of logical units than what VPLEX requested. The number does not match what the array actually transferred in the response data buffer, and does not match what the array actually has in its masking-view/storage-group for VPLEX. 2. The array is reporting that it has fewer LUs than what VPLEX requested. The number does not match what the array actually transferred in the response data buffer, and does not match what the array actually has in its masking-view/storage-group for VPLEX.
|
The retry limit is exhausted, VPLEX is proceeding with processing as many LUs as it can. If all LUs, from the array masking-view/storage-group from VPLEX, are not discovered, perform arrayre-discover. Collect VPLEX and Array logs and traces and contact Dell EMC Customer support.
|
ARRAY
|
Operational
|
0x2001d
|
scsi/171, scsi/172, scsi/173
|
0x8a2d30ab, 0x8a2d30ac, 0x8a2d30ad
|
WARNING
|
False
|
Report Luns Data Mismatch
|
A specific IT Nexus reported specific number of logical units whichdid not match the total number of logical units.
|
The Logical unit inventory reported from an array is invalid. There is a mismatch between number of LUs that the array is reporting and the number of LUs that the VPLEX requested.
|
VPLEX will retry getting logical unit inventory. Collect VPLEX and Array logs and traces and contact Dell EMC Customer support.
|
ARRAY
|
Operational
|
0x30003
|
--
|
--
|
CRITICAL
|
True
|
Virtual Volume Redundancy Loss
|
Virtual Volume redundancy has changed.
|
One or more factors have contributed to changing the redundancy ofthe given virtual volume.
|
Contact Dell EMC Customer Support for assistance with restoring theredundancy of the virtual volume.
|
VIRTUALVOLUME
|
Alarm
|
0x30004
|
amf/45, amf/96, amf/97, amf/98, amf/99, amf/100, amf/101, amf/223
|
0x8a02302d, 0x8a029060, 0x8a029061, 0x8a026062, 0x8a026063, 0x8a020064, 0x8a020065, 0x8a0230df
|
WARNING
|
True
|
Storage Volume Unreachable
|
Storage Volume accessibility has changed.
|
A storage volume attached to the system can no longer service I/O.
|
If the storage volume is unreachable seeTroubleshooting_Unreachable_Storage_Volumes.
|
STORAGEVOLUME
|
Alarm
|
0x30005
|
--
|
--
|
CRITICAL
|
True
|
System Device Redundancy Loss
|
The system device has regained full redundancy.
|
One or more factors have contributed to changing the redundancy ofthe given system device.
|
Contact Dell EMC Customer Support for assistance with restoring theredundancy of the system device.
|
METAVOLUME
|
Alarm
|
0x30006
|
--
|
--
|
CRITICAL
|
True
|
Virtual Volume Suspended
|
Virtual volume has been suspended for more than 10 seconds.
|
A suspension of I/O applied to a virtual volume has lasted longerthan a preconfigured threshold.
|
Contact Dell EMC Customer Support for analysis of the factors contributing to the extended suspension.
|
VIRTUALVOLUME
|
Alarm
|
0x30007
|
amf/20, amf/24
|
0x8a020014 0x8a020018
|
INFO
|
False
|
Device Rebuild On going
|
The given device is undergoing a rebuild on one or more of its mirrors.
|
Rebuilding work has begun on the given device in order to restore it to full redundancy.
|
No user action is required.
|
DEVICE
|
Alarm
|
0x30009
|
amf/126
|
0x8a02307e
|
WARNING
|
True
|
Logging Volume Write Failed
|
Write failed to logging volume. The mirror will be marked out of date.
|
The system has lost access to the logging-volume device on which itwas maintaining a list of changes to the reported distributed device.The specified mirror of the distributed device will be marked out ofdate and completely rebuilt when possible.
|
Check the accessibility of the logging-volume component and check theaccessibility of the underlying storage-volume of the mirror that ismarked out of date. Take corrective action, if necessary, toreconnect the inaccessible storage-volumes. Contact Dell EMC CustomerSupport for assistance.
|
LOGGINGVOLUME
|
Operational
|
0x3000a
|
amf/146
|
0x8a029092
|
CRITICAL
|
True
|
All Mirrors Out of Date
|
All mirrors of the distributed device are out of date, choosingone mirror as up to date to allow access to the device.
|
All of the mirrors of the specified distributed device had beenmarked out of date. In order to restore access to the device, and tominimize data loss, the specified mirror was marked as up to date.
|
Contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x3000b
|
amf/111
|
0x8a02606f
|
ERROR
|
True
|
Device Name Conflict
|
Name conflict detected between two devices, renaming the secondoccurrence.
|
Two discovered devices reported the same device name. This can occurif configuration changes are made when one or more storage volumesare unreachable, if storage devices from separate VPLEX systems withexisting devices are merged into one VPLEX system or if a devicebecomes visible to both clusters and happens to have the same name asa cluster-local device on the other cluster.
|
Manual intervention is required. Contact Dell EMC Customer Supportto resolve the conflict.
|
DEVICE
|
Operational
|
0x3000c
|
amf/181
|
0x8a0260b5
|
ERROR
|
True
|
Virtual Volume Name Conflict
|
Name conflict detected between two virtual volumes, renaming thesecond occurrence.
|
Two discovered virtual volumes reported the same name. This can occurif configuration changes are made when one or more storage volumesare unreachable, if storage devices from separate VPLEX systems withexisting devices are merged into one VPLEX system or if a virtualvolume becomes visible to both clusters and happens to have the samename as a cluster-local virtual volume on the other cluster.
|
Manual intervention is required. Contact Dell EMC Customer Supportto resolve the conflict.
|
VIRTUALVOLUME
|
Operational
|
0x3000e
|
amf/158
|
0x8a02609e
|
ERROR
|
True
|
Hide Storage Volume Provisioned To Other Cluster
|
Hiding storage volume at the local cluster as this storage volumeis also provisioned at the remote cluster.
|
A storage volume has been provisioned to this cluster, even thoughthe storage volume logically belongs to the other cluster. Thismessage likely indicates a back-end zoning or masking problem,because a storage volume should be provisioned to only one cluster.
|
Reconfigure the zoning and/or masking on the back-end so that eachcluster can see only its local storage volumes. Run the'array re-discover' command to remove the remote storage volumes.
|
STORAGEVOLUME
|
Operational
|
0x3000f
|
amf/221
|
0x8a0290dd
|
CRITICAL
|
True
|
Storage Volume Claimed From Multiple
|
Storage volume is claimed at both the local and remote clusters.
|
The same storage volume has been not only presented to both clusters,but claimed at both clusters as well. If more than one cluster isdoing I/O to the storage volume, data corruption is extremely likely.
|
1. Choose one cluster at which the storage volume should be used.2. Tear down all configurations involving the storage volume at theother cluster.3. Remove visibility to the storage volume from the other cluster.
|
STORAGEVOLUME
|
Operational
|
0x30010
|
amf/141
|
0x8a02308d
|
WARNING
|
True
|
Mirror Marked Out Of Date
|
A mirror of a raid-1 device has been marked fully out of date.
|
A write to a raid-1 device was successful to some mirrors, but notthe one in question. That mirror is being marked as fully out ofdate, so that a subsequent rebuild can bring the mirror back up todate.
|
If the mirror is not still showing as out of date, it has likelyauto-corrected. If it is still marked out of date, investigatethe backend issues. Once they have been resolved, the rebuildshould initiate automatically. This could take time dependingon the number of active rebuilds on the system at the time.Contact Dell EMC Customer Support if the condition persists.
|
METAVOLUME
|
Operational
|
0x30011
|
amf/197
|
0x8a0290c5
|
CRITICAL
|
True
|
Metadata Volume Write Failed
|
A write to a metadata volume was unsuccessful.
|
A write to the specified metadata volume has failed. The changes arepreserved in memory, but if the entire cluster fails or is shut downbefore the access to the metadata volume is restored and the changescan be written successfully to disk, the changes will be lost. Thesystem configuration information associated with those metadatawrites not written to the disk may be lost.
|
1. Fix the unhealthy or failed metadata volume, or underlyingstorage volumes by checking fabric connectivity and the storagearray(s).2. If the metadata volume cannot be restored to an 'ok' state,create and move to a new metadata volume as soon as possible.
|
METAVOLUME
|
Operational
|
0x30012
|
amf/233
|
0x8a0230e9
|
WARNING
|
True
|
Metadata Volume Becoming Full
|
A metadata volume has reached a preconfigured percentage of itscapacity.
|
The metadata volume is running out of available space. This eventdoes not indicate an immediate metadata volume failure.
|
Refer to the troubleshooting entry for the issue in the Generator.
|
METAVOLUME
|
Operational
|
0x30013
|
amf/251
|
0x8a0230fb
|
WARNING
|
True
|
Storage Volume Latency Events Suppressed
|
Generation of storage volume I/O latency events stopped to preventevent flooding.
|
Generation of I/O latency events has been stopped after emitting themaximum allowed number of events, to prevent event flooding.
|
1. Use the VPLEX Unisphere performance monitoring stats to verify ifthere is still high average I/O latency on the backend.2. Create storage-volume performance monitors in Vplexcli toinvestigate individual storage-volume latency stats as needed tofurther investigate the cause of the performance degradation.3. Compare the storage-volume latency stats to the latency on thestorage array for the volume(s) in question. If the latency onthe array isn't as high investigate the fabrics between thestorage array and VPLEX.4. If the issue persists and unable to determine the cause engageDELL EMC Customer Service.
|
CLUSTER
|
Operational
|
0x30014
|
amf/267
|
|
CRITICAL
|
True
|
Device Detach Full Rebuild
|
A complex sequence of failures has led to marking one mirror ofa distributed device fully out of date, and to the temporary suspensionof I/O on that distributed device.
|
A combination of failures involving metadata and disks offline atcluster followed by cluster partition and failed writes to alogging volume has led to temporary condition where the distributeddevice can no longer process I/O. Once this condition is resolved,a full rebuild will occur. This condition is necessary to avoid datacorruption.
|
A full rebuild will be started automatically once the clustersre-join. Investigate why the clusters are partitioned and take anyrequired actions to restore the WAN COM connectivity so the clusterscan re-join.
|
DISTRIBUTEDDEVICE
|
Operational
|
0x30015
|
amf/34, amf/35, amf/51, amf/52, amf/53, amf/54, amf/55, amf/56,amf/57, amf/58, amf/59, amf/60, amf/61, amf/62, amf/63, amf/64, amf/69, amf/70,amf/71, amf/72, amf/73, amf/75, amf/76, amf/77, amf/78, amf/79, amf/80, amf/81,amf/82, amf/91, amf/92, amf/122, amf/123, amf/230
|
0x8a029022, 0x8a029023, 0x8a029033, 0x8a029034, 0x8a029035, 0x8a029036, 0x8a029037, 0x8a029038, 0x8a029039, 0x8a02903a, 0x8a02903b, 0x8a02903c, 0x8a02903d, 0x8a02903e, 0x8a02903f, 0x8a029040, 0x8a029045, 0x8a029046, 0x8a029047, 0x8a029048, 0x8a029049, 0x8a02904b, 0x8a02904c, 0x8a02904d, 0x8a02904e, 0x8a02904f, 0x8a029050, 0x8a029051, 0x8a029052, 0x8a02905c, 0x8a02907a, 0x8a02907b, 0x8a0290e6
|
CRITICAL
|
True
|
Device Bad Config
|
Metadata persisted to the metadata volume relating to the deviceor storage volume in question has been detected to be inconsistent. Accessto the device or storage volume has been disabled until the problem canbe examined and remedied.
|
VPLEX has detected an inconsistency in the persisted informationrelating to the configuration of the given device or disk. This mayindicate a problem with the persisted information, or may simply bethe result of a timing issue upon cluster or system bringup.
|
Contact Dell EMC Customer Support for analysis and remedy of theproblem.
|
DEVICE
|
Alarm
|
0x30016
|
amf/215, amf/216
|
0x8a0290d7, 0x8a0290d8
|
CRITICAL
|
True
|
Virtual Volume Capacity Shrunk
|
The capacity of a virtual volume has shrunk below the capacity withwhich it was created.
|
The capacity of a virtual volume has shrunk below the capacity withwhich it was created.
|
1. Examine the back-end arrays to determine why the storage hasshrunk.2. Resize the required back-end devices to their original size.3. Contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Alarm
|
0x30017
|
amf/215, amf/216
|
0x8a0290d7, 0x8a0290d8
|
CRITICAL
|
True
|
System Device Capacity Shrunk
|
The capacity of a system device (metadata or logging) has shrunkbelow the capacity with which it was created.
|
The capacity of a system device (metadata or logging) has shrunkbelow the capacity with which it was created.
|
1. Examine the back-end arrays to determine why the storage hasshrunk.2. Resize the required back-end devices to their original size.3. Contact Dell EMC Customer Support.
|
METAVOLUME
|
Alarm
|
0x30018
|
amf/162
|
0x8a0290a2
|
CRITICAL
|
True
|
Active Metadata Volume Unhealthy
|
The active metadata volume has become unhealthy and is at risk.
|
The active meta-volume has become unhealthy and is at risk. It isin cache only, and needs to be written to storage volume.
|
The active meta-volume has become unhealthy. Either resolve theproblem on the back end, or create another meta-volume using'meta-volume create' as soon as possible, and then run the'meta-volume move' command to save the cache data to the newlycreated meta-volume. Contact Dell EMC Customer Support.
|
STORAGEVOLUME
|
Alarm
|
0x30019
|
amf/226
|
0x8a0230e2
|
WARNING
|
True
|
Active Metadata Volume Missing
|
Timed out waiting for the active metadata volume to arrive.
|
A fixed time has passed after the local cluster last booted, and theactive metadata volume has not yet been detected.
|
Fix the problem with the active metadata volume, or activate a backupif no configuration changes have taken place since the backup. Ifthere have been configuration changes, contact Dell EMC CustomerSupport to restore from the backup.
|
METAVOLUME
|
Alarm
|
0x3001a
|
amf/249, amf/250
|
0x8a0230f9, 0x8a0230fa
|
WARNING
|
True
|
Storage Volume Latency Degraded
|
Storage volume I/O latency has increased above an acceptable threshold.
|
The average I/O latency on a storage volume has exceeded theacceptable limit, likely due to increased latency on the backendstorage array or fabrics between the VPLEX and storage array.
|
1. Analyze the latency stats from the storage array for the volume inquestion at the time of the event to determine if it reported thesame high latency. Engage the storage array vendor as needed.2. If the storage array did not report the same latency at the timeof the issue for the volume then investigate the fabrics between theVPLEX and storage array.3. If the issue persists and the cause cannotbe determined engage DELL EMC Customer Service.
|
STORAGEVOLUME
|
Alarm
|
0x3001b
|
amf/250, amf/270
|
0x8a0230fa
|
WARNING
|
True
|
Remote Device Latency Degraded
|
Remote device I/O latency has increased above an acceptable threshold.
|
The average I/O latency on a remote device has exceeded theacceptable limit due to possible issues on the back-end or WAN link.
|
1. Use the VPLEX Unisphere performance monitoring stats to verify ifthere is high average I/O latency on the WAN COM links, or highaverage backend I/O latency on the cluster where the remote deviceresides, and investigate further as needed.2. Create storage-volume performance monitors in Vplexcli toinvestigate individual storage-volume latency stats as needed tofurther investigate the cause of the performance degradation.3. Compare the storage-volume latency stats to the latency on thestorage array for the volume(s) in question. If the latency on thearray isn't as high investigate the fabrics between the storage arrayand VPLEX.4. If the issue persists and the cause cannot be determined engageDELL EMC Customer Service.
|
DISTRIBUTEDDEVICE
|
Alarm
|
0x3001c
|
amf/244, amf/245, amf/246
|
0x8a0290f4, 0x8a0290f5, 0x8a0290f6
|
CRITICAL
|
True
|
Mirror Isolated
|
A mirror of a raid-1 device has been isolated due to severe performancedegradation of its storage volume components.
|
All of the storage volumes supporting this mirror are performing verypoorly, causing severe degradation in the RAID-1 performance. Toimprove the RAID-1 performance through the healthy legs, the IOs tothe poorly performing mirror leg have been blocked.
|
Determine the cause of the storage volume's poor performance byreferring to the troubleshooting entry related to degraded disks andisolated mirrors in the normal operation section of SolVe ProcedureGenerator ("Recovery -> Troubleshooting -> Problems during normaloperation -> Degraded disks and isolated mirrors problems"). If theunderlying issues cannot be fixed in a timely manner, considerdetaching the unhealthy mirror from the RAID-1 device and attaching amirror based on healthy storage volume(s) to reinstate redundancy.Contact Dell EMC Customer Support if the problem persists.
|
METAVOLUME
|
Alarm
|
0x3001d
|
amf/203
|
0x8a0200cb
|
INFO
|
False
|
Metadata Copy Succeeded
|
Successfully copied in-memory metadata to a metadata volume.
|
All in-memory metadata has been written out to the given metadata volume.
|
This is an informational event only. No action is required.
|
METAVOLUME
|
Operational
|
0x3001e
|
amf/201, amf/206, amf/219
|
0x8a0290c9, 0x8a0290ce, 0x8a0290db
|
CRITICAL
|
False
|
Metadata Copy Failed
|
Failed to copy in-memory metadata to a metadata volume.
|
An attempt to write out all in-memory metadata to a metadata volume failed.
|
Examine the metadata volume to see if there are any problems that can be corrected, contacting Dell EMC Customer Service for assistance if needed.
|
METAVOLUME
|
Operational
|
0x3001f
|
amf/205
|
0x8a0200cd
|
INFO
|
False
|
Metadata Move Succeeded
|
Successfully copied in-memory metadata to a metadata volume, which is now the active metadata volume.
|
All in-memory metadata has been written out to the given metadata volume, which is now the active metadata volume.
|
This is an informational event only. No action is required.
|
METAVOLUME
|
Operational
|
0x30020
|
amf/202, amf/207, amf/220
|
0x8a0290ca, 0x8a0290cf, 0x8a0290dc
|
CRITICAL
|
False
|
Metadata Move Failed
|
Failed to copy in-memory metadata to a metadata volume, and therefore the metadata volume has not been activated.
|
An attempt to write out all in-memory metadata to a metadata volume failed, and therefore that metadata volume has not been activated.
|
Examine the metadata volume to see if there are any problems that can be corrected, contacting Dell EMC Customer Service for assistance if needed.
|
METAVOLUME
|
Operational
|
0x30021
|
|
|
WARNING
|
False
|
Bitmap Log At Expansion Limit
|
An internal limit relating to expansion of a distributed devicehas been reached.
|
A distributed device has reached an expansion limit internal to VPLEX. Further attempts to expand this device will fail without corrective action.
|
Logging for the distributed device must be recreated. Contact Dell EMC Customer Service for assistance.
|
DISTRIBUTEDDEVICE
|
Operational
|
0x40001
|
com/11
|
0x8a0a300b
|
WARNING
|
True
|
Other Director Has Different Sw Version
|
Director running a different version of the software detected.
|
Local directors are running different versions of software.
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0x40002
|
com/40
|
0x8a0a3028
|
WARNING
|
True
|
Fewer Active Paths Than Expected
|
The number of COM paths to remote director is smaller than expected in a standard configuration.
|
Fewer than expected paths to remote director.
|
Check the state of the COM port, making sure it is enabled. Check the cable, making sure it is properly plugged in. Check the switch if applicable, making sure it is operational and the corresponding port is enabled. Contact Dell EMC Customer Support if this event persists.
|
DIRECTOR
|
Operational
|
0x40003
|
com/52
|
0x8a0a6034
|
ERROR
|
True
|
Crc Status Initialization Failed
|
Th e system failed to read an internal setting.
|
The system failed to read an internal setting. This will prevent thedirector from fully booting, thereby preventing it from processing I/O
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0x60001
|
nmg/49, nmg/50, nmg/59
|
0x8a260031, 0x8a266032, 0x8a26303b
|
ERROR
|
True
|
Cluster Partition
|
Cluster has partitioned.
|
The last director with the given cluster id has departed.This marks the loss of the indicated cluster from the point of viewof the reporting cluster.
|
The last director with the given cluster ID has departed. This marks the loss of the indicated cluster from the point of view of the reporting cluster. Check that the distributed-devices and consistency-groups are running on the winning cluster. A winner may need to be manually chosen. 1. Check for problems with the network link to the indicated cluster. 2. Check the equipment at the indicated cluster for malfunctions.
|
CLUSTER
|
Alarm
|
0x60002
|
nmg/100
|
0x8a266064
|
ERROR
|
True
|
Witness Com Node Untrustworthy
|
'Membership com connection to {remoteDirector}/{remoteDirectorId}untrustworthy: {qualifier} operational for {secondsSinceOperational}seconds.'
|
This failure exposes the cluster to a Data Unavailability condition in certain director failure/inter-director link failure scenarios.
|
Check the management network cables and the corresponding managementmodules.
|
DIRECTOR
|
Operational
|
0x60003
|
nmg/56, nmg/57
|
0x8a269038, 0x8a269039
|
CRITICAL
|
True
|
Stonith Call Failed for remote director
|
Stonith call failed for {remoteDirector}/{remoteDirectorId}
|
This director tried to kill the dentified director, but the call to do that has failed.
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0x60004
|
nmg/64, nmg/65, nmg/66, nmg/67
|
0x8a266040, 0x8a266041, 0x8a266042, 0x8a266043
|
ERROR
|
True
|
Director Running Different Firmware
|
Director running a different version of the software detected.
|
A director with a different version of firmware may have beeninserted into the cluster. Also a communications link could havebeen brought up by mistake, allowing a cluster of a differentversion to be visible.
|
If this situation arose because a director with a different versionof firmware was inserted into the cluster and booted, then shut downthat director. If the situation arose because a communications linkhas been brought up by mistake, then take down that communicationslink.
|
CLUSTER
|
Operational
|
0x60005
|
nmg/96
|
0x8a263060
|
WARNING
|
True
|
Witness Com Node From Foreign Site
|
Unexpected membership arrival uuid {directorId} appears from foreign cluster {foreignClusterId}.
|
A director from a foreign VPLEX cluster was unexpectedly discovered by this cluster.
|
Contact Dell EMC Customer Support to check if there has been amis-configuration done between clusters in the environment
|
CLUSTER
|
Operational
|
0x70001
|
nmg/107, nmg/108, nmg/109
|
0x8a26006b, 0x8a26006c, 0x8a26006d
|
WARNING
|
False
|
Cluster Witness Disabled
|
Cluster Witness is disabled.
|
This event is generated when Cluster Witness is administrativelydisabled.
|
If the Cluster Witness was disabled in error, re-enable it
|
CLUSTER
|
Alarm
|
0x70002
|
nmg/105, nmg/106
|
0x8a269069, 0x8a26906a
|
ERROR
|
True
|
Cluster Witness Server Connection Lost
|
Cluster Witness Server Connection Lost
|
The cluster reporting this event has been unable to establishcommunication with Cluster Witness Server. This may be due to the failure of the server or loss of network connectivity.
|
Check network connectivity between the local cluster and ClusterWitness Server. Check whether Cluster Witness Server VM is running.If connectivity is lost from both clusters, disable the Cluster Witness Server via VPLEXCLI until connectivity can be restored inorder to prevent data unavailability on cluster partition. If the Problem persists, contact Dell EMC Customer Support.
|
CLUSTER
|
Alarm
|
0x70003
|
nmg/112, nmg/117
|
0x8a263070, 0x8a260075
|
WARNING
|
False
|
CW Cluster Partition Guidance
|
Communication between clusters is broken.
|
Cluster Witness Server has detected and reported an inter-cluster partition. This marks the loss of connectivity between the remote cluster and the reporting cluster. This may be due to physicalfailure or congestion of the inter-cluster network.
|
If the Cluster Witness Server is present and enabled, it should have provided guidance to continue IO on the winning cluster. If theCluster Witness Server is not present, check that the distributed devices and consistency groups are running on the winning cluster.If the problem persists, contact Dell EMC Customer Support.
|
CLUSTER
|
Alarm
|
0x70004
|
nmg/113, nmg/117
|
0x8a263071, 0x8a260075
|
WARNING
|
False
|
CW Remote Cluster Failure Or Isolation Guidance
|
CW Remote Cluster Failure Or Isolation Guidance
|
Cluster Witness Server detected that the remote cluster has eitherfailed or become isolated. This could be due to site disaster or due to dual failure of inter-cluster and management networks.
|
Verify the state of the remote cluster. Also, check the state ofinter-cluster network as well as the management network connectingthe remote cluster to Cluster Witness Server. If the problempersists, contact Dell EMC Customer Support.
|
CLUSTER
|
Alarm
|
0x70005
|
nmg/116, nmg/117
|
0x8a260074, 0x8a260075
|
ERROR
|
True
|
CW Cluster Isolation Guidance Or No Guidance
|
CW Cluster Isolation Guidance Or No Guidance
|
The cluster reporting this event has been unable to receive any guidance from the Cluster Witness Server for the last 10 seconds.This may be due to failure of the Cluster Witness Server or loss of network connectivity.
|
A winner needs to be manually chosen for all suspended synchronous consistency-groups using the VPlexcli command "consistency-groupchoose-winner" in order to resume I/O at the desired cluster. Once this is complete, check network connectivity between the localcluster and Cluster Witness Server. Check whether Cluster Witness Server VM is running. If the problem persists, contact Dell EMC Customer Support.
|
CLUSTER
|
Alarm
|
0x70006
|
nmg/120
|
0x8a266078
|
Critical
|
True
|
CW Inconsistent State Message Received
|
CW Inconsistent State Message Received
|
There are no known causes for this condition. Please contact Dell EMC Customer Support for assistance.
|
Please contact Dell EMC Customer Support for assistance.
|
CLUSTER
|
Operational
|
0x70007
|
nmg/127, nmg/128
|
--
|
ERROR
|
True
|
Cluster Witness Connection Lost
|
Cluster Witness Connection Lost
|
The director reporting this event is unable to establishcommunication with Cluster Witness Server. This may be due to the failure of the server or loss of network connectivity.
|
Check network connectivity between the director and Cluster Witness Server. Check whether Cluster Witness Server is running.If connectivity is lost from other directors, disable the Cluster Witness Server until connectivity is restored in order to preventdata unavailability on cluster partition. If the problem persists, contact Dell EMC Customer Support.
|
DIRECTOR
|
Alarm
|
0x70008
|
nmg/129, nmg/130
|
--
|
WARNING
|
True
|
Cluster Witness Intermittent Communication
|
Cluster Witness Intermittent Communication
|
This event is generated when Cluster Witness is unable tosend requests or receive responses from Cluster Witness server.
|
Check network connectivity or network bandwidth between the director and Cluster Witness Server.
|
DIRECTOR
|
Alarm
|
0x90001
|
stdf/32
|
0x8a346020
|
ERROR
|
True
|
SCSI Target Reset received for host IT Nexus
|
SCSI Target Reset received for host IT Nexus.
|
Host initiator has issued target reset due to slow I/O processing.
|
Check I/O processing statistics. Identify the reason for thehost to issue Target Reset and resolve it. If the problem is withthe host, unzone the initiator in question from the fabric toprevent potential performance impact to other hosts until theissue can be resolved. Contact Dell EMC Customer Support ifproblem persists.
|
TARGETPORT
|
Operational
|
0x90002
|
stdf/53
|
0x8a349035
|
CRITICAL
|
True
|
Possible Stuck IO detected on virtual volume
|
Possible Stuck IO detected on virtual volume.
|
An IO failed to complete or be properly aborted and cleaned up.
|
Contact Dell EMC Customer Support for assistance.
|
VIRTUALVOLUME
|
Operational
|
0x90003
|
stdf/59
|
0x8a34603b
|
ERROR
|
True
|
Unmap Command Buffer Allocation Failed
|
SCSI UNMAP command failed on virtual volume due to an internal memory allocation issue.
|
A Scsi Unmap command could not be processed because of an internal firmware memory allocation failure.
|
Run the collect-diagnostics utility to collect system informationto determine why an internal firmware memory allocationfailure occurred. Please contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90004
|
stdf/25
|
0x8a346019
|
ERROR
|
True
|
Compare And Write Insufficient Resources
|
SCSI Compare And Write command failed on virtual volume due to aninternal memory allocation issue.
|
The VPLEX director had insufficient memory resources to process theScsi Compare and Write Command.
|
Run collect-diagnostics utility to collect system information to determine why an internal firmware memory allocationfailure occurred. Please contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90005
|
stdf/29
|
0x8a34601d
|
ERROR
|
True
|
Write Same 16 Insufficient Resources
|
SCSI Write Same (16) command failed on virtual volume due to aninternal memory allocation issue.
|
The VPLEX director had insufficient memory resources to process theScsi Write Same Command.
|
Run the collect-diagnostics utility to collect system informationto determine why an internal firmware memory allocation failure occurred. Please contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90006
|
stdf/39
|
0x8a346027
|
ERROR
|
True
|
Xcopy Insufficient Resources
|
Run the collect-diagnostics utility to collect system informationto determine why an internal firmware memory allocationfailure occurred. Please contact Dell EMC Customer Support.
|
The VPLEX director had insufficient memory resources to process theScsi Xcopy Command.
|
Run the collect-diagnostics utility to collect system information to determine why an internal firmware memory allocation failure occurred. Please contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90007
|
stdf/26
|
0x8a34601a
|
ERROR
|
True
|
Compare And Write Processing Failure
|
SCSI Compare and Write command failed on virtual volume it couldnot be processed by internal layers.
|
The attempt to start a transaction for a Scsi Compare and Write command failed as it could not be processed by the internal layers.
|
Please contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90008
|
stdf/30
|
0x8a34301e
|
WARNING
|
True
|
Write Same16 Processing Failure
|
SCSI Write Same (16) command failed on virtual volume as it couldnot be processed by the internal layers.
|
The attempt to start a transaction for a Scsi Write Same command failed as it could not be processed by the internallayers.
|
Consult the Troubleshooting Entry related to VAAI in VPLEX SolVe Desktop. If it does not resolve, please contact Dell EMCCustomer Support.
|
VIRTUALVOLUME
|
Operational
|
0x90009
|
stdf/23
|
0x8a343017
|
WARNING
|
True
|
Compare And Write Invalid Block Count
|
SCSI Compare and Write command failed on volume due to an invalidblock count.
|
The host application or OS requested an invalid transfer size on a Scsi Compare and Write Command.
|
This is a host side issue. Investigate why the host application or OS is not respecting the maximum transfer size advertised by the VPLEX. Consult the Troubleshooting Entry in the VAAI section of SolVe Desktop for this event. If problem persists, contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x9000a
|
stdf/24
|
0x8a343018
|
WARNING
|
True
|
Compare And Write Bad LBA
|
SCSI Compare and Write command failed on volume due to a bad Lgical Block Address.
|
The host application or OS specified an out of range Logical Block Address on a Scsi Compare and Write Command.
|
This is a host side issue. Investigate why the host application or OS is not respecting the maximum size of the volume. If problem persists, contact Dell EMC Customer Support.
|
VIRTUALVOLUME
|
Operational
|
0x9000b
|
stdf/56
|
0x8a343038
|
WARNING
|
True
|
Volume Not Thin Enabled For Unmap
|
SCSI UNMAP command received on volume which is not Thin enabled.
|
Scsi Unmap command processing is rejected by VPLEX because the volume is not thin capable or thin enabled.
|
Use the VPlexcli command "virtual-volume set-thin-enabled true-v volume_name" to enable the thin-enabled property for th virtual volume.
|
VIRTUALVOLUME
|
Operational
|
0x9000c
|
stdf/34
|
0x8a343022
|
WARNING
|
True
|
Xcopy Failed
|
Failed to process a SCSI Xcopy command on volume as xcopy-enable attribute on the storage view is disabled.
|
Scsi xcopy command processing is disabled on VPLEX.
|
Use the CLI command and set the xcopy-enabled attribute for the corresponding storage view to true. Refer to the VPlex CLI guide.
|
STORAGEVIEW
|
Operational
|
0x9000d
|
stdf/31
|
0x8a34301f
|
WARNING
|
True
|
Write Same 16 Failed
|
Failed to process a Write Same (16) command on volume as write-same-16-enabled attribute is disabled on the storage view.
|
Scsi Write Same command processing is disabled on VPLEX.
|
Use the CLI command and set the write-same-16-enabled attribute for the corresponding storage view to true. Refer to the VPlex CLI guide.
|
STORAGEVIEW
|
Operational
|
0x9000e
|
stdf/19
|
0x8a343013
|
WARNING
|
True
|
Unintentional Front End Port Link Down
|
Unintentional Link Down seen for Front End Target Port.
|
An enabled FE port has gone down as a result of FC cable pull,switch reboot or disabling switch port.
|
1. Check the FE port status in the 'ports' context of VPlexcli /clusters/cluster-*/directors/**/ports/to verify if it's still in 'no-link' state. If it is,proceed to next steps.2. Check the switch and ensure it's operational, check the switch logs for errors that will indicate the root cause of the issue.3. Check the cabling and SFPs along the path, clean/reset/replace as needed.
|
TARGETPORT
|
Operational
|
0xa0000
|
sfp/8, sfp/9
|
--
|
Critical
|
True
|
SFP Unsupported
|
The installed SFP is not a Dell EMC approved part for this interface.
|
The part number of the SFP is not approved. It is required to use Dell EMC approved SFPs.
|
Contact Dell EMC Customer Support to replace the SFP with an approved part.
|
INTERFACE
|
Alarm
|
0xa0001
|
sfp/7
|
--
|
Critical
|
True
|
SFP Absent
|
SFP is absent or malfunctioning.
|
An SFP is missing, inserted incorrectly, or faulty.
|
Contact Dell EMC Customer Support to check/replace the SFP.
|
INTERFACE
|
Alarm
|
0xa0002
|
sfp/11, sfp/12
|
--
|
Critical
|
True
|
SFP Rx Power Low
|
A port RX power is below the warning or alarm threshold.
|
A port's RX power is below the warning or alarm threshold.
|
The hardware attached to this port needs to be carefullyinvestigated, and the switch port SFP and cable should be re-seated,cleaned, and swapped as needed.
|
INTERFACE
|
Alarm
|
0xa0003
|
sfp/11, sfp/12
|
--
|
Critical
|
True
|
SFP Rx Power High
|
A port RX power is above the warning or alarm threshold
|
A port's RX power is above the warning or alarm threshold.
|
The hardware attached to this port needs to be carefully investigated, and the switch port SFP and cable should be re-seated, cleaned, and swapped as needed.
|
INTERFACE
|
Alarm
|
0xa0004
|
sfp/11, sfp/12
|
--
|
Critical
|
True
|
SFP Tx Power Low
|
A port TX power is below the warning or alarm threshold.
|
A port's TX power is below the warning or alarm threshold.
|
Contact Dell EMC Customer Support to replace the SFP.
|
INTERFACE
|
Alarm
|
0xa0005
|
sfp/11, sfp/12
|
--
|
Critical
|
True
|
SFP Tx Power High
|
A port TX power is above the warning or alarm threshold.
|
A port's TX power is above the warning or alarm threshold.
|
Contact Dell EMC Customer Support to replace the SFP.
|
INTERFACE
|
Alarm
|
0xb0001
|
dios/20
|
--
|
WARNING
|
True
|
Recovery From One Director Failure Took Too Long
|
Recovery from the failure of a single director took too long
|
Recovery from the failure of a single director took too long.
|
If the host application experienced a DU, confirm that all VPLEX volumes used by that application are in a healthy state. The applications affected will need to go through their recovery process.
|
SYSTEM
|
Operational
|
0xb0002
|
dios/13
|
--
|
INFO
|
True
|
All Failure Recovery Complete
|
Failure recovery has completed for all volumes.
|
Failure recovery has completed for all volumes.
|
This is an informational event only. No further action is required.
|
SYSTEM
|
Operational
|
0xc0001
|
utl/16
|
0x8a3a6010
|
ERROR
|
True
|
Too Many Instances of given command running at once
|
There are too many instances of the given command running at once.
|
There are already the maximum number of instances of the given management command currently running, and another one cannot be started. The previous instances of the command may be stuck,and the cause of this needs to be investigated.
|
Contact Dell EMC Customer Support.
|
DIRECTOR
|
Operational
|
0xd0001
|
vmg/1, vmg/2, vmg/3
|
0x8a523001, 0x8a523002, 0x8a523003
|
WARNING
|
True
|
Invalid Persisted Consistency Group Record
|
A persisted record relating to a consistency group had a formatunrecognized by the system. The record is thus being ignored.
|
A persisted record relating to a consistency group had a formatunrecognized by the system. The record is thus being ignored.
|
Check for any missing consistency groups, and reconstruct these as required. Contact Dell EMC Customer Support for assistance.
|
DIRECTOR
|
Operational
|
0xd0002
|
vmg/29
|
0x8a52601d
|
ERROR
|
True
|
Automatic Detach On Consistency Group Failed
|
Following a link or cluster failure, the configured winner settingson a consistency group were not able to come into effect, and I/O remains suspended on both clusters.
|
The system disallowed the automatic detach on the given consistency group, in order tpreserve consistency on the volumes in the set and avoid losing data. Cluster detach is disallowed when the cluster configured to become the winner needs data from the other cluster,and the necessary rebuild has not yet been completed.
|
If I/O needs to resume, then all required links should be brough up.
|
CONSISTENCYGROUP
|
Operational
|
0xe0001
|
ndu/3
|
0x8a250003
|
INFO
|
False
|
NDU started
|
NDU has started.
|
|
|
DIRECTOR
|
Operational
|
0xf0001
|
nvol/5
|
0x8a276005
|
ERROR
|
True
|
Scan Failed
|
Non-volatile storage filesystem corruption detected.
|
The non-volatile file system has been corrupted and must be repaired.
|
Contact Dell EMC Customer Support.
|
METAVOLUME
|
Operational
|
0xf0002
|
nvol/6
|
0x8a276006
|
ERROR
|
True
|
Header Read Failed
|
Couldn't read metavolume header - nvol not ready.
|
The non-volatile file system is either not configured or has been corrupted.
|
The file system should be reconfigured. Contact Dell EMC Customer Support.
|
METAVOLUME
|
Operational
|
0xf0003
|
nvol/7
|
0x8a276007
|
ERROR
|
True
|
Compact Failed
|
An operation on the meta-volume has failed due to lack of space.
|
The non-volatile file system is not large enough to handle the amount of information that needs to be persisted.
|
Create a larger meta-volume. Contact Dell EMC Customer Support for assistance.
|
METAVOLUME
|
Operational
|
0xf0004
|
nvol/9
|
0x8a276009
|
ERROR
|
True
|
Write Error
|
A write error occurred and data was not written to the non-volatilefile system
|
A write error occurred to the non-volatile file system and data was not written to disk.
|
Investigate why writing to the meta-volume failed. If the failure cannot be corrected, create a new meta-volume and copy the in-memorydata to the new meta-volume. Contact Dell EMC Customer Support for assistance if necessary.
|
METAVOLUME
|
Operational
|
0x120003
|
udcom/3
|
0x8a649003
|
CRITICAL
|
True
|
Path Disconnected
|
A communications path has been disconnected.
|
A communications path has been disconnected due to network connectivity issues.
|
Check the WAN COM or LOCAL COM path that was disconnected, then check the switch logs for errors that will help pinpoint the root cause.If errors point to hardware issue check/clean/replace the cables and SFPs along the path. Engage Dell EMC Customer Support if unable to determine the root cause.
|
COMMUNICATIONSPATH
|
Operational
|
0x13006f
|
tcpcom/111
|
0x8a69606f
|
ERROR
|
True
|
Discoverer Header CRC Error
|
The local port received an invalid header from the remote port (CRC check failed).
|
A tcpcom connection header failed to pass CRC checks.
|
The system should recover automatically. However, the underlying network and hardware needs to be investigated to determine the source of the corrupt packet. Please contact Dell EMC Customer Support for assistance.
|
PORT
|
Operational
|
0x1300c8
|
tcpcom/200
|
0x8a6960c8
|
ERROR
|
True
|
Path Indictment
|
A tcpcom path has been indicted.
|
A tcpcom path has been indicted.
|
The system should recover automatically. However, the underlying network and hardware needs to be investigated to determine the cause of the error. Please contact DELL EMC Customer Support for assistance.
|
COMMUNICATIONPATH
|
Operational
|
0x1300c9
|
tcpcom/201
|
0x8a6960c9
|
ERROR
|
True
|
Path Indictment CRC
|
A tcpcom path has been indicted due to a received packet failing a CRC check.
|
A tcpcom path has been indicted due to a received packet failing a CRC check.
|
The system should recover automatically. However, the underlying network and hardware needs to be investigated to determine the cause of the CRC errors. Please contact DELL EMC Customer Support for assistance.
|
COMMUNICATIONPATH
|
Operational
|
0x1300ca
|
tcpcom/202
|
0x8a6960ca
|
ERROR
|
True
|
Path Indictment Message Length
|
A tcpcom path has been indicted due to a received packet having an invalid message length.
|
A tcpcom path has been indicted due to a received packet having an invalid message length.
|
The system should recover automatically. However, the underlying network and hardware needs to be investigated to determine the cause of the invalid packets. Please contact DELL EMC Customer Support for assistance.
|
COMMUNICATIONPATH
|
Operational
|
0x1300cb
|
tcpcom/203
|
0x8a6960cb
|
ERROR
|
True
|
Path Indictment Timeout
|
A tcpcom path has been indicted due to a timeout.
|
A tcpcom path has been indicted due to a timeout.
|
The system should recover automatically. However, the underlying network and hardware needs to be investigated to determine the cause of the timeout. Please contact DELL EMC Customer Support for assistance.
|
COMMUNICATIONPATH
|
Operational
|
0x150001
|
fc/1
|
0x8a660001
|
INFO
|
False
|
Discovery Starting
|
Fabric discovery is starting due to successful fabric login.
|
Fabric discovery is starting due to successful fabric login.
|
No user action is required.
|
VIRTUALPORT
|
Operational
|
0x150008
|
fc/8
|
0x8a666008
|
ERROR
|
True
|
Unzoned Port
|
A Fibre Channel port came online, but is unzoned.
|
Fibre Channel port is connected to a switch, but the port isn't in any zone yet.
|
Please verify: 1. That the port is cabled correctly. 2. That the switch zoning is complete and correct. Additionally, some switches need to enable the zoning configuration one more time to make the just-changed zoning configuration apply. If unable to resolve the issue engage Dell EMC Customer Support.
|
VIRTUALPORT
|
Operational
|
0x15000d
|
fc/13
|
0x8a66600d
|
ERROR
|
True
|
GS Same Revision Rejected
|
When connecting to a switch, the port received a non-spec-compliantresponse indicating that the switch does not support any protocol version that VPLEX supports.
|
This switch is likely not compatible with VPLEX. VPLEX is unable toregister with the switch or perform fabric discovery. Because thisresponse is non-spec-compliant, VPLEX is unable to report on whichversion of the spec the switch does support.
|
Check the connectivity support matrix from E-Lab Navigator to verify if the switch model and firmware is supported by Dell EMC. Ifassistance is needed engage Dell EMC Customer Support.
|
VIRTUALPORT
|
Operational
|
0x15000e
|
fc/14
|
0x8a66000e
|
INFO
|
False
|
GS Same Revision Rejected Default
|
When connecting to a switch using the default protocol version, the port received a non-spec-compliant response indicating that the switch does not support the default protocol version that VPLEX supports.
|
Because this response is non-spec-compliant, VPLEX is unable todetermine which version of the spec that the switch does support. VPLEX will try a protocol version that is known to work with some switches that act in this way.
|
No user action is required. VPLEX is attempting to automatically recover from this issue. A different event will be emitted if furtherproblems are encountered.
|
VIRTUALPORT
|
Operational
|
0x15000f
|
fc/15
|
0x8a66000f
|
INFO
|
False
|
GS Revision Rejected Supported
|
When connecting to a switch, the port received a response indicating that the switch does not support the default protocol version that VPLEX supports. However, the switch responded with an older protocol version that VPLEX does support.
|
VPLEX will attempt to use the specified protocol version tocommunicate with the switch.
|
No user action is required.
|
VIRTUALPORT
|
Operational
|
0x150010
|
fc/16
|
0x8a666010
|
ERROR
|
True
|
GS Revision Rejected Unsupported
|
When connecting to a switch, the port received a response indicatingthat the switch does not support any protocol version that VPLEX supports.
|
This switch is likely not compatible with VPLEX. VPLEX is unable toregister with the switch or perform fabric discovery.
|
Check the connectivity support matrix from E-Lab Navigator to verify if the switch model and firmware is supported by Dell EMC. If assistance is needed engage Dell EMC Customer Support.
|
VIRTUALPORT
|
Operational
|
0x150011
|
fc/17
|
0x8a666011
|
CRITICAL
|
True
|
Chip Dump Detected
|
The indicated interface has encountered an internal error and has dumped diagnostics for chip vendor analysis.
|
The indicated interface has encountered an internal error and has dumped diagnostics for chip vendor analysis.
|
Contact Dell EMC Customer Support.
|
INTERFACE
|
Operational
|
0x150018
|
fc/24
|
0x8a663018
|
WARNING
|
True
|
Discovery Timeout
|
An attempt to communicate with the switch has timed out.
|
This likely indicates either a physical communication issue with theswitch or a misbehaving switch.
|
Check the physical paths to the switch and verify good connectivity through reseat/clean/replacement of cables/SFPs as needed. Check the switch logs for indications of frame drops or other problems. If unable to determine the cause engage Dell EMC Customer Support.
|
VIRTUALPORT
|
Operational
|
0x150019
|
fc/25
|
0x8a669019
|
CRITICAL
|
True
|
Chip Operation Failed
|
An internal command on this interface has failed unexpectedly.
|
An internal command on this interface has failed unexpectedly.
|
Contact Dell EMC Customer Support.
|
INTERFACE
|
Operational
|
0x15001d
|
fc/29
|
0x8a66901D
|
CRITICAL
|
True
|
Chip Reset Needed
|
Chip error requires manual reset from Dell EMC Customer Support.
|
The chip underlying the specified interface (VPLEX port) hasencountered an error condition and requires a manual reset from Dell EMC Customer Support.
|
Contact Dell EMC Customer Support to manually reset the chip.
|
INTERFACE
|
Operational
|
0x15001e
|
fc/30
|
0x8a66901E
|
CRITICAL
|
True
|
Chip Unrecoverable Error Detected
|
Unrecoverable chip error requires manual reset from Dell EMC Customer Support.
|
The chip has encountered an error condition and no automated recovery is possible. The chip may now be unresponsive resulting in stuckIO. The chip needs to be manually reset by Dell EMC Customer Support,and if that fails the director needs to be rebooted to recover from this issue.
|
Contact Dell EMC Customer Support to manually reset the chip.
|
INTERFACE
|
Operational
|
0x170192
|
febefc/402
|
0x8a673192
|
WARNING
|
True
|
Port No IO Resources Warning
|
{vportName}: In the last {seconds} seconds there were a high number of I/O allocation failures {initiatorIO} {targetIO} {targetResponses}
|
Either a large I/O spike has occurred, there are frame drop issues on the fabric, or there is an internal issue in the VPLEX.
|
Engage Dell EMC Customer Service immediately if there is an outage orextreme performance issues. Check the switch logs from the fabric todetermine if it's logged a large number of frame drops, in which casefurther investigation of the fabric is needed. Toggling the VPLEXport(s) in question one at a time by disabling and re-enabling theport in the /clusters/cluster-x/directors/director-x/ports context ofVPlexcli may relieve the issue.
|
VirtualPort
|
Operational
|
0x170193
|
febefc/403
|
0x8a673193
|
WARNING
|
True
|
Port High IO Error Rate
|
{vportName}: In the last {seconds} seconds at least {numLogins} logins observed a high I/O failure rate
|
Either there are frame drop issues on the fabric or there is aninternal issue in the VPLEX.
|
Check the physical paths to the switch and verify good connectivitythrough reseat/clean/replacement of cables/SFPs as needed. Check theswitch logs for indications of frame drops or other problems. Ifunable to determine the cause engage Dell EMC Customer Service.
|
VirtualPort
|
Operational
|
0x170194
|
febefc/404
|
|
CRITICAL
|
True
|
Port Detected Wedged
|
{vportName} has outstanding activity but made no progress in the last {seconds} seconds. Initiating chip dump and firmware abort.
|
This port has outstanding IO but failed to make progress for over60s, likely due to an internal issue.
|
Issue extended collect-diagnostics to collect the chip dump. Engage Dell EMC Customer Service.
|
VirtualPort
|
Operational
|
0x1701f5
|
febefc/501
|
0x8a6731f5
|
WARNING
|
True
|
Login High IO Error Rate
|
{vportName}: In the last {seconds} seconds {errorPercent}% of I/O failed on login (npid {npid}, wwpn {wwpn})
|
Either there are frame drop issues on the fabric or there is aninternal issue in the VPLEX.
|
Check the physical paths to the switch and verify good connectivitythrough reseat/clean/replacement of cables/SFPs as needed, includingthe connectivity of the VPLEX port. Check the switch logs forindications of frame drops or other problems. If unable to determinethe cause engage Dell EMC Customer Service.
|
VirtualPort
|
Operational
|
0x180001
|
disk/1007
|
0x8a0e33ef
|
ERROR
|
True
|
One Disk Hw After Sustained Success
|
A storage volume encountered an I/O failure due to retry exhaustion after multiple consecutive I/O completions.
|
Multiple successive I/Os to a given disk failed after retryexhaustion. There might be faulty hardware (cable, backendswitch, array).
|
Verify reported array's BE disk health, LUN masking, array configuration and physical connection. If the problem persists,contact Dell EMC Customer Support.
|
DiskId
|
Operational
|
0x180002
|
disk/1008
|
0x8a0e63f0
|
ERROR
|
True
|
Sustained Disk Hw
|
A storage volume encountered sustained I/O failures due to retry exhaustion.
|
Multiple successive I/Os to a given disk failed after retryexhaustion. There might be faulty hardware (cable, backend switch,array).
|
Verify reported array's BE disk health, LUN masking, array configuration and physical connection. If the problem persists,contact Dell EMC Customer Support.
|
DiskId
|
Operational
|
0x190001
|
scsidisk/201
|
0x8a2e60c9
|
ERROR
|
True
|
Io Busy Threshold Reached
|
The given storage volume reached the limit for the number of consecutive busy responses returned.
|
The storage volume returned too many busy responses, and may bemarked dead in VPLEX as a result, if the devices above haveredundancy.
|
Engage array vendor in determining the cause of the returned SCSI busies. If the storage volume has been marked dead, it will beautomatically resurrected by VPLEX once the array begins processing I/O for the storage volume again.
|
DiskId
|
Operational
|
0x1a0001
|
bepm/3, bepm/4
|
0x8a6b9004
|
ERROR
|
True
|
IT Flap Degraded
|
Marking IT flap degraded.
|
The back-end IT path was found cycling between degraded and un-degraded five times within a 40 minute period due to intermittent poor performance, and is now considered "flap degraded".
|
Investigate the related switch logs and array performance for the IT nexus to determine the cause for the intermittent poor performance.The I-T path will be marked "flap degraded" until the user manually restores the use of the I-T path via VPlexcli command 'back-enddegraded recover', or the default 4 hour threshold is reached, after which the IT nexus will then be marked "performance degraded" while the recovery process checks its health before un-degrading it. If the intermittent latency issue continues on the I-T path, and the user is unable to address the root cause quickly then it is advised to engage Dell EMC Customer Service to manually isolate the IT nexus path to remove it from use until the underlying issue can be resolved.
|
ARRAY
|
Alarm
|
0x1a0002
|
bepm/7, bepm/8
|
--
|
ERROR
|
False
|
Unit Degraded
|
Marking LU degraded.
|
All ITLs to a logical-unit on this director are experiencing poor performance, so the logical-unit is marked degraded.
|
Investigate the related switch logs and array performance for the logical-unit to determine the cause for the performance degradation.Once the performance improves the VPLEX will automatically restore the default outstanding IO count to the logical-unit.
|
LOGICALUNIT
|
Alarm
|
0x1a0003
|
bepm/1, bepm/2
|
0x8a6b9001
|
ERROR
|
True
|
IT Degraded
|
Marking IT degraded.
|
A critical number of ITLs (20 by default, but possibly fewer if fewer than 20 ITLs exist or the threshold was manually changed) on thisIT nexus have had multiple IOs experience IO latency of 1 second or greater and all ITLs on that IT have been taken out of service. The IT nexus is now marked degraded.
|
Investigate the related switch logs and array performance for the IT nexus to determine the cause for the degraded performance. Once theperformance improves the VPLEX will automatically restore the use of ITLs that were taken out of service.
|
ARRAY
|
Alarm
|