PowerFlex: MDM alerts are not sent over (SNMP)
Summary: Some alerts shown by MDM "showevents.py" command are not sent by the Gateway to the SNMP trap receiver.
Symptoms
Not every alert logged by the MDM in the eventlog is supposed to generate a corresponding SNMP trap - (see PowerFlex User Guide, chapter "SNMP trap support" for complete list of the events) - so it is normal and expected that some alerts listed in "showevents.py" are not sent by SNMPor Secure Remote Services (SRS).
Also, per the PowerFlex User Guide every generated alert should be open and closed:
(…) Open and closing alerts consist of the same code and issue number, except for the first digit (0 or 1) in the <ISSUE> section. For example: l SIOXX.XX0XXXXXX indicates that the alert is active l SIOXX.XX.1XXXXXX indicates that the alert has been closed (…)
The above statement relates to Secure Remote Services (SRS) events only.
When an alert is active SNMP traps are emitted repeatedly, at configurable intervals. If the alert is no longer active (i.e. error situation was fixed), SNMP traps will no longer be emitted. The PowerFlex Gateway does not emit "closing" traps.
Cause
Example of some events "missing" in SNMP: MDM eventlog:
5 2018-01-15 09:50:21.720 MDM_CLUSTER_LOST_CONNECTION WARNING The MDM, ID 0000000000000000, lost connection 6 2018-01-15 09:50:22.338 MDM_CLUSTER_BECOMING_MASTER WARNING This MDM, ID 111111111111111, took control of the cluster and is now the Master MDM. 7 2018-01-15 09:50:22.338 MDM_BECOMING_MASTER WARNING This MDM is switching to Master mode. MDM will start running. 8 2018-01-15 09:50:22.381 SDS_RECONNECTED INFO SDS: scaleio-0-5 (ID 2222222222222222) reconnected 9 2018-01-15 09:50:22.381 SDS_RECONNECTED INFO SDS: scaleio-0-6 (ID 3333333333333333) reconnected 15 2018-01-15 09:50:23.343 SDC_CONNECTED INFO SDC connected. ID: 70ad9cb000000005; IP: 192.168.1.2; GUID: 079F5643-9CCC-4F9D-836F-6865794ADA40 16 2018-01-15 09:50:26.253 MDM_CLUSTER_NOT_RESPOND WARNING The MDM, ID 000000000000000, is not responding 17 2018-01-15 09:50:28.448 MDM_DATA_DEGRADED ERROR The system is now in DEGRADED state. 18 2018-01-15 09:50:48.452 MDM_DATA_NORMAL INFO The system is now in NORMAL state.
IM Gateway scaleio-trace.log:
2018-01-15 09:52:46,567 [threadPoolTaskScheduler-1] INFO c.e.s.s.s.snmp.TrapSender - SNMP Alert for entity = [5c39c28a6ca0beae], alert = [MDM.MDM_Cluster.CLUSTER_DEGRADED] code = [SIO02.01.0000007] sent successfully 2018-01-15 09:52:46,567 [threadPoolTaskScheduler-1] INFO c.e.s.s.s.snmp.TrapSender - SNMP Alert for entity = [8b27574900000000], alert = [SDS.SDS.SDS_Disconnected] code = [SIO03.01.0000001] sent successfully
As seen in the example above, there are no "MDM_CLUSTER_LOST_CONNECTION" or "MDM_CLUSTER_BECOMING_MASTER" warnings set over SNMP - this is normal, as they do not have their corresponding SNMP traps, per User Guide.
Resolution
No impact - working as designed