Cisco MDS: some hosts execute the Link Reset Protocol without any apparent cause or other errors

Hi there,

On several MDS systems (9140 on 3.3(1c), 9506 on 4.2(1a), and others) we experience an interesting phenomenon. It seems to only occur on a specific type of server. Hopefully, somebody here recognizes the symptoms and can share experiences.

The problem is that on some links, on a link in the Active state, the host will execute the Link Reset Protocol every now and then. It will send a Link Reset primitive sequence, the switch responds with LRR, the host responds with Idle, and finally the switch sends an Idle and the ports are Active again. When doing a "show interface", such an event is visible because the output LRR counter is incremented by 1 without any other counter increasing. The SNMP counter "fcFcPortLinkResetIns" counter will also increase by 1 each time such an event occurs. This is all that happens: all login sessions over the link remain intact, no FLOGI, port login or process login happens afterward.

The puzzling thing is that no other error counters increase. No CRC errors, no InvalidTxWords, no SyncLosses, no LinkFailures, nothing. The physical links seem just fine. Hosts don't really seem to be bothered by this problem, but we do have some Exchange-servers that sometimes complain about individual I/O's taking extremely long (60 seconds), without any further effects. However: in almost every case an Exchange log entry coincides with a Link Reset. So, this is something worth investigating.

We've performed the usual stuff (replacing SFPs, cables), but as I mentioned above, the physical connections seem fine. This occurs on different switches in different datacenters.

Does anybody recognize this?

Regards,

Jurjen Oskam