Dell EMC Unity: Troubleshooting ESRS Type: Integrated connectivity issues, (User Correctable).

Here is the status of the ESRS Integrated system on the Unity "SYSTEM" > "Service" > "Overview" page.  Its current status is unknown. This KBA will attempt to assist with problem isolation of the problem and point to one of the many possible causes of this problem.
 

There is some type of undetermined problem in the environment that is resulting in the inability of the local eVE (ESRS container to properly communicate. This KBA will show you where to look in the logs to see error the messages related to the problem.

While attempting to "Troubleshoot" or "Change" the settings, you may encounter one of the following messages.  Here is a brief description on why the message may be received.

  0x6400b71 "The system was unable to verify support credentials because of a network communication problem. Verify your proxy settings are correct and proxy server is accessible if it's set. Search for the error code on the support website or product forums, or contact your service provider, if available."
  0x6400b76 "The system encountered an unexpected error while completing the ESRS configuration. Search for the error code on the support website or product forums, or contact your service provider"
  0x6400ba5 "Failed to get available sites for Remote Support."
  0x6400ba6 "Failed to get available sites for Remote Support because the access code entered is invalid."
  0x6400ba8 "Cannot get access code for Remote Support. Please check the registered email address and try again."
  0x6400bb0 "Remote Support cannot be enabled at this time"
  0x6400bba "Failed to authenticate by EMC Support for Remote Support."
  0x6400bb9 "Failed to enable Integrated Remote Support."
  0x6400bc5 "Server is currently unable to handle the request due to temporary issues or maintenance of the server. Please contact your service provider"
  0x6400bca "Unable to finalize ESRS configuration. Please relaunch the wizard to complete enablement."

Taking a deeper look at the possible cause, here is how to located & obtain the relevant error message in the Data Collection logs. 

Step 1:  Attempt to "change" the ESRS connectivity (This is the functional equivalent of running the "Update System Settings")
  - This will guide you through the initiation wizard and will either resolve your problem or fail partially though.  If the issue is resolved, then everything is good and you can stop following this KBA.

Step 2: The "change" operation failed to resolve the problem.
  - Next step is to locate the related error messages in the logs.  This can be completed by grabbing a "New set of DC's (Data Collects) from the unity array. This can be located under "SYSTEM" > "Service" > "Service Tasks" page. Section called "Collect Service Information".  Be sure to start a new data collect after forcing a failure from Step1. After several minutes a new DC will appear.  Download and save this file to your system.

Step 3: Reviewing the DC's (Data Collect's).
  - The DC's will have a helpful naming convention, Model, Serial number, Date.tar. 

Example (Unity_600_service_data_APM0016150xxxxx_2016-06-16_05_32_23.tar)

The Data Collects can be opened with any zip utility and you will want to navigate to the following location.  .\<sp>\EMC\CEM\log\eVE\ directory and look for the following files "eVE_startup.log", "ve_esrs_poll.log" & "ve_esrs.log" files.  Out of the 3 files the "ve_esrs.log" is the most helpful.

This is where the local Unity & remote ESRS systems log their communication. The communication takes place in the shape of REST API calls in JSON format. 

~~~ Here is an sample output from a local array exhibiting problems. ~~~
Tue Jun 14 21:36:58 2016 ESRS_VE.pl:8583: Headers: $VAR1 = {
'Date' => 'Thu, 02 Jun 2016 20:05:14 GMT',
'Authorization' => 'UNITY:APM0016150xxxxx:wgolQX0pcfFWcEJytXXXXX42lI=,domain=Device',
'Content-Type' => 'application/json', 'Accept' => 'application/json'  };
Tue Jun 14 21:38:45 2016 ESRS_VE.pl:8583: Check the request response for errors
Tue Jun 14 21:38:45 2016 ESRS_VE.pl:8583: $VAR1 = {
'responseCode' => '502',
'responseType' => 'text/html; charset=iso-8859-1',
'responseContent' => '<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head> <title>502 Proxy Error</title> </head><body>
<h1>Proxy Error</h1>
<p>The proxy server received an invalid response from an upstream server.<br />
The proxy server could not handle the request <em>
<ahref="/esrs/v1/devices/UNITY/APMxxxxxxxxxxx">DELETE&nbsp;/esrs/v1/devices/UNITY/APMxxxxxxxxxxx</a></em>.<p>
Reason: <strong>Error reading from remote server</strong></p></p>
</body></html>
', 'responseDate' => 'Tue, 14 Jun 2016 21:38:45 GMT' };
~~~ END sample output from a local array exhibiting problems. ~~~

In the above output, I have highlighted the area of interested (responseCode). If you are not able to find any recent messages in the log, then please check the same log location from the peer SP, one of the two SPx,service_dc.tgz files in the DC will contact the log messages. I have also highlighted in green the description on the 502 error from the same log.

Step 4: Determining the response code. 
  - At this point we have ESRS failing and we have a new set of logs with a recent error message logged as a response code.  Here is a break out of "some" of the typical response codes and what could be causing the failing message: For any 201 or 500 level error, a network check should be run and then the issue should be escalated to the EMC "RemPro" team.

This KBA is in progress, so please contact the author with more information to help complete better error codes, cause & steps to resolve.

There is a 4 hour refresh cycle on ESRS backend databases. This means that any corrected data (account permissions, serial numbers, part numbers) require a 4 hour delay before they can be accessed by the Unity array.
 
There is a required 30 minute delay for security purposes when switching from attempting to provision with one set of credentials to a new set of credentials with the same array (Example: name@companyABC.com to EMC employee credentials).

Sources of additional information:
ESRS Troubleshooting guide
https://support.emc.com/docu69327_Unity_Family_EMC_Secure_Remote_Services_Requirements_and_Configuration.pdf?language=en_US
 
2015 Youtube instructional video for ESRS setup
https://www.youtube.com/watch?v=UN8DW00ojN4

Fire Wall Ports that are required to be opened (These are exiting the customers infrastructure)
No incoming ports are required to be opened. (From the internet to the Unity system)
These are only required for the Unity Management IP address.


ESRS port requirements guide, Ensure correct customer correct Fire wall ports are opened.
https://support.emc.com/docu55301_EMC_Secure_Remote_Services_Release_3.12_Port_Requirements.pdf?language=en_US

"Complete list of network ports used by Unity" KBA487022 
https://support.emc.com/kb/487022

Dell Unity 300, Dell EMC Unity 300F, Dell EMC Unity 350F, Dell EMC Unity 400, Dell EMC Unity 400F, Dell EMC Unity 450F, Dell EMC Unity 500, Dell EMC Unity 500F, Dell EMC Unity 550F, Dell EMC Unity 600, Dell EMC Unity 600F, Dell EMC Unity 650F , Dell EMC Unity Family |Dell EMC Unity All Flash, Dell EMC Unity Family, Dell EMC Unity Hybrid ... 檢視更多內容 檢視較少內容