RecoverPoint for VMs: Upgrading From Version 5.2.x to 5.3.1 Does Not Complete

The following is seen from the RecoverPoint /home/kos/installationLogs/server.log:

XXXX/XX/XX 10:30:10.797 [pool-4-thread-1] (Task.java:98) ERROR - UpgradeClusterRestorableTask :Task failed: Executing NDU.
com.emc.recoverpoint.utils.javainfra.status.tasks.exception.TaskFailedException: could not configure repository volume.
at com.emc.recoverpoint.utils.javainfra.status.tasks.SerialTask.perform(SerialTask.java:35)
at com.emc.recoverpoint.utils.javainfra.status.tasks.Task.executeSync(Task.java:91)
at com.emc.recoverpoint.utils.javainfra.status.tasks.RestorableSerialTask.attemptToCompletePhase(RestorableSerialTask.java:94)
at com.emc.recoverpoint.utils.javainfra.status.tasks.RestorableSerialTask.perform(RestorableSerialTask.java:63)
at com.emc.recoverpoint.utils.javainfra.status.tasks.Task.executeSync(Task.java:91)
at com.emc.recoverpoint.utils.javainfra.status.tasks.Task.call(Task.java:142)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:823)
Caused by: com.emc.recoverpoint.utils.javainfra.status.tasks.exception.TaskFailedException: could not configure repository volume.
at com.emc.recoverpoint.utils.javainfra.status.tasks.SerialTask.perform(SerialTask.java:35)
at com.emc.recoverpoint.utils.javainfra.status.tasks.Task.executeSync(Task.java:91)
at com.emc.recoverpoint.utils.javainfra.status.tasks.SerialTask.perform(SerialTask.java:32)
... 9 more

XXXX/XX/XX 10:30:09,763 [CommandWorker-1] (CreateVCConnectionCommand.java:49) ERROR - Failed to connect to VCenter server.
XXXX/XX/XX 10:30:09,764 [CommandWorker-1] (Command.java:88) ERROR - Command#run() ServerException errorType[OPERATION_FAILED_NO_MESSAGE] errorMessage[Entering vCenter credentials failed. Check that all entered credentials are correct. If problem recurs, check that vCenter Server and communication are fully functional. If the problem persists, Please contact Customer Support for assistance.] failure UID: <<unique_UUID>>
com.kashya.installation.server.exceptions.CommandFailedException: Entering vCenter credentials failed. Check that all entered credentials are correct. If problem recurs, check that vCenter Server and communication are fully functional. If the problem persists, Please contact Customer Support for assistance.
at com.kashya.installation.server.commands.vsphere.CreateVCConnectionCommand.execute(CreateVCConnectionCommand.java:50) ~[classes/:?]
at com.kashya.installation.server.commands.vsphere.CreateVCConnectionCommand.execute(CreateVCConnectionCommand.java:17) ~[classes/:?]
at com.kashya.installation.server.commands.Command.runNormal(Command.java:109) [classes/:?]
at com.kashya.installation.server.commands.Command.run(Command.java:49) [classes/:?]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_265]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_265]
at com.kashya.installation.server.commands.Command.executeInternalCommand(Command.java:266) [classes/:?]

During the upgrade from RecoverPoint for VMs 5.2, Deployer attempts to save the required vCenter UUID in a tweak named "t_deployed_vc_uuid."  It does this by making a connection to the vCenter and validating whether the vRPAs are present in that particular vCenter.

However, when a vCenter has a CA Certificate with more than one chain length, the connection fails since the old Truststore is not yet copied.

Workaround:
Once this issue occurs, we must add the tweak "t_deployed_vc_uuid" with deployed VC UUID to the "tweak.params.system_layout" file on affected vRPA. The vCenter UUID value is found by running the get_storage command on a surviving vRPA at that cluster. The required value for the tweak is listed under the serial number value within the get_storage CLI command. 

Verify which RPA is missing the "t_deployed_vc_uuid."

Access the affected RPA with a root account using SSH or Web Console and perform below steps:

1. Type command:

su - admin

2. Select option [6] System management CLI
3. Type command:

get_storage

4. Enter RPA cluster name that you must add to the vCenter
5. On the displayed results, collect the value for Serial Number
6. Type command: 

quit

7. Type option Q and then type Y to go back to root
8. Once logged into RPA1 as root, type tweak and press enter
9. Type command: 

vi tweak.params.system_layout

10. At the end of the file, add the below information:

t_deployed_vc_uuid = <<Serial Number collected in step 5>>

11. Press ESC and type wq!
12. Reboot RPA1

Once you complete the above steps, vCenter can authenticate properly, and information is displayed on the RecoverPoint plug-in.

Once the Cluster Logic service is up on affected vRPA, the upgrade should start progressing from the previous step without intervention.

The deployer may also pick up and keep going if it was left open. If it was closed, check the CLI status of the other vRPAs to see if they have started upgrading as well, even with Deployer closed.

If this does not happen, contact Dell RecoverPoint Support for assistance.

Resolution:
This issue is addressed in the RecoverPoint for Virtual Machines version 5.3.1.1.

To determine whether an upgrade is appropriate for your environment, contact the Dell Customer Support, or your service representative and reference this solution ID.