Data Domain : After upgrading to DDOS / DDMC 7.1.x or later, GUI can't be accessed anymore

A customer upgrades either DDOS or DDMC to version 7.1.x or later (we will use DDOS from now on to refer to both DDOS and DDMC), for brevity, and finds out after the upgrade is complete, that it is not possible to access the GUI anymore. Restarting the HTTP / HTTPS services from the command line does not make it work either. When using a browser to access the GUI , the following error page would show : 
 

Service Not Available

The GUI Service is temporarily unavailable. Refresh browser to try again. If the problem persists, contact Dell EMC support for assistance

The GUI backend is running off a Java-based application. GUI backend is not running. If this DD has been upgraded to DDOS 7.1.x or later and the GUI has been failing since, if the DD has a trust relationship with other DDs and any of them has a cerificate with a public key shorter than 2048 bits, it may be the cause why the DD GUI is not starting up, as stricter checks in the newer bundled JDK fail to pass for some of the trusted DD hosts.

First it is necessary to ascertain this is exactly the problem being faced. For this to be the case, all of the conditions below must hold :

1. DD where GUI is not starting up is experiencing the GUI issues only since upgrading to DDOS 7.1.x or later
2. This DD has a trust relationship with other DDs, one or more of which had run a DDOS 5.4.x version (or older), or DDMC 1.1 (or older) in the past
3. This DD has a particular set of logs for the failure to start the GUI backend

Item 1 above is self-explanatory. To determine if 2 above applies, first get the list of trusted hosts in the DD :

# adminaccess trust show

For each one of the hosts this one has trust with, check their upgrade history, to see if any was installed with DDOS 5.4 (or earlier), or DDMC 1.1 (or earlier) : 

# system upgrade history

Systems installed with any of the versions above are likely to have had a CA self-signed certificate generated on install with public keys only 1024 bits long, which are no longer accepted by JDK after upgrading to DDOS / DDMC 7.1. A possible way to learn if these hosts have certificates with small public keys is by opening up the GUI to them and checking the certificate details from a browser (way to do so varies slightly across browsers).

To confirm item 3 (if the DD GUI failure logs are for this specific problem), run the following command to open the "em.info" log file :

# log view debug/sm/em.info

And search (use a forward slash) to search for these logs ("..." indicates some logs are not shown below for brevity) :

 

+-----+-----+-----+ SYSTEM (RE)START +-----+-----+-----+
...
26 Feb 2021 10:33:04,172 INFO  [main] Setting the session cookie name to 'JSESSIONID-ddem___HTTPS'
26 Feb 2021 10:33:04,172 INFO  [main] Setting the xsrf cookie name to 'DD_SSO_TOKEN___HTTPS'
26 Feb 2021 10:33:04,382 INFO  [main] Injecting the SUN provider's X.509 factory to fix validation issues
...
26 Feb 2021 10:33:05,093 INFO  [main] Re-initializing the certificates between the client and the server

26 Feb 2021 10:33:05,093 INFO  [main] Reloading the certificate stores for the system

26 Feb 2021 10:33:05,097 INFO  [main] Finished reloading the certificate stores
26 Feb 2021 10:33:05,097 ERROR [main] Exception during command execution: javax.net.ssl.SSLException - Error creating premaster secret. , will retry,  Attempt# 1
26 Feb 2021 10:33:05,243 INFO  [main] Re-initializing the certificates between the client and the server
26 Feb 2021 10:33:05,243 INFO  [main] Reloading the certificate stores for the system
26 Feb 2021 10:33:05,246 INFO  [main] Finished reloading the certificate stores

 That would indicate some of the cert this DD has imported as trusted has a short key and hence the GUI can not start.

Although DDOS 7.1 or later will continue to fail loading the GUI when presented with certtificates with small public keys, the issue has been resolved in the code for versions DDOS 6.2.1.40 and later, and DDOS 7.2.0.50 and later, so that if when upgrading to any such release the local CA certificate has a small public key, the certificate will be re-generated with a longer key.
 

Considering as of this writing (August 2022) no releases other than DDOS 6.2.1.x (for DD2200 and DD250 hardware only) and DDOS 7.x are supported anymore, no workaround is provided, although for the offending DDs you may try re-generating the host and CA certificate with longer keys, then remove and re-add trust again between the affected devices : 

# adminaccess trust del host dd-trusted-1 type mutual
# adminaccess certificate generate self-signed-cert regenerate-ca
# adminaccess trust add host dd-trusted-1 type mutual