Skip to main content
  • Place orders quickly and easily
  • View orders and track your shipping status
  • Create and access a list of your products

Integrated Data Protection Appliance - Avamar Password Out of Sync

Summary: Avamar password out of sync shown in IDPA dashboard UI

This article applies to This article does not apply to This article is not tied to any specific product. Not all product versions are identified in this article.

Symptoms

Password Out of Sync error message shows on the IDPA ACM Dashboard complaining about Avamar Server or AVProxy.

Here is a Password Out of Sync example:
ACM UI password out of sync error
 
ACM UI prompt for password update
 
Protection Software Proxy 'root' user out of sync. Please ensure 'root' password is same as 'admin' password

Cause

ACM monitors and reports all point-products which include its hardware and hypervisor platform health status. In order to do that, it keeps a copy of all current point-product login information in an encrypted password file. It constantly makes connections to those targets to check their health status. If it fails to log in to any of the monitoring target machines, it reports a password out of sync error in the ACM dashboard. 

Possible causes:
  • Password is changed or is reset directly at the Avamar server or proxy end instead of from the ACM dashboard 
  • Network latency from ACM to Avamar Server or AVProxy to query its status. (temporary issue)

Resolution

If updating the password in ACM UI is unable to resolve the password out of sync issues, following these steps:


Resolutions for Avamar server password out of sync:

Scenario #1: 

Password changed from Avamar side instead of from ACM UI:

  • Avamar password out of sync (one or more of the following Avamar user passwords have changed)

    • Avamar OS root user password
    • Avamar OS admin user password
    • Avamar Server root user password
    • Avamar Server MCUser user password
    • Avamar Server repluser user password
    • Avamar PostgreSQL database viewuser user password
    • Avamar vProxy OS root user password (Appliance Internal proxy VM)
    • Avamar vProxy OS admin user password (Appliance Internal proxy VM)
Resolution: 
We strongly recommend keeping all Avamar Server and Avamar Proxy user passwords the same as IDPA common password, however if there is a requirement to keep them different, ensure the following rules are met:
 
Password rules: 
If you change Avamar passwords from Avamar side instead of from ACM UI, IDPA requires that the Avamar OS admin and OS root accounts use the same password. The MCUser, repluser, viewuser, and Avamar server root user must share a same password which can be different than the OS admin and OS root passwords.

In the above password list, the passwords highlighted in the same color MUST share a same password, and also all passwords have to meet the IDPA global password policy shown as below:
 
 IDPA global Password policy
 
The following procedure is to verify Avamar passwords:
  1. How to verify Avamar Server and Avamar Proxy machine OS root/admin passwords are the same? 
SSH login to Avamar server and Proxy as admin, then su to root user with the same password to see if the password can log in both admin and root account. 
  1. How to verify the Avamar Server root, MCUser and repluser passwords? (Avamar Server root is different from Avamar Server OS root. OS root is a Linux OS level login user, and Avamar Server root is an Avamar application level user)
SSH log in to the Avamar server as admin and run the following command four times with a different username: (replace <Avamar_Username> with MCUser, repluser, viewuser, and root)
# avmgr logn --id=<Avamar_Username> --ap=<password>
1  Request succeeded
7161  privilege level  (enabled,create,read,backup,access,move,delete,maint,fullmanage,noticketrequired)
2  block type  (directory)
  1. How to verify the Avamar Server mcdb viewuser password?
SSH to the ACM machine and run: (if you run this command from the Avamar machine itself, it does not prompt for a password):
# psql -U viewuser -h <Avamar Server IP> -p 5555 mcdb -c "\d"
Password for user viewuser:
 
The following procedure is to change Avamar passwords:
  1. Change Avamar Server-side passwords:
Run the change-passwords command to change passwords. Here is an example to change all Avamar server passwords (in the real case, you can selectively change Avamar passwords that are set incorrectly). 
login as: admin
Password: xxxxx

admin@Avamar-svr:~/>: su -
Password: xxxxx
root@Avamar-svr:~/#:

root@Avamar-svr:~/#: change-passwords
[change-passwords version 2.1]
Identity added: /root/.ssh/rootid (/root/.ssh/rootid)
Identity added: /root/.ssh/rootid (/root/.ssh/rootid)
Identity added: /root/.ssh/rootid-save (/root/.ssh/rootid-save)

Do you wish to specify one or more additional SSH passphrase-less
    private keys that are authorized for root operations?
Answer n(o) here unless there are known inconsistencies in
    ~root/.ssh/authorized_keys files among the various nodes.
Note that the following keys will be used automatically (i.e., there is
    no need to re-specify them here):
      /root/.ssh/rootid
      /root/.ssh/rootid-save

y(es), n(o), h(elp), q(uit/exit): no
--------------------------------------------------------
The following is a test of OS root authorization with the currently
    loaded SSH key(s).

    If the authorization test fails, then you might be missing an
    appropriate private key, e.g., rootid or dpnid.
        -> In that event, re-run this program and, when prompted,
           specify as many SSH private key files as are necessary
           in order to complete root operations.

Starting root authorization test with 600 second timeout...
End of root authorization test.
--------------------------------------------------------

Change OS (login) passwords?
y(es), n(o), q(uit/exit): yes
change-passwords: INFO: Each OS password will be changed locally without further prompting as soon as you have (twice) entered a valid password.


--------------------------------------------------------
Change OS password for "admin"?
y(es), n(o), q(uit/exit): yes
Change password for user "admin".

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Enter the same OS user password again.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
BAD PASSWORD: it is too simplistic/systematic
Backup lockbox file
Backup keystore files
Backup SSV files
Flush backup
Local backup dir: /usr/local/avamar/src/lockbox_backup/2023-06-26-22_00
Flush backup dir: /usr/local/avamar/var/mc/server_data/lockbox_backup
Updated with new value under name "admin".
Backup lockbox file
Backup keystore files
Backup SSV files
Flush backup
Local backup dir: /usr/local/avamar/src/lockbox_backup/2023-06-26-22_00
Flush backup dir: /usr/local/avamar/var/mc/server_data/lockbox_backup
change-passwords: INFO: The password for OS user admin has been updated on _this_ host.
change-passwords: INFO: The password will not be reverted if you later decline to update passwords/passphrases.


--------------------------------------------------------
Change OS password for "root"?
y(es), n(o), q(uit/exit): yes
Change password for user "root".

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Enter the same OS user password again.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
BAD PASSWORD: it is too simplistic/systematic
change-passwords: INFO: The password for OS user root has been updated on _this_ host.
change-passwords: INFO: The password will not be reverted if you later decline to update passwords/passphrases.


--------------------------------------------------------
Generate new SSH keys?
y(es), n(o), h(elp), q(uit/exit): no


--------------------------------------------------------
Change Avamar Server passwords?
y(es), n(o), q(uit/exit): yes

--------------------------------------------------------
Please enter the CURRENT server password for "root"

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Checking Avamar Server root password (1200 second timeout)...
Avamar Server current root password accepted.


--------------------------------------------------------
Change Avamar Server password for "MCUser"?
y(es), n(o), q(uit/exit): yes
Please enter a new Avamar Server password for user "MCUser".

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Enter the same Avamar Server password again.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Accepted Avamar Server password for "MCUser".


--------------------------------------------------------
Change Avamar Server password for "root"?
y(es), n(o), q(uit/exit): yes
Please enter a new Avamar Server password for user "root".

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Enter the same Avamar Server password again.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Accepted Avamar Server password for "root".


--------------------------------------------------------
Change Avamar Server password for "repluser"?
y(es), n(o), q(uit/exit): yes
Please enter a new Avamar Server password for user "repluser".

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Enter the same Avamar Server password again.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
Accepted Avamar Server password for "repluser".


--------------------------------------------------------
Change the viewuser password?
y(es), n(o), h(elp), q(uit/exit): yes
Checking Administrator Server status...
Enter the NEW viewuser password.
Enter ? or help for help.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx
For verification, re-enter the NEW viewuser password.
Enter ? or help for help.

(Entering an empty (blank) line twice quits/exits.)
> xxxxx

--------------------------------------------------------
Do you wish to proceed with your changes on the selected node?
        Answering y(es) will proceed to make changes.
        Answering n(o) or q(uit) will not proceed.

y(es), n(o), q(uit/exit): yes
Changing OS passwords...
[Logging to /usr/local/avamar/var/change-passwords.log...]
Done changing OS passwords...
Changing Avamar Server passwords...
Suspending maintenance cron jobs
Checking Administrator Server status...
Stopping Administrator Server...
Changing the passwords for the local Avamar Server...
The passwords for the local Avamar Server have been changed.
Starting process of updating Administrator and Enterprise Manager configurations...
Running script to update Administrator and Enterprise Manager configurations on node 0.s...
[Logging to /usr/local/avamar/var/change-passwords.log...]
Done with updating Administrator configuration on node 0.s...
Starting process of updating client configurations...
Running script to update client configuration on all+...
[Logging to /usr/local/avamar/var/change-passwords.log...]
Updating client configuration on node 0.0...
Done updating client configuration on 0.0...
Starting process of updating mccli configuration files...
Running script to update mccli configuration files on node set "0.0"...
[Logging to /usr/local/avamar/var/change-passwords.log...]
Done with updating mccli configuration files on node 0.0...
Checking Administrator Server status...
Starting Administrator Server...
Resuming maintenance cron jobs
Starting process of updating viewuser password...
Checking Administrator Server status...
Stopping Administrator Server...
Running script to update mcdb viewuser password on node 0.0...
[Logging to /usr/local/avamar/var/change-passwords.log...]
Done with updating mcdb viewuser password on node 0.0...
Checking Administrator Server status...
Starting Administrator Server...
Stopping EMT subsystem
Starting EMT subsystem

--------------------------------------------------------
Done.
NOTES:
- If mccli (the Administrator command line interface)
      is used from any remote user accounts, then please update
      the password in each remote account's copy of the mccli
      preferences/configuration file, typically
      ~USER/.avamardata/var/mc/cli_data/prefs/mcclimcs.xml.
- Please be sure to resume schedules via the
        Administrator GUI or via 'dpnctl start sched'.

#: dpnctl start sched
Identity added: /home/admin/.ssh/admin_key (/home/admin/.ssh/admin_key)
dpnctl: INFO: Resuming backup scheduler...
dpnctl: INFO: Backup scheduler resumed.
dpnctl: INFO: No /usr/local/avamar/var/dpn_service_status exist.
  1.  Change Avamar Proxy side passwords:
Here is an example of changing both proxy admin and root passwords (login as root first, then change both admin and root passwords):
login as: admin
Password: xxxxx

su -
Password: xxxxx

# passwd admin
New password: xxxxx
BAD PASSWORD: it is too simplistic/systematic
BAD PASSWORD: is too simple
Retype new password: xxxxx
passwd: password updated successfully

# passwd root
New password: xxxxx
BAD PASSWORD: it is too simplistic/systematic
BAD PASSWORD: is too simple
Retype new password: xxxxx
passwd: password updated successfully
Once all Avamar password rules are met, go to ACM UI, refresh the ACM UI browser page, then click the Out of Sync error message, it prompts you to enter new password for one or more users, update the password accordingly. (Sometimes it still shows a password out of sync error after you entered correct password, wait for a few minutes and refresh your web browser page again. Root cause shows in Scenario #2).
 

Scenario #2 

Password out of sync error due to network latency when ACM trying to query its point-products:

  • This is a temporary issue and typically can be resolved if you refresh the ACM page after 1-2 minutes. This is a known issue, and the Dell engineering team is working on a fix in a future release.


Scenario #3 

Even though password is synchronized and works on Avamar, ACM shows password out of sync for AV, due to SSH failure or test connection failure to AV. This may be due to ACM failing to log in to AV due to SSH issues like recent changes made on AV sshconfig, cipher negotiation, and so on.

  • Run a test SSH connection from ACM to Avamar server. If it fails, log in to Avamar server and restart SSH service:
# service sshd restart
  • If this does not help, gather the error message, troubleshooting steps you have performed and raise a ticket with Dell technical support for further assistance.


Scenario #4 

Avamar MCUser or viewuser may show out of sync when ACM is unable to perform MCSDK call to Avamar to validate those user passwords. This can happen if the ACM MCSDK call fails to Avamar due to various reasons. 
If the above provided scenarios and resolutions are unable to fix the issue, do the following:
  • SSH Login to ACM as root, and stop and start ACM web application service:
# service dataprotection_webapp restart
# service dataprotection_webapp statu
  • Refresh the ACM web page and login, it shows "Appliance Startup progress." It takes some time to resync up with all the appliance components, and once done it returns to the ACM dashboard. (This is not a process of restarting Appliance) 
  • If the issue still cannot be resolved, raise a support ticket with Dell Technologies.

Avamar password related KB references:

Affected Products

PowerProtect DP4400, Integrated Data Protection Appliance Family, Integrated Data Protection Appliance Software
Article Properties
Article Number: 000217330
Article Type: Solution
Last Modified: 29 Jan 2025
Version:  4
Find answers to your questions from other Dell users
Support Services
Check if your device is covered by Support Services.