Avamar: How to Use the dump_root_hashes.rb Script to Generate a List of Clients and Backups

This article discusses features of the Avamar Engineering script "dump_root_hashes.rb" and provides the reader with basic guidance on its use.

Typical uses cases:

• Generating a list of clients and their properties (for example client ID, domain) on a system.
• Comparing a list of clients between two systems.
• Generating a list of all backups on an Avamar grid and their properties. These include root hash, size, plug-in id, domain, client name, expiration, retention, and size.
• Comparing backups that are stored on a pair of replicating Avamar grids (helpful for identifying missing or surplus backups per Avamar: A replicating pair shows different levels of capacity usage. How to investigate the causes. It may be necessary to log in to Dell Support to access this article.)

The script is aware of backups that are run from Avamar but sent to Data Domain.

Downloading the script:

See the following article for information about locating and downloading the script: Avamar: How to find and download Avamar scripts and tools from the Dell Central Avamar page.

Running the script:

The script is run by prefixing the script name with "ruby" (as seen below):

ruby dump_root_hashes.rb --help

Starting up dump_root_hashes.rb version 0.10.2 at 2019-10-21 02:17:21 +0000 (2019-10-21 02:17:21 UTC)

Usage: dump_root_hashes.rb [options]

Overview of the script's features
Running the script with the --help flag shows a list of features.  Below is output showing features of the script as of release 0.10.2.


Options:
    -q, --quiet                      Suppress progress counters
        --mode=<backuplist|backupcompare|clientlist|clientcompare|genpasswordfile|connchk>

                                     Specify the script mode. Supported modes:
                                     backuplist (default) - generate a report showing the list of all backups on the system; include the following information:
                                       -system name
                                       -client name
                                       -domain
                                       -backup create time
                                       -backup expiration time
                                       -retention tags
                                       -backup size
                                       -root hash
                                     backupcompare - generate a report showing backup information for both source and target; requires a --dstaddr parameter
                                     clientlist - generate a report showing the client information for all client accounts
                                     clientcompare - generate a report showing the client information for both source and target; requires a --dstaddr parameter
                                     genpasswordfile - generate a file containing obfuscated login credentials
                                     connchk - run a GSAN connectivity test
    -s, --srcaddr=ADDRESS            Specify the replication source. Defaults to the local system.
        --src-password-file=FILENAME Specify a file containing credentials for the source system. Only needed if source and destination passwords differ.
    -d, --dstaddr=ADDRESS            Specify the replication destination for "compare" modes
        --dst-password-file=FILENAME Specify a file containing credentials for the destination system. Only needed if source and destination passwords differ.

Filtering Options:

NOTE: Filter flags apply to BOTH source AND destination systems unless otherwise noted!

        --before=DATE                Only report on backups created before DATE.
                                     This date can be any string understood by the GNU "date" utility.
        --after=DATE                 Only report on backups created after DATE.
                                     This date can be any string understood by the GNU "date" utility.
        --srcpath=PATH               Limit reporting on the source system to domains and clients under the specified path.
        --dstpath=PATH               Limit reporting on the destination system to domains and clients under the specified path.
        --mc-deleted=<include|skip>  Skip the MC_DELETED domain by default
        --mc-retired=<include|skip>  Traverse the MC_RETIRED domain by default
        --partials=<include|skip>    Skip partial backups by default
        --expired=<include|skip>     Skip expired backups by default
        --replicate=<include|skip>   Traverse the REPLICATE domain by default
        --system-accounts=<include|skip>
                                     Skip the system accounts MC_BACKUPS, EM_BACKUPS, AVI_BACKUPS, etc. by default
        --adme=<include|skip>        Skip the ADME domain by default

Output Options:

        --outfile=FILENAME           The report is written to this file.
                                     Defaults to "<mode>.csv", where <mode> is the script mode.
                                     The file is overwritten if it exists.
        --format=<csv>               Data format to use for export. Supported formats:
                                     csv (default) - comma separated values; suitable for spreadsheet import
        --[no-]showprectime          Include the backup creation precision time in report
        --[no-]show-ddr-info         Include DDR presence information and ddrindex in report
        --[no-]show-tier-info        Include cloud tiering information in report
        --[no-]showbackuptype        Include the backup type (Full, Incremental+Full, etc. in report
        --[no-]showhybridbackups     Detect hybrid backups, where a DD backup has data on GSAN (slow)
        --hybridsearchtimeout=n      How long to wait for hybrid backup search avtar calls to return in seconds. Defaults to 300s. Set to 0 to disable timeout.
        --[no-]showlabel             Include the backup's text label in report
        --[no-]showpidnum            Include the backup's plugin id (pid number / pidnum) in report
        --[no-]showpiddesc           Include a short plugin description for each backup in report
        --compression=<none|gzip>    Use the specified compression for the output file. Defaults to gzip.
        --presence=src,dst,both      Limit output to records only on the src, only on the dst, on both systems, or some combination thereof
                                     Shows all records by default (equivalent to --presence=src,dst,both)
        --[no-]header                Include the header line in CSV files

Script Options:

        --debug                      Enable debug logging.
        --memory-limit=n             How much virtual memory the script can use in bytes. Defaults to 4GB.
        --avmgrdebug=<none|getl|getb|all>
                                     Enable debugging for avmgr queries. Defaults to none.
        --avmgr-debug-file=FILENAME  Write avmgr debug information to the specified file. Defaults to drh_avmgr_debug.log.
        --getl-timeout=n             How long to wait for avmgr getl to return in seconds. Defaults to 60s. Set to 0 to disable timeout.
        --getl-retries=n             How many times to retry an avmgr getl command before giving up. Defaults to 5.
        --getb-timeout=n             How long to wait for avmgr getb to return in seconds. Defaults to 300s (5 minutes). Set to 0 to disable timeout.
        --getb-retries=n             How many times to retry an avmgr getb command before giving up. Defaults to 5.
        --encrypt=<none|proprietary|ssl|tls|tls-sa>
                                     Set encryption mode. Defaults to tls.
        --src-encrypt=<none|proprietary|ssl|tls|tls-sa>
                                     Set source encryption mode. Overrides --encrypt if both are specified. Defaults to tls.
        --dst-encrypt=<none|proprietary|ssl|tls|tls-sa>
                                     Set destination encryption mode. Overrides --encrypt if both are specified. Defaults to tls.
    -h, --help                       Display help text
    -v, --version                    Display version information

How to use the script to compare backups that are stored on a pair of Avamar replicating grids:

The dump_root_hashes.rb script can be helpful for identifying surplus or missing backups where customers may have a pair of replicating Avamar grids configured. Generally, the expectation is that capacity levels are similar, and at least up to the previous days' backups have been replicated.

Example:
The following commands should be run on the source Avamar grid. Specify the target Avamar server name in the command and the backupcompare mode is invoked. (The most recent versions of the script automatically report on any backups detected on Data Domain):

ruby dump_root_hashes.rb --dstaddr=<target-avamar-servername>

Failed to connect to GSAN on avamar2.xxx.com. Check name resolution, network connectivity, and firewall settings.

In order for the script to run successfully, an obfuscated password file must be generated by running the script in "genpasswordfile" mode.

For security reasons, credentials cannot be passed directly to the script using the command line:

admin@avamar1:~/>: ruby dump_root_hashes.rb --mode=genpasswordfile

Starting up dump_root_hashes.rb version 0.8.3 at 2018-02-16 14:14:34 +0000 (2018-02-16 14:14:34 UTC)

Running in password file generation mode.

Password filename: avamar2.pass
Username [root]:
Password:

Encoding password
Writing encoded password to /space/home/admin/avamar2.pass

Password information successfully written to /space/home/admin/avamar2.pass

Use --src-password-file=/space/home/admin/avamar2.pass or --dst-password-file=/space/home/admin/avamar2.pass to pass these credentials into the script.

This password file can then be passed to the script using one of the parameters above (depending on whether you are attempting to connect remotely to the source or destination).

Example:

ruby dump_root_hashes.rb --dstaddr=<target-avamar-servername> --dst-password-file=<path-to-password-file> --show-ddr-info

After the script runs to completion, a .csv file is generated named with a format similar to this:
backupcompare_<source>_to_<target>.domain.com_YYYY-MM-DD_HHMM

The file can be opened in Excel and is formatted with the following column names:

• Key
• Root Hash
• On Data Domain
• Source Grid
• Source Domain
• Source Client
• src_created (UTC)
• src_expires (UTC)
• src_retention_type
• src_bytes
• src_labelnum
• src_ddrindex
• Destination Grid
• Destination Domain
• Destination Client
• dst_created (UTC)
• dst_expires (UTC)
• dst_retention_type
• dst_bytes
• dst_labelnum
• dst_ddrindex

These columns can be used to sort and filter the data.

Of particular use in this situation, is the first column 'Key'. The values here indicate whether the backup is stored on the Source system (S), on the Destination (D), or on both systems (B).

We expect the source system may have backups not (yet) present on the target. This is due to natural lag between when the backup is written to the source backup and replicated to the target (usually within the next day).

Watch this video: