Data Domain: Retention Lock Frequently Asked Questions

• Governance: The less strict of the two retention lock functions (that is locks against files can be reverted if necessary).
• Compliance: The stricter of the two functions which adhere to several common regulatory standards. That is, locks against files cannot be reverted. The DDR must be configured with a 'security officer' user who must authenticate certain commands. There are various restrictions on other functionality to prevent locked data from being removed or locks being reverted early.

• The NFS, CIFS, and DD Boost protocols are fully supported against MTrees using retention lock governance or compliance mode.
• The Virtual Tape Library (VTL) protocol is only supported against MTrees using retention lock governance mode. Automatic retention lock is not supported on Data Domain VTL. See the Data Domain Administration Guide to determine how to unlock retention-locked tapes such that they can be written to.

• SEC 17a-4(f)
• CFTC Rule 1.31b
• FDA 21 CFR Part 11
• Sarbanes-Oxley Act
• IRS 98025 and 97-22
• ISO Standard 15489-1
• MoREQ2010

• A retention lock governance license is added to the DDR.
• Retention lock governance mode is enabled against any required MTree:     

# mtree retention-lock enable mode governance mtree [mtree]

• There are specific instructions for some newer Data Domain models with iDRAC.
• A retention lock compliance license is added to the DDR.
• A user with the role of 'security' should be created (assuming such a user does not exist):     

(ADMIN USER) # user add [username] role security

•  The user with the role of 'security' should log in to the DDR and enable security user authorization:     

(SECURITY USER): # authorization policy set security-officer enabled

• The system should be configured for retention lock compliance mode. Once the following command runs to completion, the system reboots automatically:     

(ADMIN USER) # system retention-lock compliance configure

• Once the system has rebooted, retention lock compliance mode should be enabled on the system:     

(ADMIN USER) # system retention-lock compliance enable

• Retention lock compliance mode is enabled against any required MTree:

(ADMIN USER) # mtree retention-lock enable mode compliance mtree [mtree]

How is it possible to determine which MTrees have retention lock enabled?
MTrees with retention lock enabled are indicated in the output of 'mtree list', for example:     

sysadmin@ddxxxx# mtree list
Name                                Pre-Comp (GiB)   Status    Tenant-Unit
---------------------------------   --------------   -------   -----------
...
/data/col1/rich-retention-lock                 0.0   RW/RLGE   -
/data/col1/rl_test                             0.0   RW/RLGD   -
/data/col1/rl_test_comp                        0.0   RW/RLCE   -
/data/col1/test                                3.1   RW/RLGE   -
...
---------------------------------   --------------   -------   -----------
...
 RLGE : Retention-Lock Governance Enabled
 RLGD : Retention-Lock Governance Disabled
 RLCE : Retention-Lock Compliance Enabled

Can an MTree with Retention Lock Governance enabled be converted to Retention Lock Compliance?
As stated in the DDOS Administration Guide, this is not possible.

Can an MTree with Retention Lock Compliance enabled be converted to Retention Lock Governance?
As stated in the DDOS Administration Guide, this is not possible.

How are file retention or lock periods set?
Once a retention lock is enabled against an MTree, a minimum and maximum retention period must be set. These periods dictate the minimum and maximum time a file within the MTree can be locked for. For example:     

# mtree retention-lock set min-retention-period [period] mtree [mtree]
# mtree retention-lock set max-retention-period [period] mtree [mtree]

Periods can be given in various units as follows:     

• 1 min
• 1 hr
• 1 day
• 1 mo
• 1 year

How can existing retention periods be displayed?
This can be done using the following two commands:     

# mtree retention-lock show min-retention-period mtree [mtree]
# mtree retention-lock show max-retention-period mtree [mtree]

For example:     

sysadmin@dd630# mtree retention-lock show min-retention-period mtree /data/col1/rl_test
Retention-lock min-retention-period of mtree /data/col1/rl_test is: 720 minutes.
sysadmin@dd630# mtree retention-lock show max-retention-period mtree /data/col1/rl_test
Retention-lock max-retention-period of mtree /data/col1/rl_test is: 30 days.

How are files within an MTree with retention lock enabled locked?

• When retention lock is enabled against an MTree, existing files within the MTree are not automatically locked (that is, all preexisting files remain read or write).
• When a new file is written to an MTree with retention lock enabled, the file is not automatically retention locked. That is, the new file remains as read or write.

• To retention lock a specific file, the atime of the file must be modified to match the date, and time the file should be retention locked. That is the date and time until which the file should remain read-only. Until the time is modified in this way, the file cannot be retention locked (and can be modified or removed).

A file's atime can be changed from an NFS or CIFS client using the 'touch' command:

# touch -a -t [expiry time] [file to be locked]

For example, to set atime of /data/col1/rl_test/testfile to 07:05 on 8 June: 

# touch -a -t 06080705 /data/col1/rl_test/testfile

The period from current time to future atime must be within the minimum and maximum retention periods for the MTree. Otherwise, an error is generated when modifying the files atime: 

# touch -a -t 08080705 /data/col1/rl_test/testfile
touch: setting times of `/data/col1/rl_test/testfile': Permission denied

A corresponding message is also shown in the Data Domain File System (DDFS) log files:     

06/07 13:44:57.197 (tid 0x2b28ee5258c0): Attempt to set atime of /data/col1/rl_test/testfile to larger than maximum retention period of mtree.

CIFS (Windows) clients do not, by default, include a touch command or utility however several such utilities are freely available for download from various third-party websites.

Which backup applications support automatically retention locking files after writing them to a DDR?
Data Domain Retention Lock is compatible with industry-standard, NAS-based Write-Once-Read-Many (WORM) protocols. Integration is qualified with archive applications such as Symantec Enterprise Vault, SourceOne, Cloud Tiering Appliance, or DiskXtender.

For Dell NetWorker, both governance and compliance modes are supported.

As of June 2024, recent Avamar versions support both Data Domain Retention Lock Compliance and Governance) as part of Avamar "Immutable Backups" feature. See the article below and the Avamar documentation for details:
Avamar and Data Domain: Enabling Avamar Immutable Backups and Data Domain Compliance Mode Retention Lock

It is critical that the steps to enable the Avamar "Immutable Backups" feature are followed strictly. Failing to do so may end up with an Avamar MTree in the DD which cannot be further written to, or which has operational issues which force lengthy recovery. Avamar does not work with DD Automatic Retention Lock (ARL), and ARL must not be enabled for any Avamar MTree on a DD.

Customers using other backup applications which do not natively support Data Domain retention lock can also develop custom scripts to use Data Domain Retention Lock to manually set retention periods of files. In this scenario, ensure that custom scripts set file's atime such that they are unlocked before the backup application attempting to delete the file. Failure to do this can result in the backup application attempting to delete locked files (which fails); The file is left on the DDR indefinitely consuming disk space. See Data Domain administration guide.

Automatic Retention Lock
For backup applications which do not natively support the Data Domain retention lock feature, it has always been an issue for customers to leverage the feature. The backup administrator must configure scripts to set the retention lock on newly ingested files for an MTree. The lock must be to be set so that it expires shortly before the backup is to be expired (and deleted) by the backup administrator.

ARL does set a lock on each and every file written to the MTree after the feature has been enabled, preventing the file to be changed or removed for a given period of time after the cooling-off period configured has passed. That means ARL must not be enabled for workloads or backup applications that do must update some files repeatedly over time, for example, on VTL pools (VTL tape files are repeatedly written to), or for backup applications which keep metadata information alongside customer backup files themselves (such as NetWorker or Avamar). Enabling ARL in these instances to have important files locked for some time, and when these files must be written to later for other backups, the write fails, and so will any subsequent backups.

To help backup administrators lower the burden of keeping retention lock son backup files and bring the DDOS up on feature parity with other vendors, since DDOS 6.2.0.20 there is a feature that can be enabled from the CLI for each MTree with Retention Lock configured, to set a lock on every file ingested (for a given period), after some predetermined amount of time has passed since the file write to disk was completed. This way, administrators do not have to worry anymore about manually (or scripted) setting the retention lock, and this can occur automatically without the cooperation of the backup application. 
Before DDOS 7.8, Automatic Retention Lock cannot be used on DD Boost Logical Storage Units (LSU) and attempting to enable this returns an error stating that it is not supported.
From 7.8 onwards, ARL is supported for DD Boost LSUs.

(ARL used on Target DDs for Managed File Replication (MFR), like NW clone should have a long enough "automatic-lock-delay" to ensure that clone operations are complete for the backup set before lock is set on the files. Example: A small file that is part of a backup set finishes replicating quickly while a larger file takes longer, then the first file is retention locked by the time the larger file finishes replicating and encounters an error when NW attempts to move all the files of the backup set to the final archival directory.)

Automatic retention lock is not supported on Data Domain VTL.

On applicable versions, there are additional options in the "mtree retention-lock" CLI, as shown below. This feature can be configured through the UI as well by choosing "Automatic" instead of "Manual" in the "Use" option:     

# mtree retention-lock set
{min-retention-period | max-retention-period |
automatic-retention-period | automatic-lock-delay} <period>
mtree <mtree-path>

Automatic retention lock feature locks a file immediately after a preconfigured cool-off period expires (automatic-lock-delay). After the file is written to a retention lock enabled MTree, and the lock is valid for "automatic-retention-period" from the moment it was set, if the value is within the "min-retention-period" and "max-retention-period" values set for the MTree, the lock occurs.

For more usage and general details about the feature, check the corresponding DDOS Administration Guide. This feature is not well suited for situations in which the same MTree is used as the target for backups which should either have lock sets for different periods, or backups which should and backups which should have not a lock set to start with.

What can or cannot be done against a locked file?

• Retention-locked files are read-only and cannot be modified or deleted.
• Once a file's retention period expires, it is 'unlocked' - when in an unlocked state the file still cannot be modified however it can be deleted. The DDR does not automatically delete a file when its retention period expires (subsequent deletion has to be performed from a client system or backup application).
• Once set the retention period for a specific file cannot be reduced (that is the files atime cannot be brought forwards).
• Retention periods can, however, be increased (atime can be increased up to the maximum retention period for the MTree).
• Ownership and permission settings for a file can continue to be modified while the file is locked
• It is only possible to rename or delete a directory in a retention lock enabled MTree if that directory contains no files. If the directory contains files (even if these files are not retention locked) the rename, or delete of the directory fails
• Even if it does not change the file contents itself, it is not allowed to change the name (rename) files which have a lock set, but the rename is also disallowed once the file's lock has expired. For no-longer locked files, the only operation which is allowed is a delete. This is changed in DDOS 7.7.4 when, for no-longer locked files, a rename of the file is permitted.

Is it possible to completely remove the retention lock against a file or set of files?
It is possible to 'revert' (remove) the retention lock against files in MTrees using governance mode - this is done with the following command:     

# mtree retention-lock revert [path]

Once the retention lock against a file is removed, it can be modified or deleted as normal. If this command is run against a directory, then all files within that directory and all subdirectories have their retention locks removed.

It is not possible to revert a retention lock against files in MTrees using compliance mode - if this is attempted a corresponding error is displayed:     

# mtree retention-lock revert /data/col1/rl_test_comp/testfile
This operation is not allowed. Mtree is in retention-lock compliance mode.

What happens if a retention-locked file is attempted to be modified or removed?
Any attempt to modify or delete a retention-locked file causes a corresponding 'permission denied' error:     

# echo " test2" >> /data/col1/rl_test/testfile
bash: testfile: Permission denied
# rm testfile
rm: remove write-protected regular file `testfile'? y
rm: cannot remove `testfile': Permission denied

DDFS logs indicate that the operation has failed due to the file being retention locked:     

06/07 07:06:59.756 (tid 0x2b5a77605d50): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.
06/07 07:07:42.504 (tid 0x2b5a79111390): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.

Is it possible to list all files which are retention locked?
Yes this can be performed using the 'mtree retention-lock report generate retention-details' command:     

mtree retention-lock report generate retention-details
mtrees {<mtree-list> | all}
[format {text | tsv | csv}]
[output-file <file-name>]
               Report detailed information of
               retention-lock files.

For example to list details of all locked files in the /data/col1/rl_test MTree the following would be performed:     

sysadmin@ddxxxx# mtree retention-lock report generate retention-details mtrees /data/col1/jftest
Report generated on: Fri Jul  1 14:19:31 2016

Report for mtree: /data/col1/jftest
File Path        Mode        Size(Bytes)        Expiration Date
/data/col1/jftest/file1 governance 10521456 Sat Jul  2 22:35:48 2016
/data/col1/jftest/testdir/file2 governance 10521680 Sat Jul  2 22:35:42 2016
/data/col1/jftest/file3 governance 10521820 Sun Jul 10 22:36:09 2016
Total files: 3

Is it possible to completely disable retention lock for an MTree (after it is enabled)?
Yes for MTrees using governance mode, this is performed using the 'mtree retention-lock disable' command:    

# mtree retention-lock disable mtree [mtree]

For example:     
sysadmin@xxxx# mtree retention-lock disable mtree /data/col1/rl_test
Retention-lock feature is disabled (previously enabled) for mtree /data/col1/rl_test.

Once disabled, MTree list indicates that retention lock was used against the MTree but has since been disabled, that is: 
sysadmin@ddxxxx# mtree list
Name                                Pre-Comp (GiB)   Status    Tenant-Unit
---------------------------------   --------------   -------   -----------
...
/data/col1/rl_test                             0.0   RW/RLGD   -
...

sysadmin@ddxxxx# mtree retention-lock revert /data/col1/rl_test/testfile
**** Retention-lock feature is disabled (previously enabled) for mtree which contains the path /data/col1/rl_test/testfile.

• Retention lock cannot be disabled for an MTree using compliance mode:     

sysadmin@ddxxxx# mtree retention-lock disable mtree /data/col1/rl_test_comp
**** Operation is not allowed because the system is a retention-lock compliance system

• Directory replication - only supported for files using governance mode - does not replicate minimum and maximum retention periods to the destination system.
• MTree replication - can be used for either governance or compliance mode data and does replicate minimum and maximum retention periods to the destination system.
• Collection replication - can be used for either governance or compliance mode data and does replicate minimum and maximum retention periods to the destination system.

# replication break mtree://[destination system]/data/col1/[mtree]

# replication add source mtree://[source system]/data/col1/[mtree] destination mtree://[destination system/data/col1/[mtree]

# mtree retention-lock enable mode compliance mtree [mtree]

# replication resync mtree://[destination system/data/col1/[mtree]

# user reset
# filesys destroy
# filesys archive unit del [archive unit name]

# mtree delete [mtree]
# mtree retention-lock reset [min-retention-period period | max-retention-period period] mtree [mtree]
# mtree retention-lock disable mtree [mtree]
# mtree retention-lock revert

# cloud unit del <cloud unit name>