What is a retention lock?
Retention lock is a feature used on Data Domain Restorers (DDRs) to prevent modification or deletion of certain sets of files for a predetermined period. That is, retention locked files are read-only until their retention period expires.
What are the different versions of retention lock?
Retention lock is available for two different functions:
Which data access protocols are supported with retention lock?
Retention lock compliance mode meets regulatory standards:
The list of regulatory standards that retention lock compliance mode meets includes the following:
For full details of certification information, contact your contracted support provider.
How is retention lock governance enabled?
# mtree retention-lock enable mode governance mtree [mtree]
How is retention lock compliance enabled?
(ADMIN USER) # user add [username] role security
(SECURITY USER): # authorization policy set security-officer enabled
(ADMIN USER) # system retention-lock compliance configure
(ADMIN USER) # system retention-lock compliance enable
(ADMIN USER) # mtree retention-lock enable mode compliance mtree [mtree]
How is it possible to determine which MTrees have retention lock enabled?
MTrees with retention lock enabled are indicated in the output of mtree list, for example:
sysadmin@ddxxxx# mtree list Name Pre-Comp (GiB) Status Tenant-Unit --------------------------------- -------------- ------- ----------- ... /data/col1/rich-retention-lock 0.0 RW/RLGE - /data/col1/rl_test 0.0 RW/RLGD - /data/col1/rl_test_comp 0.0 RW/RLCE - /data/col1/test 3.1 RW/RLGE - ... --------------------------------- -------------- ------- ----------- ... RLGE : Retention-Lock Governance Enabled RLGD : Retention-Lock Governance Disabled RLCE : Retention-Lock Compliance Enabled
Can an MTree with Retention Lock Governance enabled be converted to Retention Lock Compliance?
As stated in the DDOS Administration Guide, this is not possible.
Can an MTree with Retention Lock Compliance enabled be converted to Retention Lock Governance?
As stated in the DDOS Administration Guide, this is not possible.
How are file retention or lock periods set?
Once a retention lock is enabled against an MTree, a minimum and maximum retention period must be set. These periods dictate the minimum and maximum time a file within the MTree can be locked for. For example:
# mtree retention-lock set min-retention-period [period] mtree [mtree]
# mtree retention-lock set max-retention-period [period] mtree [mtree]
Periods can be given in various units as follows:
How can existing retention periods be displayed?
This can be done using the following two commands:
# mtree retention-lock show min-retention-period mtree [mtree] # mtree retention-lock show max-retention-period mtree [mtree]
For example:
sysadmin@dd630# mtree retention-lock show min-retention-period mtree /data/col1/rl_test
Retention-lock min-retention-period of mtree /data/col1/rl_test is: 720 minutes.
sysadmin@dd630# mtree retention-lock show max-retention-period mtree /data/col1/rl_test
Retention-lock max-retention-period of mtree /data/col1/rl_test is: 30 days.
How are files within an MTree with retention lock enabled locked?
A file's atime can be changed from an NFS or CIFS client using the 'touch' command:
# touch -a -t [expiry time] [file to be locked]
For example, to set atime of /data/col1/rl_test/testfile to 07:05 on 8 June:
# touch -a -t 06080705 /data/col1/rl_test/testfile
The period from current time to future atime must be within the minimum and maximum retention periods for the MTree. Otherwise, an error is generated when modifying the files atime:
# touch -a -t 08080705 /data/col1/rl_test/testfile
touch: setting times of `/data/col1/rl_test/testfile': Permission denied
A corresponding message is also shown in the Data Domain File System (DDFS) log files:
06/07 13:44:57.197 (tid 0x2b28ee5258c0): Attempt to set atime of /data/col1/rl_test/testfile to larger than maximum retention period of mtree.
CIFS (Windows) clients do not, by default, include a touch command or utility however several such utilities are freely available for download from various third-party websites.
Which backup applications support automatically retention locking files after writing them to a DDR?
Data Domain Retention Lock is compatible with industry-standard, NAS-based Write-Once-Read-Many (WORM) protocols. Integration is qualified with archive applications such as Symantec Enterprise Vault, SourceOne, Cloud Tiering Appliance, or DiskXtender.
For Dell NetWorker, both governance and compliance modes are supported.
As of June 2024, recent Avamar versions support both Data Domain Retention Lock Compliance and Governance) as part of Avamar "Immutable Backups" feature. See the article below and the Avamar documentation for details:
Avamar and Data Domain: Enabling Avamar Immutable Backups and Data Domain Compliance Mode Retention Lock
It is critical that the steps to enable the Avamar "Immutable Backups" feature are followed strictly. Failing to do so may end up with an Avamar MTree in the DD which cannot be further written to, or which has operational issues which force lengthy recovery. Avamar does not work with DD Automatic Retention Lock (ARL), and ARL must not be enabled for any Avamar MTree on a DD.
Customers using other backup applications which do not natively support Data Domain retention lock can also develop custom scripts to use Data Domain Retention Lock to manually set retention periods of files. In this scenario, ensure that custom scripts set file's atime such that they are unlocked before the backup application attempting to delete the file. Failure to do this can result in the backup application attempting to delete locked files (which fails); The file is left on the DDR indefinitely consuming disk space. See Data Domain administration guide.
Automatic Retention Lock
For backup applications which do not natively support the Data Domain retention lock feature, it has always been an issue for customers to leverage the feature. The backup administrator must configure scripts to set the retention lock on newly ingested files for an MTree. The lock must be to be set so that it expires shortly before the backup is to be expired (and deleted) by the backup administrator.
ARL does set a lock on each and every file written to the MTree after the feature has been enabled, preventing the file to be changed or removed for a given period of time after the cooling-off period configured has passed. That means ARL must not be enabled for workloads or backup applications that do must update some files repeatedly over time, for example, on VTL pools (VTL tape files are repeatedly written to), or for backup applications which keep metadata information alongside customer backup files themselves (such as NetWorker or Avamar). Enabling ARL in these instances to have important files locked for some time, and when these files must be written to later for other backups, the write fails, and so will any subsequent backups.
To help backup administrators lower the burden of keeping retention lock son backup files and bring the DDOS up on feature parity with other vendors, since DDOS 6.2.0.20 there is a feature that can be enabled from the CLI for each MTree with Retention Lock configured, to set a lock on every file ingested (for a given period), after some predetermined amount of time has passed since the file write to disk was completed. This way, administrators do not have to worry anymore about manually (or scripted) setting the retention lock, and this can occur automatically without the cooperation of the backup application.
Before DDOS 7.8, Automatic Retention Lock cannot be used on DD Boost Logical Storage Units (LSU) and attempting to enable this returns an error stating that it is not supported.
From 7.8 onwards, ARL is supported for DD Boost LSUs.
(ARL used on Target DDs for Managed File Replication (MFR), like NW clone should have a long enough "automatic-lock-delay" to ensure that clone operations are complete for the backup set before lock is set on the files. Example: A small file that is part of a backup set finishes replicating quickly while a larger file takes longer, then the first file is retention locked by the time the larger file finishes replicating and encounters an error when NW attempts to move all the files of the backup set to the final archival directory.)
Automatic retention lock is not supported on Data Domain VTL.
On applicable versions, there are additional options in the mtree retention-lock CLI, as shown below. This feature can be configured through the UI as well by choosing "Automatic" instead of "Manual" in the "Use" option:
# mtree retention-lock set
{min-retention-period | max-retention-period |
automatic-retention-period | automatic-lock-delay} <period>
mtree <mtree-path>
Automatic retention lock feature locks a file immediately after a preconfigured cool-off period expires (automatic-lock-delay). After the file is written to a retention lock enabled MTree, and the lock is valid for "automatic-retention-period" from the moment it was set, if the value is within the "min-retention-period" and "max-retention-period" values set for the MTree, the lock occurs.
For more usage and general details about the feature, check the corresponding DDOS Administration Guide. This feature is not well suited for situations in which the same MTree is used as the target for backups which should either have lock sets for different periods, or backups which should and backups which should have not a lock set to start with.
What can or cannot be done against a locked file?
Is it possible to completely remove the retention lock against a file or set of files?
It is possible to 'revert' (remove) the retention lock against files in MTrees using governance mode - this is done with the following command:
# mtree retention-lock revert [path]
Once the retention lock against a file is removed, it can be modified or deleted as normal. If this command is run against a directory, then all files within that directory and all subdirectories have their retention locks removed.
It is not possible to revert a retention lock against files in MTrees using compliance mode - if this is attempted a corresponding error is displayed:
# mtree retention-lock revert /data/col1/rl_test_comp/testfile
This operation is not allowed. Mtree is in retention-lock compliance mode.
What happens if a retention-locked file is attempted to be modified or removed?
Any attempt to modify or delete a retention-locked file causes a corresponding permission denied error:
# echo " test2" >> /data/col1/rl_test/testfile
bash: testfile: Permission denied
# rm testfile
rm: remove write-protected regular file `testfile'? y
rm: cannot remove `testfile': Permission denied
DDFS logs indicate that the operation has failed due to the file being retention locked:
06/07 07:06:59.756 (tid 0x2b5a77605d50): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.
06/07 07:07:42.504 (tid 0x2b5a79111390): Atime of retention-lock file /data/col1/rl_test/testfile is not expired.
Is it possible to list all files which are retention locked?
Yes, this can be performed using the mtree retention-lock report generate retention-details command:
mtree retention-lock report generate retention-details
mtrees {<mtree-list> | all}
[format {text | tsv | csv}]
[output-file <file-name>]
Report detailed information of
retention-lock files.
For example to list details of all locked files in the /data/col1/rl_test MTree the following would be performed:
sysadmin@ddxxxx# mtree retention-lock report generate retention-details mtrees /data/col1/jftest
Report generated on: Fri Jul 1 14:19:31 2016
Report for mtree: /data/col1/jftest
File Path Mode Size(Bytes) Expiration Date
/data/col1/jftest/file1 governance 10521456 Sat Jul 2 22:35:48 2016
/data/col1/jftest/testdir/file2 governance 10521680 Sat Jul 2 22:35:42 2016
/data/col1/jftest/file3 governance 10521820 Sun Jul 10 22:36:09 2016
Total files: 3
Is it possible to completely disable retention lock for an MTree (after it is enabled)?
Yes, for MTrees using governance mode, this is performed using the mtree retention-lock disable command:
# mtree retention-lock disable mtree [mtree] For example: sysadmin@xxxx# mtree retention-lock disable mtree /data/col1/rl_test Retention-lock feature is disabled (previously enabled) for mtree /data/col1/rl_test. Once disabled,MTree listindicates that retention lock was used against the MTree but has since been disabled, that is: sysadmin@ddxxxx#mtree listName Pre-Comp (GiB) Status Tenant-Unit --------------------------------- -------------- ------- ----------- ... /data/col1/rl_test 0.0 RW/RLGD - ...
sysadmin@ddxxxx# mtree retention-lock revert /data/col1/rl_test/testfile **** Retention-lock feature is disabled (previously enabled) for mtree which contains the path /data/col1/rl_test/testfile.
sysadmin@ddxxxx# mtree retention-lock disable mtree /data/col1/rl_test_comp **** Operation is not allowed because the system is a retention-lock compliance system
# replication break mtree://[destination system]/data/col1/[mtree]
# replication add source mtree://[source system]/data/col1/[mtree] destination mtree://[destination system/data/col1/[mtree]
# mtree retention-lock enable mode compliance mtree [mtree]
# replication resync mtree://[destination system/data/col1/[mtree]
# user reset
# filesys destroy
# filesys archive unit del [archive unit name] Such systems cannot be booted to single user mode for recovery by technical support without use of a USB drive and physical access to the system.
# mtree delete [mtree] # mtree retention-lock reset [min-retention-period period | max-retention-period period] mtree [mtree] # mtree retention-lock disable mtree [mtree] # mtree retention-lock revertHowever, in DDOS 7.3 and later releases, a provision has been made for customers to be able to delete Retention Lock Compliance enabled MTrees if:
# cloud unit del <cloud unit name>
# filesys enable