PowerScale OneFS 9.6.x.x CLI Administration Guide

Datamover policies

Data transfers are coordinated according to Datamover policies and jobs. Policies specify what data is transferred, where the data is transferred to, and how often the data is transferred. You can also specify the policy expiration schedule.

There are four types of dataset policies that you can configure.

CREATION policy—the process of creating the dataset
One-time COPY policy—used for one-time data transfers
REPEAT_COPY policy—used for repeated transfers
EXPIRATION policy—how long the snapshot is stored

For detailed command syntax and descriptions, see the OneFS CLI Command Reference, which you can find under the Documentation tab on the PowerScaleOneFS support site: https://www.dell.com/support/home/en-us/product-support/product/isilon-onefs/docs.

The common options across all policies that you can specify are as follows.


Priority	Relative priority of the dataset policy: LOW \| NORMAL \| HIGH
Schedule	Specific date/times at which this policy should run, date/time of the first run of the policy, and recurrence of jobs (repetitive schedule for the policy)
Parent execution policy ID	You can link a scheduled dataset CREATION policy to a REPEAT_COPY policy. Specify this option when creating the COPY policy. The argument of the --parent-exec-policy-id is the policy ID of the dataset CREATION policy. The policy runs automatically when a job for its parent policy completes.
Run now	Create a job for a policy immediately instead of waiting for it to run as scheduled, regardless of existing schedule

The key configuration parameters for the dataset policies are as follows.

Dataset CREATION policy

The dataset creation policy is used to create datasets once—or on a schedule—that can then be moved to other systems using the isi dm policies create command. For example:

isi dm policies create test low true CREATION --creation-base-path=/ifs/home/admin --creation-account-id=local \
--creation-dataset-expiry-action=DELETE --creation-dataset-retention-period=1000 --start-time='2022-12-12 12:12:12'

NOTE: Run the isi dm policies modify --run-now=true command to manually run a policy.

The dataset creates policy and dataset copy policy can be "chained" together so that running the dataset create policy runs the copy once the dataset creation is finished. For example:

isi dm policies create test-copy-chaining NORMAL true COPY --copy-source-base-path=/ifs/data/src \
--copy-base-target-base-path=/ifs/data/test2-tgt --copy-create-dataset-on-target=true \
--copy-base-source-account-id=local --copy-base-target-account-id=local --copy-base-target-dataset-type=FILE \
--copy-base-dataset-retention-period=3600 --copy-base-policy-dataset-expiry-action=DELETE --parent-exec-policy-id=31744

For more information, see the isi dm policies create command in the PowerScale OneFS CLI Command Reference Guide and in the Datamover management tasks section of this guide.

The key configuration parameters are as follows for this policy:

Account ID: The --creation-account-id is the storage system where you want to create the dataset. It can be a local or remote account.

Base Path or Object Store Key Prefix: The --creation-base-path is the file system Base Path or Object Store Key Prefix used to create the dataset.

Expiration action: The --creation-dataset-expiry-action (DELETE) is the action to be taken after the dataset expiration. Only the DELETE action is supported in OneFS 9.4.0.0.

Retention Period: --creation-dataset-retention-period is the time in seconds after creation before the expiration action is performed by an expiration policy.

Reserve: The --creation-dataset-reserve is the number of datasets in the current tree to protect from expiration regardless of the retention period.

Dataset COPY policy

The dataset copy policy is used to define a one-time copy of a dataset to or from a remote system. For example:

isi dm policies create test-copy NORMAL true COPY --copy-source-base-path=/ifs/data/test1-src \
--copy-base-target-base-path=/ifs/data/test2-tgt --copy-create-dataset-on-target=true \
--copy-base-source-account-id=local --copy-base-target-account-id=local --copy-base-target-dataset-type=FILE \
--copy-base-dataset-retention-period=3600 --copy-base-policy-dataset-expiry-action=DELETE --start-time='2022-12-12 12:12:12'

The key configuration parameters are as follows:

Accounts: The --copy-base-source-account-id is the source storage system containing the dataset that you want to copy from.; The --copy-base-target-account-id is the target storage system where you want to copy the dataset to.

Source dataset: The --copy-dataset-id is the ID of a dataset that exists on the source storage system that you want to copy.; The --copy-source-base-path is the base path of the dataset on the source storage system.; The --copy-source-subpaths are an optional list of file system paths relative to the base path. Use this option to only copy a subset of the dataset.

Target dataset: The --copy-create-dataset-on-target is used to define a new dataset on the target storage system once the copy is complete.; The --copy-base-target-dataset-type is the dataset type from one of the following:; A file on an object store in a copy format (FILE_ON_OBJECT_COPY) or a file on file dataset (FILE).

You can run a copy policy more than once, or set an automatic recurrence using a cron string.

Dataset REPEAT_COPY policy

A repeat_copy policy copies a referenced dataset once and then runs incremental syncs to or from a remote storage system. For example:

isi dm policies create test-repeat-copy NORMAL true REPEAT_COPY --repeat-copy-source-base-path=/ifs/data/test1-src \
--repeat-copy-base-target-base-path=/ifs/data/testcopy2-tgt --copy-create-dataset-on-target=true \
--repeat-copy-base-source-account-id=local --repeat-copy-base-target-account-id=local --recurrence="* * * * *" \
--start-time="2022-12-10 12:12:12" --reconnect=false --repeat-copy-base-target-dataset-type=FILE \
--repeat-copy-base-dataset-retention-period=3600 --repeat-copy-base-dataset-expiry-action=DELETE

A repeat copy policy skips the initial baseline sync if the target base path contains a leaf dataset which is an ancestor of a source base path dataset.

The key configuration parameters for this policy:

Accounts: The --copy-base-source-account-id is the source storage system containing the dataset that you want to copy from.; The --copy-base-target-account-id is the target storage system where you want to copy the dataset to.

Source dataset: The --reconnect parameter indicates that a first-time baseline copy is not necessary.; The --repeat-copy-source-base-path is the base path (root) of the dataset on the source storage system to be used automatically.

Target dataset: These are the same options as the dataset creation policy.; The --copy-create-dataset-on-target is used to define a new dataset on the target storage system once the copy is complete.; The --copy-base-target-dataset-type is the dataset type file on file dataset (FILE).

Dataset EXPIRATION policy

The dataset expiration policy creates jobs that look for expired datasets and runs their expiration actions. For example:

isi dm policies create test-expire NORMAL true EXPIRATION --expiration-account-id=local \
--recurrence="0 0 1 * *" --start-time="2022-12-12 12:12:12"

The key configuration parameter is the --expiration-account-id <str> option, which is the storage system where you want to check for expired datasets, and the account where this expiration policy runs.

Welcome

Welcome to Dell