Hello everyone. In this session we are going to see how to set up data movement policy for Avamar backups for it to avail long-term retention feature of Data Domain to send its data to the cloud. For that I have logged into the Avamar UI, the Data Domain UI, as well as the Data Domain CLI. Here our prerequisite would be to ensure Data Domain is successfully integrated with cloud. Then, let's first check whether Data Domain has a cloud-tiered license or not.
We do see with 'elicense show' command that cloud-tier capacity license is present on Data Domain, it's active and permanent. We can check the same through UI as well, of Data Domain. You can log into the Data Domain UI, go to the 'Administration', 'Licenses' tab and you would see the cloud license present. If it's not present you've got to add the license for cloud to your capacity. Next thing we would check is the cloud profile to see which cloud provider it is integrated with. You can see the provider is Azure so it's integrated with cloud, Azure cloud provider. Next we would check the cloud unit, if it is existing, I do see the cloud unit existing with active state, same thing can be checked from Data Domain UI.
Go to the 'Data Management' tab under 'Cloud Units' tab. You would be able to see the cloud unit name, the status is enabled - that means it's active - the cloud profile details, Microsoft Azure storage and the capacity as well. As in 'Summary' tab you will be able to see the cloud tier detail. So this confirms that DD is successfully integrated with cloud. Now let us check regarding the data movement policy.
Currently we see that none of the entries on this Data Domain has a data movement policy existing. In this session we are going to set it up for the Avamar entry, and for Avamar entry it is required that the data movement policy should be app managed, so that means it needs to be set up on Avamar. So for that log in to the Avamar UI, go to the 'System' tab, under 'System' go to the 'Data Domain' tab and click on 'Edit', and put the DDboost credentials which were used for integration of Data Domain and Avamar. Click on 'Validate', it's going to get the information from Data Domain and validate.
That has successfully retrieved information, and allows us to move ahead. So the 'Next' option is now enabled, click on it to move ahead. You can verify the SNMP details, click on 'Next' to go to 'Tiering' tab. We need to come over here, tiering options, here we have a button called 'Enable Cloud Tier'. Once you select this check box you will be able to see the cloud unit as target, which we have created on the Data Domain, so let's do that.
We can see we need to select the target for our cloud backups to be the cloud unit which we have created on Data Domain. The MTree name is the Avamar MTree existing on Data Domain, or you can see the Avamar storage unit. Now if we go back to Data Domain we will be able to see the data moment would be appropriately reflected over here as app managed. Look at here, so we can see the Avamar storage unit on the MTree has a target now as the cloud unit, and the policy is 'app-managed', 'enabled'. Now we have set up the data movement policy, okay, but haven't yet defined what data would be eligible to be moved to cloud.
So that is defined by the tiering policy which is also configured on Avamar. So on Avamar UI go to this 'Cloud Tier Policy' tab. click on 'Add' and provide any policy. You can input a policy name as per your requirement, click on 'Enabled' for the policy to be enabled, then you can click on 'Next' button. Here you can choose to 'Tier All Clients' or 'Choose Membership' where you can choose either whole domain or specific clients as per your requirement.
So in our case let's 'Choose All Clients' for now. Next comes the age threshold, here you need to provide a value. This value will define what data would be eligible to be moved to cloud. So let's say for example here we have 30, so the backups that are older than 30 days would be eligible to be moved to cloud.
So the backup job here is basically the tiering policy that runs on backup only verifies what backups are present and what are eligible to be moved to cloud, and marks those backup. It doesn't actually start moving the data to the cloud, Avamar's job is only to verify and mark the eligible backups to be moved to cloud.
Here we are just defining that value by which it will be verifying what backups are eligible. The other criteria here that we could define is either 'Tier all backups' or do 'Include/Exclude backups by date type'. So if we select that you would be able to see that you have options to select daily, weekly, monthly, yearly, or no tag.
So the long term retention is basically to send or offload backups to cloud that are not frequently used, and if you want to retain them for a longer retention. So preferably you would might, you know, choose monthly and yearly to be moved to cloud. In our case we will choose that because those are not frequently used, and you would retain them for a longer time.
Then you can limit the number of backups per client as well, so depending upon your requirement if you don't want to limit you can choose the 'No limit' option. While defining age threshold always remember that your age threshold is lesser than the retention period that is existing for the backups. Because if age threshold is higher than the retention then your backups would expire even before they could be verified or marked to be eligible to be moved to cloud. So let's say your retention is 60 days make sure your age threshold doesn't exceed 60 days.
It should be lesser than the retention that is existing. Now let's move ahead. Here we can select schedule, there is 'Default Tiering Schedule' present, you can select that from the drop down. It runs every day, so the steering policy would run every day. Again remember this is not going to move the data, it is only going to verify all the backups present for whatever members you have, you know, selected for it to check.
It will check only those members, check the backups of those members and mark the backups, if they are eligible as per the criteria that you have set up in backup filters. Then 'Next', goes the retention now. Here you can either keep the retention of the backups as it is now - as the current retention - or 'Set expiration by backup type'.
That means if you want to set up a retention different than what it is now, what it is currently, for the backups that have been moved to cloud. I f you want to retain them for more longer time than the current retention then you can provide the value over here. So let's say for example the value of monthly backups is currently 60 days and you provided age threshold as 30 days.
So 30 days it would stay in active tier after that it would be marked to be moved to cloud. Once it gets moved to cloud you want to retain them for a longer time, for example let's 120 days. So you will type in here 120 and you want yearly to be retained for 180 days when they move to cloud. So I would change the retention, or you can keep the same retention and then click on 'Next'. Here we would see the Data Domain detail, the cloud unit, the target cloud unit where your backups would be sent to, and then the cloud profile, the provider and its date.
Next goes the summary. You can review it and click on 'Finish'. Once that is done this tiering policy is set up to be run daily as per our schedule, so this will verify and mark the backups that are eligible to be moved. Now what is the process that does the actual data movement? The actual data movement is done by data movement process on Data Domain. So you need to set up its schedule.
You have defined what data would be sent, now when it should be sent needs to be defined on Data Domain because the process is run on Data Domain. 'data-moment schedule show', currently we see it's not set, so we will go to the Data Domain UI, now go to the 'Data Management', 'File System', the right side settings gear icon you will see 'Data Movement' tab, here you see 'Frequency' it is currently 'Never'. ou can set it as per requirement, if you want to run it weekly once, which is yeah, which I have done it now.
So weekly, once every Saturday it would run. The other process that is required for cloud tier to be running is the cloud cleaning which allows the expired backups to be released and reclaims space on cloud tier. So you go to the 'Cleaning' tab uh you can see 'Active Tier' cleaning and 'Cloud Tier' cleaning. So we are going to set up frequency for cloud tier cleaning now which is currently 'Never'. So cloud tier cleaning depends upon the frequency of active tier, so after every nth active tier cleaning you can have the cloud tier cleaning run. So currently I would prefer to type in two, so I want after every two active tier cleaning, cloud tier cleaning would run.
Click on 'Save'. Now we can verify these schedules on CLI, 'data-movement schedule show' and 'cloud clean frequency show'. If the data movement is ongoing or if you want to check the data movement status, you can see it through the command 'data movement status'.
It will tell you if it is ongoing, it will tell you the progress percentage, if it has completed, it will tell you the completed state and everything. If you want to see what files are being moved that means you are, we want to see more details of data movement, then there is a command called 'data-movement status to-tier cloud detailed', see more details of the data moment which is ongoing.
So you can see files currently being moved if the data movement was ongoing. If you want to immediately start data movement, like if you want to start, for a data movement to start offloading data to the cloud, you know, at some instance and you want to do it manually you can run the command 'data-movement start' instead.
If you don't want to wait for you know the schedule to kick in data movement, and if you want to do it immediately you can run the command 'data-movement start' to start it. If you want to immediately start cloud cleaning then you would do 'cloud clean start cloud <unit name>', the cloud unit name. So these are the commands that you can do if you want to immediately start data movement, or you know if you want to immediately start cloud cleaning.
If you want to immediately, uh you know, for Avamar to start running the tiering policy and verify the backups that are present and mark them you will select this policy, and click on 'Run' for it to start running and then see in the activity monitor window to see its progress. So this was all about how to set up the data movement policy and the tiering policy for our backups to be sent to cloud. Hope this session was helpful, thank you everyone for joining.
