Backup strategy for a large amount of data

Hi @RimvydasLT !
What is the current load on you backup server? What type of drives do you have in you TL ?
You might want to do a dedicated storage node for this Windows host but depends on the current load on you backup server? And NVE with a storage node attached to the TL?
Please provide as much a details as possible.

@crazyrov​ Hi,

So my environment. NVE: backup server is with storage node which is attached to tape library over FC links. No load on backup server as this is completely new installation. Library has two LTO8 tape drives with 80 LTO8 tapes in it.

Windows server storage layout (which needs to be backed up). This server has a plenty of LUNs attached (FC). Each LUN is 10TB in size. Each lun is mapped to some folder in the same parent folder. When the lun is filled up, new one is added. Data grow is three new luns during one month. Client want to be able to restore three latest added luns in 1 hour if possible so I suspect that AFTD device will also be needed.

This is my first encounter with networker, so I badly need some insights from someone who knows product well. Btw. saw your videos on youtube :)

Thanks!
Now, LTO8 can go up to 750MB/s but because you have many small files the over all speed may be impacted. But even with this speed a full backup can take more than 3hrs. All this when a drive is directly attached to the Windows machine for a LAN free backup.
The optimal solution for you situation is to have a DDVE and configure blockbased backup. You can test the performance using a DDVE which is free to use upto 500GB, you should be able to figure out which option gives you a better performance for backup and restore.
Refer - https://www.dell.com/support/manuals/en-sg/networker/nw_p_nwadmin_19.9/devices-for-block-based-backups?guid=guid-d9aa5618-e736-449c-932a-0cf5f6637f94&lang=en-us

@crazyrov​ 

Hi,

My LTO8 drive speed is somewhere about 360MB/s or are you talking about two drives combined speed? I think that DDVE is out of the question, as client will not spend any additional money on licenses, so I have only two tape drives available(TS4300), 80 tapes and some disk storage for AFTD device. But when I was asking about backup strategy I asked more for insights about backup jobs configuration, expiration times, etc.. Also note, that I need to backup 500TB initially, so this will be not three hours. And looking at the tapes available, I suspect that I'll not be able to fit two full backups onto these tapes, so maybe I'll need to use forever incremental backups?:/. And also client wishes to be able to restore three last added drives in one hour (maybe AFTD device for that). So I'm more than interested about your insights regarding jobs configuration having what I have right now.

You are right on the speed, I think I saw some wrong documentation.
Incremental forever is the work configuration if you are using tapes because if ever you need to do a full data restore then it might become nearly impossible.
Can you elaborate what you mean by "restore the last added drives"? AFTD is cost wise very expensive as you will need a 1:1 space for the backup volume. How much volume you looking at restoring in 1 hour ?
Retention and schedule is more a RTO and RPO related configuration, there is no best practice around it. These need to be set as per the SLA that you have with your customer.

@crazyrov​ As I said, each drive is mapped to folder. So I have following structure: c:\folder\disk01; c:\folder\disk02; c:\folder\disk03 etc. So by saying last drive I mean c:\folder\disk03 with the condition if last drive added was mapped to c:\folder\disk03.

Another thing to note is that I have only 80 tapes, so looking at the fack that I have almost 600TB data to backup I even will not be able to have two full backups onto these tapes if I configure backup job. This server has almost 80 LUNs attached to it so my thought was maybe to archive 75 of them and then to configure 5 luns for normal backup (to AFTD device) and with staging to the same archive pool - when new LUN will be added - I'll add it to this backup job and will stage oldest one to archive backup pool and remove it from backup job,

I think about AFTD device only because of that, because client wants to be able to restore last added LUN (c:\folder\disk03, 10TB) via 1 hour. And even with AFTD I see that it will be impossible, as restore speed to disk will be no more that 600MB/s. Could you advice some solution here?

But there is one but with this strategy. I'm not able to find how can I archive these 75 LUN's from the server side:/ Can you help me with this? Was not able to find any info how to do archiving from the server's side in the manual:(

@RimvydasLT Why do you want to go with archive instead of backup with a large retention ? Id the data that is being backed never going change ?

man-o-man, this is wrong on so many levels... how can there be already backup infra and a SLA expectation, while the actual backup solution is not build around that?

And I am not even talking about the whole ever-increasing windows server? Is this a "simple" file server or are we dealing here with a specific application that is generating/collecting all this data?

In my opinion regular Windows fileservers is a bad idea. But alas as it is regarded as being way cheaper than a proper NAS, more often that not, still Windows file servers are being used.


I for one am glad that most Windows file servers are gone and more and more is located on a proper NAS (Netapp), where the nas takes care of a proper backup by having an approach of needing two NAS systems (regardless if high available or not) so that they can make remote snapshots to eachother and local snapshots. That in and by istelf is a very good solution to backup NAS data. We only make some compliance backups, once a month/quarter/year, whereas daily or higher frequency backups are ONLY made on the NAS itself. This is as it should be, not a half-baked solution using a Windows server.

Not saying this is the case here however? So what are we ever talking about here?

10TB restore within 1 hour? That entails 2.7GB per second during restore. Not something you will ever be able to achieve with two tape drives. So if that is an actual requirement, than also the proper backup infra is required for that to handle such load and satisfy the SLA RTO.

The 360MB/s is something you already achieve at the moment, or what you hope for? Restore is also typically slower than backup.

For high density filesystems, with millions of files, normally BBB (block based backups) is the way to go, to prevent the filesystem overhead to be the factor to slow things down to a gridning halt. You'd lose the ability to see individual files listed in the NW client index, but can stil restore individual files. However it requires a disk based device at first to write the backups to or be used as staging area to restore from.

Hence where the reamrk fro data domain came from. In case of AFTD, you'd need a whopping amount of disk space also to accomdate the backups, before cloning/staging them to disk...

I for one am very happy, we could let go of tape altogether in favor of only data domain.

what version of NVE do you have ? also , why set primary backup target as PTL while backing up 500tb? this metod would not utilize CBT , dedupe ... wrong solution . highly advice use AFTD at least and PTL as archining solution