This post is more than 5 years old
8 Posts
0
6865
data01 is full
Hi Dears,
My Avamar's /data01 is full, details are as below. How to fix it? (all the backup data can be droped)
admin@avamar1:/data01/>: df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda5 7.9G 3.2G 4.4G 43% /
/dev/sda1 122M 13M 103M 12% /boot
none 18G 0 18G 0% /dev/shm
/dev/sda9 1.8G 1.2G 478M 72% /space
/dev/sda8 1.5G 106M 1.3G 8% /var
/dev/sdb1 4.5T 4.4T 140G 97% /data01
The biggest dirs are /data01/cur and /data01/pool. And it seems there is no many cps.
admin@avamar1:/data01/>: cplist --lscp
cp.20170501040844 Mon May 1 12:08:44 2017 valid rol --- nodes 1/1 stripes 10047
cp.20170501123012 Mon May 1 20:30:12 2017 valid --- --- nodes 1/1 stripes 10047
admin@avamar1:/data01/>: avmaint nodelist | grep percent-full
percent-full="9.7"
fs-percent-full="74.3"
abs-percent-full="96.9"
Please kindly help to advise how to fix this issue.Thanks.
ionthegeek
2K Posts
2
May 5th, 2017 08:00
GC will fail if fs-percent-full is above 85%. As nr mentioned above, 3.3TB nodes have a file pool that consumes most of the disk space. Data will be added or removed from the pool containers as needed but the actual disk utilization of /data01 will not change. The /data01 partition looks full but it isn't.
It looks like your system is about 15% full right now (9.7 / 65) * 100 ~= 15%. There is lots of free space available for backups.
If you decide to follow the procedures nr mentioned in his previous reply, please keep in mind that both the software and hardware you are using have reached end of service life so if you run into problems, you will NOT be able to call support.
ionthegeek
2K Posts
0
May 3rd, 2017 09:00
This is expected for 3.3TB nodes. What problem are you trying to solve?
MaHongbin
8 Posts
0
May 3rd, 2017 19:00
/data01 is full. And GC will be failed if the fs usage is over 85%。 The biggest dir under /data01 is Cur and pool. How to clean /data01 so that the server will run normally？It seems there is no too much useless cps .
MaHongbin
8 Posts
0
May 3rd, 2017 22:00
Hello Anderson,
Is there any method to wipe all the useless data under /data01? All the backup data and user accounts can be dropped, I just want an empty Avamar server to restart backup jobs. This Avamar's OS version is very old,5.0.3. Thank you very much.
Avamar Exorcist
462 Posts
1
May 4th, 2017 03:00
3.3TB nodes use a 'file pool' so it is not helpful to use the df command to check the capacity level of the data partition. The "filepool" acts as a container for stripes and as Ian mentioned this behaviour (/data01 being almost 100%) is expected.
The third command you ran is the one which provides useful information
admin@avamar1:/data01/>: avmaint nodelist | grep percent-full
percent-full="9.7"
fs-percent-full="74.3"
abs-percent-full="96.9"
This tells you that the 9.7% of space within the filepool is taken up by backup data and that the space within the filepool is 74.3% consumed.
If you're looking to 'empty' your Avamar server to start afresh please review the following
KB 419031 - Methods to wipe all backup data from an Avamar server and start again - "factory reset"
MaHongbin
8 Posts
0
May 25th, 2017 10:00
Thanks guys.
Backup failed again with 2 stripes offline_media_error. I think it is caused by os capacity issue too.
MaHongbin
8 Posts
0
May 25th, 2017 10:00
Thanks for your kindly help,guys.
My backups failed again yesterday with " 2 stripes OFFLINE_MEDIA_ERROR". I guess the root cause is still capacity issue. Since it is a single node,I cannot repair it using avmaint command and have to rollback gsan to the latest cp. May I ask your help one more to advise how to fix the capacity issue? If I wipe all backup data using "factory reset", I will lose my avamar license. Thank you very much again..
ionthegeek
2K Posts
1
May 30th, 2017 11:00
It sounds like the file pool has run out of a particular size of file. This isn't something I recommend you try to fix on your own. If you contact support or your account team, you might be able to pay for a one-off engagement to get the system back up and running. I can't promise they'll help you but they might.
The hardware and software version you are using are no longer in support and will no longer receive security fixes. I recommend you move to a supported configuration as soon as you can.
MaHongbin
8 Posts
0
June 14th, 2017 21:00
OK
thanks!