What is getlogs?
The first thing that we must do is run "getlogs" on the utility node. Running this on the utility node gathers extensive logs and stores them in a tar file. The tar file is created and stored in the directory that the getlogs script is run from.
The "--server" flag allows us to specify how many days of logs we want to collect. If this flag is not used, then getlogs captures everything, and the tar file can be large.
When using the "--server" flag, ensure that it covers the timeframe of the incident that logs are being collected for. Adding an extra day or two ensures that enough data is captured, and does not add a significant amount of size to the log file. For example, if a node went offline yesterday, use "--server=2" or "--server=3" to ensure that sufficient information is collected.
What does getlogs do when run?
getlogs copies the "getnodelogs" script to every node and run it.
The getnodelogs script bundles all the logs in to a file called "nodelogs.tgz" on each node. getlogs then gathers all the "nodelogs.tgz" files, and add them into one big logs.xxxx.tar file.
The "logs.xxxx.tar" file is the file to collect. Depending on the file size, it should either be attached to the Service request, or uploaded to the MFT site (this information will be provided by your Avamar support engineer if required).
It is important to understand the distinction between getlogs vs getnodelogs, and nodelogs.tgz vs logs.xxxx.tar. Grabbing nodelogs.tgz from a single node is not sufficient, and the logs.xxxx.tar file should ALWAYS be collected.
Procedure:
1. Log in to the grid as admin, and load the ssh keys. See
Avamar: How to Log in to an Avamar Server and Load Various Keys for information about loading ssh keys.
2. From Avamar version 18.1, getlogs must be run as root (but not run from the root home directory).
a. Switch to root:
su -
Enter the root password when prompted.
b. Load the root keys:
ssh-agent bash
ssh-add .ssh/rootid
Sample output:
Identity added: .ssh/rootid (.ssh/rootid)
c. It is important to change to the /home/admin directory so that /root is not filled up
3. Run the getlogs command:
Where "x" is the number of days of logs to gather. (The "getlogs" command can be run without "--server=" and gathers all server logs)
Sample output:
Copying getnodelogs script to nodes
Running getnodelogs script on nodes
Getting nodelogs.tgz from nodes
Creating tar file: logs.130218.120930.tar
Done with getlogs
4. The file is created in the format "logs.ddmmss.hhmmss.tar". Verify that the tar file mentioned above exists in the directory the command was run in:
ls -lh logs.130218.120930.tar
Sample output:
-rw-rw-r-- 1 admin admin 324M Feb 13 12:10 logs.130213.120930.tar
(This also indicates how big the file is).
Notes:
- During the rollback process, getlogs is run automatically as a precaution.
- If getlogs was not run manually before rolling back the grid, you can obtain the copy captured during the rollback in /usr/local/avamar/var/log/rollback.
- Check the timestamp on the tar files to verify that the correct log.xxxx.tar file is being collected:
ls -ltrh /usr/local/avamar/var/log/rollback/
Sample output:
total 54G
-rw-rw-r-- 1 admin admin 18G Feb 17 11:01 logs.140217.102740.tar
-rw-rw-r-- 1 admin admin 19G Feb 19 12:57 logs.140219.122325.tar
-rw-rw-r-- 1 admin admin 18G Feb 21 12:34 logs.140217.184444.tar
- On a production grid, these logs can be quite large (2GB+).
- Once the .tar file has been created, it must be provided to Avamar support personnel for troubleshooting purpose.
- If the size of the file is less than 2 GB, it can be directly attached to the Service Request
- If the size of the file is bigger than 2 GB (likely), contact the Avamar support personnel to assist uploading the file to an MFT site.
Logs transfer to FTP general best practices:
- Generally it is necessary to have a conversation between the Avamar Support Engineer and the Customer about the environment, especially if the FTP process starts to get difficult or time consuming.
- The Customer might be able to give instructions on the best way to push files out of their environment, and could save a lot of time and effort in the process.
- For instance, the environment might have FTP access blocked on the affected grid, but have access from another grid that could be used to get the files uploaded to the MFT server.
- Depending on the situation, it might also be best to use an alternative and temporary MFT link provided by Avamar support personnel.
See this video: