Avamar: Gen4S Hardware: The SMART daemon smartd is triggering megaraid disk events on Avamar Gen4S nodes
Summary: This article aims to address the issue of the SMART daemon triggering "smartd" disk events on Avamar Gen4S nodes by disabling it.
Symptoms
The smartd daemon on an Avamar Gen4S node is triggering events when attempting to parse the configuration.
After triggering the events, it reports that they were unable to monitor the SMART devices.
less /var/log/messages
Jun 26 11:23:05 hostname smartd[8146]: smartd 5.39 2008-10-24 22:33 [x86_64-suse-linux-gnu] (openSUSE RPM) Copyright (C) 2002-8 by Bruce Allen, http://smartmontools.sourceforge.net
Jun 26 11:23:05 hostname smartd[8146]: Opened configuration file /etc/smartd.conf
Jun 26 11:23:05 hostname smartd[8146]: Drive: DEVICESCAN, implied '-a' Directive on line 26 of file /etc/smartd.conf
Jun 26 11:23:05 hostname smartd[8146]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sda, opened
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sda, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sda' to turn on SMART features
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sdb, opened
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sdb, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdb' to turn on SMART features
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sdc, opened
Jun 26 11:23:05 hostname smartd[8146]: Device: /dev/sdc, IE (SMART) not enabled, skip device Try 'smartctl -s on /dev/sdc' to turn on SMART features
Jun 26 11:23:05 hostname smartd[8146]: Unable to monitor any SMART enabled devices. Try debug (-d) option. Exiting...
The daemon can also report several events that indicate that there is a faulty component on the Avamar server:
less /var/log/messages
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_01] [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 82 to 83
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_01] [SAT], SMART Usage Attribute: 195 Hardware_ECC_Recovered changed from 118 to 119
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_02] [SAT], SMART Usage Attribute: 194 Temperature_Celsius changed from 116 to 115
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_04] [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 82 to 83
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_04] [SAT], SMART Usage Attribute: 195 Hardware_ECC_Recovered changed from 118 to 119
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_06] [SAT], SMART Prefailure Attribute: 1 Raw_Read_Error_Rate changed from 67 to 72
Sep 5 12:11:38 hostname smartd[7681]: Device: /dev/bus/0 [megaraid_disk_06] [SAT], SMART Usage Attribute: 195 Hardware_ECC_Recovered changed from 103
All the Physical Disks are online with no errors and no failure is predicted. The Gen4S hardware and logs have been carefully checked and there are no errors.
Cause
The smartd daemon was enabled manually on this Avamar server causing unexpected behavior.
Avamar Gen4S nodes do not rely on the smartd daemon to check the status of the hard drives.
By default, this daemon should not be enabled so it should be stopped.
smartd should also be disabled. Follow the resolution steps to stop and disable smartd.
Resolution
Run the following command to stop smartd service from running:
service smartd stop
Run the following command to disable the smartd service on boot:
chkconfig smartd off
The smartd events in /var/log/messages and mccli events should stop appearing.