Server not responding possible network congestion

"Server not responding" messages are informational. The message means that the Avamar client has not received a reply to any messages it has sent to the server in x seconds (x is 300 in this case). This can happen for a number of reasons. Typically we see this message if there is a network interruption or if the server is unable to respond because it's too busy or has gone read only for maintenance.

The reason for the failure is the "bad response" message. This message is essentially the server saying "You're not allowed to reference that piece of data in a backup because I don't have it." This can happen for a number of reasons as well. We have seen this message if the hash cache on the client is damaged or if the server is failing hfscheck. It's also possible that the server is processing messages from the client out of order for some reason.

The freezetimeout parameter controls volume freezing on Windows clients (e.g. VSS or OTM) so modifying this parameter is unlikely to make any difference. I would recommend opening a service request for this issue.

Hello,

Did you try to run on demand backup for this client ? if yes, did backup complete ?

This issue may occur with several reasons. If number of files on client is huge, it may take long time to create f_cache and p_cache and may fail for backup. One option would be to run backup for smaller chunks.

Regards,

Pawan

Hi Pawan,

I got the same error  Error <5775>: Internal Error: backtree: bad response and  5399 aborting.

On-demand too failed with the same error.

Could you please check the attached logs and suggest if this the issue related with large number of files.

The f_cache file started writing at 18:00:21 and went on updating till 22:04:53.

2015-03-04 18:20:02 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 18:20:04 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 18:30:46 avtar Info <8688>: Status 2015-03-04 18:30:46, 79,173 files, 10,293 folders, 112.9 GB (1,318 files, 5.815 GB, 5.15% new) 180MB 36% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\Analysis\Ramp Up Metrics\Donna Metrics (Weekly Reports)\New Ramp Up Metrics (Dec 15-18).qvw 2015-03-04 18:40:45 avtar Info <5548>: 2015-03-04 18:40:45 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 18:40:45 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 18:40:46 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 18:45:58 avtar Info <8688>: Status 2015-03-04 18:45:58, 79,182 files, 10,293 folders, 119.6 GB (1,327 files, 9.303 GB, 7.78% new) 178MB 23% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\Analysis\Ramp Up Metrics\Donna Metrics (Weekly Reports)\New Ramp Up Metrics (Jan 23-29).qvw 2015-03-04 19:01:11 avtar Info <8688>: Status 2015-03-04 19:01:11, 80,151 files, 10,312 folders, 132.5 GB (2,036 files, 12.21 GB, 9.21% new) 180MB 32% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\15\CALL_CATEGORIES_INTR.qvd 2015-03-04 19:03:19 avtar Info <5548>: 2015-03-04 19:03:19 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 19:03:19 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 19:03:20 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 19:16:24 avtar Info <8688>: Status 2015-03-04 19:16:24, 80,889 files, 10,314 folders, 139.9 GB (2,774 files, 15.77 GB, 11.28% new) 186MB 50% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\17\1272.qvd 2015-03-04 19:23:29 avtar Info <5548>: 2015-03-04 19:23:29 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 19:23:29 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 19:23:30 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 19:31:36 avtar Info <8688>: Status 2015-03-04 19:31:36, 81,542 files, 10,315 folders, 147.3 GB (3,427 files, 19.57 GB, 13.29% new) 188MB 51% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\18\EACR_INFO_PERFORMANCE.qvd 2015-03-04 19:43:49 avtar Info <5548>: 2015-03-04 19:43:49 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 19:43:49 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 19:43:51 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 19:46:49 avtar Info <8688>: Status 2015-03-04 19:46:49, 82,234 files, 10,317 folders, 155.2 GB (4,119 files, 23.30 GB, 15.01% new) 191MB 53% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\20\1291.qvd 2015-03-04 20:02:02 avtar Info <8688>: Status 2015-03-04 20:02:02, 82,884 files, 10,318 folders, 160.7 GB (4,769 files, 26.06 GB, 16.22% new) 189MB 39% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\21\EACR_INFO_SERVICE.qvd 2015-03-04 20:03:51 avtar Info <5548>: 2015-03-04 20:03:51 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 20:03:51 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 20:03:52 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 20:17:14 avtar Info <8688>: Status 2015-03-04 20:17:14, 83,975 files, 10,321 folders, 169.7 GB (5,860 files, 30.39 GB, 17.90% new) 192MB 63% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\24\1134.qvd 2015-03-04 20:24:13 avtar Info <5548>: 2015-03-04 20:24:13 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 20:24:13 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 20:24:15 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 20:32:27 avtar Info <8688>: Status 2015-03-04 20:32:27, 84,664 files, 10,322 folders, 176.8 GB (6,549 files, 33.80 GB, 19.12% new) 192MB 50% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\25\EACR_DURATION_DETAIL.qvd 2015-03-04 20:44:42 avtar Info <5548>: 2015-03-04 20:44:42 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 20:44:42 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 20:44:43 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 20:47:39 avtar Info <8688>: Status 2015-03-04 20:47:39, 85,294 files, 10,324 folders, 184.4 GB (7,179 files, 37.58 GB, 20.38% new) 196MB 53% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\27\1065.qvd 2015-03-04 21:02:52 avtar Info <8688>: Status 2015-03-04 21:02:52, 86,116 files, 10,325 folders, 192.2 GB (8,001 files, 41.38 GB, 21.53% new) 199MB 53% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201502\28\TV_DSMBG.qvd 2015-03-04 21:04:42 avtar Info <5548>: 2015-03-04 21:04:42 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 21:04:42 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 21:04:43 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 21:18:05 avtar Info <8688>: Status 2015-03-04 21:18:05, 86,770 files, 10,328 folders, 197.0 GB (8,655 files, 43.57 GB, 22.12% new) 201MB 31% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201503\2\3002.qvd 2015-03-04 21:24:43 avtar Info <5548>: 2015-03-04 21:24:43 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 21:24:43 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 21:24:45 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 21:33:17 avtar Info <8688>: Status 2015-03-04 21:33:17, 87,324 files, 10,329 folders, 203.5 GB (9,209 files, 46.82 GB, 23.01% new) 199MB 46% CPU E:\Group\EACR_DATA\EACR_INTR (EACR and INTR Data)\QVD\201503\3\EACR_ALL_FLOWS.qvd 2015-03-04 21:44:50 avtar Info <5548>: 2015-03-04 21:44:50 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 21:44:50 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 21:44:52 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 21:48:30 avtar Info <8688>: Status 2015-03-04 21:48:30, 88,224 files, 10,380 folders, 216.1 GB (9,405 files, 51.53 GB, 23.85% new) 196MB 38% CPU E:\Group\EACR_DATA\Miscellaneous\Welch\03 - TSP Transform - Qlikmart Non-TSP Match.qvw 2015-03-04 22:03:43 avtar Info <8688>: Status 2015-03-04 22:03:43, 117,990 files, 22,306 folders, 233.8 GB (9,686 files, 55.50 GB, 23.74% new) 201MB 26% CPU E:\Group\ESSAttach2\3659639\2-18-2015_12-35_PM.emaillog.txt 2015-03-04 22:04:52 avtar Info <5548>: 2015-03-04 22:04:52 Eastern Standard Time: Automatic cache flush of E:\avs\var\f_cache.dat 2015-03-04 22:04:52 avtar Info <5069>: - Writing cache file "E:\avs\var\f_cache.dat" 2015-03-04 22:04:53 avtar Info <5546>: Cache update complete E:\avs\var\f_cache.dat (88.0MB of 1534MB max) 2015-03-04 22:18:55 avtar Info <8688>: Status 2015-03-04 22:18:55, 573,259 files, 28,387 folders, 239.1 GB (10,991 files, 55.57 GB, 23.24% new) 356MB 48% CPU E:\Group\FILENET_DOCS\Processed_sublease\SubleaseArchive1.txt15068.xml.P8 2015-03-04 22:25:10 avtar Info <7694>: Server(aap2u01.aldc.att.com) not responding (possible network congestion?) (300 seconds) 2015-03-04 22:30:04 avtar Error <5775>: Internal Error: backtree: bad response 5139022bc4a9639dcd3e1997b5a6155258611c3c 1030145 ADD_HASH_DATA=2 serial=1030145 seq=0 flags=R:N:0 kind=9 rsp=MSG_ERR_INVALID_PARAMETERS 2015-03-04 22:30:04 avtar Info <5726>: Aborting backup due to error (3:EXC_HFSERROR) 2015-03-04 22:30:04 avtar FATAL <5399>: Internal error, aborting