1 Rookie
•
6 Posts
0
65
Error 132 (No buffer space available)
Hello all,
we had an issue today that caused OSPF to reset. Wondering what the actual cause was. It appears that the no buffer space available log entries pertain to IPv6 yet the IPv4 OSPF session is the one that reset.
A few log entries below. Entire issue lasted exactly 2 minutes and 9 seconds. This switch runs both OSPF and OSPF IPv6. But the OSPF IPv4 was the only one that reset. Also, this switch is actually 2 switches stacked. The OSPF link that was reset was between it and a Juniper MX router across a LACP bundle on te1/0/15 and 2/0/15 to the juniper.
<133> May 24 08:47:31 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876182 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl32. Error 132 (No buffer space available).
<133> May 24 08:47:25 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876161 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl32. Error 132 (No buffer space available).
<133> May 24 08:47:19 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876148 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl32. Error 132 (No buffer space available).
<133> May 24 08:47:13 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876147 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl747. Error 132 (No buffer space available).
<133> May 24 08:47:07 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876146 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl32. Error 132 (No buffer space available).
<134> May 24 08:47:05 CORE-SW-STACK-1 OSPF[procLOG]: spnbo.c(2737) 49876145 %% INFO [VRF ""]: OSPFv2 neighbor 10.20.30.19 on Vl32 moved from state FULL to state DOWN, on event InactivityTimer
<133> May 24 08:47:01 CORE-SW-STACK-1 OSAPI[ip6MapRadvdTask]: osapi_support.c(761) 49876144 %% NOTE Failed to send 56 byte message to ff02::1 on socket with fd 61 on interface Vl32. Error 132 (No buffer space available).
DELL-Joey C
Moderator
Moderator
•
3.3K Posts
0
June 4th, 2024 02:01
Hi,
My colleague informed me to request you to update the switches firmware, https://dell.to/3yMmyOU. There are some fixes on stacking and a major security update. Let's try to out rule the root causes.
DELL-Joey C
Moderator
Moderator
•
3.3K Posts
0
May 27th, 2024 09:42
Hi,
Can you let us know what switch model that you're having issue with, and also the firmware version.
Also, the error message, where is this generated from? Is it from the switch CLI?
Lee-S
1 Rookie
1 Rookie
•
6 Posts
0
May 30th, 2024 23:03
@DELL-Joey C
Both switches in the stack are model number N4032F. Version below:
unit active backup current-active next-active
---- ----------- ----------- -------------- --------------
1 6.5.4.21 6.5.3.3 6.5.4.21 6.5.4.21
2 6.5.4.21 6.5.3.3 6.5.4.21 6.5.4.21
The error messages were pulled from the CLI with the command: show logg
Best,
Lee
DELL-Joey C
Moderator
Moderator
•
3.3K Posts
0
May 31st, 2024 01:46
Hi,
Can you do #dir to check on the flash is out of space. Can you paste the output here, so we can identify which can be removed. The space probably is not available for OSPF routing process.
Lee-S
1 Rookie
1 Rookie
•
6 Posts
0
May 31st, 2024 16:28
@DELL-Joey C
Sure. Please see below.
-Lee
#dir
Attr Size(bytes) Creation Time Name
drwx 0 Jan 05 2024 07:00:50 .
drwx 0 Aug 21 2023 17:50:03 ..
-rw 4 May 29 2024 20:46:09 serial_speed.pending
drwx 40 Dec 27 2023 12:37:05 ramcp
drwx 60 Dec 27 2023 12:44:10 ramrc
-rw 2033 Oct 17 2020 19:20:19 sslt_cert1.pem
drwx 0 Apr 18 2001 01:14:07 crashlogs
-rw 21 Dec 27 2023 22:59:48 SupportAssistEULAAccepted
-rw 156 Oct 17 2020 19:20:19 dh512.pem
-rw 64 Dec 18 2023 21:46:53 logNvmSave.bin
-rw 225 Dec 18 2023 23:22:10 ssh_host_rsa_key.pub
-rw 92 Dec 18 2023 21:43:10 hpc_port_broad.cfg
-rw 10 Aug 24 2017 13:36:30 sshd.pid
-rw 1704 Oct 17 2020 19:20:19 sslt_key1.pem
-rw 333 Dec 18 2023 23:22:10 ssh_host_key.pub
-rw 16328 Dec 27 2023 12:37:03 log2.bin
-rw 0 Dec 27 2023 12:36:52 slog0.txt
-rw 0 Dec 18 2023 21:46:53 slog1.txt
-rw 1024 Apr 14 2017 10:57:33 sslt.rnd
-rw 592 Dec 18 2023 21:44:45 slog2.txt
--- 372 Nov 28 2023 22:27:02 watchdog.data
-rw 3889 May 29 2024 20:46:13 fastpathRun.cfg
-rw 1108 Dec 27 2023 12:37:28 hpc_broad.cfg
-rw 413 Dec 27 2023 12:37:29 coredump_regular_config
-rw 3889 May 29 2024 19:36:55 fastpath.cfg
-rw 245 Oct 17 2020 19:20:19 dh1024.pem
-rw 0 Dec 27 2023 12:36:52 olog0.txt
-rw 0 Dec 18 2023 21:46:53 olog1.txt
-rw 0 Dec 18 2023 21:43:10 olog2.txt
-rw 528 Dec 18 2023 23:22:10 ssh_host_key
-rw 601 Jan 29 2019 20:29:23 ssh_host_dsa_key.pub
-rw 5 Dec 27 2023 12:36:52 ologNdx0.txt
-rw 5 Dec 18 2023 21:46:53 ologNdx1.txt
-rw 5 Dec 18 2023 21:44:45 ologNdx2.txt
-rw 16328 Dec 26 2023 00:44:40 log-1.bin
-rw 16328 Apr 13 2019 21:30:25 log-2.bin
-rw 72 Dec 27 2023 12:37:29 coredump_regular_config.md5sum
-rw 715 Dec 27 2023 12:35:47 coredump_debug_config
-rw 96 Dec 27 2023 12:37:35 snmpOprData.cfg
-rw 8695 May 04 2018 17:16:00 running-config
drwx 0 Dec 27 2023 12:37:31 user-apps
-rw 672 Oct 17 2020 19:20:19 ssh_host_dsa_key
-rw 891 Dec 18 2023 23:22:10 ssh_host_rsa_key
-rw 70 Dec 27 2023 12:35:47 coredump_debug_config.md5sum
-rw 23865 May 29 2024 19:36:59 startup-config
-rwx 10 Oct 15 2013 19:05:24 user.start
-rw 1096 Dec 18 2023 21:43:58 hpc_add_broad.cfg
-rw 500 Jan 05 2024 07:00:50 cli.bnr
Total Size: 62914560
Bytes Used: 103084
Bytes Free: 62811476
Lee-S
1 Rookie
1 Rookie
•
6 Posts
0
May 31st, 2024 16:46
@DELL-Joey C
Also, see memory and mbuf info. Looks like a lot of allocation failures.
CORE-SW-STACK#show buffers
MBUF Utilization Monitoring Parameters
Rising Threshold............................... 0 %
Falling Threshold.............................. 0 %
Severity....................................... 5
Mbufs Total.................................... 500
Mbufs Free..................................... 384
Mbufs Rx Used.................................. 116
-------------------------------------------------------
Rx-Alloc Attempts Failures (%%)
-------------------------------------------------------
NORM 701398284 1465177973 (0%)
MID2 -922766131 150558 (0%)
MID1 62698715 5195 (0%)
MID0 14227800 185209 (1%)
HIGH 38809323 0 (0%)
-------------------------------------------------------
Tx-Alloc Attempts Failures (%%)
-------------------------------------------------------
ALL -1583207481 0 (0%)
-------------------------------------------------------
CORE-SW-STACK#show buffers total
Mbufs Total.................................... 500
Mbufs Free..................................... 384
Mbufs Rx Used.................................. 116
Total Rx Norm Alloc Attempts................... 701537925
Total Rx Mid2 Alloc Attempts................... 3372226315
Total Rx Mid1 Alloc Attempts................... 62699085
Total Rx Mid0 Alloc Attempts................... 14227845
Total Rx High Alloc Attempts................... 38809540
Total Tx Alloc Attempts........................ 2711785900
Total Rx Norm Alloc Failures................... 1465297071
Total Rx Mid2 Alloc Failures................... 150558
Total Rx Mid1 Alloc Failures................... 5195
Total Rx Mid0 Alloc Failures................... 185209
Total Rx High Alloc Failures................... 0
Total Tx Alloc Failures........................ 0
Lee-S
1 Rookie
1 Rookie
•
6 Posts
0
June 4th, 2024 16:16
@DELL-Joey C
Thank you for the link to the firmware. I have downloaded it and will schedule the upgrade to take place in the next couple days.
I will let you know if this resolves the issue.