Unsolved
This post is more than 5 years old
2 Intern
•
5.7K Posts
0
12414
Running an ISL at 8Gb produces lots of errors, 4Gb is error free. Don't understand why.
We have 2 locations, separated by 610 meters and 1245 meters dark fiber. The switches are Brocade DS-300B with 8Gb long wave (10 km) SFPs.
When the switch ports run at 8Gb I see a lot of errors and communication between 2 VNX machines is very bad. Mirrorview doesn't show any progress.
When I set the ports down to 4Gb, the ISLs are error free and mirrorview is running smoothly.
The connections between the 2 300's in each fabric is a single ISL. We have 2 fabrics, so 4 switches and 2 VNX's.
We'd like to run the ISL @8Gb, but we keep getting the errors on the link. We tried new patch cables (9 micron of course) and other outlets on the patch panel, but that wasn't it.
Does anyone have an idea of what this might be ?
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
February 8th, 2012 07:00
is there such a thing as buffer credits on Brocade ?
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
February 8th, 2012 08:00
RRR,
ISL ports set to LD ?
portCfgLongDistance
henrypan2
13 Posts
0
February 8th, 2012 08:00
Yes dynamox:>)
Henry
mennodeliege
20 Posts
0
February 8th, 2012 12:00
Still no effect.The errors only dissapear when changing to normal mode (giving me 8 b2b credits) and 4Gb fixed. absolutely no errors then. when changing to 8Gb, change to LD mode (giving me 46 b2b credits) or not and even change portcfgfillword to 3, does not resolve anything. only way of error free ISL is 4Gb Normal mode. I am the colleague of RRR
dynamox
2 Intern
2 Intern
•
20.4K Posts
0
February 8th, 2012 13:00
and it's consistent behavior on both fabrics ?
christopher_ime
2K Posts
0
February 8th, 2012 22:00
Just curious, what are the errors from porterrshow? Are they by chance class 3 discards (disc c3)? There is a Primus that may be related:
emc278061: "Brocade B-Series: 8Gb switches in Fabric OS 6.x ISL connected to Access gateway shows errors in class 3 discards disc c3 counter"
Also, just want to make sure that you are compliant to the support matrix regarding ISL's and are running at least v6.1.2b.
Also a good general read may be the "Extended Distance Technologies TechBook",
https://elabnavigator.emc.com/vault/pdf/Distance.pdf
This will also talk about the "Extended Fabric" license which is required, and a short chapter on "Data buffering and flow control" which might help troubleshoot.
mennodeliege
20 Posts
0
February 9th, 2012 00:00
and the other side when running on 8Gb:
er_enc_in 36 Encoding errors inside of frames
er_crc 19 Frames with CRC errors
er_trunc 0 Frames shorter than minimum
er_toolong 0 Frames longer than maximum
er_bad_eof 3 Frames with bad end-of-frame
er_enc_out 17043 Encoding error outside of frames
er_bad_os 17045 Invalid ordered set
er_rx_c3_timeout 0 Class 3 receive frames discardedout
er_tx_c3_timeout 0 Class 3 transmit frames discardeeout
er_c3_dest_unreach 0 Class 3 frames discarded due to unreachable
er_other_discard 0 Other discards
er_type1_miss 0 frames with FTB type 1 miss
er_type2_miss 0 frames with FTB type 2 miss
er_type6_miss 0 frames with FTB type 6 miss
er_zone_miss 0 frames with hard zoning miss
er_lun_zone_miss 0 frames with LUN zoning miss
er_crc_good_eof 16 Crc error with good eof
er_inv_arb 0 Invalid ARB
felipon1
96 Posts
0
February 9th, 2012 00:00
Hi,
According to your last post, we can see er_crc_good_eof in both ends:
er_crc_good_eof 58 Crc error with good eof
er_crc_good_eof 16 Crc error with good eof
So I suppose that there must be some element generating the errors in that link. The more speed the more sensitive and affected the switchs are by dust, a bad connection, sfp, etc.
Do you get these errors in both fabrics or just in one? If just in one, you can try to locate the faulty element by moving connections from one fabric to the other, with all that that implies...
mennodeliege
20 Posts
0
February 9th, 2012 00:00
thanks guys again, here are the errors of one side:
er_enc_in 67 Encoding errors inside of frames
er_crc 59 Frames with CRC errors
er_trunc 0 Frames shorter than minimum
er_toolong 0 Frames longer than maximum
er_bad_eof 1 Frames with bad end-of-frame
er_enc_out 375743 Encoding error outside of frames
er_bad_os 375747 Invalid ordered set
er_rx_c3_timeout 0 Class 3 receive frames discarded due to timeout
er_tx_c3_timeout 0 Class 3 transmit frames discarded due to timeout
er_c3_dest_unreach 0 Class 3 frames discarded due to destination unreachable
er_other_discard 0 Other discards
er_type1_miss 0 frames with FTB type 1 miss
er_type2_miss 0 frames with FTB type 2 miss
er_type6_miss 0 frames with FTB type 6 miss
er_zone_miss 0 frames with hard zoning miss
er_lun_zone_miss 0 frames with LUN zoning miss
er_crc_good_eof 58 Crc error with good eof
er_inv_arb 0 Invalid ARB
Richard_Butler
84 Posts
0
February 9th, 2012 03:00
Hello RRR,
Did you look at setting the 8Gb/s fillword with the command: portCfgFillword ?
Best regards.
mennodeliege
20 Posts
0
February 9th, 2012 03:00
I allready tried to use the fillword with mode 3 and disabling and enabling the ports, but it does not matter. only on 4Gb there are no errors.
felipon, the problem is on both fabrics and cables has allready been replaced and we also moved to another single mode connection
mennodeliege
20 Posts
0
February 9th, 2012 05:00
I also upgraded to FOS 7.0.1, but makes no difference..
Jun_Tan
2 Intern
2 Intern
•
605 Posts
0
February 12th, 2012 21:00
Only found some workaround to resolve this issue, but the workaround is not always working.
You can try to have a test in full Brocade Hardware environment, SFP and Switch and HBA are all shipped from Brocade.
Jun_Tan
2 Intern
2 Intern
•
605 Posts
0
February 12th, 2012 21:00
It's a 8Gb/scompatitibility problem.
mennodeliege
20 Posts
0
February 12th, 2012 23:00
Can you explain me whether you are from EMC or not? besides this, I would like to know whether it is a Brocade specific 8Gb compat issue. By the way, the ISL is not functioning on 8Gb, while both switch sides are Brocade and also the LR 8Gb SFP's are Brocade, so actually the issue is completely Brocade regarded..