Unsolved
1 Rookie
•
10 Posts
0
70
Z9100-ON Layer-3 Configuration Question
I am trying to set up a Z9100-ON as a normal Layer-3 plug and play switch. I want to plug Direct-Attach-Cables or Optics into it, run them to Mellanox 100 GBE cards and let them just work.
But the problem I face is that the default DellEMC OS sets the switch up as a Layer 2 (MAC Address-routing) switch, and the documentation is mind-boggling obtuse and scattered across dozens of documents. They really couldn’t have done a better job of making this switch super difficult to configure if they had tried — and it really looks like they tried very hard.
What documentation I have browsed has said that after I do the no shutdown and switchport commands on each port (one by one, painfully), assigning an IP address to each port (again, one by one) will give it Layer 3 properties.
But.. why should a port on the switch have an IP address? And what network should it have this IP address in? The documentation says very little at its best. It is as reluctant to actually give any helpful advice as possible.
So here I am, hoping someone can either give me some solid advice in this direction or point me to a guide that will take me there, step by step. 🙏
- Muzz -
1 Rookie
1 Rookie
•
19 Posts
0
April 3rd, 2024 21:39
If you are wanting Layer3 on OS10 then you'll need to have either:
If the IP address is on the VLAN interface (SVI) then the physical interface will need to be in Layer2 mode (Switchport) and allowed on that VLAN.
When you say "plug and play" it make me think you are looking for Layer 2 traffic to cross this switch. When you say Layer3 then that generally means you are wanting it to be the Layer3 gateway for a particular subnet; remember the golden rule of one subnet per VLAN.
This user guide should get you where you where you're trying to get.
https://dl.dell.com/content/manual39616760-dell-smartfabric-os10-user-guide-release-10-5-6.pdf?language=en-us
Further OS10 documentation can be found here:
https://www.dell.com/support/home/en-us/product-support/product/smartfabric-os10-emp-partner/docs
It is true there are many other deployment guides and the like scattered around in seemingly odd places but these two link locations should get you most of the way without specific solutions in mind.
Wentil
1 Rookie
1 Rookie
•
10 Posts
0
April 4th, 2024 03:20
@- Muzz - I want to configure this as a top-of-the-rack switch (which is what it was advertised as). So there would be two 10 Gbps feeds coming into the switch via the two SFP+ 10G ports. Intra-rack traffic will flow on a normally-routed Layer 3 IP-to-IP basis, so traffic inside the subnet stays in the rack, and rotate traffic goes out over the two uplinks. I just need to plug 100G transceivers in on each end (Switch and 100G Card) and have them work. I need to set the switch up so I don’t have to fine tune it or reconfigure it via the console if I need to move a connection from one switch port to another. Essentially, I want it to work like a normal 10G SFP+ Layer 3 Switch, just at 100G.
Wentil
1 Rookie
1 Rookie
•
10 Posts
0
April 4th, 2024 23:21
@- Muzz - Is there a way to do that? To convert it to what would be considered a “normal” switch?
In other words, when I connect it to the Mellanox 100 GBE cards, they should link and be able to send packets, either internally (to other servers connected to the switch) or externally (to other subnets or gateways over the two 10G SFP+ Uplink ports). It should be able to route on an IP / Subnet basis, i.e., Layer 3.
Right now, the switch (in Layer 2 mode) will light only when going port-to-port on the Switch. Not from Switch-to-Card, either using Dell Direct-Attach 100 GBE Cables or optical transceivers.
The documentation says I need to manually assign an IP address to each switch port or to each VLAN to get it to use Layer 3… but why? Normal switches don’t require this. What subnet should I use for this? What IP address? What is the necessary relevance needed (if any) to the subnets used by the Traffic? The documentation is silent, giving only the enigmatic bit above.
What needs to be done to get the switch to work as desired?
(edited)
- Muzz -
1 Rookie
1 Rookie
•
19 Posts
0
April 9th, 2024 16:26
I fear there are a few misconceptions occurring here. This line above makes me think that the LED on the interface doesn't light up when you connect the Server to that interface which sounds like a layer1 negotiation issue, perhaps FEC & negotiation settings need to be reviewed.
If the LED is on only when doing a loopback test, where both ends of the cable are connected to the switch, but the LED is absent (off) when connected to a server then Layer1 negotiation is the heart of the matter.
Adding an IP address to VLAN or individual physical interface on the switch will make zero difference if there is a Layer1 issue at play.
If you have the Dell part# on the cable that you are using to connect to the Server & Switch that can be looked up to confirm compatibility on the switch.
Another item that might be at play is the server NIC speed. If you are connecting a 100g cable then the server needs to advertise as able to work at 100g