UCS Fabric Interconnects are usually always run in end-host mode. At this point in the story there really isn’t that many reasons to use switch-mode on the Fabric Interconnects.
Two checks, or features that make End Host Mode possible are Reverse Path Forwarding (RPF) checks and Deja-Vu checks.
Reverse Path Forwarding Checks
Each server in the chassis is pinned dynamically (or you can set up pin groups and do it statically, but I don’t recommend that) to an uplink on Fabric Interconnect A and Fabric Interconnect B. Let’s say you have 2 uplinks on port 31 and 32 of your Fabric Interconnect. Server 1/1 (chassis 1 / blade 1) may be pinned to port 31. If a unicast packet is received for server 1/1 on uplink port 31, it will go through. But if that same packet destined for server 1/1 is received on port 32, it will be dropped. That’s because RPF checks to see if the destination for the unicast is actually forwarding its uplink traffic through that link.
Deja Vu Checks
The other check is called “Deja-Vu” . In the Cisco documentation it says: “Server traffic received on any uplink port, except its pinned uplink port is dropped“. That sounds a lot like RPF. Another presentation from Cisco live states it this way: “Packet with source MAC belonging to a server received on an uplink port is dropped”
An example to clear it up
VM A on server 1/1 wants to talk to VM B located somewhere else. The Fabric Interconnects in this case are connected to a single Nexus 5500 switch. The VM is pinned to one of the VNICs and that VNIC is pinned to go out port 31 of Fabric Interconnect A. So what happens?
First the VM will send an ARP request. An ARP request basically says: I know the IP address but I want the MAC address. (Obviously, this is in the same Layer 2 VLAN and subnet). If Fabric Interconnect A doesn’t find the IP/MAC association in its CAM table, then it will not flood the server ports down stream. That is something a switch would do. The Fabric Interconnect is different. The reason the Fabric Interconnect doesn’t send a broadcast down its server ports is because it is a source of truth and knows everyone connected on its server ports.
What it will do instead is forward the ARP request (unknown unicast) up the designated uplink (port 31). Now the Nexus switch is a switch. (And a very good one at that). It will say: “Hey, I don’t have a CAM table entry for VM B IP/MAC so I will do what we switches do best: Flood all the ports! (except the port that the unknown unicast/ARP request came in on)
Remember Fabric Interconnect A port 32 is connected to this same switch as port 31 where the unknown unicast (ARP request) went out. The Nexus 5500 will send this unknown unicast to port 32 just like every other port. But port 32 says: Wait a minute, the source address originated from me. Deja-vu! So he drops the packet.
Fabric Interconnect B has two ports 31 and 32 that will also receive the unknown unicast. If VM B is pinned to a VNIC that is pinned to port 31 on Fabric Interconnect B, he will say: I got this! And the packet will go through. Port 32, however on FI-B will look at the destination MAC and say: This is not pinned to me, so I’ll drop the packet. That is the RPF check.
To sum it up
Deja-Vu check: don’t receive a packet from the upstream switch that originated from me.
Reverse Path Forward Check: don’t receive a packet if there’s no server pinned to this uplink.
Comments are closed.