Layer 3 mode is the recommended way to configure VSM to VEM communication in the Nexus 1000v. Layer 3 mode keeps things simple and easier to troubleshoot.
I kept my design very simple. There's one VLAN (509) that I run my ESXi hosts on. The IP addresses are 192.168.40.xxx. Just to give you an example:
vCenter: 192.168.40.2
ESXi Host1: 192.168.40.101
ESXi Host2: 192.168.40.102
...
N1kv: 192.168.40.31
Using this I had a simple uplink port-profile defined:
nexus1000v(config-port-prof)# show port-profile name uplink
port-profile uplink
type: Ethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
switchport mode trunk
switchport trunk allowed vlan 1,501,506,509-510,714,3967
channel-group auto mode on mac-pinning
no shutdown
evaluated config attributes:
switchport mode trunk
switchport trunk allowed vlan 1,501,506,509-510,714,3967
channel-group auto mode on mac-pinning
no shutdown
assigned interfaces:
port-channel1
port-channel2
Ethernet3/1
Ethernet3/2
Ethernet4/1
Ethernet4/2
port-group: uplink
system vlans: 1,501,506,509-510,714
capability l3control: no
capability iscsi-multipath: no
capability vxlan: no
capability l3-vn-service: no
port-profile role: none
port-binding: static
And a simple management port-profile:
nexus1000v(config-port-prof)# show port-profile name management
port-profile management
type: Vethernet
description:
status: enabled
max-ports: 32
min-ports: 1
inherit:
config attributes:
switchport mode access
switchport access vlan 509
no shutdown
evaluated config attributes:
switchport mode access
switchport access vlan 509
no shutdown
assigned interfaces:
Vethernet1
Vethernet4
port-group: management
system vlans: 509
capability l3control: yes
capability iscsi-multipath: no
capability vxlan: no
capability l3-vn-service: no
port-profile role: none
port-binding: static
I had everything set up right... I thought. The only problem was (before, not in the output above) is that I couldn't see my VEMs! They were all hooked up in vCenter and I was even running traffic through them. But no VEMs:
nexus1000v(config)# show module vem
No Virtual Ethernet Modules found.
I finally stumbled upon
this nice document and realized I hadn't enabled l3control. Doing that:
nexus1000v(config-port-prof)# capability l3control
And Bam! Everything worked:
nexus1000v(config-port-prof)# show module vem
Mod Ports Module-Type Model Status
--- ----- -------------------------------- ------------------ ------------
3 248 Virtual Ethernet Module NA ok
4 248 Virtual Ethernet Module NA ok
Mod Sw Hw
--- ------------------ ------------------------------------------------
3 4.2(1)SV1(5.1a) VMware ESXi 5.0.0 Releasebuild-469512 (3.0)
4 4.2(1)SV1(5.1a) VMware ESXi 5.0.0 Releasebuild-469512 (3.0)
Mod MAC-Address(es) Serial-Num
--- -------------------------------------- ----------
3 02-00-0c-00-03-00 to 02-00-0c-00-03-80 NA
4 02-00-0c-00-04-00 to 02-00-0c-00-04-80 NA
Mod Server-IP Server-UUID Server-Name
--- --------------- ------------------------------------ --------------------
3 192.168.40.101 00000000-0000-0000-cafe-00000000000f 192.168.40.101
4 192.168.40.102 00000000-0000-0000-cafe-00000000000e 192.168.40.102