VMware Network Traffic Routing

March 16, 2012 11 By Eric Shanks

VMware has lots of ways to setup networking on their ESXi hosts.  In order to set this up in the best way for your needs, it’s important to understand how the traffic will be routed between VMs, virtual switches, physical switches and physical network adapters.

Before looking at an example, we should review some networking 101.  Machines on the same vlan on the same switch can communicate with one another (assuming there is no firewall type devices in the way).  Machines on different vlans on the same switch cannot communicate unless the traffic passes through a router.

Now that we’ve had a short refresher course on networking, lets look at how VMware uses virtual switches to pass traffic between VMs and the physical network.

VMs on the same vlan on the same host.

The diagram below shows two virtual machines on vlan 10, which are connected to the same virtual switch.  If the top VM wants to send data to the bottom vlan it simply sends a frame to the connected virtual switch and that switch forwards the frame to the bottom VM.  Nothing else needs to occur, in fact you don’t even need to have a physical NIC attached as an uplink port and no physical equipment is necessary to do this forwarding.

samevlan

VMs on different Vlans on the same host

Now lets look at what happens when the top VM is on a seperate vlan from the bottom vlan.  This process starts out the same, with the top VM sending a frame to the virtual switch.  Since the virtual switch doesn’t see the destination VM on the same subnet it will forward the frame to it’s uplink (physical NIC) and out to the physical network.  Once at the physical network we will hit our router which will then be able to route it back to the virtual switch on the new vlan.    This type of routing is sometimes referred to as a “router on a stick”.  Once the virtual switch gets this new frame, it can find the bottom VM and forward the frame.  A reply would then travel this entire distance in the opposite direction.

As you can see this isn’t necessarily the best use of bandwidth because now you’re limited by the physical NIC’s adapter speed.

Differentvlan

VMs on different Switches on the same host.

If we look at the next example, we have two separate virtual switches on the same host.  This example really is the same as our previous example because a vlan basically separates the traffic just like having two completely separate switches.  It doesn’t matter if the VMs are on the same VLAN or not, they can’t communicate without getting to the physical network.  The difference would be that once the frame is in the physical network, it doesn’t need a router to forward on a packet, it can just have a switch forward the frame.

DifferentSwitch

VMs on different hosts on same vlan.

Clearly since the two VMs below are on different hosts, they are going to require the frames to be sent to the physical network to get between hosts.  This is true even if you’re using distributed switches.  The diagrams that you see depicting distributed switches make it look like the switches are somehow attached to each other, but they are just trying to show you that you’ve created one distributed switch instead of individual vSwitches.

DistributedSwitch

 
This document was created using the official VMware icon and diagram library.

Copyright © 2010 VMware, Inc. All rights reserved. This product is protected by U.S. and

international copyright and intellectual property laws. VMware products are

covered by one or more patents listed at http://www.vmware.com/go/patents.

VMware does not endorse or make any representations about third party information

included in this document, nor does the inclusion of any VMware icon or diagram

in this document imply such an endorsement.