The Effect of Too Many Virtual CPUs

Many times I see new virtualization admins add too many vCPUs to virtual machines after they’ve converted their physical machines.    I believe the reason for this is a simple misunderstanding that more is not always better in this case.

With physical servers, the more is better approach seems to work fine.  If you have a quad core processor it’s better than a dual core and if you have a dual processor server it’s better than a single socket.  When it comes to virtual machines extra processors can actually make a VM perform worse than having too few processors.

To explain why this is the case, I’ll use some graphics.

First lets look at what happens when you have a virtual machine with a single virtual CPU on a physical host with multiple processors or cores.

On the right side we have physical CPUs.  The CPUs in the blue box are not currently being used for any workloads.  The CPUs in the red box are actively being used by another process.  The virtual machine on the left side has 1 vCPU and when it needs to access the processor, can request processor time from either of the two available processors.

The next example shows the same scenario, except this time the virtual machine has two vCPUs.  When it tries to request processor time, it can utilize both of the two available physical processors.

The next scenario shows where the problem occurs.  Now we have the same virtual machine as the above example, with two vCPUs but now three of the four physical processors are already busy.  Now when the virtual machine goes to request resources it has to wait for the physical processors to be available, even though there is an unused physical processor.  This VM requires two processors to be available simultaneously.  If we had a virtual machine with only one vCPU, it could run a process, but not a VM with two vCPUs.

If you were to run ESXTOP on the host that this was occurring, you might notice a high %RDY time.

I guess the moral of this story is that sometimes you need additional processing power, but if you don’t need it, don’t assign it.