Jumbo Frames

Jumbo frames can be useful to optimize IP networks, especially in storage networking.  This post should help to explain why using jumbo frames can be useful.

I’m not Jumbo, I’m just big boned!

First, let’s define what we mean by the term jumbo frame.  As you can imagine it’s bigger than a normal frame.

A Jumbo frame simply means any frame with an MTU larger than 1500 bytes.  What exactly does that mean?  To really understand that we need to look at an Ethernet frame.   The diagram below shows a hastily thrown together Ethernet frame and most of the frame we’re not concerned with for this topic.  Parts of the frame are used for determining where the frame is headed, where it came from and to make sure it arrived intact.  The section we’re looking at is the “Data” or “Payload” section of the frame.

Now we can see that every single frame is going to have the same structure, so they will all have a preamble, SFD, Destination, etc…  All of this is considered overhead in order to deliver the Data.  So if a standard Ethernet frame is 1538 bytes and 1500 of those are the payload, we can calculate the efficiency.

Efficiency = Data Size / Frame Size

Efficiency = 1500 bytes / 1538 bytes

Efficiency = 97.5%

A Jumbo frame still has to have all of the same segments of an Ethernet frame, but we can increase the size of the Data section.  If we change the Data section to 9000 bytes instead of 1500, we should get more efficiency.

Efficiency = Data Size / Frame Size

Efficiency = 9000 bytes / 9038 bytes

Efficiency = 99.5%

So now it’s fairly easy to see that a larger payload can give you some benefits with large data transfers for things like storage networking.  It should be noted that all devices in the same layer 2 network path need to be configured for the same size MTU.  If one device, such as a switch, is using a 1500 MTU frame and your server sending frames to it at 9000 MTU the switch will drop the frame.

Also, if you are using jumbo frames accross networks via a router, you’ll want to have the same size MTU or you will have IP Fragmentation.  IP Fragmentation in this case would take the 9000 MTU frame sent by the server and break it into smaller pieces in order to send the frame from the switch at 1500 MTU.

IP fragmentation would cause something like the example below, where a single frame has to be broken up into smaller frames to reach it’s destination.

This IP fragmentation only happens inside of a router.  Remember that all devices on the same layer 2 network must have the same MTU size.  So the server and the router on the same Layer 2 network need to be the same size MTU, and the router to say another router need to have the same size MTU.  That example would be below.