Scaling SDN: Is OpenFlow Ready for Prime Time?

Pica8 Says ‘Yes’ and Challenges the FUD

Up to this point, OpenFlow has mostly been deployed in research and higher-education environments. These early trials have shed some light on interesting use cases, what OpenFlow is good for, and of course, what OpenFlow might not be so good for.

This is important because OpenFlow and SDN adoption is only going to grow. It’s imperative that we understand these limitations – specifically, what’s real and what’s FUD.

One of these is scale.

If you’ve kicked the tires on OpenFlow, one question you may have heard is “How many flows does that switch support?” However, this question is only part of the story. It’s like asking only about a car’s top speed when what you should be thinking other things too – such as fuel efficiency and maintenance. So to figure out the right questions, we first need to go over a bit of background.

In its most basic terms, any network traffic, whether it’s Layer 2, Layer 3, or something else, is governed by a of forwarding rules as defined by a series of protocols. If it’s this MAC, do this. If it’s that IP, go there. Each of these “rules” is stored on the switch, in memory, in something called the Forwarding Information Base (FIB) or Router Information Base (RIB).

OpenFlow is a little different. It allows for more point-to-point connections that you can base on business logic “rules”.. These rules, or flows, are implemented in a different way. If I see packet type x, perform action y. It doesn’t have to follow the OSI networking model, and as such, gives users added flexibility to govern how traffic behaves. This works great for policy-based networking and driving business logic into the network. But at its heart, it’s another networking protocol and the basic concept is the same. The key difference with this approach is that these flows are not storied in say the FIB, but in the switch Ternary Content Addressable Memory (TCAM).

Now here is where things get interesting.

Let’s look at the switch with the Broadcom Trident II ASIC. Pretty much every major switch vendor has a switch with this ASIC including us with both pre-loaded and bare metal switch options through our Hardware Ecosystem (link). Trident II enables you to store up to two thousand entries in the TCAM. Most every other switch vendor has used this data point to drive the perception that OpenFlow will not scale.

Well, we at Pica8 agree – to an extent. Two thousand flows are not enough – if you have 400 network nodes and each of those speaks five other nodes, that already maxes out your TCAM. So what did Pica8 do to solve this? Two things:

  1. First, we made the TCAM table much more efficient. Instead of treating this as a hard limit of two thousand flow entries, we chose to slice and dice that memory into smaller chunks. Each flow entry is designed for a packet header, but in many cases, you don’t need to inspect the entire header to determine the right action. Having 3 smaller tables can go a long way – one table for port, another for MAC, and another for signature can increase the total number of rules. In some cases you might just need to match the port? Or the destination IP? Or the MAC? Or a combination of the above? Think of this as a 2000 page book – with each page having just enough room for one packet header. But if your flow doesn’t need to match the entire header, you’ll have lots of whitespace. We’ve filled up every page to the margin. In addition to that, we implemented the usage of wildcards, aggregation, de-duplication and other techniques to optimize the table. With all these enhancements, we’ve managed to effectively double the capacity of flows in the TCAM. But that’s still not enough to make an appreciable difference right? So…
  2. Second, we attached the FIB table to the TCAM. This is a capability that we have leveraged on the Trident II ASIC. In this way, we’ve vastly expanded the number of entries that use a standard IP longest prefix match algorithm, while also freeing up even more space in the TCAM by eliminating the need for IP lookups.

Both of these innovations contribute to an OpenFlow implementation that supports over two hundred THOUSAND flows – all on the exact same hardware that the other guys use. And this number makes a lot more sense as we expect more customers to roll out larger OpenFlow networks into production.

So, when you want to give OpenFlow a try – make sure you ask the right questions about the limitations you’ve been presented with. You might be surprised at the answer. To learn more about SDN scaling and Pica8, send us a note at We would love to hear your comments.