How TCAM Enables a High Speed Edge Network

by Will Chang

As we enter the era of the digital generation, where people were born with easy access to digital information and living their life online, the demand for bandwidth and speed will grow at an exponential rate. To be able to handle such a huge amount of traffic and provide nearly real-time services, the routing speed will be key. A switch with 100Gbps throughput, will need to handle every single bit that comes in within 0.01 ns, which means a 64Byte data packet is within 5 ns. Now, imagine a 1Tbps edge router. The speed will be 10 times faster!

The main job of an edge router is to take in the data packet, process it and deliver it to the proper destination. The processing generally includes packet forwarding, packet classification and policy enforcement.
 

Packet Forwarding

A packet is forwarded based on the information in the FIB (Forwarding Information Base). Within the FIB includes the destination of a packet and the interface, or port, in which the router should forward it to.
 

Packet Classification and Policy Enforcement

Especially in an edge router, it needs to be able to provide different levels of services. Service providers leverage packet classification to identify the traffic types and apply pre-defined policies to different traffic. ACL (Access Control List) can filter packets and decide whether to deny or permit to process them while QoS (Quality of Service) could identify packets and apply different priorities and rules to process.

A simplified flow is shown in picture below. The data packet goes in, a lookup is conducted on the data table to find where the action is stored and an action is picked from the action table based on the result of data table lookup. The data table usually resides in the CAM where action table is in the RAM.

 

RAM vs CAM, What's the Difference?

A table is typically put in the RAM (Random Access Memory) for a high speed manipulation. The software provides an address and fetches the data from the table that matches the address. However, using RAM for lookups may take multiple cycles to complete and that is not ideal when the goal is speed.

 

That's where CAM (Content-Addressable Memory) comes in. It does things the other way around, whereas the data is used as the key to search for the address where it is located. And by the nature of CAM's design, the search is in parallel, which means the lookup could be done in one cycle. This makes CAM much more suitable for doing a lookup for the data table.

 

Different Types of CAM

Two different types of CAM are generally used called BCAM (Binary CAM) and TCAM (Ternary CAM).

BCAM, as the name implies, is where the data is in binary states, 0 or 1. The data search has to be matched perfectly. It is suitable for data lookup with unique entries such as MAC address lookup.

 

TCAM introduces a third state (ternary means having three of something), X, which means "DON'T CARE" or "wildcard", so besides matching 0 or 1, a particular bit could be ignored when it is put as an "X", which means either 0 or 1 could be matched. In the example below, both 1110X and 111XX match the data 11101.

 

The concept of DON'T CARE could be useful in many ways when it comes to routing.

For example, when we write down 1.1.1.0/24, it describes a range of IP address from 1.1.1.0~1.1.1.255. The number 24 indicates the first 24 bits of IP address must be matched, and it doesn't matter what the last 8 bits are.

 

When dealing with traffic policy, sometimes a router should only care about parts of the traffic pattern and ignore other parameters. In the ACL configuration below, the "ANY" in first entry means data sent to any ports in destination will match the rule.

 

TCAM can easily describe these configurations in one entry with some bits marked as "X".
 

TCAM Enables Versatility and Speed

As mentioned in the beginning, there are many applications that rely on fast searches. With a powerful TCAM, routers will be capable of handling versatile applications in such as IPv4 & IPv6 Packet Classification, ACL, Policy-based Routing and QoS.

For applications such as TIP's Open BNG where versatility and a high subscribers (up to 128K for some cases) are needed, UfiSpace's S9600-72XC open aggregation router has the option for an external TCAM. 

In many cases where there are a large number of subscribers or there is a significant amount of QoS at the provider edge, an external TCAM is needed to provide larger lookup tables or extra memory.

UfiSpace's disaggregated core and edge routers all have the option for enabling an embedded external TCAM. It supports up to 16 parallel searches with up to 56.25Gbs for search. It can store a maximum of 20 million entries for IPv4 or 3 million x 160b for IPv4 ACL.