Feature Story

May 12, 2004

by Charles Waltner, News@Cisco

The Cisco 12000 Series Router just keeps on giving. When it debuted in 1997, it was the world's first router specifically built for running the Internet "backbone" arterials of major telecommunications companies, revolutionizing the industry with unprecedented performance and reliability.

But Cisco didn't stop there. In the seven years since its inception, the router's performance has increased 280 fold. The first generation 12000 Series could process 2.5 million data packets per second. The third and most recent generation of the router can manage over 700 million packets per second.

Most remarkably, the modular components of each generation have been compatible with earlier router versions. Such innovation has provided groundbreaking value for Cisco's customers. Instead of having to wholly replace older Cisco 12000 Series routers with the newer ones, customers have only needed to install modular components to enjoy new performance gains. The 12000 Series has helped service providers stay ahead of rapidly increasing Internet traffic demands while saving millions of dollars in expensive "forklift" equipment upgrades.

The Cisco 12000 Series is not only far faster now, but it is also far more sophisticated. It offers a host of features capable of providing support for an array of advanced services in both the "core" and the "edge" of the network, including quality-of-service (QoS), voice-over-IP (VoIP), virtual private networks (VPNs), security, accounting, traffic control and service-level agreements.

Thanks to such innovation, the Cisco 12000 Series has vastly exceeded sales expectations and has become the Internet's most widely used backbone router, deployed by hundreds of telecommunications companies around the world.

Garry Epps, a Cisco Distinguished Engineer and one of the primary designers of the 12000 Series, says Cisco originally estimated it would only sell a few hundred of the devices, since they were so high-performance and would appeal to only a select group of the world's largest telecommunications companies. To date Cisco has sold over 25,000 of the routers.

"We thought we saw the iceberg, but it was only the tip," Epps says.

Until the debut of the 12000 Series, service providers were using the Cisco 7500 Series router. But Cisco originally designed the 7500 Series for a wide-variety of corporate LANs and WANs, making it a "Swiss Army knife" of sorts for network management, Epps says. The 7500 Series was good at many tasks but not fine-tuned to the specialized demands of large telecommunications networks.

"Service providers couldn't keep hooking up more 7500s," Epps says. "That approach wasn't going to last too long when Internet traffic was doubling every nine months. We were up against the gun to get a new router on the market that could handle the growth of Internet traffic. Our engineering team was told we needed to develop a gigantic router, something substantially larger and more scalable than we'd ever dreamt of before. So that's what we did."

The networks of major telecommunications companies required custom-tailored machines, ones capable of continuously processing day in and day out the massive amounts of data flowing through their infrastructures. These machines also needed to continually expand their traffic-handling capacities to keep pace with the exponential growth of Internet traffic and bandwidth demands. And they had to do all of this without ever stopping for a moment, since millions of people and businesses relied on their connections.

"In the telco world, you pick up a phone and you have dial tone. It is sort of unthinkable that the phone system wouldn't work," Epps says. "That was one of the primary ideas behind the 12000 Series: to instill phone service dependability into IP networks."

Epps says his partner on the 12000 Series project, Michael Laor, a director of engineering in the Cisco Carrier Core and Multi-Service business unit, championed the idea of "future-proofing" the router. Certainly, others in Cisco and the industry were considering how networking equipment could keep up with the growth of the Internet. But many worried such growth would not continue, rendering any future-proofing costly overkill.

"Michael brought the conviction and vision we needed to build a machine capable of growing with the Internet," Epps says. "Such future-proofing is a complex engineering challenge, but Michael believed in the sustained growth of the Internet and insisted we build into the 12000 Series as much expansion capabilities as technology would allow."

Part of that future-proofing included overbuilding capacity on the "switch fabric" (the control center of a router) of the 12000 Series so it could handle ensuing fiber optic connections with greater speeds, leaving the 12000 Series another four-fold of capacity potential.

"It was sometimes hard to imagine that we would ever use that additional capacity," Epps says. "At the time we were really pushing the limits of technology. But history has shown that our strategy was the right way to go to keep the 12000 Series on pace with the growth of bandwidth demands and communication advances."

Epps says the most difficult part of creating the 12000 Series was predicting long-term requirements. The risk, he explains, was that Cisco might either under-build the router, making it insufficient for future demands, or that the company might over-build it, making it too advanced and expensive to meet the market's needs.

"In designing the 12000 Series, we had to make a hard call where this was going to go," Epps says. "We had to guess at the rate of traffic growth. It seems obvious in hindsight but the Internet was still all very new at the time. And a wrong guess could have had dramatic repercussions."

The Cisco 12000 Series Router marked a turning point for Cisco and the industry. Cisco had been creating most of its equipment through evolutionary incremental upgrades. The 12000 Series was revolutionary, built from scratch. As part of this new approach, the 12000 Series needed far greater development resources than Cisco had traditionally invested in new products. The router took two years to complete, while most Cisco products until that point only required months.

The 12000 Series also marked a shift to a fully distributed, modular computing design. Previously, Cisco routers, as well as those from other vendors, worked more like a personal computer, with packet processing done by a central unit. In the new distributed design of the 12000 Series, the router's processing function ran on as many as 16 multiple processors. A switch fabric coordinated the work of the multiple packet processors. Such a design made the router far more powerful and capable of expansion. Instead of being limited by the power of a monolithic processor unit, a distributed computing system can add more processors to continue boosting its speed and data handling capabilities.

"It's like having 16 computers working together at the same time," Epps says.

Another key advantage of the 12000 Series design is that it eliminated any point of failure. If one component stopped working, the router will continue to operate normally by automatically switching to its complementary back-up component. All components were redundant, including fans, connections, and line cards, as well as processors.

Cisco's dedication to investment protection was most severely challenged during the creation of the third generation 12800 Series router, released at the end of 2003. The second generation 12400 Series had used up the expansion capabilities of the router. Further expansion for the 12800 Series required solving some "pretty hard physics problems," Epps says. It was a task equivalent to installing a racecar engine in a four-door sedan.

Epps and Laor stepped aside on the third generation 12800 Series, turning over duties to a team lead by Yuval Bachar, Flavio Bonomi, and Chandra Joshi. The team had to pioneer several new engineering methods and chip designs to create the 12800 Series, but, again, they were able to increase the performance of the 12000 Series by another four-fold.

During the years, the engineers working on the 12000 Series not only boosted its performance but also increased the device's functional sophistication. Service providers needed to offer more than just Internet connections. Their customers wanted virtual private networks, tiered services, quality guarantees, and other more detailed networking options. Cisco evolved the 12000 Series to meet those needs. Today, the router can play a wide variety of roles in all kinds of networks, making it not only one of the most powerful routers in the industry but also one the most flexible.

Certainly, the 12000 Series' continual improvements have required intense engineering efforts and unflagging commitment to the concept of investment protection. In the end, however, all the work through the three generations of the 12000 Series router has been worth it, Epps says.

"Aside from everything else, we know the 12000 Series turned out to be a pretty good idea because our customers tell us to just keep doing what we've been doing," he says. "With that in mind, we are looking forward to matching our achievements of the past with a future full of new innovations for the 12000 Series."

Charles Waltner is a freelance journalist in Oakland, Calif.