Chief Security Officer John Stewart Explains Cisco's Procedures for Addressing Suspected Security Vulnerabilities

July 31, 2006

Given Cisco Systems' prominent role as the world's leading supplier of equipment for running the Internet and private networks, Cisco has established comprehensive processes for identifying, investigating, and addressing any and all suspected security vulnerabilities in its hardware and software. Cisco has had in place a formal security response framework since 1997. Cisco's overarching goal regarding any security issue is to help its customers. News@Cisco spoke with John Stewart, Cisco's Security Officer for corporate security programs, about how the company addresses suspected security vulnerabilities in its products.

What process does Cisco follow once it is aware that one of its products might have a security vulnerability?

John Stewart: When we receive a report from a researcher or an organization that indicates one of our products might have a security vulnerability, the information goes directly to the Cisco Product Security Incident Response Team (PSIRT), a highly specialized group of security engineers that addresses the most serious security issues, including suspected vulnerabilities in products or imminent threats to customer networks. PSIRT first validates any report by doing its due diligence and verifying that, indeed, the vulnerability or threat does exist. Next, the team conducts an investigation to find the root cause of the problem. The goal is to make sure Cisco understands all the factors and potential effects of the vulnerability. After this process, the PSIRT team then works with other engineers at Cisco to develop a fix, which is tested, examined, and re-tested as thoroughly as possible. Once that process is complete, we then notify our customers and make the fix available.

Our approach to how we disclose vulnerabilities is guided by the overriding principle to do what is best for our customers. Assuming the vulnerability is not, to the best of our knowledge, currently being exploited, and we have been able to address the vulnerability and get a fix to our customers so they are protected, we then disclose the vulnerability to the broader industry. If we announce a security vulnerability before a fix is created, that would only give malicious entities more opportunity to inflict harm. It's like a boxer with his hands down. Our customers could get clobbered without the proper network defenses in place. So our policy is to announce a vulnerability only when we have a fix available for customers. We work as cooperatively as possible with all researchers and organizations to conduct an investigation of any serious security vulnerability issue. Our process follows the framework for dealing with security threats created by the National Infrastructure Advisory Council (NIAC), which was formed by the President of the United States to provide the federal government guidance on critical infrastructure issues and to coordinate related efforts by governmental and private sector organizations.

How do you work with researchers or other security experts to uncover any potential security weaknesses in Cisco products?

John Stewart: Cisco has had a very successful and long-standing relationship with the information technology research community. Our contacts within the community span all types of organizations, from universities and independent consultants to other vendors and several industry security forums such as FIRST (Forum of Incident Response and Security Teams). The PSIRT staff communicates with the research community daily. When a researcher contacts PSIRT about a suspected security vulnerability, Cisco makes every effort to work in conjunction with the researcher to verify the problem and develop a remedy. And as I mentioned before, we are very willing to give appropriate credit for the discovery once the problem has been addressed and disclosed.

There has been speculation in the industry media that Cisco is antagonistic towards the research community or towards researchers who want to reveal security problems in Cisco products. But if you ask the security experts who have experience working with us, you will find that we have a very open, active and cooperative relationship with the research community. We are continually asking for and receiving feedback from researchers, and we work diligently to make it as easy as possible for them to work with Cisco. As I stated previously, all of our actions are guided by how we can best help our customers. If a researcher prefers to make a security vulnerability disclosure independently of Cisco, we have to respect that. Cisco, however, is very concerned that researchers do not disclose any Cisco intellectual property information that is not germane or necessary to addressing a particular security issue, regardless of whether the researcher independently uncovered the intellectual property information or gained access to it through work with Cisco on the security matter. PSIRT deals with 30 to 50 security vulnerability issues a year, and the group has been operating since 1995, so they have worked successfully many times with researchers to address security vulnerabilities while also helping us protect our customers.

How can someone contact Cisco if they suspect a security vulnerability in a Cisco product or a network that uses Cisco equipment?

John Stewart: The Cisco Product Security Incident Response Team (PSIRT) is the single point of contact at Cisco for any serious security issue. Anyone--customers, researchers, and other vendors--can contact PSIRT. Telephone numbers, email addresses, and other means of contact with PSIRT can be found on our public company Web site by clicking the link "Cisco Product Security" at http://www.cisco.com/security. The Web site provides detailed contact information and information about reporting suspected vulnerabilities, as well as outlining our process for systematically addressing these types of network security issues.

For customers who have general security concerns or issues regarding their Cisco-equipped networks, they can contact our Technical Assistance Center (TAC), which is our main technology advisory service. The experienced engineers at TAC can help customers assess their security issues. TAC works in close conjunction with PSIRT and will notify the group if TAC personnel cannot address an issue.

How can someone find out the latest information regarding security vulnerabilities related to Cisco products?

John Stewart: There are several venues. We send out "security advisories" for addressing security issues that directly involve Cisco products and require a repair, fix or other customer action. We provide "security responses" for recommendations to mitigate general problems affecting network stability or for issues that require a response to information posted to a public forum. We also send security-related "field notices" and other security information of interest to customers via email. Information about all of these methods, as well as additional information about network security can be found at Cisco's public Web site. We also send security software updates to customers with contracts, as well as providing other support to our customers, depending on the situation. All security vulnerability information provided by PSIRT is located on Cisco's public web site at http://www.cisco.com/security and http://www.cisco.com/go/psirt.

What other steps does Cisco take to address potential security vulnerabilities?

John Stewart: Cisco always strives to create the highest quality products in the industry. Certainly our success indicates we have been doing that. But, as with any company or person, Cisco is not foolproof. We do make mistakes. Our extensive security vulnerability response procedures demonstrate our recognition of this fact. But Cisco, as well as all of our customers, would prefer as few security problems as possible. Perhaps more than any other aspect of networking, security is the greatest concern for our customers. So we focus our engineering expertise on this topic accordingly to design, build, test, and test again our products as thoroughly as possible before releasing them to the public. We lead the industry with our multi-layered, integrated, systems-based approach to security, and we continue to create new and more effective tools against network-based attacks and crimes. Finally, we continue to work in close cooperation with our customers to provide them the products they need to run the most secure networks in the world. Our customers and millions of people rely on Cisco networks everyday. We take that responsibility very, very seriously.