January 23, 2003

by David Barry, News@Cisco

In the United States, American football is not so much appreciated as a sporting event by its fans as it is worshipped.

For more than 40 years, New Jersey-based film company NFL Films has preserved the most compelling moments in National Football League (NFL) history--from last-second, come-from-behind drives to awe-inspiring, victory-clinching receptions.

Each year, the company creates more than 400 hours of television programming- including their Emmy award-winning weekly series Inside the NFL, which airs on HBO, and NFL Films Presents, which airs on ESPN-as well as live Webcasts and "fantasy football" Web programs.

But the company's biggest event occurs each January, when the NFL Films crew captures the sights and sounds of the annual sporting spectacle known worldwide as the Super Bowl.

The Super Bowl is the sine qua non of American football games-pitting the two best teams in the league against each other for one, winner-take-all championship match. Watched by more viewers than any other sporting event in history, the Super Bowl is a true phenomenon. In 2002, for example, the Super Bowl game attracted 131.7 million viewers making it the fifth most-watched program in television history.

Capturing the images that will be viewed by hundreds of millions of football fans across the globe is a huge undertaking, and one on which NFL Films has earned its reputation.

NFL Film's Challenge

In July of 1999, NFL Films broke ground on a new 200,000 square foot, state-of-the-art motion picture and television studio in Mount Laurel, New Jersey. The company wanted to create a facility that would bring all its various production and post-production operations under one roof.

Expectations for the facility ran high. The complex was to be the epicenter of the company's activities, and would house the technology that would bring the company into the 21st century, ensuring a smooth transition from celluloid to digital, high-definition film production, while maintaining NFL Films' position at the forefront of filmmaking technology.

"When we began designing our new building that would provide us with the most technically advanced video and film facilities anywhere, we also saw it as an opportunity to radically change our voice communications capabilities - both at our headquarters location as well as offsite at the Super Bowl," says Dave Franza, chief information officer (CIO) and executive Internet producer at NFL Films.

Because half of NFL Films' staff attends the Super Bowl each year-for as long as three weeks at a time-staying in touch and accessible was a priority. Traditionally, communications between the NFL Films crewmembers at the Super Bowl was less than optimal: each crewmember was provided with a temporary analog phone with temporary voice-messaging mailboxes, which they were to use to communicate with other crewmembers. They also relied on pagers and cell phones to communicate, which often proved unreliable in a stadium environment.

"Traditionally, it was difficult to reach us at the Super Bowl site," says Franza. "For example, people might call my regular extension and end up being put into voice mail because they didn't know the number at the Super Bowl site."

In addition, getting the whole analog system set up invariably proved frustrating for the company.

"The phone installation was always the unknown," explains Steve Eager, director of network systems administration at NFL Films. "Typically we'd be down at the site on two Mondays before the Super Bowl, and it wouldn't be until Friday or Saturday of the week we arrived before everything was ironed out and all the analog phones were installed. Human error, configuration problems, problems with the lines--you name it, it always happened."

The company also saw the installation of a new phone system as an opportunity to realize greater employee productivity through unified messaging, which would enable NFL Films staff to retrieve all their messages--voice mail, e-mail, fax, or pages--from a single messaging mailbox.

Finally, the company wanted the ability to be able to provide its crews with in-building extension mobility, allowing them to move their communications setups anywhere they needed to work-both in the building and on location.

The IP Solution

To address these issues, NFL Films chose to implement a Cisco IP Communications solution.

The company's headquarters span two buildings on the Mount Laurel campus. Each building houses a data center with a Cisco Catalyst 6509 Switch at the core and Cisco CallManager software, the call-processing agent that extends enterprise telephony features to NFL Films' more than 475 Cisco IP phones.

In phone closets throughout the buildings, NFL Films maintains Cisco Catalyst 4006 switches with multihomed connections from each switch to the Cisco Catalyst 6509 in each building's data center. With multihome connections, each Cisco Catalyst 4006 has at least two physical connections to the core Cisco Catalyst 6509--ensuring redundancy for high availability. Further redundancy is built in at the entry point from the Verizon Synchronous Optical Network (SONET) rings. NFL Films maintains two communications entry points from Verizon--one to each building--so that if one facility fails all data is automatically routed to the other connection.

"Where your phone system is concerned, you want to build in the highest redundancy and reliability," Eager explains. "And the Cisco solution has worked flawlessly. We haven't had any downtime since the system has gone live."

The Cisco IP phones have all the functions one expects from a telephone as well as more advanced features such as the ability to access Web sites. Ease of movement has been one of the major advantages of the IP phones. Unlike traditional public branch exchange (PBX) systems, moves, adds, and changes are virtually instantaneous. Users take their IP phones with them to their new location, plug it into the Ethernet jack, and the phone registers itself with the Cisco CallManager. All user privileges and settings are automatically re-established--eliminating the cost and delay of sending vendor technicians to wiring closets. NFL Films also estimates that had it gone with a traditional phone system, the company would have needed to hire additional telecommunications personnel to handle the continual adds, moves, and changes required with a traditional PBX system.

"Because of the network intelligence, these changes are made automatically so we're able to use our staff more effectively and efficiently," says Franza. "We're able to use our data personnel to manage the phones because it's really just another application on the data network. On the server side, we just added another server. The same people managing our IP infrastructure now also manage the phones."

In addition to the ease of management, cost savings was another consideration that led NFL Films to the Cisco IP Communications solution.

"By going with IP Telephony, we needed a total of only 800,000 feet of category 6 copper cable for our converged voice and data network -- which provided a huge material and labor savings, says Eager. "If we had gone with a traditional PBX we would have needed an additional 400,000 feet of cable for the voice traffic alone. Also we didn't have to purchase a traditional PBX switch; we simply added another server to our data network. The combination of the cabling savings and eliminating the switch saved us about 40 percent of the total cost--and that was a big number."

NFL Films estimates a savings of approximately $400,000 as a result of installing a Cisco IP Communications solution rather than a traditional PBX with similar user features. This savings was realized due to the decreased need for cable, installation labor and back-end equipment. Additionally, long-distance telephone bills have been reduced by approximately 60 percent since IP Telephony was fully implemented.

The ability to increase employee productivity was another benefit of the solution. Like many organizations today, NFL Films personnel are inundated with many communications channels--telephone, fax, voice mail, e-mail, pocket pagers, mobile phones, and personal digital assistants (PDAs). While the explosion of communications choices has enhanced worker capabilities, keeping track of incoming calls and messages can be overwhelming--and extremely time consuming.

Cisco Unity^TM Unified Messaging helps workers by bringing all contact channels together into a single inbox. Workers can listen to their e-mail over the phone, check voice messages from the Internet, and when Cisco Unity is integrated with a supported third-party fax server, forward incoming faxes to wherever they may be--even at the Super Bowl thousands of miles away.

"I think a lot of our people don't even remember their voice mail password anymore because they now get their voice mail through e-mail," says Eager. "We estimate that Cisco Unity is saving our workers roughly an hour a day by eliminating all the steps previously required to access all their messages throughout the day."

The Cisco solution has improved the company's communications on location as well.

Thanks to the Cisco IP Communications solution, NFL Films is now able to extend its headquarters data network to its trailer area at the Super Bowl, making data and voice communications with headquarters virtually seamless. The company lays the cable, creates a "hub site," and installs two T1 lines for its wide area network (WAN). All communications-data, voice, and video are now able to travel over a single network.

In addition, NFL Films' IP communications network has allowed the company to eliminate connectivity and installation problems previously experienced with local phone service providers.

Each year, the Super Bowl takes place in a different city in the United States. The 2003 Super Bowl game, for example, will take place in San Diego, California. Thanks to the new IP Communications system, NFL Films is able to pay for a local primary rate interface (PRI) from the local telecommunications provider in San Diego that will enable local area code calling while at the Super Bowl, thus eliminating toll charges for calls within the San Diego area. Long-distance charges have also been greatly reduced as all calls back to headquarters, and to other long-distance destinations are carried over the same WAN used for data. NFL Films also no longer has to rent phones from a local company for the duration of their stay or pay the local phone distributor to bring in approximately 100 analog lines.

The company is also using the Cisco Survivable Remote Site Telephony (SRST) feature in the Cisco IOS. Software--which provides call processing redundancy while taking advantage of the existing remote site infrastructure to help ensure the reliability of NFL Films' "mobile office." In the unlikely event the WAN fails, the Cisco SRST solution -running on a Cisco 3640 Access Router - sends calls to the public switched telephone network (PSTN) to ensure that all calls get completed.

Unmistakable Results

NFL Films couldn't be happier with their new system.

"The Cisco IP Communications solution has helped us on so many levels, from our own in-house productivity and cost reduction to smooth operations at the Super Bowl," says Franza.

At the Super Bowl, our aim was to set up a remote office that would be very similar to the office we have in New Jersey--in essence, a virtual office. Everything is similar-you're getting your calls, voice mail, e-mail--just like at the home office. We've also cut our 'go live' time at the Super Bowl in half due to the Cisco IP Communications solution. The savings in terms of ease of use, productivity, and instant communications is incalculable."

"I guess what best sums it up is the reaction of outsiders, including many of the NFL team owners, when they tour our new facility. Here we have the most sophisticated gear--including high-definition component digital video capabilities, a satellite dish farm, a recording studio capable of seating a full orchestra, and a state-of-the-art shooting stage--but when they finish the tour, one of the most talked about features of our facility is the IP communications phone system. They're so amazed by its capabilities, the quality of sound and the look of it that a number have called their finance and IT staff soon after to talk to them about upgrading their phone system to Cisco IP Communications."

David Barry is a freelance journalist based in Princeton, NJ.