San Diego blackout highlights infrastructure vulnerabilities

Officials say outage was caused by human error

Thursday's massive power outage across parts of Arizona and southern California serves as another reminder of the vulnerabilities in the nation's power infrastructure.

The outage appears to have been caused by human error. But that fact is unlikely to comfort the growing number of people concerned about blackouts that could be triggered by cyber attacks.

Just this week for instance, the Bipartisan Policy Commission, a Washington-based think tank, expressed concern in a report about attacks against the nation's power infrastructure . The report said that attacks capable of triggering "cascading disruptions and damage" to U.S. power infrastructure are not just theoretical threats, but a very real danger.

The southern California outage started around 5.30 p.m ET Thursday and left close to 1.5 million customers of San Diego Gas & Electric (SDG&E) without power for nearly 12 hours. As of this afternoon, power had been restored to all affected areas.

In a news conference streamed live by local TV stations this afternoon, SDG&E president Mike Niggli said the problem started when an employee with APS, Arizona's largest utility company, was working on a capacitor, at a substation near Yuma.

"It was a human error that was the initiating event that took the transmission line out," Niggli said, nothing that the incident caused power flows to be redistributed throughout large portions of the western United States. "We know what the initiating event was. The question is, how did that ripple through the rest of the systems?"

In a statement APS said the outage was related to a 'procedure' at the North Gila substation northeast of Yuma. The error resulted in the 500 kV transmission line near Yuma tripping offline. Existing measures should have isolated the resulting outage to the Yuma area, the statement said.

"The reason that did not occur in this case will be the focal point of the investigation into the event," APS said. That review is already under way.

In a cascading blackout, problems in one section of a power gird ripple out over the entire gird. Similar, larger blackouts have happened elsewhere.

In 2008 for instance, a fire in a substation near Miami triggered a cascading blackout across a large swath of Florida, leaving three million people without power for hours. In 2003, a similar blackout in the northeast affected close to 15 million people in New York, Connecticut and even parts of Canada and the Midwest.

Investigators later determined the problem started when an engineer with Midwest Independent Transmission System Operator disabled a software function that allowed the utility to determine the real-time status of the power grid in its region. That problem was later exacerbated by a software failure at FirstEnergy Corp., which contributed significantly to the problem.

It's too early to say for sure what happened with yesterday's blackout, said Joseph Weiss, managing partner at Applied Control Solutions LLC and author of the book Protecting Industrial Control Systems from Electronic Threats.

But the key takeaway is that a cascading blackout can just as easily be triggered by a malicious act as by human error, he said. "The only way you can tell the difference is the intent of the individual," he said.

Often cyber security analysts tend to view threats to the power grid in the same way they view threats to information networks, he said. Any incident that results in an industrial control system being taken offline because of something happening upstream is, in a sense, a cyber incident.

In his book, Weiss says that there have been at least 170 known cyber-related outages in the U.S., including three that caused widespread regional outages. The relative lack of forensics-gathering capabilities in the utility business makes it hard to determin whether any of them might have been the result of a malicious act, he said.

"Because we have so little control systems forensics, it is very difficult to determine what happened" with many of these incidents, Weiss said.

Jaikumar Vijayan covers data security and privacy issues, financial services security and e-voting for Computerworld. Follow Jaikumar on Twitter at @jaivijayan or subscribe to Jaikumar's RSS feed . His e-mail address is jvijayan@computerworld.com .

Read more about business continuity in Computerworld's Business Continuity Topic Center.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags managementNetworkingdisaster recoverysoftwareapplicationsBusiness Continuity

Show Comments
[]