What do you do when a network administrator goes bad? That's the question IT staffers for the city of San Francisco are facing. One of their own, a net admin named Terry Childs, was arrested for sabotaging the city government's new fibre backbone network.
Now he's in jail. But he's not the only one who will pay the price.
Until recently, Childs was well regarded in his IT shop. But last month, that all began to unravel. On June 20, he reportedly went after the IT department's newly hired head of security, chasing her with a camera until she locked herself in an office to escape.
The security chief was auditing who had password access to the new network. According to the San Francisco Chronicle, that audit soon turned up the fact that Childs had changed passwords so that only he had administrative access to the network. Childs also reportedly installed software to monitor his managers' emails related to his job performance.
Childs was suspended on July 9 for insubordination. When police asked Childs for the passwords, first he gave them false information, then he refused to divulge the real passwords. On July 14, Childs was arrested for felony computer tampering.
Because prosecutors feared for the security of law enforcement documents, jail inmate bookings, payroll files and other city, Childs was jailed with bail set at a whopping US$5 million (NZ$6.5 million).
As of the middle of last week, Childs hadn't explained why he allegedly locked up the network — and he hadn't coughed up the passwords.
Meanwhile, his co-workers are now facing the possibility of having to rebuild the network and verify everything stored on it — about 60% of all government data in San Francisco.
Oh, and also having to live under a cloud for as long as they work in that IT shop. Because someone in their department was trusted too much, they'll be distrusted. By users, managers —and one another.
Is that fair? Of course not. They're not the ones who sabotaged the network and created a huge mess. They're not the ones who should have acted more quickly and effectively when it looked like there was a problem employee on the loose.
But they're the ones who'll pay.
So what to do in the wake of that net admin gone bad? First, there's the fibre backbone to unlock and sanitise, or rebuild if necessary. Then, no doubt, new technology and new layers of auditing will be added to increase security scrutiny.
That's important, but it's not enough. Childs was able to lock up the network because he was, in effect, a one-man show. Setting up a team structure for network administration, in which team members cross-train and rotate through one another's jobs, would make such an occurrence much less likely. It would also help avoid problems when a net admin doesn't go rogue, but gets hit by a bus instead.
No lone wolves. No bottlenecks. Nobody who's indispensable.
Cross-trained teams also make it easy for managers to act swiftly when an employee turns into a problem — there's no hesitation because of worrying about how he'll be immediately replaced. And teams help foster a culture of security; they police themselves in ways no security audit can.
What teams can't do is restore what Terry Childs stripped from his co-workers: the respect and trust of the people who depend on them.
The estimate for rebuilding San Francisco's fibre backbone, if it comes to that, is eight weeks.
But rebuilding respect and trust for San Francisco's IT department? That could take years.