A Ridiculous Failure of Critical Infrastructure

Since you won’t find this reported anywhere in the news (from the current looks of it), it’s left to folks on the ground to figure things out.

Some kind of combination of failure between Charter and Qwest has left tens of thousands of people in Nebraska without Internet and has disrupted the Internet and phone services for thousands more. Right now, the outage is going on 12 hours and there is no ETA for repair in sight.

The word coming down is that the outage is on a Qwest fiber, but it looks to me like both parties should be on the hot seat for not having the ability to route around the problem. There was a four hour outage on Charter a week ago that was caused by a fiber cut in Gothenburg, Nebraska. That one killed everything west of the cut, but it was small potatoes compared to this one. Is this truly the level of performance that we can expect from our major Internet backbone providers? It took me about 10 seconds to re-route my traffic to a backup provider - you would think that a couple of multimillion dollar companies would be able to sort out a problem of this nature in a reasonable amount of time. The small CLEC that I use for my backup connection had enough capacity to route around the problem and was even able to lend me a little bit after 5pm when the traffic on their network (mostly businesses) dropped off. It isn’t rocket science to figure out how to route around an outage.

Almost as frustrating is that there was NO news about the outages anywhere except on the social networking sites (Facebook, Twitter). One TV station in Hastings, NE put up a short story on their website, but I got more news from the tweets and FB posts that people where posting from their cell phones than I did from anywhere else. None of the network outage sites have any news about this.

Could this be a harbinger of things to come? I am feeling pretty thankful right now that I have a choice in backbone providers and that I kept a second one. Diversity is a good thing, and this is a great example of why we need competition and multiple options for Internet.

This is the sort of event that can start the dominoes falling toward “more regulation.” Network outages that effectively strand individuals (read “voters”) for any length of time is the one thing that politicians and regulators can understand and react to, much more readily than vague network neutrality, competition, technology or pricing issues. Network outages affect constituents immediately and obviously so politicians and regulators will feel compelled to “do something.” This instance won’t be a triggering event but if similar outages on a “critical infrastructure” occur on a regular basis and make the news, pressure will grow and grandstanding politicians will latch onto “saving the internet” as a great issue. Service providers screw up like this at their own peril.