Early on April 9th, the City of Morgan Hill (California) experienced a mass internet outage, started by a targeted infrastructure attack. Bruce Perens has a good outline of what happened, why, and the lessons to be learned. Especially important are the local services that went down:
That attack demonstrated a severe fault in American infrastructure: its centralization. The city of Morgan Hill and parts of three counties lost 911 service, cellular mobile telephone communications, land-line telephone, DSL internet and private networks, central station fire and burglar alarms, ATMs, credit card terminals, and monitoring of critical utilities. In addition, resources that should not have failed, like the local hospital’s internal computer network, proved to be dependent on external resources, leaving the hospital with a “paper system” for the day.
The City of Morgan Hill, prepared for natural disaster, reacted quickly and used local amateur (ham) radio operators to reroute emergency services and other essential communications. It is surprising (to me, at least) that the local community made it through this emergency with little harm. The warning, though, should be clear: our communities are becoming dependent upon internet access, and when access goes down, so do many of our everyday systems.
Update (4/23/09) – Scott at the CIO Weblog provides a calmer reaction to the situation in Morgan Hill, but he still sees room for improvement:
My own preference is not to avoid new technologies and outsourced services, but instead to focus on independent and redundant lines of communication with which to reach them. This approach is much less reactionary, is less costly overall, and pays much greater dividends in the event of trouble than does basing all services at your own site.
The problem with this thought is when there is only one fiber provider in town, you are stuck with the level of redundancy the telecom company builds into their infrastructure.