ATC’s CTO, Nick Enger, was recently quoted in the Wall Street Journal regarding the bombing on Christmas day in Nashville. An AT&T transmission building was squarely in the blast zone and incurred heavy damage, which exposed the data center to outside elements when an exterior wall collapsed.
A fire ensued, which was put out with water. Ordinarily, data center fires are extinguished by sucking the air out of the affected space to best protect the hardware and data. As a result, the water and fire damage disabled backup generators that were supposed to keep AT&T’s network functioning. The AT&T building is a regional network hub for various carriers to cross-connect, thus many carriers were affected, not just those with “AT&T” service.
This “significant damage” caused a multi-state internet outage, bringing down wireless and wired networks across parts of Tennessee, Kentucky, and Alabama, while disrupting cell service, some 911 networks, and communications at the Nashville International Airport.
The result was days of service outages that had a direct impact on many businesses that were not properly prepared for a full-blown network outage such as this. It wasn’t until Tuesday, December 29th, that AT&T declared all is well.
In light of this, Enger provided eight quick-hitter tips, a checklist per se, of items you need to be thinking about—as we head into 2021—when it comes to network infrastructure, disaster recovery, and business continuity.
- Identify mission-critical data and facilities; determine their risk tolerance for outages based on potential events.
- For mission-critical facilities, design a network strategy to include primary, secondary, and tertiary network connections via High-Availability (HA) network infrastructure.
- Include a dual SIM cellular connection as the tertiary connection to provide a “lifeline” to the facility’s network infrastructure; this allows mission-critical communication to continue throughout an event.
- Legacy networks (i.e. VPN or MPLS) should be migrated to next-generation network technologies such as SD-WAN or SASE.
- Mission-critical data and applications should be migrated to a hardened data center or cloud provider and replicated to a geographically diverse location.
- Voice and collaboration has significantly increased in importance for organizations and is typically deemed mission critical. These should be migrated to a cloud environment to increase stability and end user accessibility, even during catastrophic events.
- Document the disaster recovery/business continuity plan and the communication plan for each potential event.
- Review and adjust—annually—the disaster recovery/business continuity plan and the communication plan.
If you would like to run through this checklist in further detail as it pertains to your specific situation, please schedule a discovery call at your convenience here.
