Delta's Data Center Crash

Proof IT Needs to Get Involved

introduction

Last month Delta Airlines suffered a power outage in its Atlanta data center. What should have been a small problem became an operational nightmare as core data center systems failed to recover. Without control of their network, Delta was forced to ground hundreds of flights.

Delta has since reported a $100 million hit to its revenue as a direct result of the system outage.

Whose fault was it? Tough to say. Technicians on site blamed a failed “switchgear” for not routing power to a backup generator. However, the airline has yet to address broader questions about legacy technology and operational changes.

Even more concerning is that Delta isn’t the only airline who has recently had network related issues. Both Southwest and American airlines experienced similar outages during the last year. These meltdowns highlight vulnerabilities in the information and network systems which support the modern enterprise – and these systems live and breathe by the strength of the data center.

As Delta looks to reinforce their data centers they will have to consider a variety of factors. Having efficient and agile IT is critical but the industry is changing quickly. Best practices are no longer written in stone. With Delta’s collapse being front page news, I thought it appropriate to share some thoughts on the industry.

scale with stability

Stability is the name of the game for data center managers. Expected or unexpected outages (known as downtime) are a key metric they measures themselves on. For example, in a single year, the top tier data centers have 7/9 uptime meaning that they are online for 99.99999% percent of the time. This amounts to 3.15 seconds of downtime. The tricky part is maintaining this high availability as they scale. It means multiple points of failure, reliable crossover to backup systems and an airtight disaster recovery plan. The trouble for the large enterprise is that updating and scaling these systems is challenging.

For many legacy data centers, their core design comes from an era when the presumption was that the systems would go down every night. This is just not the case for many organizations – particularly airlines.

resource allocation

The golden question for Delta’s IT organization is now where to put their resources. Technology is changing so quickly that it’s no longer enough to consider short term improvements in efficiency. Long term trends need to be taken into consideration.

Many organizations are now looking to cloud computing. The economic benefits of moving to the cloud are significant. However, it does carry a host of implications.

For one, it means less reliance on physical infrastructure – they will favour smaller, more agile deployments against the monstrosities of old. It means converged infrastructure and new levels of integration for key building processes like access control and power management. But most importantly it requires the IT department to shape innovation around the strategic goals of the business.

shifting role of IT

Handling the refresh to the data center requires IT and management teams to be 100% aligned. With access to data and reliability becoming ever more relevant in today’s world, it’s the army of CIO’s and IT teams who carry the resource sword.

They understand the tech, the requirements and now have the means to implement strategies that truly affect the bottom line of a business. It’s a fascinating shift in power away from the more traditional models. The CEO still has the final say, don’t get me wrong. But the IT professional has an increasing influence and decision making power. The CEO is now looking to the IT professional for guidance.

To give you an idea of what top tier spending looks like, let’s take our lucrative friends at Google. Last year their data center related expenditure was almost 2 billion dollars. This consisted of “investments in production equipment, facilities and data center construction.” This level of investment may seem difficult to quantify, I agree, but not when you consider the repercussions of a system failure on revenue and/or company reputation.

closing remarks

The future of this industry is exciting. Volume and variety of data is increasing. Much of that driven by new ways to interact with the digital world – social, mobile, biometric wearables. How will the enterprise use the internet of everything to their advantage? Whatever the solution is you can be sure that it will rely on the strength and security of the data center.

Delta's Data Center Crash

Proof IT Needs to Get Involved

Last month Delta Airlines suffered a power outage in its Atlanta data center. What should have been a small problem became an operational nightmare as core data center systems failed to recover. Without control of their network, Delta was forced to ground hundreds of flights.

Delta has since reported a $100 million hit to its revenue as a direct result of the system outage.

Whose fault was it? Tough to say. Technicians on site blamed a failed “switchgear” for not routing power to a backup generator. However, the airline has yet to address broader questions about legacy technology and operational changes.

Even more concerning is that Delta isn’t the only airline who has recently had network related issues. Both Southwest and American airlines experienced similar outages during the last year. These meltdowns highlight vulnerabilities in the information and network systems which support the modern enterprise – and these systems live and breathe by the strength of the data center.

As Delta looks to reinforce their data centers they will have to consider a variety of factors. Having efficient and agile IT is critical but the industry is changing quickly. Best practices are no longer written in stone. With Delta’s collapse being front page news, I thought it appropriate to share some thoughts on the industry.

Stability is the name of the game for data center managers. Expected or unexpected outages (known as downtime) are a key metric they measures themselves on. For example, in a single year, the top tier data centers have 7/9 uptime meaning that they are online for 99.99999% percent of the time. This amounts to 3.15 seconds of downtime. The tricky part is maintaining this high availability as they scale. It means multiple points of failure, reliable crossover to backup systems and an airtight disaster recovery plan. The trouble for the large enterprise is that updating and scaling these systems is challenging.

For many legacy data centers, their core design comes from an era when the presumption was that the systems would go down every night. This is just not the case for many organizations – particularly airlines.

The golden question for Delta’s IT organization is now where to put their resources. Technology is changing so quickly that it’s no longer enough to consider short term improvements in efficiency. Long term trends need to be taken into consideration.

Many organizations are now looking to cloud computing. The economic benefits of moving to the cloud are significant. However, it does carry a host of implications.

For one, it means less reliance on physical infrastructure – they will favour smaller, more agile deployments against the monstrosities of old. It means converged infrastructure and new levels of integration for key building processes like access control and power management. But most importantly it requires the IT department to shape innovation around the strategic goals of the business.

Handling the refresh to the data center requires IT and management teams to be 100% aligned. With access to data and reliability becoming ever more relevant in today’s world, it’s the army of CIO’s and IT teams who carry the resource sword.

They understand the tech, the requirements and now have the means to implement strategies that truly affect the bottom line of a business. It’s a fascinating shift in power away from the more traditional models. The CEO still has the final say, don’t get me wrong. But the IT professional has an increasing influence and decision making power. The CEO is now looking to the IT professional for guidance.

To give you an idea of what top tier spending looks like, let’s take our lucrative friends at Google. Last year their data center related expenditure was almost 2 billion dollars. This consisted of “investments in production equipment, facilities and data center construction.” This level of investment may seem difficult to quantify, I agree, but not when you consider the repercussions of a system failure on revenue and/or company reputation.

The future of this industry is exciting. Volume and variety of data is increasing. Much of that driven by new ways to interact with the digital world – social, mobile, biometric wearables. How will the enterprise use the internet of everything to their advantage? Whatever the solution is you can be sure that it will rely on the strength and security of the data center.