What Data Center Managers can learn from Amazon’s failure and Netflix’s success

When I travel I sometimes find time to view one or more of the many short presentations available through the Ted.com universe. What a great concept of providing these short insights through an easy-to-access (even offline while in-air) interface. While obviously not all presentations catch my interest, most are actually quite good and some even mind-blowing.

One such presentation that caught my interest recently was Sebastian Wernicke’s “How to use data to make a hit TV show”. In this presentation Mr. Wernicke explains the difference in outcome for Amazon and Netflix in their use of (big) data analysis in their strive for creating the most successful TV show. While I was aware of some of the process behind creating Netflix’s smash hit “House of Cards“, I was blissfully unaware of Amazon’s similar attempt with “Alpha House“, that apparently never found the targeted viewers attention.

In short, while Amazon was piloting 8 different TV show ideas and deciding based on an analysis of data about user behaviour (while watching each of them), Netflix was looking at the data they already had from their huge catalog of series and movies and then drew a series of conclusions based on that. In Mr. Wernicke’s own words the reason for Netflix success was “…because they used data and brains where they belong in the process. They use data to first understand lots of pieces about their audience that they otherwise wouldn’t have been able to understand at that depth, but then the decision to take all these bits and pieces and put them back together again and make a show like “House of Cards,” that was nowhere in the data.”.

…because they used data and brains where they belong in the process.

While this might come as a surprise to some, the conclusion of Mr. Wernicke (himself an expert in computational genetics) was clear: When faced with a complex problem, the approach is always the same: First you take it apart in its pieces and then you put it back together. Data analysis is only good for the first part, while the second part requires a brain to resolve, since taking risk is a major component of that.

Now with more than a decade of experience in the data center industry, this statement made me smile a bit, since isn’t this exactly one of the problems we often face in managing our data centers? Over the years we have been deploying various kinds of (more or less sophisticated) monitoring solutions with more or less the same business case: To be able to gather enough data to make reasonable decisions.

Now here comes the problem, since can you actually make decisions purely based on aggregated data? My opinion is clear: No you can’t! Aggregated data can only tell you what has happened, not if that was good or bad and wether or not this was aligned with your intentions / expectations. Yes, you can define thresholds that will help you understand if you’re operating within normal conditions, but it will provide no answers or guidance if you’re trending outside the expected. It can’t tell you what’s going to happen either. Even the best of tending / predictive analysis tools on the market, most often gets it wrong.

Your brain however (with your accumulated experience) will be able to make these conclusions, when presented with a proper set of insights!

This is then where more and more data center managers are turning to DCIM type solutions, in order to bridge the gap between plan and actual, however in my experience too many are pursuing this for the wrong reasons. If you think a DCIM will provide you all the answers and magically improve the status of the data center operation (like some crazy vendors claim) and maybe even enable you to get rid of half your staff, then you’re off to bad start. However if your goal for implementing a DCIM is gaining a better understanding of your current operation, in order to be able to make better decisions going forward, then you’ve already taken the first step into a successful DCIM journey.

In its purest form, DCIM is a risk mitigator, that will enable you to make informed decisions for your data center(s) (like comparing plan and actual) in your strive to optimise your operation. Monitoring is obviously a big part of that, but end of the day, its your decision to make, so why not make it an informed one? Your brain holds the power, now unlock it with DCIM…