Database Insights from Archimedes to the Houston RocketsTokutek
According to a recent MIT Sloan Management Review study, top performing organizations use analytics 5 times more than lower performers. That’s pretty astounding. And while we all know about the ocean/lake/waves/(your favorite water analogy) of Big Data we struggle with everyday, information is not knowledge. So how can we get insight from data? Recent articles from O’Reilly and HBR offered some interesting clues for DBAs and software vendors.
Use what you have for “Eureka!” moments
As O’Reilly noted from the recent Economist Big Data conference, Archimedes had taken plenty of baths in his life. It was only when Hiero of Syracuse asked him an important business question about how to distinguish pure gold objects from silver-filled substitutes that he stopped to think about how to “query” his familiar dataset (in this case, tub of water) to get an answer on displacement, and ultimately volume and density. The point is, everything he needed was already familiar and available to him.
O’Reilly generalizes the argument to say that most big companies have plenty of familiar data – the problem is that it exists in silos, or isn’t being leveraged, or the right questions aren’t being asked. The example in his article is a top-end, lucrative, fairly happy United Airlines frequent business flier who all of a sudden stops flying with the airline, but keeps using his points for cars and hotels. The back story is that his company was acquired and the new place had a deal with American Airlines for flights. This would be a perfect opportunity for United to reach out to him as an advocate to try to lure in a new business contract with his new firm. Instead, United never even bothered to call to see why he had suddenly stopped using them. In short, the data was there (United was literally tracking his every move) but the right questions weren’t being asked.
Round out the bases with unique data
Daryl More, the General Manager of the Houston Rockets, notes in his piece in HBR that “success comes from better data, not better analysts”. He notes how, per Moneyball, the cash-strapped Oakland Athletics amassed a team of high talent with little budget based on rigorous analytics. However, this comparative advantage quickly eroded as team such as the Red Sox, and eventually others, started hiring top analysts as well. Realizing that this was happening now across a variety of professional sports, including basketball, Daryl has had to go one step further to stay ahead. He finds new sources of data as the new advantage his Houston Rockets. As an example, he cites NBA leaders spending small fortunes to collect new types of data such as “challenge shots” not only in their own teams, but across the league.
Impact to DBAs and Software
OK, so what does this all mean for DBAs and database software companies? In order to get fresh insights out of existing data, one has to be able to nimbly add new queries and indexes as data-sets are explored in new ways. In order to deal with ever-new and ever-growing volumes of information, one needs databases that can handle large amounts of data while keeping performance levels high. This includes being able to add new fields. Database vendors who can help with these challenges will be welcome partners to companies who are trying to become, as the MIT article noted, “transformed” business making the most of, and exceeding with, advanced analytics.