Getting to grips with Big Data and how to extract value from it
By Dr Marie Taillard, CMC Director
In the morning-after analyses following the 2012 US Presidential elections, most pundits agreed on one particular point that had gone very right for the Obama campaign and terribly wrong for the Romney campaign: their respective ability to gain real competitive advantage from the masses of voter data available to them. Both of the campaigns had bet heavily on being able to extract value from the thousands of available databases containing voters’ leanings on political and social issues, previous endorsements and donations, from sources ranging from social media to registration records to door-to-door canvassing reports. For the Democrats as well as the Republicans, data management was a key strategic focus, albeit a daunting one, as the potential of so-called Big Data as a game changer in gaining valuable insights into voter intentions and behaviour was well known to both candidates. IT professionals and analysts on both sides had been working for more than 18 months on ramping up their projects, which ironically shared one feature: they both bore the names of large cetaceans, Obama’s Narwhal and Romney’s Orca (the orca is the only predator that goes after narwhals). While interviews with Obama campaign staff and volunteers suggest how much they benefited from the Narwhal data in the months leading to the elections, the Orca team had to admit defeat early on election day when the system went bust and failed to deliver. The rest is…history.
This example illustrates rather dramatically the power of Big Data and how critical it can be to an organisation’s performance. A study of the compound annual growth rates of companies that employ Big Data for fine-grained insights into their customers’ behaviour suggests that they are growing anywhere between twice to fifty times faster than their competitors that don’t follow this practice. What makes Big Data so different and so powerful? First, its scale: not only is the sheer number of clicks, transactions, comments, mobile connections and other data points unprecedented, but the ability to store them is a new development. Timeliness of the data is also key – large ecommerce sites such as Amazon and Expedia are collecting and acting upon real time data around the clock to forecast, update, test, optimise and, in general, extract greater relevance and value every step of the way. Beyond the scale of the data, the diversity of its sources and the ability to “mash it up” is also new – social media is crossed with geolocation, search and other data to produce clearer, richer insights. Thanks to cloud solutions, Big Data is also now easily distributed and shared with partners around the world, again in real time. Finally, beyond the storage capacities, analytical capacities have also grown exponentially, and make it possible to crunch more and more data, faster and faster. Therein however lies one of the main issues. Big Data has, in many ways, become so big as to test the limits of human intelligence. How to get our heads around the magnitude, the richness, the potential, the structure, the implications of Big Data has become the billion dollar question, one that requires creativity and sure-footed leadership to answer.
Some call it “analysis paralysis”, this “caught in the headlights” reaction to too much input – the main hurdle in unleashing the power of Big Data in an organisation is a very human one that revolves around a dislike and distrust of numbers, fear of what is new and unknown, and a certain tendency by data “geeks” to keep things opaque. Getting around this requires several key initiatives: (1) transforming the organisation to adopt a data-ready culture, (2) integrating data into the decision-making process, (3) making the data itself usable and approachable. The first step is one of cultural transformation and requires both education so as to understand the nature and value of the data, and a real shift in mindset to accept its impact on the organisation. The second step is about integrating data analysts into functions such as marketing, sales and customer service in order to create an effective interface between the data itself and the valuable insights it reveals. Step three transforms data into human facts, stories and pictures that can speak to people. Starting with a highly visible, but easily controlled project may help to create proof of concept and generate interest and buy-in once clear results have been shown. We’ve seen such “easy wins” in sales organisations where Big Data allowed insights into micro-markets that were just waiting to be serviced. Once the numbers were in, internal resistance broke down very quickly.
Questions around privacy and ownership and the ethical uses of data must be addressed both internally and with key stakeholders. As usual, transparency must be the name of the game. As a matter of fact, transparency will itself become a competitive advantage as pressure increases on organisations to share data with consumers so they too can use it to their own advantage. Mobile operators for instance can compete on allowing customers to better customise their calling plans and control their usage. Supermarkets can offer financial and nutritional data and analyses. As firms embrace the full potential of Big Data, we may well do away with some of the vestiges of information asymmetry which consumers have endured for years.