Emerging Frontiers: Big Data!

Abstract

President's column “Big data,” once only the concern of database geeks or a marketing technique among retailers, is now part of our mainstream consciousness and vocabulary. Big data is having a profound impact on the upstream E&P industry as well. What is “big data”? According to a McKinsey Global Institute report, “big data refers to datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyze.” I like this definition because it does not require a dataset of any specific size. It allows datasets to increase as technology advances. This definition also varies to the extent that different industries have different standards of database and analytic software tools. To get a feel for just how big data has become, consider recent rates of data growth. In 2012, disk drives worldwide added 7 exabytes (1018) of new data. Global data volumes are projected to increase 40% per year, while global information technology (IT) spend will grow only 5%. Thirty billion pieces of content are shared on Facebook each month. Or consider that Asia now leads the world in the generation and storage of personal location data due simply to the number of mobile phones—an estimated 800 million in China alone. How mainstream has big data become? Consider these examples of wide-spread usage: High-speed supercomputers statistically analyze massive amounts of data, enabling scientific breakthroughs such as the gene sequencing of individual organisms and entire ecosystems. Coaxing the Higgs boson into a nanosecond of existence required the global distribution and analysis of roughly 200 petabytes of data. Or, more familiar to most of us, retail organizations cleverly obtain our behaviors, preferences, and product perceptions by monitoring our Facebook and Twitter information. The combination of massive amounts of data and sophisticated analytics is profoundly affecting a spectrum of industries, and oil and gas is no exception. Big Data in the Oil Field The amount of data that our industry acquires, communicates, stores, and analyzes is exploding exponentially. For example, the growth of land seismic acquisition is one of the most data-intensive phases of our industry. Channel count has roughly doubled every 3.5 years since 1970. Together with advances in the acquisition technology— which now measures vector information rather than scalar—the data acquired and stored in a typical survey has grown from gigabytes to petabytes. Furthermore, the growing popularity of permanently installed geophones for monitoring fluid fronts, passive monitoring of carbon capture sequestration sites, and observing microseismic events during hydraulic fracturing has driven an extraordinary and daunting growth in seismic data of all forms.