Survey of Big Data New SQL Database Systems for Real-Time Data Transfer in the Upstream Oil and Gas Industry

Abstract

AbstractIn less than a decade, there has been a tremendous evolution in the various storage solutions used to contain wellsite data in the upstream oil and gas industry. Many organizations from wellsite data management providers to the oil and gas operators themselves are moving away from traditional relational database management systems (RDBMS) and towards big data solutions.There are several reasons for this evolution. First, data flow has increased at an unprecedented rate as the number of rigs increase and so has the data frequency rate. Second, the types of data from the sensors and data acquisition systems (DAQ) have also grown. Third, for data analytics that requires high-speed processing, it needs to be received in a timely manner so insights can be drawn for better, faster decision- making.Big Data implementation known as NoSQL databases were proposed to handle conditions where: the data volumes are increasing exponentially, various types of data that are being transmitted or connected from various sources which are substantial for data analytics, and the last is the high-velocity streaming data that requires high-speed processing.Moving in this direction is not without sacrifice. For example, companies adopting NoSQL give up several important factors, for instance, how the database handles transactions from multiple applications. Implementing NoSQL means giving up the benefits of Atomic, Consistent, Isolated and Durable (ACID). NewSQL comes as a promising new solution which offers the same scalable performance as NoSQL for the Online Transaction Processing (OLTP) read-write workloads while still maintaining ACID guarantees for transactions.This paper will investigate, characterize and analyze the list of NewSQL databases in depth and develop comprehensive taxonomy for various critical aspects which come with them. We then will map the requirement of the data storage for handling upstream data to the proposed taxonomy so it can be used as a basis for analyzing each of the databases. At the end, we will predicate and highlight sorted databases which can be recommended for the implementation of next generation wellsite data storage in the upstream oil and gas industry.