Service-Oriented Data Processing for Dynamic Schema

Abstract

In recent years, with the rapid development of big data technology, more and more Internet enterprises have started to transform and upgrade into big data driven enterprises. In addition, in the industrial field, information services driven by industrial big data have also attracted widespread attention. Some traditional industries, such as state-owned chemical enterprises are also actively transforming to information management and big data management. These traditional chemical enterprises have been using simple data processing tools internally in the past, such as Microsoft Excel. Although these tools are relatively simple to use and do not involve much learning cost, they cannot support situations where the volume of data increases and data types become more complex. These data processing methods suffer from insufficient capacity, performance degradation, management confusion and other problems, so that they are no longer suitable for modern data management work. For these traditional chemical companies, on the one hand they have many types of specialist data to store, such as simulation data, measurement data, formulation data, component data, process data, etc. On the other hand, they are unable to determine the full database table structure from the outset, and need to change it dynamically during use. This paper designs a new service-oriented data processing for dynamic schema to meet these needs, and applies it to the development of a data center web application platform for a state-owned chemical company.