On Data Analysis and Design and Implementation of Data Preprocessing Scheme Based on Low-quality Rock Datasets

Abstract

With fast progress of deep learning technology, breakthroughs are achieved in many industries by virtue of efficient artificial intelligence models. In addition, computer hardware is cheaper, which makes it easier to acquire an excellent deep learning model. However, output of a model with good generalization rests with not only powerful hardware computing speed, but also quality of dataset involved in the calculation. Unfortunately, high-quality dataset is probably more expensive than high-end hardware, and this forces deep learning engineers or practitioners to use lower-quality dataset. Anyway, it doesn’t mean excellent deep learning programs can’t be created by such dataset. In particular, dataset preprocessing is equally important, and even engineers need to spend most of time elaborately formulating preprocessing strategies. This study mainly analyzes data and formulates preprocessing schemes of low-quality rock datasets. It aims to make deep learning programs more efficient and general-purpose at the lowest possible cost.