Data Pre-processing Techniques Research Articles

Modeling grass yields in Qinghai Province, China, based on MODIS NDVI data—an empirical comparison

Qinghai Province is one of the four largest pastoral regions in China. Timely monitoring of grass growth and accurate estimation of grass yields are essential for its ecological protection and sustainable development. To estimate grass yields in Qinghai, we used the normalized difference vegetation index (NDVI) time-series data derived from the Moderate-resolution Imaging Spectroradiometer (MODIS) and a pre-existing grassland type map. We developed five estimation approaches to quantify the overall accuracy by combining four data pre-processing techniques (original, Savitzky-Golay (SG), Asymmetry Gaussian (AG) and Double Logistic (DL)), three metrics derived from NDVI time series (VImax, VIseason and VImean) and four fitting functions (linear, second-degree polynomial, power function, and exponential function). The five approaches were investigated in terms of overall accuracy based on 556 ground survey samples in 2016. After assessment and evaluation, we applied the best estimation model in each approach to map the fresh grass yields over the entire Qinghai Province in 2016. Results indicated that: 1) For sample estimation, the cross-validated overall accuracies increased with the increasing flexibility in the chosen fitting variables, and the best estimation accuracy was obtained by the so called “fully flexible model” with R2 of 0.57 and RMSE of 1140 kg/ha. 2) Exponential models generally outperformed linear and power models. 3) Although overall similar, strong local discrepancies were identified between the grass yield maps derived from the five approaches. In particular, the two most flexible modeling approaches were too sensitive to errors in the pre-existing grassland type map. This led to locally strong overestimations in the modeled grass yields.

Low-Feedback Sampling Rate Digital Predistortion Using Deep Neural Network for Wideband Wireless Transmitters

In this paper, a low-feedback-sampling-rate digital predistortion (DPD) method is proposed for wideband wireless transmitters and radio-frequency power amplifiers (PAs). This DPD method inserts a non-ideal real band-pass filter into the feedback loop to limit the feedback bandwidth. Meanwhile, to recover the band-limited feedback signal, it introduces a signal-recovery module that is based on the deep neural network (DNN). A data-preprocessing technique is proposed to reduce the amount of input data for the DNN and thereby significantly reducing its structural complexity. Also, the use of DNN makes it feasible to implement off-line training. Both the simplicity of the proposed DNN and the availability of off-line training reduce the system complexity in DPD. Experimental validation was performed on a PA driven by wideband orthogonal-frequency-division-multiplexing (OFDM) signals. The results demonstrated the superiority of the proposed method in under-sampling applications. The proposed DPD method achieved the same linearization performance as the state-of-art DPD methods while requiring a sampling rate that was approximately 5% less. Meanwhile, the proposed DPD method has been validated to have stronger anti-noise and generalization capabilities.

On the combination of kernel principal component analysis and neural networks for process indirect control

ABSTRACT A new adaptive kernel principal component analysis (KPCA) for non-linear discrete system control is proposed. The proposed approach can be treated as a new proposition for data pre-processing techniques. Indeed, the input vector of neural network controller is pre-processed by the KPCA method. Then, the obtained reduced neural network controller is applied in the indirect adaptive control. The influence of the input data pre-processing on the accuracy of neural network controller results is discussed by using numerical examples of the cases of time-varying parameters of single-input single-output non-linear discrete system and multi-input multi-output system. It is concluded that, using the KPCA method, a significant reduction in the control error and the identification error is obtained. The lowest mean squared error and mean absolute error are shown that the KPCA neural network with the sigmoid kernel function is the best.

Improving k-Nearest Neighbors Algorithm for Imbalanced Data Classification

The k-Nearest Neighbors (k-NN) algorithm is a classic non-parametric method that has wide applications in data classification and prediction. Like many other machine learning schemes, the performance of k-NN classifiers will be significantly impacted by the imbalanced class distributions of data. That is, the data instances in the majority class tend to dominate the prediction of the test instances. In this paper, we look into the data pre-processing techniques that can be used to rebalance the training data and enhance the performance of k-NN classifiers in imbalanced data sets. We conduct extensive experiments on 14 real-world data sets collected from different application domains. We also perform statistical tests to verify the significance of different data pre-processing techniques in terms of boosting k-NN classification precision.

A study of data pre-processing techniques for imbalanced biomedical data classification

Biomedical data are widely accepted in developing prediction models for identifying a specific tumor, drug discovery and classification of human cancers. However, previous studies usually focused on different classifiers, and overlook the class imbalance problem in real-world biomedical datasets. There are a lack of studies on evaluation of data pre-processing techniques, such as resampling and feature selection, on imbalanced biomedical data learning. The relationship between data pre-processing techniques and the data distributions has never been analysed in previous studies. This article mainly focuses on reviewing and evaluating some popular and recently developed resampling and feature selection methods for class imbalance learning. We analyse the effectiveness of each technique from data distribution perspective. Extensive experiments have been done based on five classifiers, four performance measures, eight learning techniques across twenty real-world datasets. Experimental results show that: (1) resampling and feature selection techniques exhibit better performance using support vector machine (SVM) classifier. However, resampling and Feature Selection techniques perform poorly when using C4.5 decision tree and Linear discriminant analysis classifiers; (2) for datasets with different distributions, techniques such as Random undersampling and Feature Selection perform better than other data pre-processing methods with T Location-Scale distribution when using SVM and KNN (K-nearest neighbours) classifiers. Random oversampling outperforms other methods on Negative Binomial distribution using Random Forest classifier with lower level of imbalance ratio; (3) Feature Selection outperforms other data pre-processing methods in most cases, thus, Feature Selection with SVM classifier is the best choice for imbalanced biomedical data learning.

AUTO-INSURANCE FRAUD DETECTION: A BEHAVIORAL FEATURE ENGINEERING APPROACH

Every year, billions of dollars are lost in the auto insurance industry due to fraud, which forces insurance premium prices to go up annually. Although fraud detection solutions have been developed to fix the fraud detection problem, they all still face the same well-known problems of imbalanced data. There is need for a centralized claims database to gather a holistic view of fraudulent characteristic behavior. This paper proposes a data pre-processing technique, particularly a fraud behavior feature engineering approach, to improve the overall performance of prediction models. The behavior being assessed is be based on the RFM model along with an additional behavior analysis related to policy expiration. Furthermore, an ensemble feature selection and modeling is used to deal with the high dimensionality problems that the feature engineering approach brings along with it, as well as the class imbalance problems. The proposed approach shows a 56.2% increase in the F1-meaure, compared against the previous published stat-of-the-art results.

A study of data pre-processing techniques for imbalanced biomedical data classification

Biomedical data are widely accepted in developing prediction models for identifying a specific tumour, drug discovery and human cancers detection. However, previous studies usually focused on different classifiers, and overlook the class imbalance problem in real-world biomedical datasets. This paper mainly focuses on reviewing and evaluating some popular and recently developed resampling and feature selection (FS) methods for class imbalance learning with data distribution being considered. Experimental results show that: 1) resampling and FS techniques exhibit better performance using support vector machine (SVM) classifier; 2) techniques such as random undersampling and FS perform better than other data pre-processing methods with T location-scale distribution when using SVM and K-nearest neighbours (KNN) classifiers. Random oversampling outperforms other methods on negative binomial distribution using Random Forest with lower level of imbalance ratio; 3) FS outperforms other data pre-processing methods in most cases, thus, FS with SVM classifier is the best choice for imbalanced biomedical data learning.

Semi supervised data mining model for the prognosis of pre-diabetic conditions in type 2 Diabetes Mellitus.

Diabetic Mellitus is the leading disease in the world irrespective of age and geographical location. It is estimated that 43% of the overall population is affected by the disease. The reasons for the disease include inappropriate diet lifestyle with allied symptoms like obesity. Therefore, the prognosis and diagnosis of the disease are important for adequate combat and care. The prognosis related known symptoms of the disease include incontinence (inability to control urination) and frequent fatigue. Moreover, early prediction of the disease plays an important role in the prognosis of other associated conditions such as heart failure leading to chronic illness. Hence, it is of interest to describe a data mining based prediction model using known features (derived from epidemiological data collected from the public hospital using routine tests) to help in the prognosis of the disease. We used data pre-processing techniques for handling missing values and dimensionality reduction models to improve data quality. The Minimum Description Length principle (MDL) model for discretization (replacing a continuum with a finite set of points) is used to reduce high-level dimensionality of the dataset, which enabled to categorize the dataset into small groups in ordered intervals. Thus, we describe a semi-supervised learning technique (identifies promising attributes using clustering and classification methods) by combining data mining techniques for reasonable accuracy having adequate sensitivity and specificity for further discussion, cross-validation, revaluation, and application. Early prediction of the disease with improved accuracy by analysing specificity ranges in blood pressure and glucose levels will be useful to combat Diabetes Mellitus.

Modelling European small pelagic fish distribution: Methodological insights

The distribution of marine organisms is strongly influenced by climatic gradients worldwide. The ecological niche (sensu Hutchinson) of a species, i.e. the combination of environmental tolerances and resources required by an organism, interacts with the environment to determine its geographical range. This duality between niche and distribution allows climate change biologists to model potential species’ distributions from past to future conditions. While species distribution models (SDMs) have been intensively used over the last years, no consensual framework to parametrise, calibrate and evaluate models has emerged. Here, to model the contemporary (1990–2017) spatial distribution of seven highly harvested European small pelagic fish species, we implemented a comprehensive and replicable numerical procedure based on 8 SDMs (7 from the Biomod2 framework plus the NPPEN model). This procedure considers critical issues in species distribution modelling such as sampling bias, pseudo-absence selection, model evaluation and uncertainty quantification respectively through (i) an environmental filtration of observation data, (ii) a convex hull based pseudo-absence selection, (iii) a multi-criteria evaluation of model outputs and (iv) an ensemble modelling approach. By mitigating environmental sampling bias in observation data and by identifying the most ecologically relevant predictors, our framework helps to improve the modelling of fish species’ environmental suitability. Not only average temperature, but also temperature variability appears as major factors driving small pelagic fish distribution, and areas of highest environmental suitability were found along the north-western Mediterranean coasts, the Bay of Biscay and the North Sea. We demonstrate in this study that the use of appropriate data pre-processing techniques, an often-overlooked step in modelling, increase model predictive performance, strengthening our confidence in the reliability of predictions.

Improving the prediction accuracy of river inflow using two data pre-processing techniques coupled with data-driven model.

River inflow prediction plays an important role in water resources management and power-generating systems. But the noises and multi-scale nature of river inflow data adds an extra layer of complexity towards accurate predictive model. To overcome this issue, we proposed a hybrid model, Variational Mode Decomposition (VMD), based on a singular spectrum analysis (SSA) denoising technique. First, SSA his applied to denoise the river inflow data. Second, VMD, a signal processing technique, is employed to decompose the denoised river inflow data into multiple intrinsic mode functions (IMFs), each with a relative frequency scale. Third, Empirical Bayes Threshold (EBT) is applied on non-linear IMF to smooth out. Fourth, predicted models of denoised and decomposed IMFs are established by learning the feature values of the Support Vector Machine (SVM). Finally, the ensemble predicted results are formulated by adding the predicted IMFs. The proposed model is demonstrated using daily river inflow data from four river stations of the Indus River Basin (IRB) system, which is the largest water system in Pakistan. To fully illustrate the superiority of our proposed approach, the SSA-VMD-EBT-SVM hybrid model was compared with SSA-VMD-SVM, VMD-SVM, Empirical Mode Decomposition (EMD) based i.e., EMD-SVM, SSA-EMD-SVM, Ensemble EMD (EEMD) based i.e., EEMD-SVM and SSA-EEMD-SVM. We found that our proposed hybrid SSA-EBT-VMD-SVM model outperformed than others based on following performance measures: the Nash-Sutcliffe Efficiency (NSE), Mean Absolute Percentage Error (MAPE) and Root Mean Square Error (RMSE). Therefore, SSA-VMD-EBT-SVM model can be used for water resources management and power-generating systems using non-linear time series data.

Convolution Neural Networks for Binary Classification

Convolutional neural networks (CNNs) are similar to “ordinary” neural networks in the sense that they are made up of hidden layers consisting of neurons with “learnable” parameters. These neurons receive inputs, perform a dot product, and then follows it with a non-linearity. The whole network expresses the mapping between raw image pixels and their class scores. Conventionally, the Softmax function is the classifier used at the last layer of this network. However, there have been studies conducted to challenge this norm. Empirical data has shown that the CNN model was able to achieve a test accuracy of ≈99.04% using the MNIST dataset. MNIST dataset consists of 60,000 training images and 10,000 testing images. This experiment was inspired by following the experiment on MNIST dataset. The dataset we used in this experiment is collection of images consisting of cats and dogs. These images are gathered from different sources over internet. This dataset consists of 10,000 images of each class i.e., Cats and Dogs. The overall accuracy of training and validation set is 96.85%. The said results may be improved if data pre-processing techniques were employed on the datasets, and if the base CNN model was alternatively more sophisticated than the one used in this study.

A Highly Effective Data Preprocessing in Side-Channel Attack Using Empirical Mode Decomposition

Side-channel attacks on cryptographic chips in embedded systems have been attracting considerable interest from the field of information security in recent years. Many research studies have contributed to improve the side-channel attack efficiency, in which most of the works assume the noise of the encryption signal has a linear stable Gaussian distribution. However, their performances of noise reduction were moderate. Thus, in this paper, we describe a highly effective data-preprocessing technique for noise reduction based on empirical mode decomposition (EMD) and demonstrate its application for a side-channel attack. EMD is a time-frequency analysis method for nonlinear unstable signal processing, which requires no prior knowledge about the cryptographic chip. During the procedure of data preprocessing, the collected traces will be self-adaptably decomposed into sum of several intrinsic mode functions (IMF) based on their own characteristics. And then, meaningful IMF will be reorganized to reduce its noise and increase the efficiency of key recovering through correlation power analysis attack. This technique decreases the total number of traces for key recovering by 17.7%, compared to traditional attack methods, which is verified by attack efficiency analysis of the SM4 block cipher algorithm on the FPGA power consumption analysis platform.

Spatiotemporal clustering: a review

An increase in the size of data repositories of spatiotemporal data has opened up new challenges in the fields of spatiotemporal data analysis and data mining. Foremost among them is “spatiotemporal clustering,” a subfield of data mining that is increasingly becoming popular because of its applications in wide-ranging areas such as engineering, surveillance, transportation, environmental and seismology studies, and mobile data analysis. This review paper presents a comprehensive review of spatiotemporal clustering approaches and their applications as well as a brief tutorial on the taxonomy of data types in the spatiotemporal domain and patterns. Additionally, the data pre-processing techniques, access methods, cluster validation, space–time scan statistics, software tools, and datasets used by various spatiotemporal clustering algorithms are highlighted.

Data Mining Effective for Casing-Failure Prediction and Prevention

This article, written by JPT Technology Editor Chris Carpenter, contains highlights of paper IPTC 19311, “Data-Mining Approaches for Casing-Failure Prediction and Prevention,” by Christine Noshi, SPE, Samuel Noynaert, SPE, and Jerome Schubert, SPE, Texas A&M University, prepared for the 2019 International Petroleum Technology Conference, Beijing, 26–28 March. The paper has not been peer reviewed. Copyright 2019 International Petroleum Technology Conference. Reproduced by permission. Recent casing failures in the Granite Wash play in the western Anadarko Basin have sparked deep concerns for operators in North Texas and Oklahoma. Hydrostatic tests made in the field show that current API standards do not assure adequate joint and bursting strength to meet deep-well requirements. This paper is part of an ongoing effort to minimize the likelihood of failure using data-mining and machine-learning algorithms. Introduction Casing failure has long presented a challenge to the industry. The combined effects of design, dynamic borehole conditions, metallurgy, and handling have been challenging to quantify and predict accurately. Additionally, most casing-string challenges have been handled reactively instead of proactively; the total number of failures have been underreported and overlooked. The authors focus on the effects of poor cement as a primary factor; this translates into the absence of cement in a case study presented in the complete paper. Additional factors are the pumping of corrosive acids and poor standardized casing design that does not account for varied formations along with cyclical temperatures. Casing with partial cementation and sheaths with voids can contribute to excessive buckling-related collapse and tensile failures. Large pressure loadings, along with significant change in temperature, contribute to significant stresses in the intercasing annuli. In fragmentally cemented casing, tensile loading can show a great discrepancy between compression and high tension, with instances of failures in both the outer and inner strings. Additionally, cement thickening by downward flow could lack uniformity and could be prone to channeling. Air entrapment might occur, establishing bridges that hinder the process. Some authors in the literature related cementing failures with hole enlargements and washouts in long cement depths. The lack of cement support in those significant intervals exposed the casing to movement during drillpipe rotation, which triggered wear and ultimate buckling. Methodology The data were descriptively visualized using methods such as box plots, mosaic plots, and trellis charts, while predictive techniques included artificial neural networks (ANNs) and boosted-ensemble trees. A statistical software package was used along with Python coding to implement the models and choose the most-significant factors contributing to failure. Data-preprocessing techniques were implemented. The process began with data cleaning to account for missing data, re-move the bias incurred by noise, and remove outliers. For missing values, multivariate normal imputation on the basis of all samples belonging to the same class was used. Then, several parameters from different databases were integrated. Data transformation involved standardizing the data by the subtraction of the mean value and the subsequent division by standard deviation from each feature. Categorical variables were converted to numerical values because models such as neural nets, regression, and nearest-neighbor involve only numeric inputs. The compiled data set comprised 78 wells. Caution should be taken when assessing its statistical significance.

Data Analytics Application for Non-Conformance Reports in a Cabinet Manufacturing Facility

Industrialization of building construction through offsite construction and modularization is an effective approach for improving performance of construction projects. In a modular construction approach, building components are produced in a well-controlled factory environment. The components are then delivered to site, in sequence, for installation by site crews. This process reduces construction waste, improves product quality, and minimizes onsite safety incidents. As the market conditions are rapidly changing, the demand for more customized and unique products is increasing. Customers increasingly demand customized dwellings to reflect their cultural tastes and personal preferences. Cabinets in the house, kitchen or otherwise, are building components that constitute a large portion of the visible customization that customers are interested in. This paper focuses on the analysis of records in Non-Conformance Reports (NCRs) at a cabinet manufacturing facility in Alberta, Canada. An NCR record represents a defect in any product that needs a repair or rework; it captures several attributes of the defective part, such as the job number, wood species, stain, the date and time when the record is created, etc. The systematic approach presented in this study employs data analytics to the collection, cleaning, and analysis of the NCR dataset. The dataset is first analyzed as per existing operations. Various data pre-processing techniques, including attribute and instance selection and transformation, are then applied to clean the dataset. The results show that most of the “Rework” results from administrative or product handling errors, while the majority of “Repairs” result from product finishing errors. The impact of repairing the defective parts is discussed, and recommendations to reduce the number of NCRs and thereby enhance the performance of operations are presented.

Data Preprocessing: A preliminary step for web data mining

In recent years immense growth of data i.e. big data is observed resulting in a brighter and more optimized future. Big Data demands large computational infrastructure with high–performance processing capabilities. Preparing big data for mining and analysis is a challenging task and requires data to be preprocessed to improve the quality of raw data. The data instance representation and quality are foremost. Data preprocessing is preliminary data mining practice in which raw data is transformed into a format suitable for another processing procedure. Data preprocessing improves the data quality by cleaning, normalizing, transforming and extracting relevant feature from raw data. Data preprocessing significantly improve the performance of machine learning algorithms which in turn leads to accurate data mining. Knowledge discovery from noisy, irrelevant and redundant data is a difficult task therefore precise identification of extreme values and outlier, filling up missing values poses challenges. This paper discusses various big data pre–processing techniques in order to prepare it for mining and analysis tasks.

Group Method of Data Handling (GMDH) Lithology Identification Based on Wavelet Analysis and Dimensionality Reduction as Well Log Data Pre-Processing Techniques

Although the group method of data handling (GMDH) is a self-organizing metaheuristic neural network capable of developing a classification function using influential input variables, the results can be improved by using some pre-processing steps. In this paper, we propose a joint principal component analysis (PCA) and GMDH (PCA-GMDH) classifier method. We investigated well log data pre-processing techniques composed of dimensionality reduction (DR) and wavelet analysis (WA), using the southern basin of the South Yellow Sea as a case study, with the aim of improving the lithology classification accuracy of the GMDH. Our results showed that the dimensionality reduction method, which is composed of PCA and linear discriminant analysis (LDA), minimized the complexity of the classifier by reducing the number of well log suites to the relevant components and factors. On the other hand, the WA decomposed the well log signals into time-frequency wavelets for the GMDH algorithm. Of all the pre-processing methods, only the PCA was able to significantly increase the classification accuracy rate of the GMDH. Finally, the proposed joint PCA-GMDH classifier not only increased the accuracy but also was able to distinguish between all the classes of lithofacies present in the southern basin of the South Yellow Sea.

Daily river flow forecasting using ensemble empirical mode decomposition based heuristic regression models: Application on the perennial rivers in Iran and South Korea

Developing hydrologic models based on data-driven approaches (DDA) is very complicated due to the complex nature of meteorological data. For example, a high degree of irregularities, periodicities, jumps, and other forms of stochastic behavior influence the accuracy of river flow forecasting. In this study, M5 model tree (M5Tree) and multivariate adaptive regression spline (MARS) models were developed to forecast one and multi-day-ahead river flow. Moreover, ensemble empirical mode decomposition (EEMD), a robust data pre-processing technique, was used to enhance M5Tree and MARS models’ forecasting. Also, Mallows’ coefficient (CP), one of the procedures to determine the input variables, was used to obtain the optimum values of hydrological time series. The developed models were validated using two different meteorological stations (e.g., Kordkheyl in Iran and Hongcheon in South Korea).Forecasting performance of developed models (e.g., M5Tree, MARS, EEMD-M5Tree, and EEMD-MARS) was evaluated using six different statistical criteria. Comparing the results between standalone and hybrid models indicated that a data pre-processing technique can enhance the performance of standalone models (e.g., M5Tree and MARS). EEMD-MARS model (NSE = 0.819 and RMSE = 7.206 m3/s (Kordkheyl station) and NSE = 0.738 and RMSE = 50.426 m3/s (Hongcheon station)) outperformed M5Tree, MARS, and EEMD-M5Tree models based on two-day-ahead river flow forecasting in validation stage, respectively. Results showed that EEMD-MARS model was an efficient and robust tool to forecast one and multi-day-ahead (e.g., two, three, and four-day-ahead) river flow.

A hybrid model based on modified multi-objective cuckoo search algorithm for short-term load forecasting

To ensure the safe operation of electrical power systems, short-term load forecasting (STLF) plays a significant role. With the development of artificial neural network (ANN), many forecasting models based on ANN are proposed to enhance the forecasting accuracy. However, forecasting stability is also an important aspect when considering a forecasting model. Both forecasting accuracy and stability are affected heavily by the random initial values of weights and thresholds of ANN. Thus, in this paper, a new hybrid model based on the modified generalized regression neural network (GRNN) is proposed for short-term load forecasting (STLF). Meanwhile, a non-dominated sorting-based multi-objective cuckoo search algorithm (NSMOCS) is proposed to realize accurate and stable forecasting simultaneously. To utilize the similarities and reduce interference existing in the original data, some data pre-processing techniques are also incorporated. With half-hourly load data from five states in Australia, experimental results clearly show that the proposed hybrid model could obtain more accurate and stable forecasting results, compared with the comparison models.

Multi-Fault Rapid Diagnosis for Wind Turbine Gearbox Using Sparse Bayesian Extreme Learning Machine

In order to reduce operation and maintenance costs, reliability, and quick response capability of multi-fault intelligent diagnosis for the wind turbine system are becoming more important. This paper proposes a rapid data-driven fault diagnostic method, which integrates data pre-processing and machine learning techniques. In terms of data pre-processing, fault features are extracted by using the proposed modified Hilbert–Huang transforms (HHT) and correlation techniques. Then, time domain analysis is conducted to make the feature more concise. A dimension vector will then be constructed by including the intrinsic mode function energy, time domain statistical features, and the maximum value of the HHT marginal spectrum. On the other hand, as the architecture and the learning algorithm of pairwise-coupled sparse Bayesian extreme learning machine (PC-SBELM) are more concise and effective, it could identify the single- and simultaneous-fault more quickly and precisely when compared with traditional identification techniques such as pairwise-coupled probabilistic neural networks (PC-PNN) and pairwise-coupled relevance vector machine (PC-RVM). In this case study, PC-SBELM is applied to build a real-time multi-fault diagnostic system. To verify the effectiveness of the proposed fault diagnostic framework, it is carried out on a real wind turbine gearbox system. The evaluation results show that the proposed framework can detect multi-fault in wind turbine gearbox much faster and more accurately than traditional identification techniques.