Data Pre-processing Techniques Research Articles

Non-specific conducting polymer-based array capable of monitoring odour emissions from a biofiltration system in a piggery building

A commercial non-specific gas sensor array system was evaluated in terms of its capability to monitor the odour abatement performance of a biofiltration system developed for treating emissions from a commercial piggery building. The biofiltration system was a modular system comprising an inlet ducting system, humidifier and closed-bed biofilter. It also included a gravimetric moisture monitoring and water application system for precise control of moisture content of an organic woodchip medium. Principal component analysis (PCA) of the sensor array measurements indicated that the biofilter outlet air was significantly different to both inlet air of the system and post-humidifier air. Data pre-processing techniques including normalising and outlier handling were applied to improve the odour discrimination performance of the non-specific gas sensor array. To develop an odour quantification model using the sensor array responses of the non-specific sensor array, PCA regression, artificial neural network (ANN) and partial least squares (PLS) modelling techniques were applied. The correlation coefficient (r(2)) values of the PCA, ANN, and PLS models were 0.44, 0.62 and 0.79, respectively.

Theoretical and experimental approaches to the grounds of oscillator model of the brain activating system

The assessment of dependence in older adults currently requires a manual collection of data taken from questionnaires. This process is time consuming for the clinicians and intrudes the daily life of the elderly. This paper aims to semi-automate the acquisition and analysis of health data to assess and predict the dependence in older adults while executing one instrumental activity of daily living (IADL).In a mobile-health (m-health) scenario, we analyze whether the acquisition of data through wearables during the performance of IADLs, and with the help of machine learning techniques could replace the traditional questionnaires to evaluate dependence. To that end, we collected data from wearables, while older adults do the shopping activity. A trial supervisor (TS) labelled the different shopping stages (SS) in the collected data. We performed data pre-processing techniques over those SS and analyzed them with three machine learning algorithms: k-Nearest Neighbors (k-NN), Random Forest (RF) and Support Vector Machines (SVM).Our results confirm that it is possible to replace the traditional questionnaires with wearable data. In particular, the best learning algorithm we tried reported an accuracy of 97% in the assessment of dependence. We tuned the hyperparameters of this algorithm and used embedded feature selection technique to get the best performance with a subset of only 10 features out of the initial 85. This model considers only features extracted from four sensors of a single wearable: accelerometer, heart rate, electrodermal activity and temperature. Although these features are not observational, our current proposal is semi-automatic, because it needs a TS labelling the SS (with a smartphone application). In the future, this labelling process could be automatic as well.Our method can semi-automatically assess the dependence, without disturbing daily activities of elderly people. This method can save clinicians’ time in the evaluation of dependence in older adults and reduce healthcare costs.

Data mining: A tool for detecting cyclical disturbances in supply networks

Disturbances in supply chains may be either exogenous or endogenous. The ability automatically to detect, diagnose, and distinguish between the causes of disturbances is of prime importance to decision makers in order to avoid uncertainty. The spectral principal component analysis (SPCA) technique has been utilized to distinguish between real and rogue disturbances in a steel supply network. The data set used was collected from four different business units in the network and consists of 43 variables; each is described by 72 data points. The present paper will utilize the same data set to test an alternative approach to SPCA in detecting the disturbances. The new approach employs statistical data pre-processing, clustering, and classification learning techniques to analyse the supply network data. In particular, the incremental k-means clustering and the RULES-6 classification rule-learning algorithms, developed by the present authors' team, have been applied to identify important patterns in the data set. Results show that the proposed approach has the capability automatically to detect and characterize network-wide cyclical disturbances and generate hypotheses about their root cause.

Noninvasive Method for Internal Quality Evaluation of Pear Fruit Using Fiber-Optic FT-NIR Spectrometry

An experimental study on noninvasive methods for simultaneous prediction of pears' internal quality was investigated. This method is based on Fourier transform near infrared (FT-NIR) spectrometry with fiber optics in the region between 800 and 2500 nm. A total of 248 pear samples were used to develop the calibration models. The quality indices included soluble solids content (SSC) and titratable acidity (TA). Partial least squares (PLS) regression and principle component regression (PCR) were carried out describing the relationships between the data sets of laboratory data and the FT-NIR spectra. Calibration models based on the different spectral ranges and with several data pre-processing techniques (smoothing, multiplicative signal correction, standard normal variate, etc) were also compared. Performance of different calibration models was assessed in terms of root mean square errors of prediction (RMSEP) and correlation coefficients (r) of validation set of samples. The best predictive models had a RMSEP of 0.320, 0.019, and correlation coefficient (r) equal to 0.93, 0.89 for SSC and TA, respectively. It is concluded that FT- NIR spectrometry could be an easy to facilitate, reliable, accurate and fast method for noninvasive prediction of pears' internal quality.

Real-life applications of the MULVADO software package for processing DOSY NMR data

MULVADO is a newly developed software package for DOSY NMR data processing, based on multivariate curve resolution (MCR), one of the principal multivariate methods for processing DOSY data. This paper will evaluate this software package by using real-life data of materials used in the printing industry: two data sets from the same ink sample but of different quality. Also a sample of an organic photoconductor and a toner sample are analysed. Compared with the routine DOSY output from monoexponential fitting, one of the single channel algorithms in the commercial Bruker software, MULVADO provides several advantages. The key advantage of MCR is that it overcomes the fluctuation problem (non-consistent diffusion coefficient of the same component). The combination of non-linear regression (NLR) and MCR can yield more accurate resolution of a complex mixture. In addition, the data pre-processing techniques in MULVADO minimise the negative effects of experimental artefacts on the results of the data. In this paper, the challenges for analysing polymer samples and other more complex samples will also be discussed.

Forensic Classification of Paper with Infrared Spectroscopy and Principal Components Analysis

Fourier transform infrared (FT-IR) spectra of six document papers were classified by the chemometric technique of soft independent modelling of class analogy (SIMCA) using principal component analysis (PCA). The data were split into two regions, mid infrared (MIR, 2500–4000 cm−1) and the near infrared region (NIR, 4000–9000 cm−1). The raw data failed to produce an appreciable separation; hence it was pre-processed using derivatives and the multiple wavenumber ratios technique. The aim of this research was to determine the spectral region with the best discriminating power and evaluate the impact of data pre-processing techniques on the classification of paper. It was found that MIR (both raw and ratios) had higher discriminating ability than the NIR. The first derivatives and two ratios produced the best classifications. These results indicate that IR spectroscopy coupled with chemometric analyses can be effectively employed for the forensic classification of paper.

Automatic feature selection for unsupervised clustering of cycle-based signals in manufacturing processes

Recent developments in sensing and computer technology have resulted in most manufacturing processes becoming a data-rich environment. A cycle-based signal refers to an analog or digital signal that is obtained during each repetition of an operation cycle in a manufacturing process. It is a very important class of in-process sensing signals for manufacturing processes because it contains extensive information on the process condition and product quality (e.g., the forming force signal in forging processes). In contrast with currently available supervised classification approaches that heavily depend on the training dataset or engineering field knowledge, this paper aims to develop an automatic feature selection method for the unsupervised clustering of cycle-base signals. First, principal component analysis is applied to the raw signals. Then a new method is proposed to select information containing principal components to allow clustering to be performed. The dimension of the problem can be significantly reduced through the use of these two steps. Finally, a model-based clustering method is applied to the selected principal components to find the clusters in the cycle-based signals. A numerical example and a real-world example of a forging process are used to illustrate the effectiveness of the proposed method. The proposed technique is an important data pre-processing technique for the monitoring and diagnostic system development using cycle-based signals for manufacturing processes.

VIBRATION MONITORING FOR FAULT DIAGNOSIS OF HELICOPTER PLANETRY GEARS

In this paper, vibration data analysis techniques are investigated for fault diagnosis of helicopter planetary gears. A data pre-processing technique is introduced that achieves the same result as the commonly used Time Synchronous Averaging with much lower computational complexity since interpolation is not required. A notion of using raw vibration data instead of the Time Synchronous Averaged data is also presented that is more suitable for the analysis of vibration data produced by planetary gearboxes and for the purposes of detecting carrier plate crack fault. Based on this notion, features such as the Harmonic Index in the frequency domain and the Intra-Revolution Energy Variance in the wavelet domain are derived. The features are used as inputs to fault classifiers and are shown to detect the fault successfully based on the test data that is available.

Application of principal component analysis and artificial neural networks to recognize the individual VOCs of methanol/2-propanol in a binary mixture by SAW multi-sensor array

This work reports on the performance of a volatile organic compounds (VOCs) identification system based on a surface acoustic wave (SAW) multi-sensor array with four acoustic sensing elements, developed in dual configuration as multiplexed two-port resonator 433.92 MHz oscillators and a reference SAW element, in order to recognize the different individual components in a binary mixture of VOCs such as methanol (CH 3OH) and 2-propanol (C 3H 7OH), in the range 20–140 and 5–70 ppm, respectively. The SAW sensors, operating at room temperature, have been specifically coated by sensing thin films belonging to various chemical classes such as arachidic acid (fatty acids), carbowax (stationary phases), triethanolamine (amines), acrylated polysiloxane (polysiloxanes) to ensure cross-sensitivity towards VOCs under test. By using the relative frequency change as the output signal of the SAW multi-sensor array with an artificial neural network (ANN), a recognition system has been realized for the identification of tested VOCs. The features extraction from output signals of the SAW multi-sensor array, exposed to the binary component mixture of methanol and 2-propanol, has been also performed by pattern recognition techniques such as principal component analysis (PCA). The feedforward multi-layer neural network with a hidden layer and trained by a back-propagation learning algorithm has been implemented in order to classify and identify the tested VOCs patterns. Both the normalized responses of four SAW sensors array and the selected principal components (PCs) scores have been used as inputs to the multi-layer perceptron ANN by resulting in a 100% success recognition rate with the four SAW sensors normalized responses and with the first two principal components scores of the original data of the primary matrix. The different strategies used to recognize the VOCs patterns by the ANNs such as the ‘Leave-one-out’ method and ‘Training-and-Test’ method are discussed. Our experimental results have evidenced that the proposed binary vapor mixture classifier based on the electronic nose system, developed by inexpensive and poorly selective chemical SAW sensors, is highly effective in the identification of tested VOCs of methanol and 2-propanol. Moreover, the combination of PCA, as data pre-processing technique, and ANN, as patterns classification technique, provides a rapid and accurate recognition method of the individual components in the tested binary mixture of methanol and 2-propanol.

Interactive simulation of teeth cleaning

In this paper, an interactive simulation system for teeth cleaning is presented. This simulation system offers assistance for optimizing design and manufacturing of new toothbrushes. Data acquisition and pre-processing techniques for the model generation are shown, an editor for modelling of the elastic behaviour of the toothbrush parts is presented and the mathematical method for modelling of flexible parts of the toothbrush is explained. After the introduction of the PHANToM haptoid and the explanation of a new approach to collision detection, first results of the project are shown.

A method for evaluating data-preprocessing techniques for odour classification with an array of gas sensors

The performance of a pattern recognition system is dependent on, among other things, an appropriate data-preprocessing technique, In this paper, we describe a method to evaluate the performance of a variety of these techniques for the problem of odour classification using an array of gas sensors, also referred to as an electronic nose. Four experimental odour databases with different complexities are used to score the data-preprocessing techniques. The performance measure used is the cross-validation estimate of the classification rate of a K nearest neighbor voting rule operating on Fisher's linear discriminant projection subspace.

Near Infrared Spectroscopy of Fuel Contaminated Sand and Soil. I. Preliminary Results and Calibration Study

Regional areas may be contaminated by the past activities of the chemical and oil industries and of the military. Therefore, the present study was undertaken to test the possibilities of near infrared (NIR) reflectance spectroscopy for direct detection and determination of oil and fuel contaminations. If reliable results are obtained NIR reflectance spectroscopy could be a valuable part of land remediation processes. Preliminary investigations showed that it is possible to distinguish samples of stone chippings, sand, cultivated soil, humus and potting soil by multivariate data analysis. After spiking with gasoline, diesel, motor oil and synthetic hydrocarbon mixtures (BTEX) sand rather than cultivated soil shows obvious spectral absorptions due to contaminations higher than 1% (w/w). The influence of particle size fractions has been investigated in detail using dry sand sieved to < 500 μm (fine), 500–800 μm (medium) and >800 μm (coarse). Contaminations in fine and medium fractions often can be modelled with only one intensity at sufficiently low calibration error, SEC. With coarse fractions SEC is three times higher. Models based on derivative spectra have no significant advantage. In general, mean centring results in more pronounced error minima than multiplicative scatter correction (MSC). Partial least squares (PLS) models can be fitted to obtain any wanted SEC even by cross-validation. For comparable SEC, PLS models in general do not need more factors if samples become more inhomogeneous. Data pre-processing techniques such as Kubelka–Munk transformation, Saunderson correction, MSC and combinations thereof have been tested. Adequate sample variation of the diffuse reflectance fraction of detected light according to the Saunderson model could improve the performance of calibration models. The best values for standard error of prediction, SEP, are obtained if calibration models are derived from sets of spectra of sieved samples and used for contamination prediction of natural samples, and not vice versa. Spectra of contaminated soil and humus need cleverer spectral selection and pre-processing for better performance of calibration models.

Compression of nth‐order data arrays by B‐splines. Part 2: Application to second‐order FT‐IR spectra

AbstractIn order to improve the storage and CPU time in the numerical analysis of large two‐dimensional (hyphenated, second‐order) infrared spectra, a data‐preprocessing technique (compression) is presented which is based on B‐splines. B‐splines have been chosen as the compression method since they are wellsuited to model smooth curves. There are two primary goals of compression: a reduction of file size and a reduction of computation when analyzing the compressed representation. The compressed representation of the spectra is used as a substitute for the original representation. For the particular example used here, approximately 0.16 bit per data element was required for the compressed representation in contrast with 16 bits per data element in the uncompressed representation. The compressed representation was further analysed using principal component analysis and compared with a similar analysis on the original data set. The results shows that the principal compotent model of the compressed representation is directly comparable with the principal component model of the original data.