Combination Of Preprocessing Techniques Research Articles

Urdu Documents Clustering with Unsupervised and Semi-Supervised Probabilistic Topic Modeling

Document clustering is to group documents according to certain semantic features. Topic model has a richer semantic structure and considerable potential for helping users to know document corpora. Unfortunately, this potential is stymied on text documents which have overlapping nature, due to their purely unsupervised nature. To solve this problem, some semi-supervised models have been proposed for English language. However, no such work is available for poor resource language Urdu. Therefore, document clustering has become a challenging task in Urdu language, which has its own morphology, syntax and semantics. In this study, we proposed a semi-supervised framework for Urdu documents clustering to deal with the Urdu morphology challenges. The proposed model is a combination of pre-processing techniques, seeded-LDA model and Gibbs sampling, we named it seeded-Urdu Latent Dirichlet Allocation (seeded-ULDA). We apply the proposed model and other methods to Urdu news datasets for categorizing. For the datasets, two conditions are considered for document clustering, one is “Dataset without overlapping” in which all classes have distinct nature. The other is “Dataset with overlapping” in which the categories are overlapping and the classes are connected to each other. The aim of this study is threefold: it first shows that unsupervised models (Latent Dirichlet Allocation (LDA), Non-negative matrix factorization (NMF) and K-means) are giving satisfying results on the dataset without overlapping. Second, it shows that these unsupervised models are not performing well on the dataset with overlapping, because, on this dataset, these algorithms find some topics that are neither entirely meaningful nor effective in extrinsic tasks. Third, our proposed semi-supervised model Seeded-ULDA performs well on both datasets because this model is straightforward and effective to instruct topic models to find topics of specific interest. It is shown in this paper that the semi-supervised model, Seeded-ULDA, provides significant results as compared to unsupervised algorithms.

Parallel pre-processing through orthogonalization (PORTO) and its application to near-infrared spectroscopy

Data generated from spectroscopy may be deformed by artefacts due to a range of physical, chemical and environmental factors that are not of interest for the characterization of the samples under study. For example, data acquired by near-infrared (NIR) spectroscopy in the diffuse reflectance mode can be affected by light scattering. This artefact, if not reduced or removed by spectral pre-processing, can complicate the multivariate data analysis. However, different pre-processing approaches correct these effects in different ways. For example, differentiation can reveal underlying bands, while spectral normalization techniques such as standard normal variate (SNV) can correct for multiplicative and additive effects. Combining multiple pre-processing techniques can lead to better results. However, it is not feasible for a user to explore all possible combinations of pre-processing techniques. In the present work, a new pre-processing fusion approach, based on the framework of separating common and distinct components in multi-block multivariate data analysis, is demonstrated. The approach utilizes parallel and orthogonalized partial least squares (PO-PLS) regression for the parallel fusion of multiple pre-processing techniques applied to the same data. The results obtained on 4 different NIR spectroscopic data sets related to the assessment of fruit quality and used as benchmark are compared to those of the recently developed sequential pre-processing through orthogonalization (SPORT) approach: it is found that, in all the cases, the PO-PLS approach leads to slightly better performances. Furthermore, a clear understanding of the common and distinct information present in the data sets after each pre-treatment was obtained. Parallel pre-processing through orthogonalization (PORTO) can be seen as parallel boosting of multiple pre-processing techniques to improve model performances.

Comparison of Individual and Integrated Inline Raman, Near-Infrared, and Mid-Infrared Spectroscopic Models to Predict the Viscosity of Micellar Liquids.

In many industries, viscosity is an important quality parameter which significantly affects consumer satisfaction and process efficiency. In the personal care industry, this applies to products such as shampoo and shower gels whose complex structures are built up of micellar liquids. Measuring viscosity offline is well established using benchtop rheometers and viscometers. The difficulty lies in measuring this property directly in the process via on or inline technologies. Therefore, the aim of this work is to investigate whether proxy measurements using inline vibrational spectroscopy, e.g., near-infrared (NIR), mid-infrared (MIR), and Raman, can be used to predict the viscosity of micellar liquids. As optical techniques, they are nondestructive and easily implementable process analytical tools where each type of spectroscopy detects different molecular functionalities. Inline fiber optic coupled probes were employed; a transmission probe for NIR measurements, an attenuated total reflectance probe for MIR and a backscattering probe for Raman. Models were developed using forward interval partial least squares variable selection and log viscosity was used. For each technique, combinations of pre-processing techniques were trialed including detrending, Whittaker filters, standard normal variate, and multiple scatter correction. The results indicate that all three techniques could be applied individually to predict the viscosity of micellar liquids all showing comparable errors of prediction: NIR: 1.75 Pa s; MIR: 1.73 Pa s; and Raman: 1.57 Pa s. The Raman model showed the highest relative prediction deviation (RPD) value of 5.07, with the NIR and MIR models showing slightly lower values of 4.57 and 4.61, respectively. Data fusion was also explored to determine whether employing information from more than one data set improved the model quality. Trials involved weighting data sets based on their signal-to-noise ratio and weighting based on transmission curves (infrared data sets only). The signal-to-noise weighted NIR–MIR–Raman model showed the best performance compared with both combined and individual models with a root mean square error of cross-validation of 0.75 Pa s and an RPD of 10.62. This comparative study provides a good initial assessment of the three prospective process analytical technologies for the measurement of micellar liquid viscosity but also provides a good basis for general measurements of inline viscosity using commercially available process analytical technology. With these techniques typically being employed for compositional analysis, this work presents their capability in the measurement of viscosity—an important physical parameter, extending the applicability of these spectroscopic techniques.

Model based characterisation of delamination by means of thermographic inspection

The objective of the work presented in this paper is to propose a fast and accurate approach for the characterisation of the delamination in single-layer metallic planar pieces using infrared thermography. This approach is based on a combination of pre-processing techniques and inversion. The inversion has been done in a reduced parameter space since the preprocessing phase gives us information about the location and the shape of the flaws in the transverse plane. The estimation of the flaws’ parameters has been carried out by an iterative data fitting method involving a fast semi-analytical three-dimensional model. The robustness of the proposed approach is numerically assessed in presence of synthetic noisy data sets in different configurations.