Preprocessing Strategies Research Articles

Quantification of alloying elements in steel targets: The LIBS 2022 regression contest

We present the results of the regression contest organized for the LIBS 2022 conference. While the motivation and design of the contest are briefly presented, the work focuses on the methodologies of the three best-performing teams. The employed spectral preprocessing strategies, choice of regression models and its optimization are detailed for each team separately. The aim of the contest reflects the long-term challenges faced by quantitative laser-induced breakdown spectroscopy (LIBS) analysis. Thus, the contest was designed with the purpose of providing a transparent platform for comparing and evaluating the large range of data processing tools available in the LIBS literature. Namely, the contest consisted of the quantification of two major (Cr, Ni) and two minor (Mn, Mo) elements in 15 steel targets. For constructing an appropriate regression model, spectra of 42 targets were provided. The spectra were collected using a commercially available laboratory-based LIBS system and made publicly available. The contest lasted 53 days during which the teams did not receive feedback. In total, 21 teams participated out of which the three best-performing methodologies are presented here. A single linear partial least squares model and two artificial neural network regression models are presented. The corresponding feature selection strategies included emission line selection, spectral range selection, and automatized wavelength selection. Various spectral normalization strategies and data augmentation strategies are also presented.

Optimizing Quantum Classification Algorithms on Classical Benchmark Datasets.

The discovery of quantum algorithms offering provable advantages over the best known classical alternatives, together with the parallel ongoing revolution brought about by classical artificial intelligence, motivates a search for applications of quantum information processing methods to machine learning. Among several proposals in this domain, quantum kernel methods have emerged as particularly promising candidates. However, while some rigorous speedups on certain highly specific problems have been formally proven, only empirical proof-of-principle results have been reported so far for real-world datasets. Moreover, no systematic procedure is known, in general, to fine tune and optimize the performances of kernel-based quantum classification algorithms. At the same time, certain limitations such as kernel concentration effects-hindering the trainability of quantum classifiers-have also been recently pointed out. In this work, we propose several general-purpose optimization methods and best practices designed to enhance the practical usefulness of fidelity-based quantum classification algorithms. Specifically, we first describe a data pre-processing strategy that, by preserving the relevant relationships between data points when processed through quantum feature maps, substantially alleviates the effect of kernel concentration on structured datasets. We also introduce a classical post-processing method that, based on standard fidelity measures estimated on a quantum processor, yields non-linear decision boundaries in the feature Hilbert space, thus achieving the quantum counterpart of the radial basis functions technique that is widely employed in classical kernel methods. Finally, we apply the so-called quantum metric learning protocol to engineer and adjust trainable quantum embeddings, demonstrating substantial performance improvements on several paradigmatic real-world classification tasks.

An automatic generation of pre-processing strategy combined with machine learning multivariate analysis for NIR spectral data

Pre-processing near-infrared (NIR) spectral data is indispensable in multivariate analysis, since the measured spectra of complex samples are often subject to overwhelming background, light scattering, varying noises, and other unexpected factors. Various pre-processing methods have been developed to remove or reduce the interference of these effects. Until now, most applications of NIR spectra pre-processing in multivariate calibration have been trial-and-error, with selecting a proper method depending on the nature of the data, expertise, and practitioner experience. Thus, it is usually challenging to determine the best pre-processing method for a given data. In order to tackle these problems, this study proposes a new concept of data pre-processing, namely, automatically generating a pre-processing strategy (AGoES). This concept belongs to the ensemble pre-processing method, where machine learning algorithms (PLSR, SVM, k-NN, DT, AB, and GPR) built on differently preprocessed data are combined by 5-fold cross-validation and grid search optimization. To investigate our concept, a public NIR spectral dataset was used to predict three responses, including dry matter content (DM), organic matter content (OM) and ammonium nitrogen content (AN) from manure organic waste. The results show that SVM is the best algorithm combined with the AGoES pre-processing to predict DM and AN with a ratio of prediction to deviation (RPD) of 3.619 and 2.996, respectively. The AB tandem with AGoES pre-processing is the best strategy for predicting OM with an RPD of 3.185. Therefore, in the framework of the AGoES concept, it is unsupervised pre-processing, more simple, and feasible to apply multivariate analysis using machine learning algorithms.

Vehicles Speed Estimation Model from Video Streams for Automatic Traffic Flow Analysis Systems

Image and video processing have been widely used to provide traffic parameters, which will be used to improve certain areas of traffic operations. This research aims to develop a model for estimating vehicle speed from video streams to support traffic flow analysis (TFA) systems. Subsequently, this paper proposes a vehicle speed estimation model with three main stages of achieving speed estimation: (1) pre-processing, (2) segmentation, and (3) speed detection. The model uses a bilateral filter in the pre-processing strategy to provide free-shadow image quality and sharpen the image. Gaussian filter and active contour are used to detect and track objects of interest in the image. The Pinhole model is used to assess the real distance of the item within the image sequence for speed estimation. Kalman filter and optical flow are used to flatten vehicle speed and acceleration uncertainties. This model is evaluated with a dataset that consists of video recordings of moving vehicles at traffic light junctions on the urban roadway. The average percentage for speed estimation error is 20.86%. The average percentage for accuracy obtained is 79.14%, and the overall average precision of 0.08.

A Double-Stage 3D U-Net for On-Cloud Brain Extraction and Multi-Structure Segmentation from 7T MR Volumes

The brain is the organ most studied using Magnetic Resonance (MR). The emergence of 7T scanners has increased MR imaging resolution to a sub-millimeter level. However, there is a lack of automatic segmentation techniques for 7T MR volumes. This research aims to develop a novel deep learning-based algorithm for on-cloud brain extraction and multi-structure segmentation from unenhanced 7T MR volumes. To this aim, a double-stage 3D U-Net was implemented in a cloud service, directing its first stage to the automatic extraction of the brain and its second stage to the automatic segmentation of the grey matter, basal ganglia, white matter, ventricles, cerebellum, and brain stem. The training was performed on the 90% (the 10% of which served for validation) and the test on the 10% of the Glasgow database. A mean test Dice Similarity Coefficient (DSC) of 96.33% was achieved for the brain class. Mean test DSCs of 90.24%, 87.55%, 93.82%, 85.77%, 91.53%, and 89.95% were achieved for the brain structure classes, respectively. Therefore, the proposed double-stage 3D U-Net is effective in brain extraction and multi-structure segmentation from 7T MR volumes without any preprocessing and training data augmentation strategy while ensuring its machine-independent reproducibility.

Optimizing strategies of Raman spectra model combining pre-processing and classification for diagnosis of lung cancer

The analysis of biological sample data based on Raman spectroscopy involves applying the relevant chemometrics pre-processing and creating the statistical model. In this work, genetic algorithms and pipelines were used to study the steps and sequences of the pre-processing for human blood serum Raman spectra form 76 healthy individuals and 84 lung cancer patients for different data analysis models. The models used in this study include support vector machine (linear kernel and nonlinear kernel), multilayer perceptron, and partial least squares discriminant analysis. The results show that the steps and sequence of pre-processing are not immutable for different models. These optimized pipelines are evaluated by execution time and optimization results. The conclusions are that genetic algorithms can optimize the pipeline of pre-processing strategies and classification models to improve data analysis accuracy, and support vector machine models are more suitable for the classification of our lung cancer serum data.

Impact of sensor data pre-processing strategies and selection of machine learning algorithm on the prediction of metritis events in dairy cattle

With all the sensor data currently generated at high frequency in dairy farms, there is potential for earlier diagnosis of postpartum diseases compared with traditional monitoring methodologies. Our objectives were 1) to compare the impact of sensor data pre-processing on classifier performance by using multiple time windows before a given metritis event, while considering other cow-level factors and farm-scheduled activities; 2) to compare the performance of random forest (RF), k-nearest neighbors (k-NN), and support vector machine (SVM) classifiers at different decision thresholds using different number of past observations (time-lags) for the detection of behavioral patterns associated with changes in metritis scores; and 3) to compare classifier performance between each one of the five behaviors registered every hour by an ear-tag 3-axis accelerometer (CowManager, Agis Autimatisering, Harmelen, Netherlands). A total of 239 metritis events were created by comparing metritis scores between two consecutive clinical evaluations from cows that were retrospectively selected from a dataset containing sensor data and health information during the first 21 days postpartum from June 2014 to May 2017. Hourly sensor data classified by the accelerometer as either ruminating, eating, not active (including both standing or lying), and two different levels of activity (active and high activity) behaviors corresponding to the 3 days before each metritis event were aggregated every 24-, 12-, 6-, and 3-hour time windows. Multiple time-lags were also used to determine the optimal number of past observations needed for optimal classification. Similarly, different decision thresholds were compared in terms of model performance. Depending on the classifier, algorithm hyperparameters were optimized using grid search (RF, k-NN, SVM) and random search (RF). All behaviors changed throughout the study period and showed distinct daily patterns. From the three algorithms, RF had the highest F1 score followed by k-NN and SVM. Furthermore, sensor data aggregated every 6- or 12-h time windows had the best model performance at multiple time-lags. We concluded that the data from the first 3 days post-partum should be discarded when studying metritis, and either one of the five behaviors measured with CowManager could be used when predicting metritis when sensor data were aggregated every 6- or 12-hour time windows, and using time-lags corresponding to 2–3 days before a given event, depending on the time window used. This study shows how to maximize sensor data in their potential for disease prediction, enhancing the performance of algorithms used in machine learning.

Machine Learning Techniques to Detect a DDoS Attack in SDN: A Systematic Review

The recent advancements in security approaches have significantly increased the ability to identify and mitigate any type of threat or attack in any network infrastructure, such as a software-defined network (SDN), and protect the internet security architecture against a variety of threats or attacks. Machine learning (ML) and deep learning (DL) are among the most popular techniques for preventing distributed denial-of-service (DDoS) attacks on any kind of network. The objective of this systematic review is to identify, evaluate, and discuss new efforts on ML/DL-based DDoS attack detection strategies in SDN networks. To reach our objective, we conducted a systematic review in which we looked for publications that used ML/DL approaches to identify DDoS attacks in SDN networks between 2018 and the beginning of November 2022. To search the contemporary literature, we have extensively utilized a number of digital libraries (including IEEE, ACM, Springer, and other digital libraries) and one academic search engine (Google Scholar). We have analyzed the relevant studies and categorized the results of the SLR into five areas: (i) The different types of DDoS attack detection in ML/DL approaches; (ii) the methodologies, strengths, and weaknesses of existing ML/DL approaches for DDoS attacks detection; (iii) benchmarked datasets and classes of attacks in datasets used in the existing literature; (iv) the preprocessing strategies, hyperparameter values, experimental setups, and performance metrics used in the existing literature; and (v) current research gaps and promising future directions.

Classifying Brain Tumors on Magnetic Resonance Imaging by Using Convolutional Neural Networks

The study of neuroimaging is a very important tool in the diagnosis of central nervous system tumors. This paper presents the evaluation of seven deep convolutional neural network (CNN) models for the task of brain tumor classification. A generic CNN model is implemented and six pre-trained CNN models are studied. For this proposal, the dataset utilized in this paper is Msoud, which includes Fighshare, SARTAJ, and Br35H datasets, containing 7023 MRI images. The magnetic resonance imaging (MRI) in the dataset belongs to four classes, three brain tumors, including Glioma, Meningioma, and Pituitary, and one class of healthy brains. The models are trained with input MRI images with several preprocessing strategies applied in this paper. The CNN models evaluated are Generic CNN, ResNet50, InceptionV3, InceptionResNetV2, Xception, MobileNetV2, and EfficientNetB0. In the comparison of all CNN models, including a generic CNN and six pre-trained models, the best CNN model for this dataset was InceptionV3, which obtained an average Accuracy of 97.12%. The development of these techniques could help clinicians specializing in the early detection of brain tumors.

Pursuit of the Ultimate Regression Model for Samarium(III), Europium(III), and LiCl Using Laser-Induced Fluorescence, Design of Experiments, and a Genetic Algorithm for Feature Selection.

Laser-induced fluorescence spectroscopy, Raman scattering, and partial least squares regression models were optimized for the quantification of samarium (0-150 μg mL-1), europium (0-75 μg mL-1), and lithium chloride (0.1-12 M) with a transformational preprocessing strategy. Selecting combinations of preprocessing methods to optimize the prediction performance of regression models is frequently a major bottleneck for chemometric analysis. Here, we propose an optimization tool using an innovative combination of optimal experimental designs for selecting preprocessing transformation and a genetic algorithm (GA) for feature selection. A D-optimal design containing 26 samples (i.e., combinations of preprocessing strategies) and a user-defined design (576 samples) did not statistically lower the root mean square error of the prediction (RMSEP). The greatest improvement in prediction performance was achieved when a GA was used for feature selection. This feature selection greatly lowered RMSEP statistics by an average of 53%, resulting in the top models with percent RMSEP values of 0.91, 3.5, and 2.1% for Sm(III), Eu(III), and LiCl, respectively. These results indicate that preprocessing corrections (e.g., scatter, scaling, noise, and baseline) alone cannot realize the optimal regression model; feature selection is a more crucial aspect to consider. This unique approach provides a powerful tool for approaching the true optimum prediction performance and can be applied to numerous fields of spectroscopy and chemometrics to rapidly construct models.

CT Lung Nodule Segmentation: A Comparative Study of Data Preprocessing and Deep Learning Models

The number of deaths from lung cancer reached 1.8 million in 2020, ranking first among all cancers. Early diagnosis has been found to improve the survival rate of lung cancer patients after treatment in clinical care. Computed tomography (CT) is a technique commonly used in the early detection of lung cancer to determine the benignity or malignancy of lung nodules. Manual analysis of CT results is less efficient and its accuracy is affected by physicians’ experience levels. Segmenting lung nodules in a computer-aided diagnosis (CAD) system can effectively improve the efficiency and accuracy of the diagnosis. In this paper, we evaluate several deep learning segmentation models (including UNet, SegNet, GCN, FCN, DeepLabV3+, PspNet TransUNet, SwinNet) and examine the effects of different preprocessing methods on the models to explore the best preprocessing and training strategies for lung nodule segmentation. Specifically, we investigate the effects of two different data preprocessing methods (adding a lung mask and croping the region of interest) on the segmentation results, where better segmentation results are achieved by including the nodal data of the region of interest without the lung mask. Through a comprehensive comparison, TransUNet achieves the best segmentation accuracy, with DICE indices of 0.887, 0.871, 0.75, and 0.744 tested on four datasets, respectively.

BiDETECT: BiLSTM with BERT for hate speech detection in tweets

The utilization of online platforms for spreading hate speech has become a major concern. The conventional techniques used to identify hate speech, such as relying on keywords and manual moderation, frequently fall short and can lead to either missed detections or incorrect identifications. In response, researchers have developed various deeplearning strategies for locating hate speech in text. This paper covers a wide range of Deep Learning approaches, encompassing Convolutional Neural Networks and especially transformer-based models. It also discusses the key factors that influence the performance of these methods, such as the choice of datasets, the use of pre-processing strategies, and the design of the model architecture. In conjunction with summarizing existing research, it also identifies a selection of key hurdles and limitations of Deep Learning for discovering hate speech and has proposed a novel method to overcome them. In Bidirectional Long Short-Term Memory and BERT for Hate Speech Detection (BiDETECT), which involves adding a Bidirectional Long Short-Term Memory (BiLSTM) layer to Bidirectional Encoder Representations from Transformers (BERT) for classification, the hurdles include the difficulties in defining hate speech, the limitations of current datasets, and the challenges of generalizing models to new domains. It also discusses the ethical implications of employing Deep Learning to pinpoint hate speech and the need for responsible and transparent research in this area.

Impact of Recursive Feature Elimination with Cross-validation in Modeling the Spatial Distribution of Three Mosquito Species in Morocco

Many studies in ecology are interested in characterizing the ecological factors; determining the distribution of animal species. The classical approach consists in identifying the combination of ecological factors that allow reproducing observations of the presence and absence of the species of interest. The major difficulty lies in the imbalance between a considerable quantity of ecological factors to be tested and a relatively limited number of presence/absence observations. Selection of the most influential ecological features is a classical data pre-processing strategy that aims to overcome this imbalance and improve model performance. In this paper, we applied recursive feature elimination with cross-validation (RFECV) approach on presence/absence mosquito data in Morocco; to select optimal subsets of ecological features, in order to improve the performance of the predictive models. This method demonstrated the best ability to improve the performance of the predictive models, and can be recommended as a modeling improvement technique for large datasets.

Robust image compression‐encryption via scrambled block bernoulli sampling with diffusion noise

AbstractThis paper proposed an image compression‐encryption scheme based on compressive sensing theory, which achieves high security, strong robustness, and high rate‐distortion performance. First, the denoising preprocessing strategy is applied at the encoder side, which can enhance the rate‐distortion performance without sacrificing security and robustness. Second, the preprocessed image is randomly down‐sampled using scrambled block Bernoulli sampling with diffusion noise (SBBS‐DN), which is generated by combining a hyper‐chaotic system and SHA256 hash of the plain image. Third, a deep‐learned plug‐and‐play is embedded prior for plain image reconstruction at the decoder side. Simulation results show that the proposed scheme has desirable security performance (being resistant to different attacks), high R‐D performance (PSNR gains over 1.3 dB than JPEG at 0.50 bpp compression ratio), and high error resilience (reconstructed 29.92 dB at 0.50 bpp compression ratio even with 50% bit loss).

Preprocessing: A method For Reducing Time Complexity

Data can be processed quickly if it is in some order, whereas unsequenced data can take more time to obtain results. Sorting is used for data arrangement. It is also one of the essential requirement for most applications and this step helps to boost performance. Sorting is also a prerequisite in several computer applications like databases. Over time computer scientists have not only introduced new sorting techniques considering various factors to be improved but they have also presented enhanced variants of existing sorting methods. The main objective has always been to reduce the execution time and space of the sorting algorithms. With every passing day, digital content is growing rapidly, which is a significant cause that encourages researchers to design new time-space efficient sorting algorithms. This paper presents some preprocessing strategies for quicksort and insertion sort to improve their performances. Tha main idea of using these preprocessings is to make input data more suitable for sorting algorithm, as most sorting function performs extraordinary for a specific type of input, such as insertion sort works better on nearyly sorted data. To authenticate the efficiency of existing sorting algorithms, these have been compared with proposed preprocessing strategies. The results with proposed techniqes outperforms the results of original sorting methods. It also helps to convert worst case into average case. By using this approch complexity of many algorithms can be reduced, therfore this is very important.

CoT-XNet: contextual transformer with Xception network for diabetic retinopathy grading

Objective. Diabetic retinopathy (DR) grading is primarily performed by assessing fundus images. Many types of lesions, such as microaneurysms, hemorrhages, and soft exudates, are available simultaneously in a single image. However, their sizes may be small, making it difficult to differentiate adjacent DR grades even using deep convolutional neural networks (CNNs). Recently, a vision transformer has shown comparable or even superior performance to CNNs, and it also learns different visual representations from CNNs. Inspired by this finding, we propose a two-path contextual transformer with Xception network (CoT-XNet) to improve the accuracy of DR grading. Approach. The representations learned by CoT through one path and those by the Xception network through another path are concatenated before the fully connected layer. Meanwhile, the dedicated pre-processing, data resampling, and test time augmentation strategies are implemented. The performance of CoT-XNet is evaluated in the publicly available datasets of DDR, APTOS2019, and EyePACS, which include over 50 000 images. Ablation experiments and comprehensive comparisons with various state-of-the-art (SOTA) models have also been performed. Main results. Our proposed CoT-XNet shows better performance than available SOTA models, and the accuracy and Kappa are 83.10% and 0.8496, 84.18% and 0.9000 and 84.10% and 0.7684 respectively, in the three datasets (listed above). Class activation maps of CoT and Xception networks are different and complementary in most images. Significance. By concatenating the different visual representations learned by CoT and Xception networks, CoT-XNet can accurately grade DR from fundus images and present good generalizability. CoT-XNet will promote the application of artificial intelligence-based systems in the DR screening of large-scale populations.

Noninvasive Classification of Glioma Subtypes Using Multiparametric MRI to Improve Deep Learning.

Background: Deep learning (DL) methods can noninvasively predict glioma subtypes; however, there is no set paradigm for the selection of network structures and input data, including the image combination method, image processing strategy, type of numeric data, and others. Purpose: To compare different combinations of DL frameworks (ResNet, ConvNext, and vision transformer (VIT)), image preprocessing strategies, magnetic resonance imaging (MRI) sequences, and numerical data for increasing the accuracy of DL models for differentiating glioma subtypes prior to surgery. Methods: Our dataset consisted of 211 patients with newly diagnosed gliomas who underwent preoperative MRI with standard and diffusion-weighted imaging methods. Different data combinations were used as input for the three different DL classifiers. Results: The accuracy of the image preprocessing strategies, including skull stripping, segment addition, and individual treatment of slices, was 5%, 10%, and 12.5% higher, respectively, than that of the other strategies. The accuracy increased by 7.5% and 10% following the addition of ADC and numeric data, respectively. ResNet34 exhibited the best performance, which was 5% and 17.5% higher than that of ConvNext tiny and VIT-base, respectively. Data Conclusions: The findings demonstrated that the addition of quantitatively numeric data, ADC images, and effective image preprocessing strategies improved model accuracy for datasets of similar size. The performance of ResNet was superior for small or medium datasets.

Exploiting Domain Knowledge to Address Class Imbalance in Meteorological Data Mining

We deal with the problem of class imbalance in data mining and machine learning classification algorithms. This is the case where some of the class labels are represented by a small number of examples in the training dataset compared to the rest of the class labels. Usually, those minority class labels are the most important ones, implying that classifiers should primarily perform well on predicting those labels. This is a well-studied problem and various strategies that use sampling methods are used to balance the representation of the labels in the training dataset and improve classifier performance. We explore whether expert knowledge in the field of Meteorology can enhance the quality of the training dataset when treated by pre-processing sampling strategies. We propose four new sampling strategies based on our expertise on the data domain and we compare their effectiveness against the established sampling strategies used in the literature. It turns out that our sampling strategies, which take advantage of expert knowledge from the data domain, achieve class balancing that improves the performance of most classifiers.

A multi-frame network model for predicting seizure based on sEEG and iEEG data.

Analysis and prediction of seizures by processing the EEG signals could assist doctors in accurate diagnosis and improve the quality of the patient's life with epilepsy. Nowadays, seizure prediction models based on deep learning have become one of the most popular topics in seizure studies, and many models have been presented. However, the prediction results are strongly related to the various complicated pre-processing strategies of models, and cannot be directly applied to raw data in real-time applications. Moreover, due to the inherent deficiencies in single-frame models and the non-stationary nature of EEG signals, the generalization ability of the existing model frameworks is generally poor. Therefore, we proposed an end-to-end seizure prediction model in this paper, where we designed a multi-frame network for automatic feature extraction and classification. Instance and sequence-based frames are proposed in our approach, which can help us simultaneously extract features of different modes for further classification. Moreover, complicated pre-processing steps are not included in our model, and the novel frames can be directly applied to the raw data. It should be noted that the approaches proposed in the paper can be easily used as the general model which has been validated and compared with existing model frames. The experimental results showed that the multi-frame network proposed in this paper was superior to the existing model frame in accuracy, sensitivity, specificity, F1-score, and AUC in the classification performance of EEG signals. Our results provided a new research idea for this field. Researchers can further integrate the idea of the multi-frame network into the state-of-the-art single-frame seizure prediction models and then achieve better results.

Computer assisted diagnosis of skin cancer: A survey and future recommendations

Skin cancer is amid the most frequent types of cancer, accounting for approximately 2 to 3 million cases being diagnosed each year worldwide. Abnormal cell development on the skin causes skin lesions and manual inspection of skin lesions is a difficult, challenging, instinctive, and tedious task. Computer Aided Diagnosis (CAD) techniques can assist doctors to enhance their investigation skills and reduce the time it takes to get a precise diagnosis. Furthermore, the lack of advanced, user-friendly CAD techniques has raised serious concerns about the noninvasive, precise, and rapid identification of diseases. CAD systems can help to make an early diagnosis of skin lesions to plan timely treatment schedules for the patients to increase their survival rates. However, due to the distinctive and complex properties of skin lesion images, examination of skin lesion images still poses significant difficulties. The motivation behind this study is to discuss several preprocessing, segmentation, and classification strategies for analyzing skin lesions to differentiate between cancerous and non-cancerous images. The primary goal is to provide an overview for naïve researchers to commence their research in this field. Moreover, this manuscript will also highlight open challenges and future recommendations which further calls the distinct researchers to begin their research in this domain.