Feature Engineering Process Research Articles

Time Series Classification for Predicting Biped Robot Step Viability

The prediction of the stability of future steps taken by a biped robot is a very important task, since it allows the robot controller to adopt the necessary measures in order to minimize damages if a fall is predicted. We present a classifier to predict the viability of a given planned step taken by a biped robot, i.e., if it will be stable or unstable. The features of the classifier are extracted from a feature engineering process exploiting the useful information contained in the time series generated in the trajectory planning of the step. In order to state the problem as a supervised classification one, we need the ground truth class for each planned step. This is obtained using the Predicted Step Viability (PSV) criterion. We also present a procedure to obtain a balanced and challenging training/testing dataset of planned steps that contains many steps in the border between stable and non stable regions. Following this trajectory planning strategy for the creation of the dataset we are able to improve the robustness of the classifier. Results show that the classifier is able to obtain a 95% of ROC AUC for this demanding dataset using only four time series among all the signals required by PSV to check viability. This allows to replace the PSV stability criterion, which is safe, robust but impossible to apply in real-time, by a simple, fast and embeddable classifier that can run in real time consuming much less resources than the PSV.

A demand forecasting system of product categories defined by their time series using a hybrid approach of ensemble learning with feature engineering

Retail companies face major problems in the estimation of their product’s future demand due to the high diversity of sales behavior that each good presents. Different forecasting models are implemented to meet the demand requirements for efficient inventory management. However, in most of the proposed works, a single model approach is applied to forecast all products, ignoring that some methods are better adapted for certain features of the demand time series of each product. The proposed forecasting system addresses this problem, by implementing a two-phase methodology that initially clusters the products with the application of an unsupervised learning approach using the extracted demand features of each good, and then, implements a second phase where, after a feature engineering process, a set of different forecasting methods are evaluated to identify those with best performs for each cluster. Finally, ensemble machine learning models are implemented using the top-performing models of each cluster to carry out the demand estimation. The results indicate that the proposed forecasting system improves the demand estimation over the single forecasting approaches when evaluating the R2, MSE, and MASE quality measures.

Explainable ResNet50 learning model based on copula entropy for cotton plant disease prediction

This paper presents a novel Deep Learning (DL) framework for cotton plant disease prediction based on Explainable Artificial Intelligence (XAI) and Copula entropy based-Grey Wolf Optimization (GWO) algorithm. The suggested framework uses a cotton plant image as input and pre-processes it to produce an image with enhanced contrast. Features are extracted from the leaf images with the ResNet50 CNN model. A crucial pre-processing model known as feature selection helps to improve the effectiveness of image classification by deleting extraneous or irrelevant features. Therefore, the Gray Wolf optimization (GWO) algorithm which is a global search method with potential use in feature selection is employed in this paper. The proposed framework introduces Copula entropy (CE) as an indicator of association to create the GWO’s initial population and enhance the GWO feature engineering process. For the GWO initialization procedure, CE has been used to choose the most significant features which substantially improved the quality of the GWO starting population and as a result improved the performance of the proposed CE-based GWO algorithm by 78.57 % faster than the traditional GWO as stated by the time complexity analysis. In addition, Feature importance explanation is determined using XAI layer. The final classification is achieved using Random Forests (RF) classifier which is an ensemble learning approach. According to the experimental results, the suggested model has a classification accuracy of 99 % and a mean squared error of 0.0383. Furthermore, the proposed model has been compared to state-of-the-art algorithms and the results showed that the proposed model has the superiority performance. The proposed model can therefore be used to track a variety of cotton areas to enable faster analysis and response, resulting in higher productivity.

A semantic-based model with a hybrid feature engineering process for accurate spam detection

Detecting spam emails is essential to maintaining the security and integrity of email communication. Existing research has made significant progress in developing effective spam detection models, but challenges remain in improving classification performance and adaptability to evolving spamming techniques. In this study, we propose a novel spam detection model with a comprehensive feature engineering approach that combines term frequency-inverse document frequency (TF-IDF) vectorizer and word embedding features to optimize the feature space. Our contribution lies in integrating semantic-based word embeddings, leveraging pre-existing knowledge to capture the semantic meaning of words and enhance the representation of email texts. To identify the most suitable word embedding technique for our model, we evaluated GloVe, Word2Vec, and FastText. GloVe was selected for its better performance, which is the result of its pre-training on a large and diverse text corpus. Furthermore, the model was evaluated without word embeddings, which did not exhibit the same effectiveness level as our word embedding-based model. Additionally, we utilized the support vector machine as a classifier and hyperparameter tuning technique to identify our model’s most effective parameter values. The proposed model was tested on two datasets. The experimental results showed that our model outperformed the other models discussed in the literature, achieving an accuracy of 99.5% on the SpamAssassin dataset, and 99.28% on the Enron-Spam dataset.

Identifying grain size in ASTM A36 steel using ultrasonic backscattered signals and machine learning

Ultrasonic nondestructive techniques can be useful tools in the microstructural classification of metallic alloys. Among the most common techniques for characterizing materials, backscattered signal analysis stands out because it does not require a back surface echo. This study classifies five ASTM A36 steel samples with varying grain sizes using ultrasonic waves. For each sample, ultrasound signals were obtained using two ultrasonic array probes with center frequencies of 5 and 10 MHz. An extensive feature engineering process was conducted using the tfresh package in Python, which extracts a wide range of features from time series data, transforming raw ultrasound signals into a structured format. Seven machine learning models were evaluated based on accuracy, precision, recall, and F1 score, with performances ranging from 70 to 100%. The XGBoost model exhibited the best performance with 10 MHz probe signals, achieving 100% accuracy. An additional validation test was conducted to evaluate the models’ generalization capability, considering inter-specimen variabilities. Despite a slight reduction in prediction metrics, the XGBoost model maintained good performance, with accuracy between 89% and 100% across all frequencies. Such findings demonstrate that backscattered grain noise can effectively identify grain sizes provided that an adequate machine learning model is used.

Audio Classification on the FSDKaggle2018 Dataset using Tensorflow, Keras

This project explores audio classification leveraging representation learning within the TensorFlow framework. The methodology focuses on the initial engineering of wave features derived from raw audio data, which are subsequently used to train convolutional neural networks (CNNs) for effective representation learning. By transforming the audio signals into a structured format amenable to convolutional processing, our system is designed to capture the intrinsic properties and patterns embedded in the sound waves. The feature engineering process is detailed where various envelope features such as the homomorphic envelogram, hilbert envelogram and wavelet decompositions help in extracting meaningful information from the raw audio signals. These engineered features provide a robust foundation for the subsequent layers of convolutional networks. The CNNs are meticulously architected to learn hierarchical representations, effectively capturing both low-level and high-level audio characteristics.This study attempts to reinforces the significance of tailored feature engineering in deep learning and demonstrate an effective audio classification pipeline using representation learning and open up new avenues for research in audio signal processing and machine learning.

MLCompare: Platform to Compare Machine Learning Models

Abstract: The field of machine learning (ML) continues to advance, the demand for efficient and effective methods for model development and selection becomes increasingly pressing. Automated Machine Learning (AutoML) platforms have emerged as powerful tools to address this need by automating the process of model selection, hyperparameter tuning, and feature engineering. By integrating a wide array of ML techniques and leveraging cloud computing resources, the platform enables researchers and practitioners to efficiently explore and compare the performance of various algorithms across diverse datasets. Key features include a user-friendly interface, a robust evaluation framework encompassing traditional metrics and advanced techniques, and parallel processing capabilities. Through experiments on benchmark datasets from multiple domains, the platform demonstrates its ability to streamline algorithm selection and provide valuable insights into algorithm performance. This contribution advances the field of AutoML by facilitating informed decision-making and accelerating innovation in ML applications.

CYBER BULLYING DETECTION: AN ENSEMBLE BASED MACHINE LEARNING APPROACH

Research on cyberbullying detection.To identify control and reduce the bullying contents spread over social media sites. Though data collection and feature engineering process has been elaborated, yet most of the emphasis is on feature selection algorithms. Thus there is an extensive need to identify, control and reduce the bullying contents spread over social media sites, which has motivated us to conduct this research to automate the detection process of offensive language or cyberbullying. In our work, Logistic Regression and Bagging ensemble model classifier have performed individually best in detecting cyberbullying which has been outperformed by our proposed SLE and DLE voting classifiers. Our proposed SLE and DLE models yield the best performance of 96% when TF-IDF (Unigram) feature extraction is applied with K-Fold cross-validation.

Evaluating the Impact of Different Feature as a Counter Data Aggregation approaches on the Performance of NIDSs and Their Selected Features

Abstract There is much effort nowadays to protect communication networks against different cybersecurity attacks (which are more and more sophisticated) that look for systems’ vulnerabilities they could exploit for malicious purposes. Network Intrusion Detection Systems (NIDSs) are popular tools to detect and classify such attacks, most of them based on ML models. However, ML-based NIDSs cannot be trained by feeding them with network traffic data as it is. Thus, a Feature Engineering (FE) process plays a crucial role transforming network traffic raw data onto derived one suitable for ML models. In this work, we study the effects of applying one such FE technique in different ways on the performance of two ML models (linear and non-linear) and their selected features. This the Feature as a Counter approach. The derived observations are computed from either with the same number of raw samples, (batch-based approaches) or by aggregating them by time intervals (timestamp-based approach). Results show that there is no significant differences between the proposed approaches neither in the performance of the models nor in the selected features that validate our proposal making it feasible to be widely used as a standard FE method.

Explanatory Cognitive Diagnosis Models Incorporating Item Features.

Item quality is crucial to psychometric analyses for cognitive diagnosis. In cognitive diagnosis models (CDMs), item quality is often quantified in terms of item parameters (e.g., guessing and slipping parameters). Calibrating the item parameters with only item response data, as a common practice, could result in challenges in identifying the cause of low-quality items (e.g., the correct answer is easy to be guessed) or devising an effective plan to improve the item quality. To resolve these challenges, we propose the item explanatory CDMs where the CDM item parameters are explained with item features such that item features can serve as an additional source of information for item parameters. The utility of the proposed models is demonstrated with the Trends in International Mathematics and Science Study (TIMSS)-released items and response data: around 20 item linguistic features were extracted from the item stem with natural language processing techniques, and the item feature engineering process is elaborated in the paper. The proposed models are used to examine the relationships between the guessing/slipping item parameters of the higher-order DINA model and eight of the item features. The findings from a follow-up simulation study are presented, which corroborate the validity of the inferences drawn from the empirical data analysis. Finally, future research directions are discussed.

KolamNetV2: efficient attention-based deep learning network for tamil heritage art-kolam classification

In India, kolam, commonly referred to as rangoli, is a traditional style of art. It involves using rice flour, chalk, or coloured powders to create elaborate patterns and motifs on the ground. Kolam is a common daily ritual in many regions of India, especially in South India, where it is seen as a significant cultural tradition and a means to greet visitors. Unfortunately, as a result of people’s hectic lives nowadays, the habit of drawing kolam on a regular basis is dwindling. The art of making kolam patterns is in danger of disappearing as so many individuals no longer have the time or space to do it on a regular basis. Therefore, it is imperative that ancient art be conserved and digitally documented in order to enlighten our next generation about kolam and its classifications. Deep learning has become a powerful technique because of its ability to learn from raw image data without the aid of a feature engineering process. In this article, we attempted to understand the types of Kolam images using the proposed deep architecture called KolamNetV2. KolamNetV2 comprises EfficientNet and attention layers, ensuring high accuracy with minimal training data and parameters. We evaluated KolamNetV2 to reveal its ability to learn the various types in our challenging Kolam dataset. The experimental findings show that the proposed network achieves fine enhancement in performance metrics viz, precision-0.7954, recall-0.7846, F1score-0.7854 and accuracy-81%. We compared our results with state-of-the-art deep learning methodologies, proving the astounding capability.Graphical

Time-Series Analysis on AIDE IoT Attack Data Unraveling Trends and Patterns for Enhanced Security

With the growing use of IoT devices, cyberattacks have increased against these devices, using various exploitable vulnerabilities. Cyberattacks are potentially ruinous events for business owners and the cost of a cyber-attack is not only financial, but companies also must spend time on recovering from the attacks. Through applied research we set out the latest findings and trends to provide new insights into the security of IoTs, focusing on the emerging nature of IoT attacks, frequency, and usual vulnerabilities, thus providing significance to cybersecurity and IoT defense, which will be beneficial to researchers, manufacturers, individuals, organizations, and governments. Although the literature on IoTs is quite rich to this day, however, there is currently no study that provides an in-depth analysis of patterns and frequencies of IoT attacks to discover insights into the most vulnerable times and systems. Compared to other related research on the security of IoTs, this applied research encompasses much more technical angles to the security of IoT. It begins with the comprehensive data acquisition from the Global Cyber Alliance’s (GCA) Automated IoT Defense Ecosystem (AIDE) and preprocessing to ensure data integrity and relevance. Followed by a thorough feature selection and engineering process. Python scripts and libraries were used for efficient data analysis. Unraveling Trends and Patterns for Enhanced Security are sensitive areas that have remained untouched by previous research works. Time-Series Analysis on AIDE IoT Attack Data used in this study provided critical insights into attack patterns by fields such as timestamps, attack duration, and login credentials, implying a huge scientific and technological uncover of an intricate and evolving landscape of IoT security threats and attacks.

DESIGN AND DEVELOPMENT OF FEATURE ENGINEERING MODEL FOR CVD MULTI-DIMENSIONAL DATASETS STANDARDIZATION

Cardio Vascular Diseases (CVD) are most common medical abnormalities in modern times. According to World Health Organization (WHO), the CVD related deaths spot is higher and increased due to chronical reasons such as unhealthy lifestyle, food habits, geographical changes and comorbidities from genetic history. The studies are focused on one particular causes as it trends to eliminate the comorbidities influencing the secondary causes. Hence a large unbalanced prediction and independent datasets are created and customized. In this research, the authors have aimed to propose dataset standardization on CVD. The proposed standardization is based on attribute interdependency mapping and indexing. The attribute interdependency from one influencing parameter is aligned and coordinated with secondary attributes from relationship/dependency evaluation. Further, the dependency mapping is simplified by layering and customizing the data attributes. The multi-dimensional CVD datasets extracted in this process is mapped and tracked from feature engineering process. The dataset standardization of CVD is novel and first of its kind in data analytics and processing.

COVID-ECG-RSNet: COVID-19 classification from ECG images using swish-based improved ResNet model

Millions of humansworldwide have been impacted by consecutive COVID-19waves during the past threeyears. The correct treatment for COVID-19 has not yet been identified because vaccine receivers have also contracted the illness. People’s lives can be saved, and patients can be shielded from laborious treatments with the quick and accurate diagnosis of coronavirus. Investigators have utilized various clinical imaging modalities i.e.,CT-Scan and X-ray for the identification of coronavirus, while, less emphasis is spent on the utilization ofECG images in recognizing the COVID-19-affected patients. In contrast to CT-Scan and X-ray images, ECG samples arereadily accessible, hence we utilize them to diagnose coronavirus. Since scientists generally transform the ECG data into integer representation before using any approach, which indicates higher computing overhead. Accurateand quickcoronavirus identification from the ECG modalityis a difficult and cumbersome operation. In this study, we attempted to address these issues by leveraging the ECG filesdirectly in an improved deep learning (DL)methodology called the COVID-ECG-RSNet. Descriptively, a novel DL model is presented by altering the ResNet-50 model by introducing an optimized activation approach called the Swish method during the feature engineering process. Moreover, we introduced multiple dense layers at the end of the proposed CNN structure to ensure more robust sample features for classification purposes. The introduced COVID-ECG-RSNet model is proficient in categorizing the ECG samples as being normal, coronavirus, myocardial infarction (MI), abnormal heartbeats (AHB), and victims with Previous history of Myocardial Infarction (PMI) groups. A vast evaluation with the help of a complicated and publically accessible data sample of ECG images is accomplished to show the effectiveness of our approach in recognizing the coronavirus images in contrast to other categories of heart diseases and healthy samples via using the ECG modality. We have attained an accuracy score of 98.8%, along with precision, and recall scores of 98.9%, and 98.7% respectively indicating the efficacy of suggested approach.

Copy-Move Forgery Detection in Medical Images Using Handcrafted Features

Abstract: In the realm of medical imaging, the authenticity and integrity of images are paramount for accurate diagnosis and treatment planning. Copy-move forgery, a prevalent form of image tampering, poses a significant threat to the reliability of medical images. This research project focuses on the development and implementation of a robust copy-move forgery detection system tailored specifically for medical images. The proposed methodology leverages handcrafted features, extracting distinctive characteristics from the images to detect instances of forgery. Through a meticulous process of feature engineering and selection, the algorithm aims to enhance sensitivity and specificity in identifying manipulated regions within medical images. The study explores the application of advanced image processing techniques and pattern recognition algorithms to achieve a high level of accuracy in forgery detection.

Developing an Intelligent Decision Support System for large-scale smart grid communication network planning

The placement of routers and gateways in wireless communication networks for Smart Grid Advanced Metering Infrastructure (AMI) is a complex problem often addressed in the literature through heuristics, clustering, or linear programming approaches. In large-scale scenarios, the complexity is directly related to the region’s number of smart meters and poles. The terrain profile can introduce signal quality degradation between a smart meter and the routers and gateways, further complicating the problem. This study presents the AIDA-ML method, representing a preliminary step toward developing an Intelligent Decision Support System (IDSS) for an effective positioning strategy. AIDA-ML incorporates a feature engineering-based strategy and utilizes machine learning algorithms to learn from the results computed by a heuristic method called AIDA. The heuristic approach is too time-consuming, relying on external resources like numerous terrain profile API calls and a deep understanding of wireless communication loss models. To implement a machine learning-based and more straightforward approach, a feature engineering process is employed to capture the operational characteristics and results of the analytical method while requiring less processing time. Through experiments conducted with real-world data from 26 cities in the state of Paraná, Brazil, comprising 466,237 smart meters and 352,867 poles, the results obtained using machine learning algorithms suggest that AIDA-ML can ensure the connection coverage of smart meters while meeting the same minimum requirements established for the analytical method. Moreover, AIDA-ML offers the advantage of reducing the processing time by 87.60% and by 96.86% the overall search space size compared to the heuristic approach.

Enhancing Intrusion Detection Systems with a Hybrid Deep Learning Model and Optimized Feature Composition

Systems for detecting intrusions (IDS) are essential for protecting network infrastructures from hostile activity. Advanced methods are required since traditional IDS techniques frequently fail to properly identify sophisticated and developing assaults. In this article, we suggest a novel method for improving IDS performance through the use of a hybrid deep learning model and feature composition optimization. RNN and CNN has strengths that the proposed hybrid deep learning model leverages to efficiently capture both spatial and temporal correlations in network traffic data. The model can extract useful features from unprocessed network packets using CNNs and RNNs, giving a thorough picture of network behaviour. To increase the IDS's ability to discriminate, we also offer feature optimization strategies. We uncover the most pertinent and instructive features that support precise intrusion detection through a methodical feature selection and engineering process. In order to reduce the computational load and improve the model's efficiency without compromising detection accuracy, we also use dimensionality reduction approaches. We carried out extensive experiments using a benchmark dataset that is frequently utilized in intrusion detection research to assess the suggested approach. The outcomes show that the hybrid deep learning model performs better than conventional IDS methods, obtaining noticeably greater detection rates and lower false positive rates. The performance of model is further improved by the optimized feature composition, which offers a more accurate depiction of network traffic patterns.

Remaining Useful Life Estimation of Turbofan Engines with Deep Learning Using Change-Point Detection Based Labeling and Feature Engineering

Accurate remaining useful life (RUL) prediction is one of the most challenging problems in the prognostics of turbofan engines. Recently, RUL prediction methods for turbofan engines mainly involve data-driven models. Preprocessing the sensor data is essential for the performance of the prognostic models. Most studies on turbofan engines use piecewise linear (PwL) labeling, which starts with a constant initial RUL value in normal/healthy operating time. In this study, we designed a prognostic procedure that includes difference-based feature construction, change-point-detection-based PwL labeling, and a 1D-CNN-LSTM (one-dimensional–convolutional neural network–long short-term memory) hybrid neural network model for RUL prediction. The procedure was evaluated on the subset FD001 of the C-MAPSS dataset. The proposed procedure was compared with machine learning and deep learning models with and without the new difference feature. Also, the results were compared with the studies that used similar labeling approaches. Our analysis of the numerical results underscores the clear superiority of the proposed 1D-CNN-LSTM model with the difference feature in RUL prediction, with a score of 437.2 and an RMSE value of 16.1. This result illustrates the superior predictive capability of the 1D-CNN-LSTM model, which outperformed traditional machine learning methods and one of the earliest deep learning methods. These findings emphasize the superior predictive capability of the 1D-CNN-LSTM model and underline the potential of the feature engineering process for more accurate and robust RUL prediction in the context of turbofan engine prognostics.

Privacy‐preserving peak time forecasting with Learning to Rank XGBoost and extensive feature engineering

AbstractIn modern power systems, predicting the time when peak loads will occur is crucial for improving efficiency and minimising the possibility of network sections becoming overloaded. However, most works in the load forecasting field are not focusing on a dedicated peak time forecast and are not dealing with load data privacy. At the same time, developing methods for forecasting peak electricity usage that protect customers' data privacy is essential since it could encourage customers to share their energy usage data, leading to more data points for the effective management and planning of power grids. Hence, the authors employ a dedicated Learning to Rank XGBoost algorithm to forecast peak times with only ranks of loads instead of absolute load magnitudes as input data, thereby offering potential privacy‐preserving properties. We show that the presented Learning to Rank XGBoost model yields comparable results to a benchmark XGBoost load forecasting model. Additionally, we describe our extensive feature engineering process and a state‐of‐the‐art Bayesian hyperparameter optimisation for selecting model parameters, which leads to a significant improvement of forecasting accuracy. Our method was used in the context of the final round of the international BigDEAL load forecasting challenge 2022, where we consistently achieved high‐ranking results in the peak time track and an overall fourth rank in the peak load forecasting track with our general XGBoost model.

Exploring interpretable features of hardness for intermetallic compounds prepared by spark plasma sintering

Intermetallic compounds, known for their excellent hardness, conductivity, and strength, have significant applications in aerospace and automotive industries. Hardness is a crucial mechanical property in the development and optimization of intermetallic compounds (IMCs), and meanwhile, spark plasma sintering (SPS) serves as a prevalent technique for preparing IMCs. In this study, a dataset of Vickers hardness of binary intermetallic compounds prepared by SPS and potential feature sets influencing the target performance (HV) were collected. Three machine-learning strategies were developed and comprehensively evaluated. The first strategy focuses on processing parameters and compositions, the second incorporates physical properties in addition to the features considered in the first strategy, and the third one employs a combined feature engineering based on the second strategy. The third strategy, which includes three screened features through a rigorous feature engineering process, achieves the highest predictive accuracy. Subsequently, a symbolic regression (SR) model based on genetic programming (GP) was employed to develop a physically interpretable formula linking the target performance with the selected features. The findings of this study are of significance for developing high-performance intermetallic compounds.