Feature Engineering Process Research Articles

Analisa dan Perancangan Machine Learning Untuk Mendeteksi Kegagalan Job di Apache Spark

A collection of data stored in a database, so the longer the data, the bigger the data, because the data processed is very large, processing time in Apache Spark can take up to a dozen or tens of hours. Sometimes, the Apache Spark application even fails. Therefore, to minimize the waiting time that could have been avoided or reduced, artificial intelligence through Machine Learning will be used to detect whether an Apache Spark application will fail or run smoothly. Factors to determine this failure are called features and are generated through the feature engineering process. The purpose of this research is to design Machine Learning so that it is able to find out what features will determine the success or failure of the Apache Spark application. The research method used is the Prototyping process model.

Improved Prediction of Body Mass Index in Real-World Administrative Healthcare Claims Databases.

To continue closing the gap between the predictive modeling and its real-world application, we report a new data-to-prediction pipeline that advanced the state-of-the-art predictive performance of body mass index (BMI) classifications by integrating siloed claims databases via a common data model. This study adapted the ensemble-based methodology of the baseline prediction model and focused on removing the silos in the claims databases. We applied the Super Learner machine learning algorithm (SLA) to learn a combined dataset consisting of 50% data from the Optum Date of Death database and 50% data from the IBM MarketScan Commercial Claims and Encounters (CCAE), and omitted the commonly used one-hot-encoding step and used multi-categorical variables directly in the feature engineering process. These developments were then optimized via a standard cross-validation scheme and the performance was evaluated on a holdout test set. Sociodemographic and clinical characteristics were used with (denoted as SLA1) and without (denoted as SLA2) baseline BMI values to predict BMI classifications (≥ 30, ≥ 35, and ≥ 40kg/m2). Although the newly implemented SLA1 performed similarly to the previous model, with the area under the receiver operating characteristic curve (ROC AUC) being approximately 88% for all BMI classifications, specificity ranging from 90% to 96%, and accuracy ranging from 88% to 93%. The new SLA2 achieved consistently better performance on all metrics across all BMI classes. In particular, the new SLA2 achieved 77-79% in ROC AUC, increasing from the previously reported level (73%). Its specificity improved to the range of 76-90% from 71-86%. Its accuracy improved to the range of 77-86% from 73-80%. Its recall (i.e., sensitivity) improved to the range of 64-78% from 60-76%. This study demonstrates dramatic improvements in the prediction of BMI across classifications using integrated databases in a common data model for the generation of real-world evidence.

Optimization Simulation of Dance Technical Movements and Music Matching Based on Multifeature Fusion.

Music and dance videos have been popular among researchers in recent years. Music is one of the most important forms of human communication; it carries a wealth of emotional information, and it is studied using computer tools. In the feature engineering process, most present machine learning approaches suffer from information loss or insufficient extracted features despite the relevance of computer interface and multimedia technologies in sound and music matching tasks. Multifeature fusion is widely utilized in education, aerospace, intelligent transportation, biomedicine, and other fields, and it plays a critical part in how humans get information. In this research, we offer an effective simulation method for matching dance technique movements with music based on multifeature fusion. The initial step is to use music beat extraction theory to segment the synchronized dance movements and music data, then locate mutation points in the music, and dynamically update the pheromones based on the merits of the dance motions. The audio feature sequence is obtained by extracting audio features from the dancing video's accompanying music. Then, we combine the two sequences to create an entropy value sequence based on audio variations. By comparing the consistency of several approaches for optimizing dance movement simulation trials, the optimized simulation method described in this research has an average consistency of 87%, indicating a high consistency. As a result, even though the background and the subject are readily confused, the algorithm in this research can keep a consistent recognition rate for more complicated dance background music, and the approach in this study can still guarantee a certain accuracy rate.

Early-stage lifetime prediction for lithium-ion batteries: A deep learning framework jointly considering machine-learned and handcrafted data features

Predicting the battery lifetime at its early stage is a promising technology for accelerating the battery development, production, and design optimization. However, it is a challenging task for most existing prediction methods because information is too limited in early life cycles, and the early-cycle capacity data exhibits a weak correlation with the target battery lifetime. In this paper, to realize an accurate battery lifetime prediction via data obtained from just first few life cycles, we propose a three-stage deep learning framework. First, we develop an emerging two-channel data feature engineering process, which jointly consider a convolutional neural network based latent feature extraction and domain knowledge based handcrafted features. Next, a wrapper feature selection method is adopted to further compress the dimension of developed features via eliminating linearly correlated ones. Finally, processed data features are fed into a data-driven model to realize the early-stage battery lifetime prediction. Results of computational experiments show that our proposed joint consideration of machine-learned features and handcrafted features can improve early-stage battery lifetime predictions via comparing with state-of-the-art benchmarks. We also show that the proposed framework can be generalized to batteries cycled under different operation conditions.

Op2Vec: An Opcode Embedding Technique and Dataset Design for End-to-End Detection of Android Malware

Android is one of the leading operating systems for smartphones in terms of market share and usage. Unfortunately, it is also an appealing target for attackers to compromise its security through malicious applications. To tackle this issue, domain experts and researchers are trying different techniques to stop such attacks. All the attempts of securing the Android platform are somewhat successful. However, existing detection techniques have severe shortcomings, including the cumbersome process of feature engineering. Designing representative features require expert domain knowledge. There is a need for minimizing human experts’ intervention by circumventing handcrafted feature engineering. Deep learning could be exploited by extracting deep features automatically. Previous work has shown that operational codes (opcodes) of executables provide key information to be used with deep learning models for the detection process of malicious applications. The only challenge is to feed opcodes information to deep learning models. Existing techniques use one-hot encoding to tackle the challenge. However, the one-hot encoding scheme has severe limitations. In this paper, we introduce (1) a novel technique for opcodes embedding, which we name Op2Vec, and (2) based on the learned Op2Vec, we have developed a dataset for end-to-end detection of Android malware. Introducing the end-to-end Android malware detection technique avoids expert-intensive handcrafted feature extraction and ensures automation. Some of the recent deep learning-based techniques showed significantly improved results when tested with the proposed approach and achieved an average detection accuracy of 97.47%, precision of 0.976, and F1 score of 0.979.

FeatureEnVi: Visual Analytics for Feature Engineering Using Stepwise Selection and Semi-Automatic Extraction Approaches.

The machine learning (ML) life cycle involves a series of iterative steps, from the effective gathering and preparation of the data-including complex feature engineering processes-to the presentation and improvement of results, with various algorithms to choose from in every step. Feature engineering in particular can be very beneficial for ML, leading to numerous improvements such as boosting the predictive results, decreasing computational times, reducing excessive noise, and increasing the transparency behind the decisions taken during the training. Despite that, while several visual analytics tools exist to monitor and control the different stages of the ML life cycle (especially those related to data and algorithms), feature engineering support remains inadequate. In this paper, we present FeatureEnVi, a visual analytics system specifically designed to assist with the feature engineering process. Our proposed system helps users to choose the most important feature, to transform the original features into powerful alternatives, and to experiment with different feature generation combinations. Additionally, data space slicing allows users to explore the impact of features on both local and global scales. FeatureEnVi utilizes multiple automatic feature selection techniques; furthermore, it visually guides users with statistical evidence about the influence of each feature (or subsets of features). The final outcome is the extraction of heavily engineered features, evaluated by multiple validation metrics. The usefulness and applicability of FeatureEnVi are demonstrated with two use cases and a case study. We also report feedback from interviews with two ML experts and a visualization researcher who assessed the effectiveness of our system.

Root cause analysis of COVID-19 cases by enhanced text mining process

<p>The main focus of this research is to find the reasons behind the fresh cases of COVID-19 from the public’s perception for data specific to India. The analysis is done using machine learning approaches and validating the inferences with medical professionals. The data processing and analysis is accomplished in three steps. First, the dimensionality of the vector space model (VSM) is reduced with improvised feature engineering (FE) process by using a weighted term frequency-inverse document frequency (TF-IDF) and forward scan trigrams (FST) followed by removal of weak features using feature hashing technique. In the second step, an enhanced K-means clustering algorithm is used for grouping, based on the public posts from Twitter®. In the last step, latent dirichlet allocation (LDA) is applied for discovering the trigram topics relevant to the reasons behind the increase of fresh COVID-19 cases. The enhanced K-means clustering improved Dunn index value by 18.11% when compared with the traditional K-means method. By incorporating improvised two-step FE process, LDA model improved by 14% in terms of coherence score and by 19% and 15% when compared with latent semantic analysis (LSA) and hierarchical dirichlet process (HDP) respectively thereby resulting in 14 root causes for spike in the disease.</p>

MPSAutodetect: A Malicious Powershell Script Detection Model Based on Stacked Denoising Auto-Encoder

PowerShell is an important tool used to automate administrative tasks. It is an open-source tool that is pre-installed on Windows machines and is accessible on many other operating systems. Administrators generally utilize PowerShell to carry out a range of typical management tasks, such as adding and deleting accounts, editing groups, and accessing hard-to-find user information. However, researchers recently found that PowerShell was used to execute various attacks. These attacks leveraged PowerShell’s vast number of properties to access privileged information, gain control of entire machines or spread over an organization. Because of the obfuscation and complex natures of these malicious scripts, detection is costly and difficult. Here, we present Malicious PowerShell Script Autodetect (MPSAutodetect), a detection model that relies on machine learning techniques to detect malicious PowerShell scripts. Our model was built with the use of stacked denoising auto-encoders (SdAs) to extract meaningful features. These valuable, easily attained features were fed to the eXtreme gradient boosting (XGBoost) classifier. Two substantial datasets (labelled and unlabelled) were collected to train and test MPSAutodetect in supervised and semi-supervised manners. The dataset contained malicious and benign obfuscated scripts. The results showed that regardless of the features extracted from the SdA, the supervised approach resulted in better detection, with a significant 98% true-positive rate and a low 0.6% false-positive rate. Thus, the analysis of MPSAutodetect showed that the model achieves respectable performance without the hassling process of manual feature engineering.

Knowledge-driven feature engineering to detect multiple symptoms using ambulatory blood pressure monitoring data

Background: Hypertension is a major health concern across the globe and needs to be properly diagnosed to so it can be treated and to mitigate for this critical health condition. In this context, ambulatory blood pressure monitoring is essential to provide for a proper diagnosis of hypertension, which may not be possible otherwise due to the white coat effect or masked hypertension. In this paper, the objective is to develop a model which incorporates expert’s knowledge in the feature engineering process so as to accurately predict multiple medical conditions. As a case study, we have considered multiple symptoms related to hypertension and used an ambulatory blood pressure monitoring method to continuously acquire hypertension relevant data from a patient. The goal is to train a model with a minimum set of the most effective knowledge-driven features which are useful to detect multiple symptoms simultaneously using multi-class classification techniques.Method: Artificial intelligence-based blood pressure monitoring techniques introduce a new dimension in the diagnosis of hypertension by enabling a continuous (24hours) analysis of systolic and diastolic blood pressure levels. In this work, we present a model that entails a knowledge-driven feature engineering method and implemented an ambulatory blood pressure monitoring system to diagnose multiple cardiac parameters and associated conditions simultaneously these include morning surge, circadian rhythm, and pulse pressure. The knowledge-driven features are extracted to improve the interpretability of the classification model and machine learning techniques (Random Forest, Naive Bayes, and KNN) were applied in a multi-label classification setup using RAkEL to classify multiple conditions simultaneously.Results: The results obtained (F 1 = 0.918) show that the Random forest technique has performed well for multilabel classification using knowledge-driven features. Our technique has also reduced the complexity of the model by reducing the number of features required to train a machine learning model.Conclusion: Considering these results, we conclude that knowledge-driven feature engineering enhances the learning process by reducing the number of features given as input to the machine learning algorithm. The proposed feature engineering method considers expert’s knowledge to develop better diagnosis models which are free from misleading data-driven noisy features in some situations. It is a white-box approach in which clinicians can under stand the importance of a feature while looking at its value.

AI Based Traffic Flow Prediction Model for Connected and Autonomous Electric Vehicles

There is a paradigm shift happening in automotive industry towards electric vehicles as environment and sustainability issues gained momentum in the recent years among potential users. Connected and Autonomous Electric Vehicle (CAEV) technologies are fascinating the automakers and inducing them to manufacture connected autonomous vehicles with self-driving features such as autopilot and self-parking. Therefore, Traffic Flow Prediction (TFP) is identified as a major issue in CAEV technologies which needs to be addressed with the help of Deep Learning (DL) techniques. In this view, the current research paper presents an artificial intelligence-based parallel autoencoder for TFP, abbreviated as AIPAE-TFP model in CAEV. The presented model involves two major processes namely, feature engineering and TFP. In feature engineering process, there are multiple stages involved such as feature construction, feature selection, and feature extraction. In addition to the above, a Support Vector Data Description (SVDD) model is also used in the filtration of anomaly points and smoothen the raw data. Finally, AIPAE model is applied to determine the predictive values of traffic flow. In order to illustrate the proficiency of the model’s predictive outcomes, a set of simulations was performed and the results were investigated under distinct aspects. The experimentation outcomes verified the effectual performance of the proposed AIPAE-TFP model over other methods.

Feature Engineering Process on Well Log Data for Machine Learning-Based Sagd Performance Prediction

Feature Engineering Process on Well Log Data for Machine Learning-Based Sagd Performance Prediction

A Lightweight Hybrid Intrusion Detection Framework using Machine Learning for Edge-Based IIoT Security

Due to the development of cloud computing and Internet of Things (IoT) environments, such as healthcare systems, telecommunications and Industry 4.0 or Industrial IoT (IIoT) many daily services are transformed. Therefore, Security issues become useful to better protect these novel technologies. IIoT security represents a real challenge for industry actors and academic research. A set of security approaches, such as intrusion detection are integrated to improve IIoT environments security. Hence, an Intrusion Detection System (IDS) aims to monitor, detect an intrusion in real time and then make reliable decisions. Many recent IDS incorporate Machine Learning (ML) techniques to improve their Accuracy (ACC), precision and Detection Rate (DR). This paper presents a hybrid IDS for Edge-Based IIoT Security using ML techniques. This new hybrid framework is based on misuse and anomaly detection using K-Nearest Neighbor (K-NN) and Principal Component Analysis (PCA) techniques. Specifically, the K-NN classifier has been incorporated to improve detection accuracy and make effective decision and the PCA is used for an enhanced feature engineering and training process. The obtained results have proven that our proposed Framework presents many advantages compared with other recent models. It gives good results with 99.10% ACC, 98.4% DR 2.7% False Alarm Rate (FAR) on NSL-KDD dataset and 98.2% ACC, 97.6% DR, 2.9% FAR on Bot-IoT dataset.

A Novel Capacity Estimation Method for Li-Ion Battery Cell by Applying Ensemble Learning to Extremely Sparse Significant Points

Accurate capacity estimation is important for safe operation of battery. Existing advanced researches heavily rely on feature engineering to model capacity degradation, where features are difficult to design and extract. In this paper, a novel purely data-driven capacity estimation method is proposed by applying ensemble learning to extremely sparse significant points on voltage and/or temperature curve. The significant points are just raw points evenly distributed throughout the charging/discharging process, or points corresponding to specific SoC, which is easy to extract without complicated feature engineering process on raw data. A novel ensemble learning framework incorporating light gradient boosting decision tree (LightGBM) and neural network is employed to find the regression relationship between significant points and battery capacity. Public battery dataset collected by Oxford university is used to verify the effectiveness of the proposed method. Results show that for Oxford dataset, the maximum and mean capacity estimation error could be controlled within 1.25% and 0.5% respectively, which is superior to most of existing capacity estimation methods. Moreover, robustness, generalization ability and model application are well discussed.

EEG based Drowsiness Prediction Using Machine Learning Approach

Drowsiness is the main cause of road accidents and it leads to severe physical injury, death, and significant economic losses. To monitor driver drowsiness various methods like Behaviour measures, Vehicle measures, Physiological measures and Hybrid measures have been used in previous research. This paper mainly focuses on physiological methods to predict the driver’s drowsiness. Several physiological methods are used to predict drowsiness. Among those methods, Electroencephalography is one of the non-invasive physiological methods to measure the brain activity of the subject. EEG brain signal extracted from the human scalp is analysed with various features and used for various health application like predicting drowsiness, fatigue etc. The main objective of the proposed system is to early predict the driver drowsiness with high accuracy so that we have divided our work into two steps. The first step is to collect the publicly available dataset of EEG based Eye state as (Eye open and Eye closed) where the signal acquisition process was done from Emotiv EEG Neuroheadset (14 electrodes) and analysed various feature engineering techniques and statistical techniques. The second step was applied with the machine learning classification model as K-NN and performance-based predicting models are used. In the Existing System, they used various machine learning classification models like K-NN and SVM for EEG Eye state classification and produced results around 80% -97%. Compared to the Existing system our proposed method produced better classification models for predicting driver drowsiness using different Feature engineering process and classification models as K-NN produced 98% of accuracy.

AFGSL: Automatic Feature Generation based on Graph Structure Learning

Feature engineering relies on domain knowledge and human intervention. To automate the process of feature engineering, automated feature construction methods use deep neural networks to capture feature interactions and attention coefficients to quantify the relationship between features. However, these methods ignore the influence of insignificant features that introduce noise and degrade the performance of the model. In this paper, we study the problem of feature interactions from the perspective of graph and propose a novel Automatic Feature Generation model based on Graph Structure Learning, called AFGSL. In this model, the adjacency matrix reflects the relationships between features, so that it can be used to filter out insignificant features. Furthermore, Q-learning is used to train the policy of stacking interaction layers, which enables it to make full use of both local and global information in the process of feature generation. The results of experiments on four real-world datasets show that AFGSL outperforms the state-of-the-art methods.

Precise Learning of Source Code Contextual Semantics via Hierarchical Dependence Structure and Graph Attention Networks

Deep learning is being used extensively in a variety of software engineering tasks, e.g., program classification and defect prediction. Although the technique eliminates the required process of feature engineering, the construction of source code model significantly affects the performance on those tasks. Most recent works was mainly focused on complementing AST-based source code models by introducing contextual dependencies extracted from CFG. However, all of them pay little attention to the representation of basic blocks, which are the basis of contextual dependencies.In this paper, we integrated AST and CFG and proposed a novel source code model embedded with hierarchical dependencies. Based on that, we also designed a neural network that depends on the graph attention mechanism. Specifically, we introduced the syntactic structural of the basic block, i.e., its corresponding AST, in source code model to provide sufficient information and fill the gap. We have evaluated this model on three practical software engineering tasks and compared it with other state-of-the-art methods. The results show that our model can significantly improve the performance. For example, compared to the best performing baseline, our model reduces the scale of parameters by 50% and achieves 4% improvement on accuracy on program classification task.

Novel architecture with selected feature vector for effective classification of mitotic and non-mitotic cells in breast cancer histology images

The paper focuses on the detection of mitosis in breast cancer. Detection methods in vogue rely heavily on visual inspection and assessment of histology images by trained pathologists, so the validation of results is a time-consuming process. Even accuracy wise, there remains room for improvement. This calls for high accuracy automated systems to replace manual assessments. Mitotic count is important because it provides us information about the aggressiveness of the tumour cells i.e. how rapidly it is spreading in the human body. To address this research gap, the current study is devoted to construct a novel framework utilizing neural network-based concepts along with reduced feature vectors and multiple machine learning techniques to classify the mitotic and non-mitotic cells. The dominant features in the detection of mitosis are texture features, therefore the texture of cells is used to derive the efficient reduced feature vectors which plays a significant role in identifying cancerous cells. During the feature engineering process, the features that distinguish a cancer cell from the normal cells were identified. Using these features as the base features, extraction with multiple techniques like GLCM, LBP and LTP along with classification algorithms like SVM, Naive Bayes and Random Forest were applied to the source dataset of the images, whose ground truth reference was available. Inspiring from Neural Networks, the proposed architecture allocates different weights to different features which increases the productivity of the model. The proposed framework is a new and intelligent version of ensemble learning which makes ensemble learning a more powerful tool for accurate and high-throughput results. The proposed method outperforms techniques already reported in the literature for the detection of mitotic cells in MITOS-12, AMIDA 13, MITOS-14 and TUPAC16.

Prediction of stock price direction using a hybrid GA-XGBoost algorithm with a three-stage feature engineering process

The stock market has performed one of the most important functions in a laissez-faire economic system by gathering people, companies, and flows of money for several centuries. There have been numerous studies on the stock market among researchers to predict stock prices, and a growing number of studies employed machine learning or deep learning techniques on the stock market predictions with the advent of big data and the rapid development of artificial intelligence techniques. However, making accurate predictions of stock price direction remains difficult because stock prices are inherently complex, nonlinear, nonstationary, and sometimes too irrational to be predictable. Despite the wealth of information, previous prediction systems often overlooked key indicators and the importance of feature engineering. This study proposes a hybrid GA-XGBoost prediction system with an enhanced feature engineering process consisting of feature set expansion, data preparation, and optimal feature set selection using the hybrid GA-XGBoost algorithm. This study experimentally verifies the importance of feature engineering process in stock price direction prediction by comparing obtained feature sets to original dataset as well as improving prediction performance to outperform benchmark models. Specifically, the most significant accuracy increment comes from feature expansion that adds 67 technical indicators to the original historical stock price data. This study also produces a parsimonious optimal feature set using the GA-XGBoost algorithm that can achieve the desired performance with substantially fewer features. Consequently, this study empirically proves that a successful prediction performance largely depends on a deliberate combination of feature engineering processes with a baseline learning model to make a good balance and harmony between the curse of dimensionality and the blessing of dimensionality.

Efficiently Classifying Lung Sounds through Depthwise Separable CNN Models with Fused STFT and MFCC Features.

Lung sounds remain vital in clinical diagnosis as they reveal associations with pulmonary pathologies. With COVID-19 spreading across the world, it has become more pressing for medical professionals to better leverage artificial intelligence for faster and more accurate lung auscultation. This research aims to propose a feature engineering process that extracts the dedicated features for the depthwise separable convolution neural network (DS-CNN) to classify lung sounds accurately and efficiently. We extracted a total of three features for the shrunk DS-CNN model: the short-time Fourier-transformed (STFT) feature, the Mel-frequency cepstrum coefficient (MFCC) feature, and the fused features of these two. We observed that while DS-CNN models trained on either the STFT or the MFCC feature achieved an accuracy of 82.27% and 73.02%, respectively, fusing both features led to a higher accuracy of 85.74%. In addition, our method achieved 16 times higher inference speed on an edge device and only 0.45% less accuracy than RespireNet. This finding indicates that the fusion of the STFT and MFCC features and DS-CNN would be a model design for lightweight edge devices to achieve accurate AI-aided detection of lung diseases.

Diagnosis and prognosis of mental disorders by means of EEG and deep learning: a systematic mapping study

Electroencephalography (EEG) is used in the diagnosis and prognosis of mental disorders because it provides brain biomarkers. However, only highly trained doctors can interpret EEG signals due to its complexity. Machine learning has been successfully trained with EEG signals for classifying mental disorders, but a time consuming and disorder-dependant feature engineering (FE) and subsampling process is required over raw EEG data. Deep Learning (DL) is positioned as a prominent research field to process EEG data because (i) it features automated FE by taking advantage of raw EEG signals improving results and (ii) it can be trained over the vast amount of data generated by EEG. In this work, a systematic mapping study has been performed with 46 carefully selected primary studies. Our goals were (i) to provide a clear view of which are the most prominent study topics in diagnosis and prognosis of mental disorders by using EEG with DL, and (ii) to give some recommendations for future works. Some results are: epilepsy was the predominant mental disorder present in around half of the studies, convolutional neural networks also appear in approximate 50% of the works. The main conclusions are (i) processing EEG with DL to detect mental disorders is a promising research field and (ii) to objectively compare performance between studies: public datasets, intra-subject validation, and standard metrics should be used. Additionally, we suggest to pay more attention to ease the reproducibility, and to use (when possible) an available framework to explain the results of the created DL models.