Input Feature Selection Research Articles

Performance evaluation of Indian banks using feature selection data envelopment analysis: A machine learning perspective

The early signal of the potential risk of bank failure is imperative for various stakeholders such as management personnel, lenders, and shareholders. This study has developed a new feature selection‐based data envelopment analysis (DEA) model to calculate the efficiencies and predict the stress of Indian banks. The feature selection‐based data envelopment analysis (DEA) model combines feature selection methodology (a machine learning technique), with the traditional data envelopment analysis. The proposed model is able to map the DEA ranks, and the status of both failure and success of banks, and other similar decision‐making units (DMUs). It also helps in solving the problems associated with the selection of appropriate input and output features as well as the time‐dependent data points which usually have a lagged effect. The proposed model is applicable only when appropriate samples of past data of DMUs performance are available, whereby it maps both the input and output features with performance. Importantly, while dealing with the past data for selecting the appropriate inputs and outputs, it is imperative to select all the indicators that affect the performance of DMUs and thereby reduce the number of features using the machine learning approach. The proposed model is one of its kind, integrating a machine learning technique to a nonparametric frontier decision model.

Evolutionary algorithm for automated formation of decision-making models for predicting the safety of opioid therapy

In this paper, an evolutionary algorithm for solving the problem of predicting the safety of opioid therapy for patients with pancreatic cancer is proposed. Opioid analgesics such as fentanyl and morphine are used as a therapy for pain syndromes. Using the patient database, based on the results of the therapy applied to them, it is determined whether there is a correlation between the outcome and the combination of input data taken into account. To find a set of informative features, it is proposed to use the genetic algorithm for multi-criterion optimization, in which two criteria are reduced to one generalized criterion using the method of “additive convolution”. The formed combination of the selected input features, which affects the outcome, is used to build a decision support model and to evaluate it afterwards.

BE-BLC: BERT-ELMO-Based Deep Neural Network Architecture for English Named Entity Recognition Task

One of the most important factors which considerably affects the quality of the neural sequence labeling model is the selection and encoding of input features to generate rich semantic and grammatical word representation vectors. Recently, pre-training text encoders have significantly ameliorated the performance on various Natural Language Processing (NLP) tasks. In this paper, we introduce a deep neural network model to address a challenging task of the sequence labeling problem, the task of named entity recognition. The model consists of three sub-networks to fully exploit current Google’s representations of the most powerful NLP transfer learning BERT and ELMO model, as well as the traditional state-of-the-art BiLSTM-CRF (BLC) model. The model outperforms the state-of-the-art systems on CoNLL-2003 by achieving a 95.56% F1-score and getting results comparable to the previous best-performing system models on OntoNotes 5.0 by achieving an 89.21% F1-score without using any external knowledge.

Modified Fuzzy-Q-Learning (MFQL)-Based Mechanical Fault Diagnosis for Direct-Drive Wind Turbines Using Electrical Signals

In this paper, a self-learning multi-class intelligent model for wind turbine fault diagnosis is proposed by using MFQL (Modified-Fuzzy-Q-Learning) technique. The MFQL is adaptive in nature and extension of fuzzy-Q-learning method where look-up table of Q-learning is conquered by fuzzy based approximation strategy to reduce the curse of dimensionality of the Q-learning. The proposed MFQL classifier diagnoses the mechanical and imbalance faults without using mechanical sensors. Proposed methodology is addressed with relying on PMSG (Permanent Magnet Synchronous Generator) stator current signals, which is already being used by protection system of wind turbines. According to the aforementioned description, non-stationary current signals of PMSG have been pre-processed to extract the input features by empirical mode decomposition followed with J48 algorithm based most relevant input feature selection. For the one-step ahead performance demonstration of the proposed MFQL approach, results have been compared with neural network, support vector machines, fuzzy logic, and conventional Fuzzy-Q-Learning techniques. Demonstrated results outperform the capability of proposed MFQL approach. Moreover, MFQL is developed first time to implement in the area of WTGS fault diagnosis in the literature.

Optainet-based technique for SVR feature selection and parameters optimization for software cost prediction

The software cost prediction is a crucial element for a project’s success because it helps the project managers to efficiently estimate the needed effort for any project. There exist in literature many machine learning methods like decision trees, artificial neural networks (ANN), and support vector regressors (SVR), etc. However, many studies confirm that accurate estimations greatly depend on hyperparameters optimization, and on the proper input feature selection that impacts highly the accuracy of software cost prediction models (SCPM). In this paper, we propose an enhanced model using SVR and the Optainet algorithm. The Optainet is used at the same time for 1-selecting the best set of features and 2-for tuning the parameters of the SVR model. The experimental evaluation was conducted using a 30% holdout over seven datasets. The performance of the suggested model is then compared to the tuned SVR model using Optainet without feature selection. The results were also compared to the Boruta and random forest features selection methods. The experiments show that for overall datasets, the Optainet-based method improves significantly the accuracy of the SVR model and it outperforms the random forest and Boruta feature selection methods.

Securely and Efficiently Outsourcing Decision Tree Inference

Outsourcing machine learning inference services to the cloud is getting increasingly popular. However, this also entails privacy risks to the provider's proprietary model and the client's sensitive data. Focusing on inference with decision trees, this article proposes a framework for securely and efficiently outsourcing decision tree inference. Targeting both privacy and efficiency, we propose a customized protocol using only lightweight cryptography in the online execution of secure inference. We resort to additive secret sharing and tackle the problems in various components including secure input feature selection, decision node evaluation, and inference result generation. Our protocol requires no interaction from the provider and client during online secure inference, a distinct advantage over prior works for practical deployment as they all operate under the client-provider setting where synchronous and continuous interaction is required. Performance evaluation demonstrates our security design's efficiency, as well as substantial performance benefits for the client (up to four orders of magnitude in computation and 163 times in communication), as opposed to prior art in the non-outsourcing setting. To facilitate the practical usage for meeting more service demands, we also investigate the extensions for secure outsourced inference of random forests and categorical feature-based decision trees.

Impact of Input Feature Selection on Groundwater Level Prediction From a Multi-Layer Perceptron Neural Network

With the growing use of machine learning (ML) techniques in hydrological applications, there is a need to analyze the robustness, performance, and reliability of predictions made with these ML models. In this paper we analyze the accuracy and variability of groundwater level predictions obtained from a Multilayer Perceptron (MLP) model with optimized hyperparameters for different amounts and types of available training data. The MLP model is trained on point observations of features like groundwater levels, temperature, precipitation, and river flow in various combinations, for different periods and temporal resolutions. We analyze the sensitivity of the MLP predictions at three different test locations in California, United States and derive recommendations for training features to obtain accurate predictions. We show that the use of all available features and data for training the MLP does not necessarily ensure the best predictive performance at all locations. More specifically, river flow and precipitation data are important training features for some, but not all locations. However, we find that predictions made with MLPs that are trained solely on temperature and historical groundwater level measurements as features, without additional hydrological information, are unreliable at all locations.

Reviewing trip purpose imputation in GPS-based travel surveys

Abstract The global positioning system (GPS) has motivated rapid advances in mobility data collection. A massive amount of spatio-temporal information has made it possible to know where a person was and when, but not how and why (s)he travelled, creating the need for inference models. Compared with mode detection, purpose imputation has been insufficiently studied. However, the relative lack of attention to purpose identification does not mean that this field has not emerged. For this paper, which is the first review dedicated to inferring trip purposes from GPS data, 1162 non-duplicate papers from four databases (Scopus, Web of Science, ScienceDirect and TRID) were screened, and a corpus of 25 publications was selected for examination. Based on these papers, the purpose imputation problem is defined in the contexts of the evolution of GPS-based travel surveys and two research domains, transportation science (TS) and human geography (HG). Subsequently, three steps of the purpose detection process, namely trip end detection, input feature selection and main algorithms and validation, are surveyed. During these procedures, the differences between studies in TS and those in HG are highlighted. Finally, unresolved issues related to data and feature selection, algorithms and assessment are discussed substantially to provide potential research directions. This review may be an informative reference for those newly accessing the GPS-based purpose imputation field and/or intending to develop solutions to this problem.

Input map and feature selection for soil legacy data

Techniques that disaggregate complex soil-terrain polygons from legacy maps are becoming more relevant, as cost effective highly detailed soil information is required to advise agriculture, hydrology, ecology, engineering, and a variety of other disciplines. Disaggregation involves the spatial placement of individual soil classes from soil legacy polygons which have multiple soil classes, while specifying the approximate proportion of each soil class and verbally or diagrammatically explaining their distribution in the landscape. One of the most common disaggregation approaches is known as DSMART (“Disaggregation and Harmonisation of Soil Map Units through Resampled Classification Trees”). However, DSMART is computationally intensive and has many parameters that must be optimised. This study aimed to address these drawbacks including input map selection, feature selection, and resample size optimisation. The research site was selected in the upper reaches of the Mvoti river catchment covering 317 km2 in KwaZulu Natal province, South Africa. The catchment consists of 20 soil-terrain polygons drawn at a 1:250,000 scale from the South African Land Type Survey (LTS). First, the optimal input map derived from landform elements (geomorphons) was selected through a spatially resampled Cramer’s V test to determine the association between the legacy polygons (proportion of terrain) and the geomorphon units. This was done for five different aggregated geomorphons with different parameters. Second, three feature selection algorithms (FSAs) were embedded into DSMART to determine if the algorithms could improve accuracy and computational efficiency. Third, the FSAs were compared using 25, 50, 100, and 200 resamples per polygon. The results indicate that the Cramer’s V test is a rapid method to determine the optimal input map. All FSAs achieved a significantly greater accuracy then when disaggregating the original legacy polygons and were more computationally efficient than when using all 52 covariates. This study has implications when disaggregating large and small datasets by improving computational efficiency while maintaining an acceptable accuracy.

Fuel properties of hydrochar and pyrochar: Prediction and exploration with machine learning

Conversion of wet organic wastes into renewable energy is a promising way to substitute fossil fuels and avoid environmental deterioration. Hydrothermal carbonization and pyrolysis can convert wet organic wastes into hydrochar and pyrochar, which are potential fossil fuel alternatives due to their comparable fuel properties. Machine learning (ML) has strong prediction ability after being trained with historic dataset and facilitates good understanding of the impact of input features on output targets through a data-driven approach. In this study, ML models for multi-task prediction of fuel properties of the chars were developed and optimized based on two datasets for hydrochar and pyrochar. Feature importance and correlation were explored based on optimized ML model, and feature re-examination was conducted for model improvement. Results showed that support vector regression model with optimal hyper-parameters exhibited better generalized performance for prediction of both hydrochar and pyrochar properties with the best average R2 of 0.90 and 0.94. ML-based feature analysis indicated that process temperature and carbon content in the feedstock were the significant features impacting fuel properties of both chars, while nitrogen content was another important input feature for hydrochar and hydrogen content for pyrochar. The accuracy (especially for pyrochar), generalization ability, and computational speed of models were further improved after feature re-examination. The intuitions obtained from feature analysis provided meaningful insights to select input features for prediction performance improvement and computational cost saving, and might guide experiments to produce chars with desired quality.

A priori analysis on deep learning of subgrid-scale parameterizations for Kraichnan turbulence

In the present study, we investigate different data-driven parameterizations for large eddy simulation of two-dimensional turbulence in the \emph{a priori} settings. These models utilize resolved flow field variables on the coarser grid to estimate the subgrid-scale stresses. We use data-driven closure models based on localized learning that employs multilayer feedforward artificial neural network (ANN) with point-to-point mapping and neighboring stencil data mapping, and convolutional neural network (CNN) fed by data snapshots of the whole domain. The performance of these data-driven closure models is measured through a probability density function and is compared with the dynamic Smagorinksy model (DSM). The quantitative performance is evaluated using the cross-correlation coefficient between the true and predicted stresses. We analyze different frameworks in terms of the amount of training data, selection of input and output features, their characteristics in modeling with accuracy, and training and deployment computational time. We also demonstrate computational gain that can be achieved using the intelligent eddy viscosity model that learns eddy viscosity computed by the DSM instead of subgrid-scale stresses. We detail the hyperparameters optimization of these models using the grid search algorithm.

Identification of new long-lived particles using deep neural networks

We present the development of a deep neural network for identifying generic displaced jets arising from the decays of exotic long-lived particles in data recorded by the CMS detector at the CERN LHC. Various jet features including detailed information about each clustered particle candidate as well as reconstructed secondary vertices are refined through the use of 1-dimensional convolution layers before being combined with high-level engineered features and passed through a series of fully-connected layers. The proper lifetime of the long-lived particle,cτ0, is treated as a parameter of the neural network model, which allows for hypothesis testing over several orders of magnitude ranging fromcτ0=1 µm to 10 m. Domain adaptation by backward propagation is performed to construct domain-independent features at an intermediate layer of the network to mitiage difference between simulation and data. The training is performed by streaming ROOT trees containingO(100M) jets directly into the TensorFlow queue system, which allows for a flexible selection of input features and asynchronous preprocessing. The application of the tagger is showcased in a search for long-lived gluinos as predicted by split supersymmetric models demonstrating significant gains in sensitivity over a reference analysis.

GIS Application in Mapping and Development of Trout Fisheries Resources along Yargyap Chu Drainage in Eastern Himalayas

A study was carried out by using Geo-informatics on the basic morphometry of the aquatic resources of Shi Yomi district of Arunachal Pradesh in the Eastern Himalayas with an objective to develop strategic plans for trout fisheries development along the drainage system of river Yargyapchuat Menchukha circle. The river Yargyapchu flows a distance of 53 km and have a stream frequency of fourth order. The upland lake resources accounts 6 in numbers and covers an area of 39.15ha. Eight categories of Land Use Land Cover (LULC) were classified for the Menchukha circle where the major percentage of land is covered by forest (80percent) followed by snow area (10percent), wasteland (9percent) and agricultural land (1percent). The slope class developed from Digital Elevation Model (DEM) shows 41percentof the total geographic area was within 0–21 degree slope, 50percent area in 21–35 degree slope and 9percent area within 36–78 degree slope. Furthermore, the elevation class up to 2000m MSL and above encompasses 76percent and 24percent of the total geographic area respectively. These selected input feature classes were superimposed in the spatial analyst ArcGIS v 10.7 to determine 27.84 ha in the slope range of 0–10 degreeas highly suitable and 22.34 ha in the slope range of 10–20 degree as moderately suitablefor trout fish farmingin the Menchukha circle of Shi Yomi district.

Forecasting Implied Volatility Smile Surface via Deep Learning and Attention Mechanism

The implied volatility smile surface is the basis of option pricing, and the dynamic evolution of the option volatility smile surface is difficult to predict. In this paper, attention mechanism is introduced into LSTM, and a volatility surface prediction method combining deep learning and attention mechanism is pioneeringly established. LSTM's forgetting gate makes it have strong generalization ability, and its feedback structure enables it to characterize the long memory of financial volatility. The application of attention mechanism in LSTM networks can significantly enhance the ability of LSTM networks to select input features. The experimental results show that the two strategies constructed using the predicted implied volatility surfaces have higher returns and Sharpe ratios than that the volatility surfaces are not predicted. This paper confirms that the use of AI to predict the implied volatility surface has theoretical and economic value. The research method provides a new reference for option pricing and strategy.

Long-term forest mapping from classification of MODIS time series: best practices

Coarse spatial resolution (CSR) time series have been successfully used at regional scale to produce homogeneous and up-to-date forest cover maps. This study aims to classify CSR time series using a nomenclature as detailed as national forest inventories nomenclatures. To identify best practices for classifying time series, three algorithms are compared: maximum likelihood, support vector machine, and random forest. For each algorithm, training, temporal compositing, and selection of input features have been optimized. Results establish a clear improvement in classification accuracy when red, near-infrared, and short-wave infrared spectral bands are used instead of vegetation indices. Temporal compositing has a major impact when the whole phenological cycle is used for 3 consecutive years. Random forest produces the best classification, support vector machine appears to be sensitive to overtraining, and maximum likelihood is unable to deal with the complex characteristics of forest and natural vegetation classes.

An ensemble system to predict the spatiotemporal distribution of energy security weaknesses in transmission networks

The security of energy supplies requires the depletion of potential fault events in power transmission networks. To achieve this, sufficient lead time before the happening of a fault event is indispensable for preparing countermeasures. With this inspiration, this paper establishes the fuzzy inference with rare association rule learning system. This ensemble system is designed for the long-term prediction of the spatiotemporal distribution of such energy security weaknesses, that is, to predict when and where these events are more expected to appear. In this system, merely the environmental features rather than the electrical features are needed as inputs. All the selected input features are divided into discrete and continuous features, and are evaluated separately. For the discrete features, the rare association rule learning model is implemented so that the rarely distributed environmental elements are extracted and diagnosed specifically. The risk indices of each element on the overall reliability are worked out as well. For the continuous features, a hierarchical fuzzy inference system along with the rare association rule learning model is deployed to calculate the corresponding risk indices of all the elements. In the hierarchical fuzzy inference system, the probabilistic fuzzy risks are employed instead of the direct fuzzy risks. Then the relative weights of these two sides are optimized. At last, an empirical case based on a practical transmission network is conducted, and the flexibility and the robustness of the proposed system during real applications can be validated consequently.

Predicting the quality of surgical exposure using spatial and procedural features from laparoscopic videos.

PURPOSE: Evaluating the quality of surgical procedures is a major concern in minimally invasive surgeries. We propose a bottom-up approach based on the study of Sleeve Gastrectomy procedures, for which we analyze what we assume to be an important indicator of the surgical expertise: the exposure of the surgical scene. We first aim at predicting this indicator with features extracted from the laparoscopic video feed, and second to analyze how the extracted features describing the surgical practice influence this indicator. METHOD: Twenty-nine patients underwent Sleeve Gastrectomy performed by two confirmed surgeons in a monocentric study. Features were extracted from spatial and procedural annotations of the videos, and an expert surgeon evaluated the quality of the surgical exposure at specific instants. The features were used as input of a classifier (linear discriminant analysis followed by a support vector machine) to predict the expertise indicator. Features selected in different configurations of the algorithm were compared to understand their relationships with the surgical exposure and the surgeon's practice. RESULTS: The optimized algorithm giving the best performance used spatial features as input ([Formula: see text]). It also predicted equally the two classes of the indicator, despite their strong imbalance. Analyzing the selection of input features in the algorithm allowed a comparison of different configurations of the algorithm and showed a link between the surgical exposure and the surgeon's practice. CONCLUSION: This preliminary study validates that a prediction of the surgical exposure from spatial features is possible. The analysis of the clusters of feature selected by the algorithm also shows encouraging results and potential clinical interpretations.

Improved supervised classification of bedrock in areas of transported overburden: Applying domain expertise at Kerkasha, Eritrea

A regional bedrock map provides a foundation from which to build geological interpretations. However, rapid and accurate bedrock mapping in an area that lacks outcrop is a common problem, especially in regions with sparse data. A historic bedrock map from an Au and base metal project in the Kerkasha district, Eritrea, is significantly improved by predicting bedrock distribution in areas previously mapped as transported overburden. Publicly-available remote sensing data (DTM and ASTER) were combined with airborne geophysical data (magnetics and radiometrics) to provide features for bedrock prediction. Remote sensing data were pre-processed using Principal Components Analysis (PCA) to yield an equal number of principal components (PC) as input features. Four iterations were trialled, using different combinations of remote sensing PC features. The two initial trials used all available remote sensing data but compared results when feature ranking and selection is applied to reduce the number of PCs used for training and classification. The subsequent two trials used subsets of available remote-sensing data, selected based on domain expertise (i.e., the domain-specific knowledge of a geologist), with all respective PCs were retained. Five-fold cross-validation scores were highest when a DTM, magnetics, and radiometrics data were included as input features. However, qualitative visual appraisal of predicted results across trials, complemented by maps of class membership uncertainty (using a measure of entropy), indicate that geologically-meaningful results are also produced when radiometrics are omitted and only the DTM and magnetics are used. The study concludes with a generalised workflow to assist geologists who are seeking to improve the bedrock interpretation of areas under cover in a single area of interest. Domain expertise is shown to be critical for the selection of appropriate input features and validation of results during predictive lithologic mapping.

Arrhythmia Classification with Single Beat ECG Evaluation and Support Vector Machine

Abnormal electrical activity of the human heart indicates cardiac dysfunction. The Electrocardiogram (ECG) is one of the non-invasive diagnostic techniques to detect cardiac abnormalities. Irregularity and non-stationarity in the ECG signal impose difficulties to clinicians for accurate diagnosis of heart diseases only by visual inspection. Automatic recognition of abnormal ECG beats aids in early detection of heart diseases. This paper explores the ECG single beat analysis to identify the cardiac abnormality. In this work, seven classes of arrhythmia are considered as recommended by AAMI(Association for the Advancement of Medical Instrumentation) standard. Beat feature database is generated from 44 recordings of the MIT-BIH arrhythmia database to support the arrhythmia classification. Classification is implemented with Multiclass Support Vector Machine (SVM) for non-linearly separable data effectively. Classification accuracy up to 93% is achieved for the selected input feature sets. This work assesses the suitability of the ECG input features for multi-class classification of arrhythmia.

Cost Forecasting Model of Transformer Substation Projects Based on Data Inconsistency Rate and Modified Deep Convolutional Neural Network

Precise and steady substation project cost forecasting is of great significance to guarantee the economic construction and valid administration of electric power engineering. This paper develops a novel hybrid approach for cost forecasting based on a data inconsistency rate (DIR), a modified fruit fly optimization algorithm (MFOA) and a deep convolutional neural network (DCNN). Firstly, the DIR integrated with the MFOA is adopted for input feature selection. Simultaneously, the MFOA is utilized to realize parameter optimization in the DCNN. The effectiveness of the MFOA–DIR–DCNN has been validated by a case study that selects 128 substation projects in different regions for training and testing. The modeling results demonstrate that this established approach is better than the contrast methods with regard to forecasting accuracy and robustness. Thus, the developed technique is feasible for the cost prediction of substation projects in various voltage levels.