Sequential Forward Feature Selection Algorithm Research Articles

Wind power plants hybridised with solar power: A generation forecast perspective

The association of different variable renewable technologies in hybrid power plants and the benefits of their aggregation for the operation of power systems is an area of recent research. Accurate forecasts are crucial for extracting those benefits and promote an optimal integration of such plants into power systems and electricity markets. This study focuses on the hybridisation of existing wind power plants with different shares of solar photovoltaic capacity and investigates how these power plants can reduce their combined forecast errors and thus, increasing profitability in electricity markets.The work uses a forecast methodology based on a sequential forward feature selection algorithm which employs two different objective functions and an artificial neural network approach previously presented but, in this case, it is applied to the specific case of hybrid power plants. The methodology uses as input data from a numerical weather prediction model and iteratively selects meteorological features to achieve the different objective functions implemented, namely i) minimisation of the root mean square error; or ii) maximisation of the market remuneration.The methodology developed was applied to three case studies in Portugal with different levels of wind and solar generation complementarity. The results show that the hybrid power plants can increase market value by up to 5% and total remuneration can increase by up to 30% when compared with the existing wind power plant, while it is possible to reduce the forecast errors by nearly 4%. The obtained results highlight the need to select the most relevant meteorological features to maximise the accuracy of the power forecast and the renewable power producers revenues in a market environment.

Enhancing wind power forecast accuracy using the weather research and forecasting numerical model-based features and artificial neuronal networks

Forecasting with accuracy the quantity of energy produced by wind power plants is crucial to enabling its optimal integration into power systems and electricity markets. Despite the remarkable improvements in the wind forecasting systems in recent years, large errors can still be observed, especially for longer time horizons. This work focuses on identifying new numerical weather prediction (NWP)-based features aiming to improve the overall quality of wind power forecasts.The methodology also incorporates a sequential forward feature selection algorithm. This algorithm was designed to select iteratively the meteorological features which minimize the wind forecast errors.The methodology was applied separately to seven wind parks in Portugal with different climate characteristics. The proposed approach allowed a reduction between 13% and 37% in the root mean square errors of wind power forecasts, compared with a baseline scenario. While the meteorological features identified for each wind park showed similarities within regions with analogous wind power generation profiles, each wind park required specific meteorological parameters as input data to obtain the best performance. Thus, the results show to be crucial to select the most relevant features of a specific site to maximize the accuracy of a wind power forecast.

Load2Load: Day-ahead load forecasting at aggregated level

A reliable and accurate short-term load forecasting (STLF) helps utilities and energy providers deal with the challenges posed by supply and demand balance, higher penetration of renewable energies and the development of electricity markets with increasingly complex pricing strategies in future smart grids. Recent advances in deep learning have been successively utilized to STLF. However, there is no certain study that evaluates the performances of different STLF methods at an aggregated level on different datasets with different numbers of daily measurements.In this study, a deep learning STLF architecture called Load2Load is proposed for day-ahead forecasting. Different forecasting methods have been evaluated and compared on two datasets with different temporal resolutions and features. An additive ensemble method as well as a selective ensemble method that selects the outputs of different forecasters in an hourly manner are proposed. Moreover; a modified sequential forward feature selection algorithm is proposed, resulting in better performance with a much smaller number of features.Numerical results show that the proposed Load2Load architecture has a competing performance compared to other advanced forecasters. When used together with the proposed ensemble methods, Load2Load can significantly improve the forecasting performance. The proposed feature selection algorithm results in better performance for the majority of the cases while reducing the dimensionality. According to the results with two different datasets; the proposed methods are shown to be robust to temporal resolutions, feature types and sequence lengths.

Computational prediction of Calu-3-based in vitro pulmonary permeability of chemicals

Pulmonary is a potential route for drug delivery and exposure to toxic chemicals. The human bronchial epithelial cell line Calu-3 is generally considered to be a useful in vitro model of pulmonary permeability by calculating the apparent permeability coefficient (Papp) values. Since in vitro experiments are time-consuming and labor-intensive, computational models for pulmonary permeability are desirable for accelerating drug design and toxic chemical assessment. This study presents the first attempt for developing quantitative structure-activity relationship (QSAR) models for addressing this goal. A total of 57 chemicals with Papp values based on Calu-3 experiments was first curated from literature for model development and testing. Subsequently, eleven descriptors were identified by a sequential forward feature selection algorithm to maximize the cross-validation performance of a voting regression model integrating linear regression and nonlinear random forest algorithms. With applicability domain adjustment, the developed model achieved high performance with correlation coefficient values of 0.935 and 0.824 for cross-validation and independent test, respectively. The preliminary results showed that computational models could be helpful for predicting Calu-3-based in vitro pulmonary permeability of chemicals. Future works include the collection of more data for further validating and improving the model.

A Data-Driven Short-Term PV Generation and Load Forecasting Approach for Microgrid Applications

The data-driven (DD) is a systematic approach to improve the data and model by deriving/adding features to address the problem identified during the iterative loop of forecasting model development. This article proposes a DD framework for forecasting short-term PV generation and load demand. A framework of three stages with a unique contribution in each stage, such as generalizing data preprocessing steps (stage-1), multivariate feature generation and selection (stage-2), and model hyperparameter tuning (stage-3) for further improvement in forecasting is proposed. It focuses on data as well as forecasting models. The whole process is analyzed using the time-series measured data collected from a real-life demonstration project in Ireland. Data preprocessing is generalized for both generation and demand forecasting under the same framework. The relevant features are selected with the help of the proposed random forest sequential forward feature selection algorithm. Hyperparameters are tuned through tree-structured Parzen estimator algorithm for further improvement. In addition, the performance of the classical autoregressive integrated moving average model is compared with the machine learning-based gate recurrent unit, long short term memory, recurrent neural network, and convolutional neural network models. Results show that the data-driven forecasting model framework systematically improves the model performance. The seasonal variation has also a high impact on the model performances.

Interpretable Machine Learning Models for Malicious Domains Detection Using Explainable Artificial Intelligence (XAI)

With the expansion of the internet, a major threat has emerged involving the spread of malicious domains intended by attackers to perform illegal activities aiming to target governments, violating privacy of organizations, and even manipulating everyday users. Therefore, detecting these harmful domains is necessary to combat the growing network attacks. Machine Learning (ML) models have shown significant outcomes towards the detection of malicious domains. However, the “black box” nature of the complex ML models obstructs their wide-ranging acceptance in some of the fields. The emergence of Explainable Artificial Intelligence (XAI) has successfully incorporated the interpretability and explicability in the complex models. Furthermore, the post hoc XAI model has enabled the interpretability without affecting the performance of the models. This study aimed to propose an Explainable Artificial Intelligence (XAI) model to detect malicious domains on a recent dataset containing 45,000 samples of malicious and non-malicious domains. In the current study, initially several interpretable ML models, such as Decision Tree (DT) and Naïve Bayes (NB), and black box ensemble models, such as Random Forest (RF), Extreme Gradient Boosting (XGB), AdaBoost (AB), and Cat Boost (CB) algorithms, were implemented and found that XGB outperformed the other classifiers. Furthermore, the post hoc XAI global surrogate model (Shapley additive explanations) and local surrogate LIME were used to generate the explanation of the XGB prediction. Two sets of experiments were performed; initially the model was executed using a preprocessed dataset and later with selected features using the Sequential Forward Feature selection algorithm. The results demonstrate that ML algorithms were able to distinguish benign and malicious domains with overall accuracy ranging from 0.8479 to 0.9856. The ensemble classifier XGB achieved the highest result, with an AUC and accuracy of 0.9991 and 0.9856, respectively, before the feature selection algorithm, while there was an AUC of 0.999 and accuracy of 0.9818 after the feature selection algorithm. The proposed model outperformed the benchmark study.

Integration of Healthcare domain Ontologies using Bayesian Networks

Semantic Web (SW) was created with the vision of knowledge sharing. Knowledge from the past and present help predict the future with the use of Machine Learning (ML) algorithms. SW powered with ontologies help in realizing machine interactions supporting automated knowledge extraction. Healthcare as a field of medical domain gives lot of importance for timely accurate decisions with the available features. Representing existing information in terms of ontologies, retrieving the decisions upon establishing interaction between the relevant ontologies within the same domain, knowledge sharing & reusing the existing facts are of great benefit to the medical practitioners and researchers which has lot of open challenges to be resolved in order to realize the same. To address the stated issues, an algorithmic approach – Ontologies Integration algorithm using Bayesian Networks (OIBN) based on Bayesian Belief Networks (BBN) working on Naïve beliefs has been proposed which works on symptoms through the attributes of related ontologies within the same domain exploring the symptom dependencies and their probability of occurrences in combination. Selection of features for integration will follow the steps proposed in Sequential Forward Feature Selection algorithm (SFFS). The observation on the correctness of the presented method over diabetic datasets represented in ontological form with integration of relevant features reveals that the knowledge graphs have been efficiently explored discovering the facts based on the probability theory. The experimental results conclude that the proposed technique is showing enhanced prediction accuracy of 80.95% which is better compared to accuracies of the individual ontologies prior to integration and existing state-of-art technique.

Automatic Detection of Obstructive Sleep Apnea Using Wavelet Transform and Entropy-Based Features From Single-Lead ECG Signal

Obstructive sleep apnea (OSA) is a prevalent sleep disorder and highly affects the quality of human life. Currently, gold standard for OSA detection is polysomnogram. Since this method is time consuming and cost inefficient, practical systems focus on the usage of electrocardiogram (ECG) signals for OSA detection. In this paper, a novel automatic OSA detection method using a single-lead ECG signal has been proposed. A nonlinear feature extraction using wavelet transform (WT) coefficients obtained by an ECG signal decomposition is employed. In addition, different classification methods are investigated. ECG signals are decomposed into eight levels using a Symlet function as a mother Wavelet function with third order. Then, the entropy-based features including fuzzy/approximate/sample/correct conditional entropy as well as other nonlinear features including interquartile range, mean absolute deviation, variance, Poincare plot, and recurrence plot are extracted from WT coefficients. The best features are chosen using the automatic sequential forward feature selection algorithm. In order to assess the introduced method, 95 single-lead ECG recordings are used. The support vector machine classifier having a radial basis function kernel leads to an accuracy of 94.63% (sensitivity: 94.43% and specificity: 94.77%) and 95.71% (sensitivity: 95.83% and specificity: 95.66%) for minute-by-minute and subject-by-subject classifications, respectively. The results show that applying entropy-based features for extracting hidden information of the ECG signals outperforms other available automatic OSA detection methods. The results indicate that a highly accurate OSA detection is attained by just exploiting the single-lead ECG signals. Furthermore, due to the low computational load in the proposed method, it can easily be applied to the home monitoring systems.

Discrete wavelength selection for the optical readout of a metamaterial biosensing system for glucose concentration estimation via a support vector regression model.

In this contribution, a method to select discrete wavelengths that allow an accurate estimation of the glucose concentration in a biosensing system based on metamaterials is presented. The sensing concept is adapted to the particular application of ophthalmic glucose sensing by covering the metamaterial with a glucose-sensitive hydrogel and the sensor readout is performed optically. Due to the fact that in a mobile context a spectrometer is not suitable, few discrete wavelengths must be selected to estimate the glucose concentration. The developed selection methods are based on nonlinear support vector regression (SVR) models. Two selection methods are compared and it is shown that wavelengths selected by a sequential forward feature selection algorithm achieves an estimation improvement. The presented method can be easily applied to different metamaterial layouts and hydrogel configurations.

The Emotion Recognition System Based on Autoregressive Model and Sequential Forward Feature Selection of Electroencephalogram Signals

Electroencephalogram (EEG) is one of the useful biological signals to distinguish different brain diseases and mental states. In recent years, detecting different emotional states from biological signals has been merged more attention by researchers and several feature extraction methods and classifiers are suggested to recognize emotions from EEG signals. In this research, we introduce an emotion recognition system using autoregressive (AR) model, sequential forward feature selection (SFS) and K-nearest neighbor (KNN) classifier using EEG signals during emotional audio-visual inductions. The main purpose of this paper is to investigate the performance of AR features in the classification of emotional states. To achieve this goal, a distinguished AR method (Burg's method) based on Levinson-Durbin's recursive algorithm is used and AR coefficients are extracted as feature vectors. In the next step, two different feature selection methods based on SFS algorithm and Davies-Bouldin index are used in order to decrease the complexity of computing and redundancy of features; then, three different classifiers include KNN, quadratic discriminant analysis and linear discriminant analysis are used to discriminate two and three different classes of valence and arousal levels. The proposed method is evaluated with EEG signals of available database for emotion analysis using physiological signals, which are recorded from 32 participants during 40 1 min audio visual inductions. According to the results, AR features are efficient to recognize emotional states from EEG signals, and KNN performs better than two other classifiers in discriminating of both two and three valence/arousal classes. The results also show that SFS method improves accuracies by almost 10-15% as compared to Davies-Bouldin based feature selection. The best accuracies are %72.33 and %74.20 for two classes of valence and arousal and %61.10 and %65.16 for three classes, respectively.

Identification of Biomarkers for Esophageal Squamous Cell Carcinoma Using Feature Selection and Decision Tree Methods

Esophageal squamous cell cancer (ESCC) is one of the most common fatal human cancers. The identification of biomarkers for early detection could be a promising strategy to decrease mortality. Previous studies utilized microarray techniques to identify more than one hundred genes; however, it is desirable to identify a small set of biomarkers for clinical use. This study proposes a sequential forward feature selection algorithm to design decision tree models for discriminating ESCC from normal tissues. Two potential biomarkers of RUVBL1 and CNIH were identified and validated based on two public available microarray datasets. To test the discrimination ability of the two biomarkers, 17 pairs of expression profiles of ESCC and normal tissues from Taiwanese male patients were measured by using microarray techniques. The classification accuracies of the two biomarkers in all three datasets were higher than 90%. Interpretable decision tree models were constructed to analyze expression patterns of the two biomarkers. RUVBL1 was consistently overexpressed in all three datasets, although we found inconsistent CNIH expression possibly affected by the diverse major risk factors for ESCC across different areas.

Hybrid support vector regression – Particle swarm optimization for prediction of compressive strength and RCPT of concretes containing metakaolin

This paper develops a hybrid support vector regression (SVR) – particle swarm optimization (PSO) model to predict the compressive strength (CS) and rapid chloride penetration test (RCPT) results of concretes containing metakaolin. The predictive accuracy of SVR models is highly dependent on their learning parameters. Therefore, PSO is exploited to seek the optimal hyper-parameters for SVR in order to improve its generalization capability. Moreover, a SVR-based sequential forward feature selection algorithm is proposed to disclose the most dominant input variables for the prediction of CS and RCPT results. The performance of the hybrid model is compared with the well-known system modeling method of adaptive neural-fuzzy inference system (ANFIS) by using 100 data samples with 25 different mix proportions established by experiments. The results show that the hybrid model has strong potential to predict material properties with high degree of accuracy and robustness.

Intelligent Identification of Childhood Musical Murmurs

Heart murmurs are often the first signs of heart valvular disorders. However, most heart murmurs detected in children are innocent musical murmurs (also called Still′s murmurs), which should be distinguished from other murmur types that are mostly pathological, such as regurgitant, obstructive, and flow murmurs. In order to reduce both unnecessary healthcare expenditures and parental anxiety, this study aims to develop algorithms for intelligently identifying musical murmurs in children. Discrete wavelet transform was applied to phonocardiographic signals to extract features. Singular value decomposition was applied on the matrix derived from continuous wavelet transform to extract extra features. The sequential forward feature selection algorithm was then utilized to select significant features. Musical murmurs were subsequently differentiated via a classification procedure consisting of three classification techniques: discriminant analysis, support vector machine, and artificial neural network. The results of 89.02% sensitivity, 84.76% specificity and 87.36% classification accuracy were achieved.

Multifeature Prostate Cancer Diagnosis and Gleason Grading of Histological Images

We present a study of image features for cancer diagnosis and Gleason grading of the histological images of prostate. In diagnosis, the tissue image is classified into the tumor and nontumor classes. In Gleason grading, which characterizes tumor aggressiveness, the image is classified as containing a low- or high-grade tumor. The image sets used in this paper consisted of 367 and 268 color images for the diagnosis and Gleason grading problems, respectively, and were captured from representative areas of hematoxylin and eosin-stained tissue retrieved from tissue microarray cores or whole sections. The primary contribution of this paper is to aggregate color, texture, and morphometric cues at the global and histological object levels for classification. Features representing different visual cues were combined in a supervised learning framework. We compared the performance of Gaussian, k-nearest neighbor, and support vector machine classifiers together with the sequential forward feature selection algorithm. On diagnosis, using a five-fold cross-validation estimate, an accuracy of 96.7% was obtained. On Gleason grading, the achieved accuracy of classification into low- and high-grade classes was 81.0%.