Forward Feature Selection Method Research Articles

Comparative Analysis of Classification Methods in Sentiment Analysis: The Impact of Feature Selection and Ensemble Techniques Optimization

Optimizing classification methods (forward selection, backward elimination, and optimized selection) and ensemble techniques (AdaBoost and Bagging) are essential for accurate sentiment analysis, particularly in political contexts on social media. This research compares advanced classification models with standard ones (Decision Tree, Random Tree, Naive Bayes, Random Forest, K- NN, Neural Network, and Generalized Linear Model), analyzing 1,200 tweets from December 10-11, 2023, focusing on "Indonesia" and "capres." It encompasses 490 positive, 355 negative, and 353 neutral sentiments, reflecting diverse opinions on presidential candidates and political issues. The enhanced model achieves 96.37% accuracy, with the backward selection model reaching 100% accuracy for negative sentiments. The study suggests further exploration of hybrid feature selection and improved classifiers for high-stakes sentiment analysis. With forward feature selection and ensemble method, Naive Bayes stands out for classifying negative sentiments while maintaining high overall accuracy (96.37%).

Deep Learning Based Hybrid Intrusion Detection Systems to Protect Satellite Networks

Despite the fact that satellite-terrestrial systems have advantages such as high throughput, low latency, and low energy consumption, as well as low exposure to physical threats and natural disasters and cost-effective global coverage, their integration exposes both of them to particular security challenges that can arise due to the migration of security challenges from one to another. Intrusion Detection Systems (IDS) can also be used to provide a high level of protection for modern network environments such as satellite-terrestrial integrated networks (STINs). To optimize the detection performance of malicious activities in network traffic, four hybrid intrusion detection systems for satellite-terrestrial communication systems (SAT-IDSs) are proposed in this paper. All the proposed systems exploit the sequential forward feature selection (SFS) method based on random forest (RF) to select important features from the dataset that increase relevance and reduce complexity and then combine them with a machine learning (ML) or deep learning (DL) model; Random Forest (RF), Long Short-Term memory (LSTM), Artificial Neural Networks (ANN), and Gated Recurrent Unit (GRU). Two datasets—STIN, which simulates satellite networks, and UNSW-NB15, which simulates terrestrial networks—were used to evaluate the performance of the proposed SAT-IDSs. The experimental results indicate that selecting significant and crucial features produced by RF-SFS vastly improves detection accuracy and computational efficiency. In the first dataset (STIN), the proposed hybrid ML system SFS-RF achieved an accuracy of 90.5% after using 10 selected features, compared to 85.41% when using the whole dataset. Furthermore, the RF-SFS-GRU model achieved the highest performance of the three proposed hybrid DL-based SAT-IDS with an accuracy of 87% after using 10 selected features, compared to 79% when using the entire dataset. In the second dataset (UNSW-NB15), the proposed hybrid ML system SFS-RF achieved an accuracy of 78.52% after using 10 selected features, compared to 75.4% when using the whole dataset. The model with the highest accuracy of the three proposed hybrid DL-based SAT-IDS was the RF-SFS-GRU model. It achieved an accuracy of 79% after using 10 selected features, compared to 74% when using the whole dataset.

Performance Analysis and Assessment of Type 2 Diabetes Screening Scores in Patients with Non-Alcoholic Fatty Liver Disease

Type 2 diabetes (T2D) and non-alcoholic fatty liver disease (NAFLD) are worldwide chronic diseases that have strong relationships with one another and commonly exist together. Type 2 diabetes is considered one of the risk factors for NAFLD, so its occurrence in people with NAFLD is highly likely. As the high and increasing number of T2D and NAFLD, which potentially followed by existing together number, an analysis and assessment of T2D screening scores in people with NAFLD is necessary to be done. To prevent this potential case, an effective early prediction model is also required to be developed, which could help the patients avoid the dangers of both existing diseases. Therefore, in this study, analysis and assessment of T2D screening scores in people with NAFLD and the early prediction model utilizing a forward logistic regression-based feature selection method and multi-layer perceptrons are proposed. Our analysis and assessment results showed that the prevalence of T2D among patients with NAFLD was 8.13% (for prediabetes) and 37.19% (for diabetes) in two population-based NAFLD datasets. The variables related to clinical tests, such as alanine aminotransferase (ALT), aspartate aminotransferase (AST), alkaline phosphatase (ALP), gamma-glutamyl transferase (GGT), and systolic blood pressure (SBP), were found to be statistically significant predictors (p-values < 0.001) that indicate a strong association with T2D among patients with NAFLD in both the prediabetes and diabetes NAFLD datasets. Finally, our proposed model showed the best performance in terms of all performance evaluation metrics compared to existing various machine learning models and also the models using variables recommended by WHO/CDC/ADA, with achieved accuracy as much as 92.11% and 83.05% and its improvement scores after feature selection of 1.35% and 5.35%, for the first and second dataset, respectively.

Diagnosis of Alzheimer’s Disease using Wrapper Feature Selection Method

Alzheimer’s disease (AD) symptoms are being treated by early diagnosis, where we can only slow the symptoms and research is still undergoing. In consideration, using T1-weighted images several classification models are proposed in Machine learning to identify AD. In this paper, we consider the improvised feature selection, to reduce the complexity by using wrapping techniques and Restricted Boltzmann Machine (RBM). This present work used the subcortical and cortical features of 278 subjects from the ADNI dataset to identify AD and sMRI. Multi-class classification is used for the experiment i.e., AD, EMCI, LMCI, HC. The proposed feature selection consists of Forward feature selection, Backward feature selection, and Combined PCA & RBM. Forward and backward feature selection methods use an iterative method starting being no features in the forward feature selection and backward feature selection with all features included in the technique. PCA is used to reduce the dimensions and RBM is used to select the best feature without interpreting the features. We have compared the three models with PCA to analysis. The following experiment shows that combined PCA &RBM, and backward feature selection give the best accuracy with respective classification model RF i.e., 88.65, 88.56% respectively.

Prediction of antioxidant peptides using a quantitative structure−activity relationship predictor (AnOxPP) based on bidirectional long short-term memory neural network and interpretable amino acid descriptors

Antioxidant peptides can protect against free radical-mediated diseases, especially food-derived antioxidant peptides are considered as potential competitors among synthetic antioxidants due to their safety, high activity and abundant sources. However, wet experimental methods can not meet the need for effectively screening and clearly elucidating the structure-activity relationship of antioxidant peptides. Therefore, it is particularly important to build a reliable prediction platform for antioxidant peptides. In this work, we developed a platform, AnOxPP, for prediction of antioxidant peptides using the bidirectional long short-term memory (BiLSTM) neural network. The sequence characteristics of peptides were converted into feature codes based on amino acid descriptors (AADs). Our results showed that the feature conversion ability of the combined-AADs optimized by the forward feature selection method was more accurate than that of the single-AADs. Especially, the model trained by the optimal descriptor SDPZ27 significantly outperformed the existing predictor on two independent test sets (Accuracy = 0.967 and 0.819, respectively). The SDPZ27-based AnOxPP learned four key structure-activity features of antioxidant peptides, with the following importance as steric properties > hydrophobic properties > electronic properties > hydrogen bond contributions. AnOxPP is a valuable tool for screening and design of peptide drugs, and the web-server is accessible at http://www.cqudfbp.net/AnOxPP/index.jsp.

Deep Learning Approach with LSTM for Daily Streamflow Prediction in a Semi-Arid Area: A Case Study of Oum Er-Rbia River Basin, Morocco

Daily hydrological modelling is among the most challenging tasks in water resource management, particularly in terms of streamflow prediction in semi-arid areas. Various methods were applied in order to deal with this complex phenomenon, but recently data-driven models have taken a better space, given their ability to solve prediction problems in time series. In this study, we have employed the Long Short-Term Memory (LSTM) network to simulate the daily streamflow over the Ait Ouchene watershed (AIO) in the Oum Er-Rbia river basin in Morocco, based on a temporal sequence of in situ and remotely sensed hydroclimatic data ranging from 2001 to 2010. The analysis adopted in this work is based on three-dimension input required by the LSTM model (1); the input samples used three splitting approaches: 70% of the dataset as training, splitting the data considering the hydrological year and the cross-validation method; (2) the sequence length; (3) and the input features using two different scenarios. The prediction results demonstrate that the LSTM performs poorly using the default data input scenario, whereas the best results during the testing were found in a sequence length of 30 days using approach 3 (R2 = 0.58). In addition, the LSTM fed with the lagged data input scenario using the Forward Feature Selection (FFS) method provides high performance accuracy using approach 2 (R2 = 0.84) in a sequence length of 20 days. Eventually, in applications related to water resources management where data are limited, the use of the deep learning technique is able to create high predictive accuracy, which can be enhanced with the right combination subset of features by using FFS.

A machine learning-based nano-photocatalyst module for accelerating the design of Bi2WO6/MIL-53(Al) nanocomposites with enhanced photocatalytic activity.

It is a great challenge to acquire novel Bi2WO6/MIL-53(Al) (BWO/MIL) nanocomposites with excellent catalytic activity by the trial-and-error method in the vast untapped synthetic space. The degradation rate of Rhodamine B dye (DRRhB) can be used as the main parameter to evaluate the catalytic activity of BWO/MIL nanocomposites. In this work, a machine learning-based nano-photocatalyst module was developed to speed up the design of BWO/MIL with desirable performance. Firstly, the DRRhB dataset was constructed, and four key features related to the synthetic conditions of BWO/MIL were filtered by the forward feature selection method based on support vector regression (SVR). Secondly, the SVR model with radical basis function for predicting the DRRhB of BWO/MIL was established with the key features and optimal hyperparameters. The correlation coefficients (R) between predicted and experimental DRRhB were 0.823 and 0.884 for leave-one-out cross-validation (LOOCV) and the external test, respectively. Thirdly, potential BWO/MIL nanocomposites with higher DRRhB were discovered by inverse projection, the prediction model, and virtual screening from the synthesis space. Meanwhile, an online web service (http://1.14.49.110/online_predict/BWO2) was built to share the model for predicting the DRRhB of BWO/MIL. Moreover, sensitivity analysis was brought into boosting the model's explainability and illustrating how the DRRhB of BWO/MIL changes over the four key features, respectively. The method mentioned here can provide valuable clues to develop new nanocomposites with the desired properties and accelerate the design of nano-photocatalysts.

Predicting perinatal mortality based on maternal health status and health insurance service using homogeneous ensemble machine learning methods

BackgroundPerinatal mortality in Ethiopia is the highest in Africa, with 68 per 1000 pregnancies intrapartum deaths. It is mainly associated with home delivery, which contributes to more than 75% of perinatal deaths. Financial constraints significantly impact timely access to maternal health care. Financial incentives, such as health insurance, may address the demand- and supply-side factors. This study, hence, aims to predict perinatal mortality based on maternal health status and health insurance service using homogeneous ensemble machine learning methods.MethodsThe data was collected from the Ethiopian demographic health survey from 2011 to 2019 G.C. The data were pre-processed to get quality data that are suitable for the homogenous ensemble machine-learning algorithms to develop a model that predicts perinatal mortality. We have applied filter (chi-square and mutual information) and wrapper (sequential forward and sequential backward) feature selection methods. After selecting all the relevant features, we developed a predictive model using cat boost, random forest, and gradient boosting algorithms and evaluated the model using both objective (accuracy, precision, recall, F1_score, ROC) and subjective (domain expert) based evaluation techniques.ResultsPerinatal mortality prediction models were developed using random forest, gradient boosting, and cat boost algorithms with the overall accuracy of 89.95%, 90.24%, and 82%, respectively. Risk factors of perinatal mortality were identified using feature importance analysis and relevant rules were extracted using the best performing model.ConclusionsA prediction model that was developed using gradient boosting algorithms was selected for further use in the risk factor analysis, generating relevant rules, development of artifacts, and model deployment because it has registered better performance with 90.24% accuracy. The most determinant risk factors of perinatal mortality were identified using feature importance and some of them are community-based health insurance, mother's educational level, region and place of residence, age, wealth status, birth interval, preterm, smoking cigarette, anemia level, hemoglobin level, and marital status.

Delta Radiomic Signature from Neoadjuvant Pancreas Cancer Radiation Planning Volume is Superior to Tumor Proper Volume to Predict Surgical Failures

<h3>Purpose/Objective(s)</h3> Predicting outcomes in pancreatic ductal adenocarcinoma (PDAC) patients who underwent surgical resection following neoadjuvant therapy is critical for decision making to optimize therapy. Delta radiomics may be a meaningful way to exploit changes in tumor features after preoperative therapy. Given the heterogeneity and infiltrative nature of the tumor, there is a lack of a standard methodology for identifying the tumor volume to be used for feature extraction. We hypothesize that a radiation planning volume inclusive of the tumor infiltrative regions is more predictive than a tumor proper volume that only includes the homogenous part of the tumor. <h3>Materials/Methods</h3> We reviewed our institutional database for patients with the following inclusion criteria: 1- histopathological diagnosis of PDAC, 2- received neoadjuvant chemotherapy and/or radiation therapy followed by surgical resection, and 3- have available CT scans before and after neoadjuvant therapy. We adopted two different methods to contour the tumor: A- "Tumor proper": including only the core mass component in the pancreas and immediately extending from the pancreas with homogeneous Hounsfield units, 2- "GTV": Including tumor proper in addition to the heterogonous infiltrating part of the tumor surrounding the pancreas, soft tissue and vessels if involved. We extracted 257 radiomic features from the segmented volumes and used the net changes between pre- and post-therapy as the delta-radiomic features. We built Cox Proportional Hazards models with a step forward feature selection method using each of the two volumes independently and C-index was used to assess the models. Outcomes were disease free and distant failure free survivals. <h3>Results</h3> Fifty-eight patients met the inclusion criteria. Thirty-five (60%) were of male gender with a median age of 67 years. Forty-eight patients (83%) presented with ECOG performance status of 0-1. Borderline and locally advanced tumors were reported in 25 patients (43%). All patients receive neoadjuvant chemotherapy, while 12 patients (21%) received neoadjuvant radiation therapy. FOLFIRNOX and gemcitabine/nab-paclitaxel were administered in 39 (67%) and 16 (28%) patients respectively. Pancreatoduodenectomy was done in 41 patients (71%). Following neoadjuvant therapy, median CA 19-9 dropped from 291 to 29 U/mL. For the disease-free survival, C-indices for the radiomic model were 0.662 (95% CI: 0.656, 0.668) and 0.736 (95% CI: 0.731, 0.741) for the Tumor proper and GTV respectively. For the distant failure free survival, the C-indices for the radiomic model were 0.677 (95% CI: 0.669, 0.684) and 0.729 (95% CI: 0.723, 0.736) for the Tumor proper and GTV respectively. <h3>Conclusion</h3> Inclusion of the infiltrative volumes consistently resulted in a more predictive delta radiomic model in all tested outcomes when compared to features extracted from the homogenous intrapancreatic part of the tumor only. External and prospective validation is needed to further investigate this finding.

An interpretable stacking ensemble learning framework based on multi-dimensional data for real-time prediction of drug concentration: The example of olanzapine.

Background and Aim: Therapeutic drug monitoring (TDM) has evolved over the years as an important tool for personalized medicine. Nevertheless, some limitations are associated with traditional TDM. Emerging data-driven model forecasting [e.g., through machine learning (ML)-based approaches] has been used for individualized therapy. This study proposes an interpretable stacking-based ML framework to predict concentrations in real time after olanzapine (OLZ) treatment. Methods: The TDM-OLZ dataset, consisting of 2,142 OLZ measurements and 472 features, was formed by collecting electronic health records during the TDM of 927 patients who had received OLZ treatment. We compared the performance of ML algorithms by using 10-fold cross-validation and the mean absolute error (MAE). The optimal subset of features was analyzed by a random forest-based sequential forward feature selection method in the context of the top five heterogeneous regressors as base models to develop a stacked ensemble regressor, which was then optimized via the grid search method. Its predictions were explained by using local interpretable model-agnostic explanations (LIME) and partial dependence plots (PDPs). Results: A state-of-the-art stacking ensemble learning framework that integrates optimized extra trees, XGBoost, random forest, bagging, and gradient-boosting regressors was developed for nine selected features [i.e., daily dose (OLZ), gender_male, age, valproic acid_yes, ALT, K, BW, MONO#, and time of blood sampling after first administration]. It outperformed other base regressors that were considered, with an MAE of 0.064, R-square value of 0.5355, mean squared error of 0.0089, mean relative error of 13%, and ideal rate (the percentages of predicted TDM within ± 30% of actual TDM) of 63.40%. Predictions at the individual level were illustrated by LIME plots, whereas the global interpretation of associations between features and outcomes was illustrated by PDPs. Conclusion: This study highlights the feasibility of the real-time estimation of drug concentrations by using stacking-based ML strategies without losing interpretability, thus facilitating model-informed precision dosing.

Feature selection using Decomposed Mutual Information Maximization

Feature selection has been recognized for long as an important preprocessing technique to reduce dimensionality and improve the performance of regression and classification tasks. The class of sequential forward feature selection methods based on Mutual Information (MI) is widely used in practice, mainly due to its computational efficiency and independence from the specific classifier. A recent work introduced a theoretical framework for this class of methods which explains the existing proposals as approximations to an optimal target objective function. Such framework made clear the advantages and drawbacks of each proposal. Methods accounting for the redundancy of candidate features using a maximization function and considering the so-called complementary effect are among the best ones. However, they still penalize the complementarity, which is an important drawback. This paper proposes the Decomposed Mutual Information Maximization (DMIM) method, which keeps the good theoretical properties of the best methods proposed so far but overcomes the complementarity penalization by applying the maximization separately to the inter-feature and class-relevant redundancies. DMIM was extensively evaluated and compared with other methods, both theoretically and using two synthetic scenarios and 20 publicly available real datasets applied to specific classifiers. Our results show that DMIM achieves a better classification performance than the remaining forward feature selection methods based on MI.

Forward feature selection for toxic speech classification using support vector machine and random forest

&lt;span lang="EN-US"&gt;This study describes the methods for eliminating irrelevant features in speech data to enhance toxic speech classification accuracy and reduce the complexity of the learning process. Therefore, the wrapper method is introduced to estimate the forward selection technique based on support vector machine (SVM) and random forest (RF) classifier algorithms. Eight main speech features were then extracted with derivatives consisting of 9 statistical sub-features from 72 features in the extraction process. Furthermore, Python is used to implement the classifier algorithm of 2,000 toxic data collected through the world's largest video sharing media, known as YouTube. Conclusively, this experiment shows that after the feature selection process, the classification performance using SVM and RF algorithms increases to an excellent extent. We were able to select 10 speech features out of 72 original feature sets using the forward feature selection method, with 99.5% classification accuracy using RF and 99.2% using SVM.&lt;/span&gt;

Classification of surface electromyography and gyroscopic signals of finger gestures acquired by Myo armband using machine learning methods

Gestures of the human hand can be identified through processing of surface electromyography (sEMG) signals. The human hand can perform many gestures via manipulation of the fingers. With correct classification of finger gestures, the mobility of a prosthetic hand can be increased and provide greater functionally. In this study, reliable classification was obtained for sEMG finger data acquired from a Myo armband placed on the lower forearm. In order to improve classification, gyroscopic signals, not previously used in other studies, were investigated in the sEMG finger data. Data was acquired from ten normal subjects using the Myo armband to identify 6 finger gestures: thumb, index finger, middle finger, little finger, ring finger and rest. Participants repeated each gesture thirty times. sEMG signals were preprocessed to extract features. 17 features were used in the feature matrix. By using the sequential forward feature selection method, the highest performance feature set was determined. Support Vector Machine, K-Nearest Neighbor and multilayer artificial neural network were used as classification algorithm. The classification was made using the Classification Learner Application and Neural Network Pattern Recognition Tool in Matlab®. The best performance with the features extracted only from sEMG data was 94.40% using the Artificial Neural Networks (ANN) method. The best performance with the features extracted from both sEMG and gyroscopic data was 96.30% (p-value < 0.05)with the ANN method. It is seen that gyroscopic signals can increase classification performance.

Prediction of vegetation phenology with atmospheric reanalysis over semiarid grasslands in Inner Mongolia

Vegetation phenology is a sensitive indicator of climate change and vegetation growth. In the present study, two phenological phases with respect to vegetation growth at the initial and mature stages, namely, the start of the season (SOS) and the peak of the season (POS), were estimated from a satellite-derived normalized difference vegetation index (NDVI) dataset over a long-term period of 32 years (1983 to 2014) and used to explore their responses to atmospheric variables, including air temperature, precipitation, solar radiation, wind speed and soil moisture. First, the forward feature selection method was used to determine whether each independent variable was linear or nonlinear to the SOS and POS. In addition, a generalized additive model (GAM) was used to analyze the correlation between the phenological phases and each independent variable at different temporal scales. The results show that soil moisture and precipitation are linearly correlated with the SOS, whereas the other variables are nonlinearly correlated. Meanwhile, soil moisture, wind speed and solar radiation are found to be nonlinearly correlated with the POS. However, air temperature and precipitation reveal a significant negative correlation with the POS. Furthermore, it was concluded that the aforementioned independent variables from the previous year could contribute to approximately 63%–85% of the SOS variations in the present year, whereas the atmospheric variables from April to June could contribute to approximately 70%–85% of the POS variations in the same year. Finally, the SOS and POS predicted by the GAM exhibit significant agreement with those derived from the satellite NDVI dataset, with the root mean square error of approximately 3 to 5 days.

Automatic detection of epileptic seizures using Riemannian geometry from scalp EEG recordings.

This paper proposes a new framework for epileptic seizure detection using non-invasive scalp electroencephalogram (sEEG) signals. The major innovation of the current study is using the Riemannian geometry for transforming the covariance matrices estimated from the EEG channels into a feature vector. The spatial covariance matrices are considered as features in order to extract the spatial information of the sEEG signals without applying any spatial filtering. Since these matrices are symmetric and positive definite (SPD), they belong to a special manifold called the Riemannian manifold. Furthermore, a kernel based on Riemannian geometry is proposed. This kernel maps the SPD matrices onto the Riemannian tangent space. The SPD matrices, obtained from all channels of the segmented sEEG signals, have high dimensions and extra information. For these reasons, the sequential forward feature selection method is applied to select the best features and reduce the computational burden in the classification step. The selected features are fed into a support vector machine (SVM) with an RBF kernel to classify the feature vectors into seizure and non-seizure classes. The performance of the proposed method is evaluated using two long-term scalp EEG (CHB-MIT benchmark and private) databases. Experimental results on all 23 subjects of the CHB-MIT database reveal an accuracy of 99.87%, a sensitivity of 99.91%, and a specificity of 99.82%. In addition, the introduced algorithm is tested on the private sEEG signals recorded from 20 patients, having 1380 seizures. The proposed approach achieves an accuracy, a sensitivity, and a specificity of 98.14%, 98.16%, and 98.12%, respectively. The experimental results on both sEEG databases demonstrate the effectiveness of the proposed method for automated epileptic seizure detection, especially for the private database which has noisier signals in comparison to the CHB-MIT database. Graphical Abstract Block diagram of the proposed epileptic seizure detection algorithm.

Prediction of Hypertension Outcomes Based on Gain Sequence Forward Tabu Search Feature Selection and XGBoost.

For patients with hypertension, serious complications, such as myocardial infarction, a common cause of heart failure, occurs in the late stage of hypertension. Hypertension outcomes can lead to complications, including death. Hypertension outcomes threaten patients’ lives and need to be predicted. In our research, we reviewed the hypertension medical data from a tertiary-grade A class hospital in Beijing, and established a hypertension outcome prediction model with the machine learning theory. We first proposed a gain sequence forward tabu search feature selection (GSFTS-FS) method, which can search the optimal combination of medical variables that affect hypertension outcomes. Based on this, the XGBoost algorithm established a prediction model because of its good stability. We verified the proposed method by comparing other commonly used models in similar works. The proposed GSFTS-FS improved the performance by about 10%. The proposed prediction method has the best performance and its AUC value, accuracy, F1 value, and recall of 10-fold cross-validation were 0.96. 0.95, 0.88, and 0.82, respectively. It also performed well on test datasets with 0.92, 0.94, 0.87, and 0.80 for AUC, accuracy, F1, and recall, respectively. Therefore, the XGBoost with GSFTS-FS can accurately and effectively predict the occurrence of outcomes for patients with hypertension, and can provide guidance for doctors in clinical diagnoses and medical decision-making.

Effect of environmental covariable selection in the hydrological modeling using machine learning models to predict daily streamflow

There are different methods for predicting streamflow, and, recently machine learning has been widely used for this purpose. This technique uses a wide set of covariables in the prediction process that must undergo a selection to increase the precision and stability of the models. Thus, this work aimed to analyze the effect of covariable selection with Recursive Feature Elimination (RFE) and Forward Feature Selection (FFS) in the performance of machine learning models to predict daily streamflow. The study was carried out in the Piranga river basin, located in the State of Minas Gerais, Brazil. The database consisted of an 18-year-old historical series (2000–2017) of streamflow data at the outlet of the basin and the covariables derived from the streamflow of affluent rivers, precipitation, land use and land cover, products from the MODIS sensors, and time. The highly correlated covariables were eliminated and the selection of covariables by the level of importance was carried out by the RFE and FFS methods for the Multivariate Adaptive Regression (EARTH), Multiple Linear Regression (MLR), and Random Forest (RF) models. The data were partitioned into two groups: 75% for training and 25% for validation. The models were run 50 times and had their performance evaluated by the Nash Sutcliffe efficiency coefficient (NSE), Determination coefficient (R2), and Root of Mean Square Error (RMSE). The three models tested showed satisfactory performance with both covariable selection methods, however, all of them proved to be inaccurate for predicting values associated with maximum streamflow events. The use of FFS, in most cases, improved the performance of the models and reduced the number of selected covariables. The use of machine learning to predict daily streamflow proved to be efficient and the use of FFS in the selection of covariables enhanced this efficiency.

An Exponential Factorization Machine with Percentage Error Minimization to Retail Sales Forecasting

This article proposes a new approach to sales forecasting for new products (stock-keeping units [SKUs]) with long lead time but short product life cycle. These SKUs are usually sold for one season only, without any replenishments. An exponential factorization machine (EFM) sales forecast model is developed to solve this problem which not only takes into account SKU attributes, but also pairwise interactions. The EFM model is significantly different from the original Factorization Machines (FM) from two fold: (1) the attribute-level formulation for explanatory/input variables; and (2) exponential formulation for the positive response/output/target variable. The attribute-level formation excludes infeasible intra-attribute interactions and results in more efficient feature engineering comparing with the conventional one-hot encoding, while the exponential formulation is demonstrated more effective than the log-transformation for the positive but not skewed distributed responses. In order to estimate the parameters, percentage error squares (PES) and error squares (ES) are minimized by a proposed adaptive batch gradient descent method over the training set. To overcome the over-fitting problem, a greedy forward stepwise feature selection method is proposed to select the most useful attributes and interactions. Real-world data provided by a footwear retailer in Singapore are used for testing the proposed approach. The forecasting performance in terms of both mean absolute percentage error (MAPE) and mean absolute error (MAE) compares favorably with not only off-the-shelf models but also results reported by extant sales and demand forecasting studies. The effectiveness of the proposed approach is also demonstrated by two external public datasets. Moreover, we prove the theoretical relationships between PES and ES minimization, and present an important property of the PES minimization for regression models; that it trains models to underestimate data. This property fits the situation of sales forecasting where unit-holding cost is much greater than the unit-shortage cost (e.g., perishable products).

Two accurate hybrid islanding detection schemes for distribution network

In this paper, two accurate hybrid islanding detection schemes are proposed based on Wavelet Transform and Stockwell transform (S-transform). The proposed methods use the potential of sequence voltage (negative) retrieved at the target Distributed Generation (DG) location of the distribution network under study. In one of the schemes, Discrete Wavelet transform (DWT) is applied to process the negative sequence voltage signal and for its decomposition, which is further used to extract six statistical features like energy, entropy, mean, kurtosis, standard deviation, and skewness from the reconstructed DWT coefficients. Test and train data sets are generated with the wide variation of loading conditions, and optimal features are chosen from the full feature set by forward feature selection method (FFS) during the training process by an artificial neural network (ANN). After that, the trained system is tested to get the detection result. Another scheme presented in this paper for islanding detection is based on S-transform, which is used to decompose the negative sequence voltage signal. Amplitude, frequency, and phase are the three coefficients acquired from the pre-processing of the raw signal by S-transform. Then the cumulative sums of the energy content of the S-transform coefficients are determined and are compared with a threshold value to get the detection result. The proposed schemes are tested in a distribution network consisting of two 9 MW wind farm driven by six 1.5 MW wind turbine connected to 120 kV main grid through a 25 kV, 30 km feeder. Several cases have been investigated like normal condition, islanding, DG line trip, disconnection of point of common coupling, and sudden change in load to test the performance of the proposed schemes. It can be observed from the results that both the approaches gave high accuracy in the detection of islanding conditions and demarcates properly from the non-islanding state. However, results show that the S-transform based approach provides a better resolution and quick detection of islanding than the wavelet transform approach.

Ensemble of artificial neural networks and K-nearest neighbour for classification of granite images

This study attempted to develop a granite quality classification model by comparing colour, texture, and ensemble of colour and texture. An average of 120 images was taken for each verities of grade A, grade B, grade C. A greyscale coexistence matrix was used for texture extraction and a colour histogram for colour extraction. Five textures and six colour features were extracted from each granite image for classification. To build the classification KNN, ANN and ensemble of KNN and ANN are examined. Based on the experiment ensemble of ANN and KNN model outperform good result by the combined texture and colour features using sequential forward feature selection (SFFS) methods. An average accuracy of 85.3%, 93.6%, and 95.8% is achieved for KNN, ANN and ensemble of KNN and ANN respectively. Granite fractures and vines of the images have a strong impact on the performance of the classifier.