Combination Of Multiple Classifiers Research Articles

Speech emotion recognition using a combination of variational mode decomposition and Hilbert transform

The objective of automated speech emotion recognition (SER) is to recognize each emotion of the speech signal uniquely and efficiently using machines like computers, mobile devices, etc. The popularity of SER has been widely acknowledged among researchers due to its extensive applicability in practical contexts. The use of SER has shown to be advantageous in several domains, including medical treatment, enhancement of security systems, surveillance operations, online marketing strategies, online educational platforms, online search engines, personal communication, customer relationship management, reinforcement of machine and human connection, and numerous other areas. Numerous authors have used various techniques, including the combination of multiple features (acoustic, non-acoustic, or both) and classifiers (machine learning, deep learning, or both), in order to enhance the efficacy of emotion categorization. In the present work, our objective is to enhance the performance of emotion classification (PECL) by using a combined approach of the variational mode decomposition (VMD) and Hilbert transform (HT) techniques. Using the VMD method, we decomposed the speech signal frame into many sub-signals, or intrinsic mode functions (IMFs). Then HT is applied to each VMD-based IMF signal to find the mode instantaneous amplitude (MIA) and mode instantaneous frequency (MIF) signal vectors. We extracted proposed features such as HT-based approximate entropy (HTAE), HT-based permutation entropy (HTPE), HT-based increment entropy (HTIE), and HT-based sample entropy (HTSE) using each MIA and MIF signal vector. The combination of the proposed HT-based features is called HT-based entropy (HTE) features. Then, we accessed the PECL using the HTE and MFCC features alone and in conjunction with a deep neural network (DNN) classifier. The experiment's results showed that the combinations of the proposed feature (MFCC + HTE) using a DNN classifier outperformed the individual features and obtained a SER accuracy of 86.92% for the EMOVO dataset and 91.63% for the EMO-DB dataset.

Adaboost algorithm combined multiple random forest models (Adaboost-RF) is employed for fluid prediction using well logging data

Well logging fluid prediction is of great significance in oil and gas exploration. Based on data mining technology, this paper proposes an adaptive boosting random forest (Adaboost-RF) method for well logging fluid prediction. First, we use the Adaboost algorithm for feature selection, train a weak classifier by repeatedly weighting observations and correcting hard-to-classify samples, and obtain a combination of multiple weak classifiers. This method can effectively improve the accuracy and robustness of the classifier and can reduce the risk of overfitting. Then, we use random forest (RF) as a basic classifier to build an Adaboost-RF model for well logging fluid prediction. The combination of Adaboost and RF can further improve the stability and accuracy of the classifier. To verify the performance of this method, we performed experimental evaluation using real well logging data. Experimental results show that the Adaboost-RF method can have higher accuracy and stability in log fluid prediction than the traditional method (backpropagation neural network) and the method using RF alone. In summary, this method combines the characteristics of Adaboost and RF, which can improve the accuracy and stability of the classifier and is easy to implement and generalize, providing a new, efficient, and accurate fluid prediction method for the field of oil and gas exploration.

Evaluation of Multiple Classifier Systems for Mapping Different Hierarchical Levels of Forest Ecosystems in the Mediterranean Region Using Sentinel-2, Sentinel-1, and ICESat-2 Data

The conservation and management of forest areas require knowledge about their extent and attributes on multiple scales. The combination of multiple classifiers has been proposed as an attractive classification approach for improved accuracy and robustness that can efficiently exploit the complementary nature of diverse remote sensing data and the merits of individual classifiers. The aim of this study was to develop and evaluate multiple classifier systems (MCSs) within a cloud-based computing environment for multi-scale forest mapping in Northeastern Greece using passive and active remote sensing data. Five individual machine learning base classifiers were used for class discrimination across the three different hierarchy levels, and five ensemble approaches were used for combining them. In the case of the binary classification scheme in the upper level of the hierarchy for separating woody vegetation (forest and shrubs) from other land, the overall accuracy (OA) slightly increased with the use of the MCS approach, reaching 94%. At the lower hierarchical levels, when using the support vector machine (SVM) base classifier, OA reached 84.13% and 74.89% for forest type and species mapping, respectively, slightly outperforming the MCS approach. Yet, two MCS approaches demonstrated robust performance in terms of per-class accuracy, presenting the highest average F1 score across all classification experiments, indicating balanced misclassification errors across all classes. Since the competence of individual classifiers is dependent on individual scene settings and data characteristics, we suggest that the adoption of MCS systems in efficient computing environments (i.e., cloud) could alleviate the need for algorithm benchmarking for Earth’s surface cover mapping.

A hybrid quantum annealing method for generating ensemble classifiers

Quantum annealing has been widely used to optimize machine learning such as ensemble classifiers. This ensemble classifier enhances classification performance through the combination of multiple accurate and diverse base classifiers. It benefits from the “perturb and combine” strategy which involves using the random subspace as the prevalent approach to perturb input data. There are several methods to generate a random subspace which include bagging, boosting, clustering, and, more recently, clustering balancing. The main contribution of this research was to improve the accuracy of the ensemble classifier using the hybrid quantum annealing method. The process involved making larger, stronger, and more balanced clusters through 1) the modification of the range of values from K in incremental clustering, 2) the adjustment of the clusters to be stronger and more balanced, and 3) the optimization of pure-class clusters using quantum annealing. The proposed method was tested and evaluated on 10 benchmark datasets from the UCI repository and the results were compared with current approaches. The results showed that the proposed method has better accuracy than the others due to the larger, stronger, and more balanced clusters generated as well as the better trade-off between accuracy and diversity.

Research on the Application of Multi-Source Information Fusion in Multiple Gait Pattern Transition Recognition.

Multi-source information fusion technology is a kind of information processing technology which comprehensively processes and utilizes multi-source uncertain information. It is an effective scheme to solve complex pattern recognition and improve classification performance. This study aims to improve the accuracy and robustness of exoskeleton gait pattern transition recognition in complex environments. Based on the theory of multi-source information fusion, this paper explored a multi-source information fusion model for exoskeleton gait pattern transition recognition in terms of two aspects of multi-source information fusion strategy and multi-classifier fusion. For eight common gait pattern transitions (between level and stair walking and between level and ramp walking), we proposed a hybrid fusion strategy of multi-source information at the feature level and decision level. We first selected an optimal feature subset through correlation feature extraction and feature selection algorithm, followed by the feature fusion through the classifier. We then studied the construction of a multi-classifier fusion model with a focus on the selection of base classifier and multi-classifier fusion algorithm. By analyzing the classification performance and robustness of the multi-classifier fusion model integrating multiple classifier combinations with a number of multi-classifier fusion algorithms, we finally constructed a multi-classifier fusion model based on D-S evidence theory and the combination of three SVM classifiers with different kernel functions (linear, RBF, polynomial). Such multi-source information fusion model improved the anti-interference and fault tolerance of the model through the hybrid fusion strategy of feature level and decision level and had higher accuracy and robustness in the gait pattern transition recognition, whose average recognition accuracy for eight gait pattern transitions reached 99.70%, which increased by 0.15% compared with the highest average recognition accuracy of the single classifier. Moreover, the average recognition accuracy in the absence of different feature data reached 97.47% with good robustness.

A framework of dynamic selection method for user classification in touch-based continuous mobile device authentication

Continuous authentication can provide a mechanism to continuously monitor mobile devices while a user is actively using it, after passing the initial-login authentication phase. Touch biometric is one of the promising modality to realise continuous authentication on mobile devices by distinguishing between the touch strokes performed by the legitimate and illegitimate users through classification algorithms. While the benefit of the scheme is promising, the effectiveness of different classification methods are not thoroughly understood. Little consideration has been given on the combination of multiple classifiers to perform continuous authentication. In this paper, we propose a novel classification framework for touch-based continuous mobile device authentication (CMDA), utilising dynamic selection of classifiers (DS). Instead of classifying all touch strokes using the same classifier, the proposed framework classifies each touch sample using the most promising classifier(s) from a pool of classifiers. Based on the proposed framework, we evaluated various DS methods in multiple scenarios across four touch datasets. The aim of this evaluation is to assess the feasibility of DS on touch-based CMDA. We then compared these DS methods with well-known single classifiers and static ensemble methods. The experimental results show the potential and feasibility of the DS methods to improve the authentication performance of touch-based CMDA against the benchmark methods. We found that DS methods are capable of producing promising results with relatively low equal error rate (EER) in many scenarios of the datasets, with relatively high consistencies. The obtained results would be valuable for further enhancement of existing user classification methods and the development of new DS methods in touch-based CMDA.

Facial Expression Recognition Using Convolution Neural Network Fusion and Texture Descriptors Representation

Facial expression recognition is an interesting research direction of pattern recognition and computer vision. It has been increasingly used in artificial intelligence, human–computer interaction and security monitoring. In recent years, Convolution Neural Network (CNN) as a deep learning technique and multiple classifier combination method has been applied to gain accurate results in classifying face expressions. In this paper, we propose a multimodal classification approach based on a local texture descriptor representation and a combination of CNN to recognize facial expression. Initially, in order to reduce the influence of redundant information, the preprocessing stage is performed using face detection, face image cropping and texture descriptors of Local Binary Pattern (LBP), Local Gradient Code (LGC), Local Directional Pattern (LDP) and Gradient Direction Pattern (GDP) calculation. Second, we construct a cascade CNN architecture using the multimodal data of each descriptor (CNNLBP, CNNLGC, CNNGDP and CNNLDP) to extract facial features and classify emotions. Finally, we apply aggregation techniques (sum and product rule) for each modality to combine the four multimodal outputs and thus obtain the final decision of our system. Experimental results using CK[Formula: see text] and JAFFE database show that the proposed multimodal classification system achieves superior recognition performance compared to the existing studies with classification accuracy of 97, 93% and 94, 45%, respectively.

Corporate failure prediction using threshold‐based models

AbstractCorporate failure prediction literature indicates that models' performance depends on more than the complexity of the prediction method. Recent advances cite the relevance of sampling approaches to model performance. Therefore, this study proposes a novel approach that implements threshold models for corporate failure prediction efforts. Threshold models estimate asymptotically conservative confidence regions, in which the samples are split by size. Then, single‐based classifiers and a combination of multiple classifiers are employed in each region to estimate the prediction accuracy. This article offers a comparison of the proposed threshold‐based corporate failure model with two benchmark models, previously used in studies in the same context. The empirical results show that the proposed threshold‐based model clearly outperforms conventional models, in particular with the combination of multiple classifiers. The superiority of the threshold‐based model stems from its ability to discern failed firms, which represent the most important class in financial terms. This study thus provides initial evidence of the utility of threshold models in corporate failure prediction efforts.

Optimization of Small Sized File Access Efficiency in Hadoop Distributed File System by Integrating Virtual File System Layer

Storage for large datasets, handling data in different formats and data getting generated with high speed are the major highlights of the Hadoop because of which the Hadoop got invented. Hadoop is the solution for the big data problems as discussed above. In order to give the improved solution (in terms of access efficiency and time) for small sized files, this solution is proposed. A novel approach called VFS-HDFS architecture is designed in which the focus is on optimization of small sized files access problems with significant development compared with the existing solutions i.e. HDFS sequence files, HAR, NHAR. In the proposed work a Virtual file system layer has been added as a wrapper over the top of existing HDFS architecture. However, the research work is carried out without altering the existing HFDS architecture. In this paper drawbacks of existing techniques i.e. Flat File Technique and Table Chain Technique which are implemented in HDFS HAR, NHAR, sequence file is overcome by using Bucket Chain Technique. The files to merge in a single bucket are selected using ensemble classifier which is a combination of different classifiers. Combination of multiple classifiers gives the better accurate results. Using this proposed system, better results are obtained compared with the existing system in terms of access efficiency of small sized files in HDFS.

Diagnosis of Infantile Hip Dysplasia With B-Mode Ultrasound via Two-Stage Meta-Learning Based Deep Exclusivity Regularized Machine.

The B-mode ultrasound (BUS) based computer-aided diagnosis (CAD) has shown its effectiveness for developmental dysplasia of the hip (DDH) in infants. In this work, a two-stage meta-learning based deep exclusivity regularized machine (TML-DERM) is proposed for the BUS-based CAD of DDH. TML-DERM integrates deep neural network (DNN) and exclusivity regularized machine into a unified framework to simultaneously improve the feature representation and classification performance. Moreover, the first-stage meta-learning is mainly conducted on the DNN module to alleviate the overfitting issue caused by the significantly increased parameters in DNN, and a random sampling strategy is adopted to self-generate the meta-tasks; while the second-stage meta-learning mainly learns the combination of multiple weak classifiers by a weight vector to improve the classification performance, and also optimizes the unified framework again. The experimental results on a DDH ultrasound dataset show the proposed TML-DERM algorithm achieves the superior classification performance with the mean accuracy of 85.89%, sensitivity of 86.54%, and specificity of 85.23%.

Realistic Smile Expression Recognition Approach Using Ensemble Classifier with Enhanced Bagging

A robust smile recognition system could be widely used for many real-world applications. Classification of a facial smile in an unconstrained setting is difficult due to the invertible and wide variety in face images. In this paper, an adaptive model for smile expression classification is suggested that integrates a fast features extraction algorithm and cascade classifiers. Our model takes advantage of the intrinsic association between face detection, smile, and other face features to alleviate the over-fitting issue on the limited training set and increase classification results. The features are extracted taking into account to exclude any unnecessary coefficients in the feature vector; thereby enhancing the discriminatory capacity of the extracted features and reducing the computational process. Still, the main causes of error in learning are due to noise, bias, and variance. Ensemble helps to minimize these factors. Combinations of multiple classifiers decrease variance, especially in the case of unstable classifiers, and may produce a more reliable classification than a single classifier. However, a shortcoming of bagging as the best ensemble classifier is its random selection, where the classification performance relies on the chance to pick an appropriate subset of training items. The suggested model employs a modified form of bagging while creating training sets to deal with this challenge (error-based bootstrapping). The experimental results for smile classification on the JAFFE, CK+, and CK+48 benchmark datasets show the feasibility of our proposed model.

Preoperative ultrasound radiomics analysis for expression of multiple molecular biomarkers in mass type of breast ductal carcinoma in situ

BackgroundThe molecular biomarkers of breast ductal carcinoma in situ (DCIS) have important guiding significance for individualized precision treatment. This study was intended to explore the significance of radiomics based on ultrasound images to predict the expression of molecular biomarkers of mass type of DCIS.Methods116 patients with mass type of DCIS were included in this retrospective study. The radiomics features were extracted based on ultrasound images. According to the ratio of 7:3, the data sets of molecular biomarkers were split into training set and test set. The radiomics models were developed to predict the expression of estrogen receptor (ER), progesterone receptor (PR), human epidermal growth factor receptor 2 (HER2), Ki67, p16, and p53 by using combination of multiple feature selection and classifiers. The predictive performance of the models were evaluated using the area under the curve (AUC) of the receiver operating curve.ResultsThe investigators extracted 5234 radiomics features from ultrasound images. 12, 23, 41, 51, 31 and 23 features were important for constructing the models. The radiomics scores were significantly (P < 0.05) in each molecular marker expression of mass type of DCIS. The radiomics models showed predictive performance with AUC greater than 0.7 in the training set and test set: ER (0.94 and 0.84), PR (0.90 and 0.78), HER2 (0.94 and 0.74), Ki67 (0.95 and 0.86), p16 (0.96 and 0.78), and p53 (0.95 and 0.74), respectively.ConclusionUltrasonic-based radiomics analysis provided a noninvasive preoperative method for predicting the expression of molecular markers of mass type of DCIS with good accuracy.

Advancing synthesis of decision tree-based multiple classifier systems: an approximate computing case study

So far, multiple classifier systems have been increasingly designed to take advantage of hardware features, such as high parallelism and computational power. Indeed, compared to software implementations, hardware accelerators guarantee higher throughput and lower latency. Although the combination of multiple classifiers leads to high classification accuracy, the required area overhead makes the design of a hardware accelerator unfeasible, hindering the adoption of commercial configurable devices. For this reason, in this paper, we exploit approximate computing design paradigm to trade hardware area overhead off for classification accuracy. In particular, starting from trained DT models and employing precision-scaling technique, we explore approximate decision tree variants by means of multiple objective optimization problem, demonstrating a significant performance improvement targeting field-programmable gate array devices.

Multiple Features Extraction and Classifiers Combination Based Handwriting Digit Recognition

Multiple Features Extraction and Classifiers Combination Based Handwriting Digit Recognition

Credit Card Fraud Detection Using Fuzzy Rough Nearest Neighbor and Sequential Minimal Optimization with Logistic Regression

&lt;p&gt;The global online communication channel made possible with the internet has increased credit card fraud leading to huge loss of monetary fund in their billions annually for consumers and financial institutions. The fraudsters constantly devise new strategy to perpetrate illegal transactions. As such, innovative detection systems in combating fraud are imperative to curb these losses. This paper presents the combination of multiple classifiers through stacking ensemble technique for credit card fraud detection. The fuzzy-rough nearest neighbor (FRNN) and sequential minimal optimization (SMO) are employed as base classifiers. Their combined prediction becomes data input for the meta-classifier, which is logistic regression (LR) resulting in a final predictive outcome for improved detection. Simulation results compared with seven other algorithms affirms that ensemble model can adequately detect credit card fraud with detection rates of 84.90% and 76.30%.&lt;/p&gt;

A hybrid ensemble learning method for the identification of gang-related arson cases

Arson is one of the most common crimes, and it has the characteristics of low cost and great harm. In addition to causing casualties and property damage, arson can often have huge social impacts and cause psychological panic in the public. Since arson is more harmful when conducted by a gang, how to effectively identify gang crimes in arson cases has become an important issue. In this paper, we propose a hybrid method that combines ensemble learning and intelligent optimization algorithms to solve this problem. First, we develop the recursive feature elimination (RFE)-based feature selection method to remove redundant features. Second, for the data imbalance problem, we determine the optimal processing algorithm from 18 candidate algorithms. Third, after trying a combination of multiple base classifiers, we obtain the optimal base classifier combination. Fourth, when integrating the prediction results of the base classifier, we propose a weighted ensemble strategy. Finally, we use the differential evolution (DE) algorithm to optimize the parameters of the base classifier and the weight of the combination, which further enhances the identification ability of the model. To verify the actual performance of the proposed method, we conducted experiments on the US National Fire Incident Reporting System (NFIRS) database. The results show that the proposed method is significantly superior to other popular machine learning methods, which proves that this method can provide a more reliable decision basis in the detection of arson cases.

A new synergistic approach for crop discrimination in a semi-arid region using Sentinel-2 time series and the multiple combination of machine learning classifiers

Accurate monitoring of agricultural lands and crop types is a crucial tool for sustainable food production. Therefore, to provide reliable and updated crop maps, the improvement of satellite image classification approaches is essential. In this context, machine learning algorithms present a potential tool for efficient and effective classification of remotely sensed data. The main strengths of machine learning algorithms are the capacity to handle data of high dimensionality, and mapping classes characterized by strong complex dynamics. The main objective of this work was to develop a new synergistic approach for crop discrimination in the semi-arid region of Chichaoua province, located in the Marrakesh-Safi region, Morocco, using high spatio-temporal resolution imagery and a multiple combination of machine learning classifiers. This approach was developed based on 10m spatial resolution open access Sentinel-2 (S2) images and machine learning algorithms. The atmospherically corrected S2 images were accessed through the Theia Land Data Center. Reference dataset was collected from a field survey carried out during the 2018 agricultural season in order to train the classifiers. Artificial Neural Networks, Support Vector Machine, K-Nearest Neighbors, Bagged Trees, Naive Bayes, Discriminant Analysis and Decision Trees classifiers were trained over the study area and the accuracy metrics, mainly Overall Accuracy (OA) and Kappa coefficient (K), were assessed. The trained models were single classifiers to build the ensemble classifier system. The obtained results showed high OA and K values up to 96% and 0.95 respectively, achieved by the developed approach. Therefore, based on these results, the approach we developed using the combination of multiple classifiers has a significant impact on crop classification quality.

Feature Selection and Ensemble Learning Techniques in One-Class Classifiers: An Empirical Study of Two-Class Imbalanced Datasets

Class imbalance learning is an important research problem in data mining and machine learning. Most solutions including data levels, algorithm levels, and cost sensitive approaches are derived using multi-class classifiers, depending on the number of classes to be classified. One-class classification (OCC) techniques, in contrast, have been widely used for anomaly or outlier detection where only normal or positive class training data are available. In this study, we treat every two-class imbalanced dataset as an anomaly detection problem, which contains a larger number of data in the majority class, i.e. normal or positive class, and a very small number of data in the minority class. The research objectives of this paper are to understand the performance of OCC classifiers and examine the level of performance improvement when feature selection is considered for pre-processing the training data in the majority class and ensemble learning is employed to combine multiple OCC classifiers. Based on 55 datasets with different ranges of class imbalance ratios and one-class support vector machine, isolation forest, and local outlier factor as the representative OCC classifiers, we found that the OCC classifiers are good at high imbalance ratio datasets, outperforming the C4.5 baseline. In most cases, though, performing feature selection does not improve the performance of the OCC classifiers in most. However, many homogeneous and heterogeneous OCC classifier ensembles do outperform the single OCC classifiers, with some specific combinations of multiple OCC classifiers, both with and without feature selection, performing similar to or better than the baseline combination of SMOTE and C4.5.

Combination of multiple classifiers for automatic recognition of diseases and damages on plant leaves

In this paper, we present an automatic recognition system of diseases and damages on plant leaves. The proposed system is based on classifiers combination technique, in which we have two variants of combination: serial combination of two classifiers, and hybrid combination of three classifiers including a serial combination of two classifiers in parallel with an individual classifier. Three types of features are adopted including color, texture and shape. The tests of this study to evaluate the three variants of combination are carried out on a database of 600 images of six classes (Leaf miners, Tuta absoluta and Thrips, Early blight, Late blight and powdery mildew). The comparison of results between the two methods serial and hybrid of the proposed system indicates that significant performances were obtained by applying the hybrid method for the recognition of diseases and damages on plant leaves.

Voting‐based intrusion detection framework for securing software‐defined networks

SummarySoftware‐defined networking (SDN) is an emerging paradigm in enterprise networks because of its flexible and cost‐effective nature. By decoupling control and data plane, SDN can provide various defense solutions for securing futuristic networks. However, the architectural design and characteristics of SDN attract several severe attacks. Distributed denial of service (DDoS) is considered as a major destructive cyber attack that makes the services of controller unavailable for its legitimate users. In this research article, an intrusion detection framework is proposed to detect DDoS attacks against SDN. The proposed framework relies on voting‐based ensemble model for the attack detection. Ensemble model is a combination of multiple machine learning classifiers for prediction of final results. In this research article, we propose and analyze three ensemble models named as Voting‐CMN, Voting‐RKM, and Voting‐CKM particularly to benchmarking datasets such as UNSW‐NB15, CICIDS2017, and NSL‐KDD, respectively. For validation of the proposed models, a cross‐validation technique is used with the prediction algorithms. The effectiveness of proposed models is evaluated in terms of prominent metrics (accuracy, precision, recall, and F‐measure). Experimental results indicate that the proposed models achieve better performance in terms of accuracy as compared with other existing models.