Single Classifier Approach Research Articles

SSC-SFN: spectral-spatial non-local segment federated network for hyperspectral image classification with limited labeled samples

ABSTRACT Hyperspectral image (HSI) classification methods based on deep learning (DL) have performed well in numerous investigations. Although many modified superpixel-wise neural networks are utilized to enhance spatial information, their ability to mine spectral information in graph structures is insufficient. Moreover, single classifier approaches are unable to extract adequate spatial and spectral information simultaneously. For the classification of large-scale research areas, many works have relied on the use of a large number of labeled samples, leading to low efficiency and weak generalization. To address these issues, an effective spectral-spatial HSI classification approach based on spectral-spatial non-local segment federated network (SSC-SFN) was developed in this study. In this framework, deconvolution is employed to recover the data size, while the lost spatial information is replaced by up-pooling. The spectral dimensional features are updated through the generation of non-Euclidean graph structures and the non-local segment smoothing technique. The convolutional neural network and graph convolutional network techniques are coupled to exploit the available spectral and spatial structure information fully. Extensive experimental results obtained using four public benchmark datasets show that the classification accuracy of SSC-SFN can exceed 90% for large-scale HSIs with limited samples.

Machine Learning-Based Ensemble Classifiers for Anomaly Handling in Smart Home Energy Consumption Data

Addressing data anomalies (e.g., garbage data, outliers, redundant data, and missing data) plays a vital role in performing accurate analytics (billing, forecasting, load profiling, etc.) on smart homes' energy consumption data. From the literature, it has been identified that the data imputation with machine learning (ML)-based single-classifier approaches are used to address data quality issues. However, these approaches are not effective to address the hidden issues of smart home energy consumption data due to the presence of a variety of anomalies. Hence, this paper proposes ML-based ensemble classifiers using random forest (RF), support vector machine (SVM), decision tree (DT), naive Bayes, K-nearest neighbor, and neural networks to handle all the possible anomalies in smart home energy consumption data. The proposed approach initially identifies all anomalies and removes them, and then imputes this removed/missing information. The entire implementation consists of four parts. Part 1 presents anomaly detection and removal, part 2 presents data imputation, part 3 presents single-classifier approaches, and part 4 presents ensemble classifiers approaches. To assess the classifiers' performance, various metrics, namely, accuracy, precision, recall/sensitivity, specificity, and F1 score are computed. From these metrics, it is identified that the ensemble classifier "RF+SVM+DT" has shown superior performance over the conventional single classifiers as well the other ensemble classifiers for anomaly handling.

Multi-class imbalanced enterprise credit evaluation based on asymmetric bagging combined with light gradient boosting machine

Most existing research on multi-class imbalanced enterprise credit evaluation modeling has been built on data-level imbalance processing methods and single classifier approaches. Using the one-versus-one (OVO) decomposition and fusion method to dispose multi-class classification, this paper proposes two new credit evaluation ensemble models by combining the asymmetric bagging (AB) and the light gradient boosting machine (LightGBM). Based on a multi-class imbalanced dataset of Chinese enterprises that issue corporate bonds from 2014 through 2020, this study conducts a series of empirical experiments for multi-class imbalanced enterprise credit evaluation. The experimental results demonstrate that our proposed models can significantly outperform the benchmark models, which integrate the OVO and one-versus-all decomposition and fusion method respectively with the random under sampling LightGBM, random over sampling LightGBM, synthetic minority over sampling technique LightGBM and the AB decision tree. In addition, the proposed models can carry out analysis on feature importance, which provides decision-making basis for enterprise stakeholders.

Camera model identification based on forensic traces extracted from homogeneous patches

A crucial challenge in digital image forensics is to identify the source camera model used to generate given images. This is of prime importance, especially for Law Enforcement Agencies in their investigations of Child Sexual Abuse Material found in darknets or seized storage devices.In this work, we address this challenge by proposing a solution that is characterized by two main contributions. It relies on the extraction of rather small homogeneous regions that we extract very efficiently from the integral image, and on a hierarchical classification approach with convolutional neural networks as the underlying models. We rely on homogeneous regions as they contain camera traces that are less distorted than regions with high-level scene content. The hierarchical approach that we propose is important for scaling up and making minimal modifications when new cameras are added. Furthermore, this scheme performs better than the traditional single classifier approach. By means of thorough experimentation on the publicly available Dresden data set, we achieve an accuracy of 99.01% with 5-fold cross-validation on the ‘natural’ subset of this data set. To the best of our knowledge, this is the best result ever reported for Dresden data set.

An enhanced Predictive heterogeneous ensemble model for breast cancer prediction

Breast Cancer is one of the most prevalent tumors after lung cancer and is common in both women and men. This disease is mostly asymptomatic in the early stages thus detection is difficult, and it becomes complicated and expensive to be treated in later stages resulting in increased fatality rates. There are comparatively very few pieces of literature that investigated breast cancer employing an ensemble learning for cancer prediction as compared to single classifier approaches. This paper presents a heterogeneous ensemble machine learning approach, to detect breast cancer in the early stages. The proposed approach follows the CRISP-DM process and uses Stacking for building the ensemble model using three different algorithms – K-Nearest Neighbors (KNN), Support Vector Machine (SVM), and Decision Tree (DT). The performance of this meta classifier is compared with the individual performances of its base classifiers (KNN, SVM, DT) and other single classifiers – Logistic Regression (LR), Artificial Neural Network (ANN), Naïve Bayes (NB), Stochastic Gradient Descent (SGD) and a homogenous ensemble model of Random Forest (RF). The top 5 features – Glucose, Resistin, HOMA, Insulin, and BMI are derived by using Chi-Square. Evaluation of the model helps in estimating its consideration for early breast cancer prediction just by using the anthropometric data of humans. Performances of models are compared using metrics such as accuracy, AUC, ROC Curve, f1-score, precision, recall, log loss, and specificity using K-fold cross-validation of 2, 3, 5, 10, and 20 folds. The proposed ensemble model achieved the greatest accuracy of 78 % with the lowest log-loss of 0.56, at K = 20, thus rejecting the Null hypothesis. The derived p-value is 0.014, from the one-tailed t-test, which provides lower significance at ∝ = 0.05.

Construction of Accurate Crack Identification on Concrete Structure using Hybrid Deep Learning Approach

In general, several conservative techniques are available for detecting cracks in concrete bridges but they have significant limitations, including low accuracy and efficiency. Due to the expansion of the neural network method, the performance of digital image processing based crack identification has recently diminished. Many single classifier approaches are used to detect the cracks with high accuracy. The classifiers are not concentrating on random fluctuation in the training dataset and also it reflects in the final output as an over-fitting phenomenon. Though this model contains many parameters to justify the training data, it fails in the residual variation. These residual variations are frequent in UAV recorded photos as well as many camera images. To reduce this challenge, a noise reduction technique is utilized along with an SVM classifier to reduce classification error. The proposed technique is more resourceful by performing classification via SVM approach, and further the feature extraction and network training has been implemented by using the CNN method. The captured digital images are processed by incorporating the bending test through reinforced concrete beams. Moreover, the proposed method is determining the widths of the crack by employing binary conversion in the captured images. The proposed model outperforms conservative techniques, single type classifiers, and image segmentation type process methods in terms of accuracy. The obtained results have proved that, the proposed hybrid method is more accurate and suitable for crack detection in concrete bridges especially in the unmanned environment.

Learning From the Optical Spectrum: Failure Detection and Identification

The availability of coarse-resolution cost-effective optical spectrum analyzers (OSAs) allows their widespread deployment in operators’ networks. In this paper, we explore several machine learning approaches for soft-failure detection, identification, and localization that take advantage of OSAs. In particular, we present three different solutions for the two most common filter-related soft-failures: filter shift and tight filtering, which noticeably deform the expected shape of the optical spectrum. However, filter cascading is a key challenge as it affects the shape of the optical spectrum similarly to the tight filtering; the approaches are specifically designed to avoid the misclassification of properly operating signals when normal filter cascading effects are present. The proposed solutions are as follows: First, the multi-classifier approach, which uses features extracted directly from the optical spectrum; second, the single-classifier approach, which uses preprocessed features to compensate for filter cascading; and third, the residual-based approach, which uses a residual signal computed from subtracting the signal acquired by OSAs from an expected signal synthetically generated. Extensive numerical results are ultimately presented to compare the performance of the proposed approaches in terms of accuracy and robustness.

Fusion Approaches for Land Cover Map Production Using High Resolution Image Time Series without Reference Data of the Corresponding Period

Optical sensor time series images allow one to produce land cover maps at a large scale. The supervised classification algorithms have been shown to be the best to produce maps automatically with good accuracy. The main drawback of these methods is the need for reference data, the collection of which can introduce important production delays. Therefore, the maps are often available too late for some applications. Domain adaptation methods seem to be efficient for using past data for land cover map production. According to this idea, the main goal of this study is to propose several simple past data fusion schemes to override the current land cover map production delays. A single classifier approach and three voting rules are considered to produce maps without reference data of the corresponding period. These four approaches reach an overall accuracy of around 80% with a 17-class nomenclature using Formosat-2 image time series. A study of the impact of the number of past periods used is also done. It shows that the overall accuracy increases with the number of periods used. The proposed methods require at least two or three previous years to be used.

Novelty detection in data streams

In massive data analysis, data usually come in streams. In the last years, several studies have investigated novelty detection in these data streams. Different approaches have been proposed and validated in many application domains. A review of the main aspects of these studies can provide useful information to improve the performance of existing approaches, allow their adaptation to new applications and help to identify new important issues to be addresses in future studies. This article presents and analyses different aspects of novelty detection in data streams, like the offline and online phases, the number of classes considered at each phase, the use of ensemble versus a single classifier, supervised and unsupervised approaches for the learning task, information used for decision model update, forgetting mechanisms for outdated concepts, concept drift treatment, how to distinguish noise and outliers from novelty concepts, classification strategies for data with unknown label, and how to deal with recurring classes. This article also describes several applications of novelty detection in data streams investigated in the literature and discuss important challenges and future research directions.

Patient-Specific Variations in Biomarkers across Gingivitis and Periodontitis

This study investigates the use of saliva, as an emerging diagnostic fluid in conjunction with classification techniques to discern biological heterogeneity in clinically labelled gingivitis and periodontitis subjects (80 subjects; 40/group) A battery of classification techniques were investigated as traditional single classifier systems as well as within a novel selective voting ensemble classification approach (SVA) framework. Unlike traditional single classifiers, SVA is shown to reveal patient-specific variations within disease groups, which may be important for identifying proclivity to disease progression or disease stability. Salivary expression profiles of IL-1ß, IL-6, MMP-8, and MIP-1α from 80 patients were analyzed using four classification algorithms (LDA: Linear Discriminant Analysis [LDA], Quadratic Discriminant Analysis [QDA], Naïve Bayes Classifier [NBC] and Support Vector Machines [SVM]) as traditional single classifiers and within the SVA framework (SVA-LDA, SVA-QDA, SVA-NB and SVA-SVM). Our findings demonstrate that performance measures (sensitivity, specificity and accuracy) of traditional classification as single classifier were comparable to that of the SVA counterparts using clinical labels of the samples as ground truth. However, unlike traditional single classifier approaches, the normalized ensemble vote-counts from SVA revealed varying proclivity of the subjects for each of the disease groups. More importantly, the SVA identified a subset of gingivitis and periodontitis samples that demonstrated a biological proclivity commensurate with the other clinical group. This subset was confirmed across SVA-LDA, SVA-QDA, SVA-NB and SVA-SVM. Heatmap visualization of their ensemble sets revealed lack of consensus between these subsets and the rest of the samples within the respective disease groups indicating the unique nature of the patients in these subsets. While the source of variation is not known, the results presented clearly elucidate the need for novel approaches that accommodate inherent heterogeneity and personalized variations within disease groups in diagnostic characterization. The proposed approach falls within the scope of P4 medicine (predictive, preventive, personalized, and participatory) with the ability to identify unique patient profiles that may predict specific disease trajectories and targeted disease management.

Robust and complex approach of pathological speech signal analysis

This paper presents a study of the approaches in the state-of-the-art in the field of pathological speech signal analysis with a special focus on parametrization techniques. It provides a description of 92 speech features where some of them are already widely used in this field of science and some of them have not been tried yet (they come from different areas of speech signal processing like speech recognition or coding). As an original contribution, this work introduces 36 completely new pathological voice measures based on modulation spectra, inferior colliculus coefficients, bicepstrum, sample and approximate entropy and empirical mode decomposition. The significance of these features was tested on 3 (English, Spanish and Czech) pathological voice databases with respect to classification accuracy, sensitivity and specificity. To our best knowledge the introduced approach based on complex feature extraction and robust testing outperformed all works that have been published already in this field. The results (accuracy, sensitivity and specificity equal to 100.0±0.0%) are discussable in the case of Massachusetts Eye and Ear Infirmary (MEEI) database because of its limitation related to a length of sustained vowels, however in the case of Príncipe de Asturias (PdA) Hospital in Alcalá de Henares of Madrid database we made improvements in classification accuracy (82.1±3.3%) and specificity (83.8±5.1%) when considering a single-classifier approach. Hopefully, large improvements may be achieved in the case of Czech Parkinsonian Speech Database (PARCZ), which are discussed in this work as well. All the features introduced in this work were identified by Mann–Whitney U test as significant (p<0.05) when processing at least one of the mentioned databases. The largest discriminative power from these proposed features has a cepstral peak prominence extracted from the first intrinsic mode function (p=6.9443×10−32) which means, that among all newly designed features those that quantify especially hoarseness or breathiness are good candidates for pathological speech identification. The paper also mentions some ideas for the future work in the field of pathological speech signal analysis that can be valuable especially under the clinical point of view.

Bayesian Context-Dependent Learning for Anomaly Classification in Hyperspectral Imagery

Many remote sensing applications involve the classification of anomalous responses as either objects of interest or clutter. This paper addresses the problem of anomaly classification in hyperspectral imagery (HSI) and focuses on robustly detecting disturbed earth in the long-wave infrared (LWIR) spectrum. Although disturbed earth yields a distinct LWIR signature that distinguishes it from the background, its distribution relative to clutter may vary over different environmental contexts. In this paper, a generic Bayesian framework is proposed for training context-dependent classification rules from wide-area airborne LWIR imagery. The proposed framework combines sparse classification models with either supervised or discriminative context identification to pool information across contexts and improve classification overall. Experiments are performed with data from a LWIR landmine detection system. Contexts are learned from endmember abundances extracted from the background near each detected anomaly. Classification performance is compared with single-classifier approaches using the same information as well as other baseline anomaly detectors from the literature. Results indicate that utilizing context for classifying anomalies in HSI could lead to more robust performance over varying terrain.

Discovering high quality answers in community question answering archives using a hierarchy of classifiers

In community-based question answering (CQA) services where answers are generated by human, users may expect better answers than an automatic question answering system. However, in some cases, the user generated answers provided by CQA archives are not always of high quality. Most existing works on answer quality prediction use the same model for all answers, despite the fact that each answer is intrinsically different. However, modeling each individual QA pair differently is not feasible in practice. To balance between efficiency and accuracy, we propose a hybrid hierarchy-of-classifiers framework to model the QA pairs. First, we analyze the question type to guide the selection of the right answer quality model. Second, we use the information from question analysis to predict the expected answer features and train the type-based quality classifiers to hierarchically aggregate an overall answer quality score. We also propose a number of novel features that are effective in distinguishing the quality of answers. We tested the framework on a dataset of about 50 thousand QA pairs from Yahoo! Answer. The results show that our proposed framework is effective in identifying high quality answers. Moreover, further analysis reveals the ability of our framework to classify low quality answers more accurately than a single classifier approach.

Hybrid Artificial Neural Networks Modeling for Faults Identification of a Stochastic Multivariate Process

Due to the recent rapid growth of advanced sensing and production technologies, the monitoring and diagnosis of multivariate process operating performance have drawn increasing interest in process industries. The multivariate statistical process control (MSPC) chart is one of the most commonly used tools for detecting process faults. However, an out-of-control MSPC signal only indicates that process faults have intruded the underlying process. Identifying which of the monitored quality variables is responsible for the MSPC signal is fairly difficult. Pinpointing the responsible variable is vital for process improvement because it effectively determines the root causes of the process faults. Accordingly, this identification has become an important research issue concerning recent multivariate process applications. In contrast with the traditional single classifier approach, the present study proposes hybrid modeling schemes to address problems that involve a large number of quality variables in a multivariate normal process. The proposed scheme includes multivariate adaptive regression splines (MARS), logistic regression (LR), and artificial neural network (ANN). By applying MARS and LR techniques, we may obtain fewer but more significant quality variables, which can serve as inputs to the ANN classifier. The performance of our proposed approaches was evaluated by conducting a series of experiments.

Bayesian sample size determination for binary regression with a misclassified covariate and no gold standard

Covariate misclassification is a common problem in epidemiology, genetics, and other biomedical areas. Because this form of misclassification is known to bias estimators, accounting for it at the design stage is of high importance. In this paper, we extend on previous work applied to response misclassification by developing a Bayesian approach to sample size determination for a covariate misclassification model with no gold standard. Our procedure considers both conditionally independent tests and tests in which dependence exists between classifiers. We specifically consider a Bayesian power criterion for the sample size determination scheme, and we demonstrate the improvement in model power for our dual classifier approach compared to a naïve single classifier approach.

Hierarchical and parallel processing of auditory and modulation frequencies for automatic speech recognition

This paper investigates from an automatic speech recognition perspective, the most effective way of combining Multi Layer Perceptron (MLP) classifiers trained on different ranges of auditory and modulation frequencies. Two different combination schemes based on MLP are considered. The first one operates in parallel fashion and is invariant to the order in which feature streams are introduced. The second one operates in hierarchical fashion and is sensitive to the order in which feature streams are introduced. The study is carried on a Large Vocabulary Continuous Speech Recognition system for transcription of meetings data using the TANDEM approach. Results reveal that (1) the combination of MLPs trained on different ranges of auditory frequencies is more effective if performed in parallel fashion; (2) the combination of MLPs trained on different ranges of modulation frequencies is more effective if performed in hierarchical fashion moving from high to low modulations; (3) the improvement obtained from separate processing of two modulation frequency ranges (12% relative WER reduction w.r.t. the single classifier approach) is considerably larger than the improvement obtained from separate processing of two auditory frequency ranges (4% relative WER reduction w.r.t. the single classifier approach). Similar results are also verified on other LVCSR systems and on other languages. Furthermore, the paper extends the discussion to the combination of classifiers trained on separate auditory–modulation frequency channels showing that previous conclusions hold also in this scenario.