Two-stage Classification System Research Articles

A detailed study of resampling algorithms for cyberattack classification in engineering applications.

The evolution of engineering applications is highly relevant in the context of protecting industrial systems. As industries are increasingly interconnected, the need for robust cybersecurity measures becomes paramount. Engineering informatics not only provides tools for knowledge representation and extraction but also affords a comprehensive spectrum of developing sophisticated cybersecurity solutions. However, safeguarding industrial systems poses a unique challenge due to the inherent heterogeneity of data within these environments. Together with this problem, it's crucial to acknowledge that datasets that simulate real cyberattacks within these diverse environments exhibit a high imbalance, often skewed towards certain types of traffics. This study proposes a system for addressing class imbalance in cybersecurity. To do this, three oversampling (SMOTE, Borderline1-SMOTE, and ADASYN) and five undersampling (random undersampling, cluster centroids, NearMiss, repeated edited nearest neighbor, and Tomek Links) methods are tested. Particularly, these balancing algorithms are used to generate one-vs-rest binary models and to develop a two-stage classification system. By doing so, this study aims to enhance the efficacy of cybersecurity measures ensuring a more comprehensive understanding and defense against the diverse range of threats encountered in industrial environments. Experimental results demonstrates the effectiveness of proposed system for cyberattack detection and classification among nine widely known cyberattacks.

Two-Stage Traffic Sign Classification System

Traffic Sign Recognition (TSR) has received widespread attention due to increased traffic accidents caused by a failure to recognize road traffic signs. Most Traffic Sign Recognition remains challenging due to the illumination of natural scenes, and the diversity in traffic signs [31]. To solve the problem, we proposed a two-stage classification system to overcome the above-mentioned challenges and classify the traffic sign boards efficiently. During the first phase, colour based histogram equalizations applied to eliminates the illumination factors and then traffic sign boards are classified into many shapes, which includes circular, triangular, hexagonal, and diamond traffic signs boards based on the modified HOG feature descriptors and various machine learning classifiers are compared [12]. In the second phase, circular and triangular traffic signs are further categorized into specific subclasses using circular and triangular signs trained Convolutional Neural Network (CNN) respectively to overcome intra-class variation among the similar sign board. Experimentation was conducted on German Traffic Sign Recognition Benchmark dataset (GTSRB) [30] and the results show 99.96% top-1 accuracy during the first stage of classification. In the second stage, there is accuracy of 99.40% for circular traffic signs and 97.50% for triangular traffic signs.

A Weighted Ensemble Approach with Multiple Pre-trained Deep Learning Models for Classification of Stroke

Stroke ranks as one of the deadliest diseases globally, emphasizing the crucial need for early diagnosis. This study aims to create a two-stage classification system for stroke and non-stroke images to support early clinical detection. Deep learning, a cornerstone of diagnosis, detection, and prompt treatment, is the primary methodology. Transfer learning adapts successful deep learning architectures for diverse problems, and ensemble learning combines multiple classifiers for enhanced results. These two techniques are applied to classify stroke using a dataset of stroke and normal images. In the initial stage, six pre-trained models are fine-tuned, with DenseNet, Xception, and EfficientNetB2 emerging as the top performers, achieving validation accuracies of 98.4%, 98.4%, and 98%, respectively. These models serve as base learners within an ensemble framework. A weighted average ensemble method combines them, resulting in a remarkable 99.84% accuracy on a reserved test dataset. This approach exhibits promise for stroke detection, a life-threatening condition, while also demonstrating the effectiveness of ensemble techniques in enhancing model performance.

Predicting the Level of Respiratory Support in COVID-19 Patients Using Machine Learning.

In this paper, a machine learning-based system for the prediction of the required level of respiratory support in COVID-19 patients is proposed. The level of respiratory support is divided into three classes: class 0 which refers to minimal support, class 1 which refers to non-invasive support, and class 2 which refers to invasive support. A two-stage classification system is built. First, the classification between class 0 and others is performed. Then, the classification between class 1 and class 2 is performed. The system is built using a dataset collected retrospectively from 3491 patients admitted to tertiary care hospitals at the University of Louisville Medical Center. The use of the feature selection method based on analysis of variance is demonstrated in the paper. Furthermore, a dimensionality reduction method called principal component analysis is used. XGBoost classifier achieves the best classification accuracy (84%) in the first stage. It also achieved optimal performance in the second stage, with a classification accuracy of 83%.

Machine learning modeling of genome-wide copy number alteration signatures reliably predicts IDH mutational status in adult diffuse glioma

Knowledge of 1p/19q-codeletion and IDH1/2 mutational status is necessary to interpret any investigational study of diffuse gliomas in the modern era. While DNA sequencing is the gold standard for determining IDH mutational status, genome-wide methylation arrays and gene expression profiling have been used for surrogate mutational determination. Previous studies by our group suggest that 1p/19q-codeletion and IDH mutational status can be predicted by genome-wide somatic copy number alteration (SCNA) data alone, however a rigorous model to accomplish this task has yet to be established. In this study, we used SCNA data from 786 adult diffuse gliomas in The Cancer Genome Atlas (TCGA) to develop a two-stage classification system that identifies 1p/19q-codeleted oligodendrogliomas and predicts the IDH mutational status of astrocytic tumors using a machine-learning model. Cross-validated results on TCGA SCNA data showed near perfect classification results. Furthermore, our astrocytic IDH mutation model validated well on four additional datasets (AUC = 0.97, AUC = 0.99, AUC = 0.95, AUC = 0.96) as did our 1p/19q-codeleted oligodendroglioma screen on the two datasets that contained oligodendrogliomas (MCC = 0.97, MCC = 0.97). We then retrained our system using data from these validation sets and applied our system to a cohort of REMBRANDT study subjects for whom SCNA data, but not IDH mutational status, is available. Overall, using genome-wide SCNAs, we successfully developed a system to robustly predict 1p/19q-codeletion and IDH mutational status in diffuse gliomas. This system can assign molecular subtype labels to tumor samples of retrospective diffuse glioma cohorts that lack 1p/19q-codeletion and IDH mutational status, such as the REMBRANDT study, recasting these datasets as validation cohorts for diffuse glioma research.

A two-stageclassification method for borehole-wall images with support vector machine.

Analyzing geological drilling hole images acquired by Axial View Panoramic Borehole Televiewer (APBT) is a key step to explore the geological structure in a geological exploration. Conventionally, the borehole images are examined by technicians, which is inefficient and subjective. In this paper, three dominant types of borehole-wall images on coal-rock mass structure, namely, border images, fracture images and intact rock mass images are mainly studied. The traditional image classification methods based on unified feature extraction algorithm and single classifier is not effect for the borehole images. Therefore, this paper proposes a novel two-stage classification approach to improve the classification performance of borehole images. In the first-stage classification, the border images are identified from three kinds of images based on texture features and gray-scale histograms features. For the remaining two types of images, in the second-stage classification, Gabor filter is first applied to segment the region of interest (ROI) (such as microfracture, absciss layer and horizontal cracks, etc.) and the central interference region. Then, using the same feature vector after eliminating the central interference region, fracture images are separated from intact rock mass images. We test our two-stage classification system with real borehole images. The results of experimental show that the two-stage classification method can effectively classify three major borehole-wall images with the correction rate of 95.55% in the first stage and 95% in the second stage.

Raman spectroscopic detection and identification of Burkholderia mallei and Burkholderia pseudomallei in feedstuff.

Burkholderia mallei (the etiologic agent of glanders in equines and rarely humans) and Burkholderia pseudomallei, causing melioidosis in humans and animals, are designated category B biothreat agents. The intrinsically high resistance of both agents to many antibiotics, their potential use as bioweapons, and their low infectious dose, necessitate the need for rapid and accurate detection methods. Current methods to identify these organisms may require up to 1 week, as they rely on phenotypic characteristics and an extensive set of biochemical reactions. In this study, Raman microspectroscopy, a cultivation-independent typing technique for single bacterial cells with the potential for being a rapid point-of-care analysis system, is evaluated to identify and differentiate B. mallei and B. pseudomallei within hours. Here, not only broth-cultured microbes but also bacteria isolated out of pelleted animal feedstuff were taken into account. A database of Raman spectra allowed a calculation of classification functions, which were trained to differentiate Raman spectra of not only both pathogens but also of five further Burkholderia spp. and four species of the closely related genus Pseudomonas. The developed two-stage classification system comprising two support vector machine (SVM) classifiers was then challenged by a test set of 11 samples to simulate the case of a real-world-scenario, when "unknown samples" are to be identified. In the end, all test set samples were identified correctly, even if the contained bacterial strains were not incorporated in the database before or were isolated out of animal feedstuff. Specifically, the five test samples bearing B. mallei and B. pseudomallei were correctly identified on species level with accuracies between 93.9 and 98.7%. The sample analysis itself requires no biomass enrichment step prior to the analysis and can be performed under biosafety level 1 (BSL 1) conditions after inactivating the bacteria with formaldehyde.

Identification of Bacillus anthracis via Raman Spectroscopy and Chemometric Approaches

Raman micro-spectroscopy was applied to compile a large-scale database of Raman spectra of single Bacillus endospores and to calculate classification functions, which were trained to discriminate between endospores of 66 strains from 13 Bacillus and Bacillus-related species including B. anthracis. The developed two-stage classification system comprising two support vector machines and one linear discriminant analysis classifier was then challenged by a test set of 27 samples to simulate the case of a real-world-scenario, when "unknown samples" are to be identified. In the end, all 27 test set samples including six B. anthracis strains were identified correctly. The samples thereby covered a diverse selection of species within the phylogenetically broad Bacillus genus and also included strains, which were not incorporated in the database before. All of them were correctly identified on the species level with accuracies between 88 and 100%. The sample analysis itself requires no biomass enrichment step prior to the analysis and qualifies the presented Raman spectroscopic approach to be a rapid analysis system in term of Bacillus endospore typing.

A preliminary two-stage alarm correlation and filtering system using SOM neural network and K-means algorithm

Intrusion Detection Systems (IDSs) play a vital role in the overall security infrastructure. Although the IDS has become an essential part of corporate network infrastructure, the art of detecting intrusion is still far from perfect. A significant problem is that of false alarms, as generating a huge volume of such alarms could render the system inefficient. In this paper, we propose a new method to reduce the number of false alarms. We develop a two-stage classification system using a SOM neural network and K-means algorithm to correlate the related alerts and to further classify the alerts into classes of true and false alarms. Preliminary experiments show that our approach effectively reduces all superfluous and noisy alerts, which often contribute to more than 50% of false alarms generated by a common IDS.

Arabic handwritten digit recognition

In this paper, we fill a gap in the literature by studying the problem of Arabic handwritten digit recognition. The performances of different classification and feature extraction techniques on recognizing Arabic digits are going to be reported to serve as a benchmark for future work on the problem. The performance of well known classifiers and feature extraction techniques will be reported in addition to a novel feature extraction technique we present in this paper that gives a high accuracy and competes with the state-of-the-art techniques. A total of 54 different classifier/features combinations will be evaluated on Arabic digits in terms of accuracy and classification time. The results are analyzed and the problem of the digit ‘0’ is identified with a proposed method to solve it. Moreover, we propose a strategy to select and design an optimal two-stage system out of our study and, hence, we suggest a fast two-stage classification system for Arabic digits which achieves as high accuracy as the highest classifier/features combination but with much less recognition time.

Effectiveness of global features for automatic medical image classification and retrieval – The experiences of OHSU at ImageCLEFmed

In 2006 and 2007, Oregon Health and Science University (OHSU) participated in the automatic image annotation task for medical images at ImageCLEF, an annual international benchmarking event that is part of the cross language evaluation forum (CLEF). The goal of the automatic annotation task was to classify 1000 test images based on the image retrieval in medical applications (IRMA) code, given a set of 10,000 training images. There were 116 distinct classes in 2006 and 2007. We evaluated the efficacy of a variety of primarily global features for this classification task. These included features based on histograms, gray level correlation matrices and the gist technique. A multitude of classifiers including k-nearest neighbors, two-level neural networks, support vector machines, and maximum likelihood classifiers were evaluated. Our official error rates for the 1000 test images were 26% in 2006 using the flat classification structure. The error count in 2007 was 67.8 using the hierarchical classification error computation based on the IRMA code in 2007. Confusion matrices as well as clustering experiments were used to identify visually similar classes. The use of the IRMA code did not help us in the classification task as the semantic hierarchy of the IRMA classes did not correspond well with the hierarchy based on clustering of image features that we used. Our most frequent misclassification errors were along the view axis. Subsequent experiments based on a two-stage classification system decreased our error rate to 19.8% for the 2006 dataset and our error count to 55.4 for the 2007 data.

Detection of coronary calcifications from computed tomography scans for automated risk assessment of coronary artery disease

A fully automated method for coronary calcification detection from non-contrast-enhanced, ECG-gated multi-slice computed tomography (CT) data is presented. Candidates for coronary calcifications are extracted by thresholding and component labeling. These candidates include coronary calcifications, calcifications in the aorta and in the heart, and other high-density structures such as noise and bone. A dedicated set of 64 features is calculated for each candidate object. They characterize the object's spatial position relative to the heart and the aorta, for which an automatic segmentation scheme was developed, its size and shape, and its appearance, which is described by a set of approximated Gaussian derivatives for which an efficient computational scheme is presented. Three classification strategies were designed. The first one tested direct classification without feature selection. The second approach also utilized direct classification, but with feature selection. Finally, the third scheme employed two-stage classification. In a computationally inexpensive first stage, the most easily recognizable false positives were discarded. The second stage discriminated between more difficult to separate coronary calcium and other candidates. Performance of linear, quadratic, nearest neighbor, and support vector machine classifiers was compared. The method was tested on 76 scans containing 275 calcifications in the coronary arteries and 335 calcifications in the heart and aorta. The best performance was obtained employing a two-stage classification system with a k-nearest neighbor (k-NN) classifier and a feature selection scheme. The method detected 73.8% of coronary calcifications at the expense of on average 0.1 false positives per scan. A calcium score was computed for each scan and subjects were assigned one of four risk categories based on this score. The method assigned the correct risk category to 93.4% of all scans.

Combining Model-based and Discriminative Approaches in a Modular Two-stage Classification System: Application to Isolated Handwritten Digit Recognition

The motivation of this work is based on two key observations. First, the classification algorithms can be separated into two main categories: discriminative and model-based approaches. Second, two types of patterns can generate problems: ambiguous patterns and outliers. While, the first approach tries to minimize the first type of error, but cannot deal effectively with outliers, the second approach, which is based on the development of a model for each class, make the outlier detection possible, but are not sufficiently discriminant. Thus, we propose to combine these two different approaches in a modular two-stage classification system embedded in a probabilistic framework. In the first stage we pre-estimate the posterior probabilities with a model-based approach and we re-estimate only the highest probabilities with appropriate Support Vector Classifiers (SVC) in the second stage. Another advantage of this combination is to reduce the principal burden of SVC, the processing time necessary to make a decision and to open the way to use SVC in classification problem with a large number of classes. Finally, the first experiments on the benchmark database MNIST have shown that our dynamic classification process allows to maintain the accuracy of SVCs, while decreasing complexity by a factor 8.7 and making the outlier rejection available.

Plant performance studies on a hydraulic cone classifier as a secondary classifier

Hydrocyclones have found wide acceptance as classifiers because of their low cost and flexibility in operation. However, they are not very efficient as classifiers in closed circuit grinding. To improve their classification efficiency, a two-stage classification system involving two hydrocyclones in series has been successfully implemented at Rakha Concentrator of M/s Hindustan Copper Limited. This involved extra pumping to the second stage of classification. In order to avoid this, a modified “gravitational cone classifier” has been installed in place of the secondary cyclone. This paper describes briefly the modified system and the observations made during the plant trials.