Hierarchical Classification Model Research Articles

A clustering machine learning approach for improving concrete compressive strength prediction

AbstractThis study investigates the application of clustering techniques to enhance the accuracy of hierarchical classification and regression (HCR) models for predicting concrete compressive strength (CCS). Following the hypothesis that integrating clustering at the initial levels of model hierarchy reduces classification errors and prevents their propagation to subsequent levels, HCR models were developed utilizing both the unweighted pair group method with arithmetic mean (UPGMA) and hard clustering (HC) methods. Findings demonstrate that models using UPGMA significantly outperform those based on HC. Furthermore, it was demonstrated that further hierarchical clustering allows for multilayered HCR models that improve predictive performance by further leveraging parent–child relationships within data clusters. Overall, this study demonstrates that the proposed methodology can be introduced in the model development pipeline to enhance the prediction accuracy of CCS models.

On the importance of integrating convolution features for Indian medicinal plant species classification using hierarchical machine learning approach

This work proposes a novel hierarchical classification framework designed to categorize hundred Indian medicinal plant species. The innovation lies in introducing a comprehensive feature representation by integrating convolutional features with geometric, texture, shape, and multispectral features for classification tasks. In this study, a two-level hierarchical plant classification model is proposed to address the challenges of inter-class similarity and intra-class variations. The first level classifies 100 medicinal plant species into 11 groups based on visual similarities among the plants. At level two, the specific plant species containing in each group are predicted using Random Forest classifier. The evaluation is performed at two levels to analyze the effectiveness of the proposed model. The performance analysis compares the effectiveness of individual feature types against the composite feature model. Performance is also evaluated based on specific groups that demonstrate high similarity between classes and intra-class variations among the plant species separately. Furthermore, the generality of the model is tested using two self-created datasets-RTL80 and RTP40, requiring more than 300 man-hours to collect. Experimental results demonstrate a promising accuracy of 94.54% on GSL100 leaf dataset and 75.46% on RTL80 and RTP40 real-time datasets reflecting the superiority of the proposed hierarchical model over state-of-the-art methods.

Abnormal line loss identification and category classification of distribution networks based on semi-supervised learning and hierarchical classification

Line loss refers to the electrical energy that is dissipated as heat during the transmission and distribution of electricity through power lines. However, unusual causes, such as grid topology mismatch and communication failure, can cause abnormal line loss. Efficient abnormal line loss detection contributes not only to minimizing energy wastage and reducing carbon emissions but also to maintaining the stability and reliability of the entire distribution network. In actual situations, the cause of abnormal line loss is not labeled due to the expensive labor cost. This paper proposes a hierarchical abnormal line loss identification and category classification model, considering the unlabeled and unbalanced sample problem. First, an abnormal line loss identification model-based random forest is established to detect whether the line loss is abnormal. Then, an abnormal line loss category classification model is developed with semi-supervised learning for line loss abnormal category classification, considering the unlabeled samples. The real dataset in China is utilized to validate the performance of the proposed model. Its reliability implies the potential to be applied to real-world scenarios to improve the management level and safety of the power grid.

DeepMedFeature: An Accurate Feature Extraction and Drug-Drug Interaction Model for Clinical Text in Medical Informatics

Drug-drug interactions (DDIs) are an important biological phenomenon which can result in medical errors from medical practitioners. Drug interactions can change the molecular structure of interacting agents which may prove to be fatal in the worst case. Finding drug interactions early in diagnosis can be pivotal in side-effect prevention. The growth of big data provides a rich source of information for clinical studies to investigate DDIs. We propose a hierarchical classification model which is double-pass in nature. The first pass predicts the occurrence of an interaction and then the second pass further predicts the type of interaction such as effect, advice, mechanism, and int. We applied different deep learning algorithms with Convolutional Bi-LSTM (ConvBLSTM) proving to be the best. The results show that pre-trained vector embeddings prove to be the most appropriate features. The F1-score of the ConvBLSTM algorithm turned out to be 96.39% and 98.37% in Russian and English language respectively which is greater than the state-of-the-art systems. According to the results, it can be concluded that adding a convolution layer before the bi-directional pass improves model performance in the automatic classification and extraction of drug interactions, using pre-trained vector embeddings such as Fasttext and Bio-Bert.

A Geographic Object-Based Image Approach Based on the Sentinel-2 Multispectral Instrument for Lake Aquatic Vegetation Mapping: A Complementary Tool to In Situ Monitoring

Aquatic vegetation is an essential component of lake ecosystems, used as a biological indicator for in situ monitoring within the Water Framework Directive. We developed a hierarchical object-based image classification model with multi-seasonal Sentinel-2 imagery and suitable spectral indices in order to map the aquatic vegetation in a Mediterranean oligotrophic/mesotrophic deep lake; we then applied the model to another lake with similar abiotic and biotic characteristics. Field data from a survey of aquatic macrophytes, undertaken on the same dates as EO data, were used within the accuracy assessment. The aquatic vegetation was discerned into three classes: emergent, floating, and submerged aquatic vegetation. Geographic object-based image analysis (GEOBIA) proved to be effective in discriminating the three classes in both study areas. Results showed high effectiveness of the classification model in terms of overall accuracy, particularly for the emergent and floating classes. In the case of submerged aquatic vegetation, challenges in their classification prompted us to establish specific criteria for their accurate detection. Overall results showed that GEOBIA based on spectral indices was suitable for mapping aquatic vegetation in oligotrophic/mesotrophic deep lakes. EO data can contribute to large-scale coverage and high-frequency monitoring requirements, being a complementary tool to in situ monitoring.

Classification and Value Assessment of Sports Intangible Cultural Heritage Resources Combined with Digital Technology

Abstract This paper is dedicated to designing and constructing a knowledge ontology framework for sports intangible cultural heritage (ICH) resources, aiming to support their preservation and inheritance by integrating and mining sports ICH resources. The study includes the collection of multiple data of sports ICH from various data sources, and constructing ICH knowledge ontology using CIDOC CRM metadata reference model and seven-step method. To enrich the content of the ontology, the TextRank algorithm is used to extract textual critical information and design a domain-specific NER model for sports NRL. In addition, the study adopts the VSM vector space model for text representation and uses an improved hierarchical classification model for text categorization to improve classification accuracy. The study also explores the similarity calculation of concepts in the sports NRM ontology and proposes a semantic similarity calculation formula based on the ontology concepts. Respondents’ willingness to pay was investigated through the conditional value method (CVM) to assess the value of sports NRM tourism resources. Finally, the influencing factors of respondents’ willingness to pay were analyzed using statistical analysis and Logistic regression model, and it was found that they were mainly influenced by the degree of understanding of sports non-heritage resources and the level of education. The results of this study not only provide theoretical and methodological support for the effective integration and excavation of sports non-heritage resources and a new perspective for their protection, inheritance and sustainable development.

A Bloom Cognitive Hierarchical Classification Model for Chinese Exercises Based on Improved Chinese-RoBERTa-wwm and BiLSTM

A Bloom Cognitive Hierarchical Classification Model for Chinese Exercises Based on Improved Chinese-RoBERTa-wwm and BiLSTM

Bayesian workflow for the investigation of hierarchical classification models from tau-PET and structural MRI data across the Alzheimer's disease spectrum.

Alzheimer's disease (AD) diagnosis in its early stages remains difficult with current diagnostic approaches. Though tau neurofibrillary tangles (NFTs) generally follow the stereotypical pattern described by the Braak staging scheme, the network degeneration hypothesis (NDH) has suggested that NFTs spread selectively along functional networks of the brain. To evaluate this, we implemented a Bayesian workflow to develop hierarchical multinomial logistic regression models with increasing levels of complexity of the brain from tau-PET and structural MRI data to investigate whether it is beneficial to incorporate network-level information into an ROI-based predictive model for the presence/absence of AD. This study included data from the Translational Biomarkers in Aging and Dementia (TRIAD) longitudinal cohort from McGill University's Research Centre for Studies in Aging (MCSA). Baseline and 1 year follow-up structural MRI and [18F]MK-6240 tau-PET scans were acquired for 72 cognitive normal (CN), 23 mild cognitive impairment (MCI), and 18 Alzheimer's disease dementia subjects. We constructed the four following hierarchical Bayesian models in order of increasing complexity: (Model 1) a complete-pooling model with observations, (Model 2) a partial-pooling model with observations clustered within ROIs, (Model 3) a partial-pooling model with observations clustered within functional networks, and (Model 4) a partial-pooling model with observations clustered within ROIs that are also clustered within functional brain networks. We then investigated which of the models had better predictive performance given tau-PET or structural MRI data as an input, in the form of a relative annualized rate of change. The Bayesian leave-one-out cross-validation (LOO-CV) estimate of the expected log pointwise predictive density (ELPD) results indicated that models 3 and 4 were substantially better than other models for both tau-PET and structural MRI inputs. For tau-PET data, model 3 was slightly better than 4 with an absolute difference in ELPD of 3.10 ± 1.30. For structural MRI data, model 4 was considerably better than other models with an absolute difference in ELPD of 29.83 ± 7.55 relative to model 3, the second-best model. Our results suggest that representing the data generating process in terms of a hierarchical model that encompasses both ROI-level and network-level heterogeneity leads to better predictive ability for both tau-PET and structural MRI inputs over all other model iterations.

Prediction of fractional flow reserve with enhanced ant lion optimized support vector machine

The evaluation of coronary morphology provides important guidance for the treatment of coronary heart disease (CHD). A chaotic Gaussian mutation antlion optimizer algorithm (CGALO) is proposed in the paper, and it is combined with SVM to construct a classification prediction model for Fractional flow reserve (FFR). To overcome the limitations of the original antlion optimizer (ALO) algorithm, the chaotic Gaussian mutation strategy is introduced, which leads to an improvement in its convergence speed and accuracy. To evaluate the proposed algorithm's performance, comparative experiments were conducted on 23 benchmark functions alongside 12 other cutting-edge optimization algorithms. The experimental outcomes demonstrate that the proposed algorithm achieves superior convergence accuracy and speed compared to the alternative comparison algorithms. Additionally, it is combined with SVM and FS to construct a hierarchical FFR classification model, which is utilized to make effective predictions for 84 patients at the affiliated hospital of medical school, Ningbo university. The experimental results demonstrate that the proposed model achieves an average accuracy of 92%. Moreover, it concludes that smoking history, number of lesion vessels, lesion location, diffuse lesions and ST segment changes, and other factors are the most critical indicators for FFR. Therefore, the model that has been established is a new FFR intelligent classification prediction technology that can effectively assist doctors in making corresponding decisions and evaluation plans.

ENHANCING NON-NATIVE ACCENT RECOGNITION THROUGH A COMBINATION OF SPEAKER EMBEDDINGS, PROSODIC AND VOCAL SPEECH FEATURES

The transcription accuracy of automatic speech recognition (ASR) system may suffer when recognizing accented speech. The resulting bias in ASR system towards a specific accent due to under representation of that accent in the training dataset. Accent recognition of existing speech samples can help with the preparation of the training datasets, which is an important step toward closing the accent gap and eliminating biases in ASR system. For that we built a system to recognize accent from spoken speech data. In this study, we have explored some prosodic and vocal speech features as well as speaker embeddings for accent recognition on our custom English speech data that covers speakers from around the world with varying accents. We demonstrate that our selected speech features are more effective in recognizing nonnative accents. Additionally, we experimented with a hierarchical classification model for multi-level accent classification. To establish an accent hierarchy, we employed a bottom-up approach, combining regional accents and categorizing them as either native or non-native at the top level. Furthermore, we conducted a comparative study between flat classification and hierarchical classification using the accent hierarchy structure.

Traffic Sign Detection and Recognition Using Multi-Frame Embedding of Video-Log Images

The detection and recognition of traffic signs is an essential component of intelligent vehicle perception systems, which use on-board cameras to sense traffic sign information. Unfortunately, issues such as long-tailed distribution, occlusion, and deformation greatly decrease the detector’s performance. In this research, YOLOv5 is used as a single classification detector for traffic sign localization. Afterwards, we propose a hierarchical classification model (HCM) for the specific classification, which significantly reduces the degree of imbalance between classes without changing the sample size. To cope with the shortcomings of a single image, a training-free multi-frame information integration module (MIM) was constructed, which can extract the detection sequence of traffic signs based on the embedding generated by the HCM. The extracted temporal detection information is used for the redefinition of categories and confidence. At last, this research performed detection and recognition of the full class on two publicly available datasets, TT100K and ONCE. Experimental results show that the HCM-improved YOLOv5 has a mAP of 79.0 in full classes, which exceeds that of state-of-the-art methods, and achieves an inference speed of 22.7 FPS. In addition, MIM further improves model performance by integrating multi-frame information while only slightly increasing computational resource consumption.

Contextual Embeddings based on Fine-tuned Urdu-BERT for Urdu threatening content and target identification

Identification of threatening text on social media platforms is a challenging task. Contrary to the high-resource languages, the Urdu language has very limited such approaches and the benchmark approach has an issue of inappropriate data annotation. Therefore, we present robust threatening content and target identification as a hierarchical classification model for Urdu tweets. This study investigates the potential of the Urdu-BERT (Bidirectional Encoder Representations from Transformer) language model to learn universal contextualized representations aiming to showcase its usefulness for binary classification tasks of threatening content and target identification. We propose to exploit a pre-trained Urdu-BERT as a transfer learning model after fine-tuning its parameters on a newly designed Urdu corpus from the Twitter platform. The proposed dataset contains 2,400 tweets manually annotated as threatening or non-threatening at the first level and threatening tweets are further categorized into individual or group at the second level. The performance of fine-tuned Urdu-BERT is compared with the benchmark study and other feature models. Experimental results show that the fine-tuned Urdu-BERT model achieves state-of-the-art performance by obtaining 87.5% accuracy and 87.8% F1-score for threatening content identification and 82.5% accuracy and 83.2% F1-score for target identification task. Furthermore, the proposed model outperforms the benchmark study.

A Testlet Diagnostic Classification Model with Attribute Hierarchies.

In this article, a testlet hierarchical diagnostic classification model (TH-DCM) was introduced to take both attribute hierarchies and item bundles into account. The expectation-maximization algorithm with an analytic dimension reduction technique was used for parameter estimation. A simulation study was conducted to assess the parameter recovery of the proposed model under varied conditions, and to compare TH-DCM with testlet higher-order CDM (THO-DCM; Hansen, M. (2013). Hierarchical item response models for cognitive diagnosis (Unpublished doctoral dissertation). UCLA; Zhan, P., Li, X., Wang, W.-C., Bian, Y., & Wang, L. (2015). The multidimensional testlet-effect cognitive diagnostic models. Acta Psychologica Sinica, 47(5), 689. https://doi.org/10.3724/SP.J.1041.2015.00689). Results showed that (1) ignoring large testlet effects worsened parameter recovery, (2) DCMs assuming equal testlet effects within each testlet performed as well as the testlet model assuming unequal testlet effects under most conditions, (3) misspecifications in joint attribute distribution had an differential impact on parameter recovery, and (4) THO-DCM seems to be a robust alternative to TH-DCM under some hierarchical structures. A set of real data was also analyzed for illustration.

BADM-Net: Hierarchical Classification Network for Identifying Anomalous Trends in Bridge Monitoring Data Patterns

SHM systems have been widely implemented in long-span bridges, and seas of field measurement data have been accumulated. Due to the imperfect sensors, data transmission and acquisition, various anomalies inevitably exist in the SHM data, which may lead to unreliable structural condition assessment. Thus, an effective approach for detecting data anomalies is highly desirable. Due to the imbalanced data, some anomalous patterns are undertrained in popular end-to-end deep neural network models, resulting in a reduction in detection precision. In this paper, a hierarchical classification model with deep neural network tree is proposed for imbalanced data. The DNN tree contains three levels: (1) CNN to divide seven types of data into four categories (134, 2, 5, 67), denoted as C4; (2) two DNNs to classify to two classes separately (1, 34, 6, 7), denoted as D2D2; (3) DNNs to classify to two classes (3, 4). So, the DNN tree is presented as C4_D2D2_D2. The DNN tree is an open framework and can be defined based on the data characteristics. In the data processing, three data sets are built for training, namely single-channel data set, dual-channel data set and statistical data set. To validate our work, we considered the effects of balanced and imbalanced training sets and training ratios. The results show that our model can detect the multi-pattern anomalies of SHM data efficiently with 95.5% high accuracy. Besides, the proportion of abnormal data classified to normal data has been reduced, especially 3-minor. This model successfully solves the problem in a simple and easy to understand way, which has certain reference significance for the bridge structure anomaly judgment in the future.

Galaxy image classification using hierarchical data learning with weighted sampling and label smoothing

ABSTRACT With the development of a series of Galaxy sky surveys in recent years, the observations increased rapidly, which makes the research of machine learning methods for galaxy image recognition a hot topic. Available automatic galaxy image recognition researches are plagued by the large differences in similarity between categories, the imbalance of data between different classes, and the discrepancy between the discrete representation of Galaxy classes (DDRGC) and the essentially gradual changes from one morphological class to the adjacent class. These limitations have motivated several astronomers and machine learning experts to design projects with improved galaxy image recognition capabilities. Therefore, this paper proposes a novel learning method, ‘Hierarchical Imbalanced data learning with Weighted sampling and Label smoothing’ (HIWL). The HIWL consists of three key techniques respectively dealing with the above-mentioned three problems: (1) designed a hierarchical galaxy classification model based on an efficient backbone network; (2) utilized a weighted sampling scheme to deal with the imbalance problem; and (3) adopted a label smoothing technique to alleviate the DDRGC problem. We applied this method to galaxy photometric images from the Galaxy Zoo-The Galaxy Challenge, exploring the recognition of completely round smooth, in between smooth, cigar-shaped, edge-on, and spiral. The overall classification accuracy is 96.32 per cent, and some superiorities of the HIWL are shown based on recall, precision, and F1-Score in comparing with some related works. In addition, we also explored the visualization of the galaxy image features and model attention to understand the foundations of the proposed scheme.

Hierarchical Semantic Risk Minimization for Large-Scale Classification

Hierarchical structures of labels usually exist in large-scale classification tasks, where labels can be organized into a tree-shaped structure. The nodes near the root stand for coarser labels, while the nodes close to leaves mean the finer labels. We label unseen samples from the root node to a leaf node, and obtain multigranularity predictions in the hierarchical classification. Sometimes, we cannot obtain a leaf decision due to uncertainty or incomplete information. In this case, we should stop at an internal node, rather than going ahead rashly. However, most existing hierarchical classification models aim at maximizing the percentage of correct predictions, and do not take the risk of misclassifications into account. Such risk is critically important in some real-world applications, and can be measured by the distance between the ground truth and the predicted classes in the class hierarchy. In this work, we utilize the semantic hierarchy to define the classification risk and design an optimization technique to reduce such risk. By defining the conservative risk and the precipitant risk as two competing risk factors, we construct the balanced conservative/precipitant semantic (BCPS) risk matrix across all nodes in the semantic hierarchy with user-defined weights to adjust the tradeoff between two kinds of risks. We then model the classification process on the semantic hierarchy as a sequential decision-making task. We design an algorithm to derive the risk-minimized predictions. There are two modules in this model: 1) multitask hierarchical learning and 2) deep reinforce multigranularity learning. The first one learns classification confidence scores of multiple levels. These scores are then fed into deep reinforced multigranularity learning for obtaining a global risk-minimized prediction with flexible granularity. Experimental results show that the proposed model outperforms state-of-the-art methods on seven large-scale classification datasets with the semantic tree.

Visualization of confusion matrices with network graphs

AbstractThe use of network analysis as a means of visualizing the off‐diagonal (misclassified) elements of a confusion matrix is demonstrated, and the potential to use the network graphs as a guide for developing hierarchical classification models is presented. A very brief summary of graph theory is described. This is followed by an explanation and code with examples of how these networks can then be used for visualization of confusion matrices. The use of network graphs to provide insight into differing model performance is also addressed.

Probabilistic Cascading Classifier for Energy-Efficient Activity Monitoring in Wearables

Advances in embedded systems have given rise to integrating several small-size health monitoring devices within daily human life. This trend led to an ongoing extension of wearable sensors in a broad range of applications. Wearable technologies, which are firmly connected with the human body, utilize sensors and machine learning to describe individuals’ physical or psychological routines through activity recognition and human movement. Since wearables are used all day long, the power consumption of these systems needs to be reasonably low. Current research considers that such machine learning methods are trained with fixed properties, including sensor sampling rate and statistical features computed from the time series data. However, in reality, wearables require continuous reconfiguration of their computational algorithms due to the personalized nature of human gait and movement. Furthermore, computational algorithms must become energy- and memory-efficient due to these embedded sensors’ limited power and memory. In this paper, we propose a resource-efficient framework for real-time, continuous, and on-node human activity recognition. Typically activity recognition problem is a multi-class classification problem. However, we suggest transforming this problem based on MET (Metabolic Equivalent of Task) into a hierarchical classification model, providing personalized structure for each individual. We discuss the design and construction of this new configurable classification paradigm. Our results demonstrate that the proposed probabilistic cascading system accuracy for different personalized scenarios varies between 94.5% and 96.9% in detecting activities using a limited memory, while power usage of the system is reduced by as high as 17.2% compared to the traditional methods.

Advanced content-based retrieval for digital correspondence documents with ontology classification

The growth of digital correspondence documents with various types, different naming rules, and no sufficient search system complicates the search process with certain content, especially if there are unclassified documents, the search becomes inaccurate and takes a long time. This research proposed archiving method with automatic hierarchical classification and the content-based search method which displays ontology classification information as the solution to the content-based search problems. The method consists of preprocessing (creation of automatic hierarchical classification model using a combination of convolutional neural network (CNN) and regular expression method), archiving (document archiving with automatic classification), and retrieval (content-based search by displaying ontology relationships from the document classification). The archiving of 100 documents using the automatic hierarchical classification was found to be 79% accurate as indicated by the 99% accuracy for CNN and 80% for Regex. Moreover, the search results for classified content-based documents through the display of ontology relationships were discovered to be 100% accurate. This research succeeded in improving the quality of search results for digital correspondence documents as indicated by its higher specificity, accuracy, and speed compared to conventional methods based on file names, annotations, and unclassified content.

A Hierarchical Genotyping Framework Using DNA Melting Temperatures Applied to Adenovirus Species Typing.

Known genetic variation, in conjunction with post-PCR melting curve analysis, can be leveraged to provide increased taxonomic detail for pathogen identification in commercial molecular diagnostic tests. Increased taxonomic detail may be used by clinicians and public health decision-makers to observe circulation patterns, monitor for outbreaks, and inform testing practices. We propose a method for expanding the taxonomic resolution of PCR diagnostic systems by incorporating a priori knowledge of assay design and sequence information into a genotyping classification model. For multiplexed PCR systems, this framework is generalized to incorporate information from multiple assays to increase classification accuracy. An illustrative hierarchical classification model for human adenovirus (HAdV) species was developed and demonstrated ~95% cross-validated accuracy on a labeled dataset. The model was then applied to a near-real-time surveillance dataset in which deidentified adenovirus detected patient test data from 2018 through 2021 were classified into one of six adenovirus species. These results show a marked change in both the predicted prevalence for HAdV and the species makeup with the onset of the COVID-19 pandemic. HAdV-B decreased from a pre-pandemic predicted prevalence of up to 40% to less than 5% in 2021, while HAdV-A and HAdV-F species both increased in predicted prevalence.