Interpretable Classification Models Research Articles

Classification of Caries Based on CBCT: A Deep Learning Network Interpretability Study.

This study aimed to create a caries classification scheme based on cone-beam computed tomography (CBCT) and develop two deep learning models to improve caries classification accuracy. A total of 2713 axial slices were obtained from CBCT images of 204 carious teeth. Both classification models were trained and tested using the same pretrained classification networks on the dataset, including ResNet50_vd, MobileNetV3_large_ssld, and ResNet50_vd_ssld. The first model was used directly to classify the original images (direct classification model). The second model incorporated a presegmentation step for interpretation (interpretable classification model). Performance evaluation metrics including accuracy, precision, recall, and F1 score were calculated. The Local Interpretable Model-agnostic Explanations (LIME) method was employed to elucidate the decision-making process of the two models. In addition, a minimum distance between caries and pulp was introduced for determining the treatment strategies for type II carious teeth. The direct model that utilized the ResNet50_vd_ssld network achieved top accuracy, precision, recall, and F1 score of 0.700, 0.786, 0.606, and 0.616, respectively. Conversely, the interpretable model consistently yielded metrics surpassing 0.917, irrespective of the network employed. The LIME algorithm confirmed the interpretability of the classification models by identifying key image features for caries classification. Evaluation of treatment strategies for type II carious teeth revealed a significant negative correlation (p < 0.01) with the minimum distance. These results demonstrated that the CBCT-based caries classification scheme and the two classification models appeared to be acceptable tools for the diagnosis and categorization of dental caries.

Unraveling the relationship between coastal landscapes and sentiments: An integrated approach based on social media data and interpretable machine learning methods

AbstractCoastal landscapes exert a significant impact on the human sentimental perceptions and physical and mental well‐being of people. However, little is known about explicitly linking between the landscape characteristics and people's sentimental preferences expressed in social media data. The main objective of this study was to explore the nonlinear and interaction effects of key factors that influenced sentiments in the coastal areas of Hong Kong, considering both subjective landscape preferences and objective landscape patterns. We quantified users' sentiment polarity based on the crowdsourcing textual data of Flickr. To study users' subjective landscape preferences, we computed various visual landscape objects' proportion in images. Meanwhile, eight user clusters and nine image clusters were detected by the identified visual object labels. We quantified objective landscape patterns considering the land use pattens and the availability of public service facilities. Finally, we utilized an interpretable classification model to analyze the factors that may affect sentiments and their interplay interactions. We found that ecotourism‐related clusters exhibited the most positive sentiment. The proportion of floor and sky pixels in images exhibits the highest global relative importance when predicting sentiments. This study extends a new insight on the relationship between landscape characteristics and sentiments from both subjective and objective perspectives based on social media data and interpretable machine learning methods. This research may help decision‐makers in designing landscapes that aptly satisfy to the needs of the public and promote sustainable management of the coastal environment.

Differentiating Pressure Ulcer Risk Levels through Interpretable Classification Models Based on Readily Measurable Indicators.

Pressure ulcers carry a significant risk in clinical practice. This paper proposes a practical and interpretable approach to estimate the risk levels of pressure ulcers using decision tree models. In order to address the common problem of imbalanced learning in nursing classification datasets, various oversampling configurations are analyzed to improve the data quality prior to modeling. The decision trees built are based on three easily identifiable and clinically relevant pressure ulcer risk indicators: mobility, activity, and skin moisture. Additionally, this research introduces a novel tabular visualization method to enhance the usability of the decision trees in clinical practice. Thus, the primary aim of this approach is to provide nursing professionals with valuable insights for assessing the potential risk levels of pressure ulcers, which could support their decision-making and allow, for example, the application of suitable preventive measures tailored to each patient's requirements. The interpretability of the models proposed and their performance, evaluated through stratified cross-validation, make them a helpful tool for nursing care in estimating the pressure ulcer risk level.

Omics-CNN: A comprehensive pipeline for predictive analytics in quantitative omics using one-dimensional convolutional neural networks

Background and objectiveThe development of machine learning-based models that can be used for the prediction of severe diseases has been one of the main concerns of the scientific community. The current study seeks to expand a highly sophisticated tool, the Convolutional Neural Networks, making it applicable in multidimensional omics data classification problems and testing the newly introduced method on publicly available transcriptomics and proteomics data. MethodsIn this study, we introduce Omics-CNN, a Convolutional Neural Network-based pipeline, which couples Convolutional Neural Networks with dimensionality reduction, preprocessing, clustering, and explainability techniques to make them suitable to build highly accurate and interpretable classification models from high-throughput omics data. The developed tool has the potential to classify patients depending on the expression of genetic and clinical factors and identify features that can act as diagnostic biomarkers. Regarding dimensionality reduction, univariate and multivariate techniques were explored and compared. Gradient Weighted Class Activation Mapping analysis was performed to determine the most important features in the classification of the samples after training the model. ResultsThe newly introduced pipeline was applied to one transcriptomics and one proteomics dataset for the identification of diagnostic models and biosignatures for Ischemic Stroke (IS) and COVID-19 infection, reporting highly accurate biosignatures with accuracies of 96 % and 95.41 %, respectively. Meanwhile, classification models based solely on a small part of attributes provided lower predictive accuracy, but identified compact transcript biosignature (KRT15, VPRBP, TNFRSF4, GORASP2) for Ischemic Stroke and protein biosignature (ADGRB3, VNN2, AGER, CIAPIN1) for Covid-19 infection diagnosis, respectively. ConclusionsOmics-CNN, overcame the inherent problems of applying Convolutional Neural Networks for the training diagnostic models with quantitative omics data, outperforming previous models of machine learning developed using the same datasets for Ischemic Stroke and Covid-19 infection diagnosis, determining the most contributing biomarkers for both diseases.

Excitation-emission fluorescence spectroscopy and sparse chemometric methods for grape seed oil classification and authentication

Grape seed oil (GSO) authentication and classification have recently become the most important aspects of edible oil quality control due to its high nutritional values. In the present contribution, excitation-emission fluorescence spectroscopy and sparse chemometric methods were used for the classification of GSO obtained from different Iranian grape genotypes and the detection of GSO adulteration with refined sunflower oil (SFO). The fluorescence spectra were collected in the wavelength ranges of λex = 200–500 nm and λem = 200–800 nm. To develop multivariate models, more than 200 samples from five different varieties of GSO were used. The sparse version of N-way partial least squares discriminant analysis (sNPLS-DA) was utilized to build an interpretable classification model. Classification accuracies for the sNPLS-DA model were 1.00 for predicting all grape genotypes. The collected fluorescence data revealed a distinct intensity difference between the Chafteh GSO and other GSOs in λex = 270–310 nm, and λem = 300–350 nm. To simulate the adulteration of GSO with SFO, 35 binary blends of Chafteh GSO samples (as a higher-quality oil) were prepared at adulterant levels from 10% to 50%. Sparse multivariate regression techniques, including the least absolute shrinkage and selection operator (Lasso), Ridge, and Elastic net were applied for quantitative analysis of GSO adulteration. The coefficient of multiple determination (R2) and the RMSE for the Lasso model were 0.914 and 0.013, respectively, for the external test sets. The results showed that the sparse chemometric methods coupled with excitation-emission fluorescence spectroscopy can be used as interpretable and reliable tools for quality control and adulteration detection of oil samples in the food industry and commerce.

An integrated approach for fuzzy rule generation in dataset classification using hybrid grid partitioning and rough set theory

This research presents an integrated approach for fuzzy rule generation in dataset classification by combining hybrid grid partitioning and rough set theory. The objective is to enhance the accuracy and interpretability of classification models. The approach leverages hybrid grid partitioning to achieve localized rule generation, capturing the local characteristics and patterns within different regions of the feature space. Furthermore, rough set theory is applied for attribute reduction, identifying the most relevant features and reducing the complexity of the classification problem. The generated fuzzy rules provide interpretable and understandable classification rules that facilitate domain expert interpretation. The research contributes to the field by proposing a comprehensive framework that improves both accuracy and interpretability of dataset classification. The findings demonstrate the effectiveness of the integrated approach, although certain limitations exist. Future research should focus on parameter selection, scalability challenges, and the applicability of the approach to diverse problem domains. The integrated approach presents a promising methodology for enhancing the accuracy and interpretability of dataset classification, with potential applications in various domains where accurate and interpretable classification models are crucial.

Skin cancer detection using effective optical parameters and the classification and regression tree algorithm: A novel framework

Early detection of skin cancer matters because diagnosis, prognosis and treatment plan differ for each skin cancer type at their stages. Medical imaging taking the advantages of the non-invasive and non-ionizing polarized light is emerging as a tool for the development of screening and diagnotic tests. In this work, we proposed a novel framework to classify human melanoma and nonmelanoma skin cancer using the Classification and Regression Tree algorithm (CART). The samples were prepared from twenty-four non-melanoma skin cancer samples (consisting of twelve squamous cell carcinoma and twelve basal cell carcinoma samples); and three melanoma skin cancer samples. We calculated ten optical parameters from anisotropic biological tissues, namely the LB orientation angle (α), the LB phase retardance (β), the CB optical rotation angle (γ), the LD orientation angle (θd), the linear dichroism (D), the circular dichroism (R), the degrees of linear depolarization (e1 and e2), the degree of circular depolarization (e3), and the depolarization index (∆) using Stokes-Mueller matrix formalism. All effective optical parameters of biological tissue were then input into the CART classifier as predictors. The model yielded an accuracy of 92.6%, which is desirable for any robust and interpretable classification model. The results showed that for biological tissue samples, linear polarization properties dominate over circular ones due to the cellular microstructural composition of tissue, especially under anomalous growth as seen in skin cancer. This novel framework can potentially assist physicians in making timely and well-informed medical decisions.

Interpretable classification of pathology whole-slide images using attention based context-aware graph convolutional neural network

Background and ObjectiveWhole slide image (WSI) classification and lesion localization within giga-pixel slide are challenging tasks in computational pathology that requires context-aware representations of histological features to adequately infer nidus. The existing weakly supervised learning methods mainly treat different locations in the slide as independent regions and cannot learn potential nonlinear interactions between instances based on i.i.d assumption, resulting in the model unable to effectively utilize context-ware information to predict the labels of WSIs and locate the region of interest (ROI). MethodsHere, we propose an interpretable classification model named bidirectional Attention-based Multiple Instance Learning Graph Convolutional Network (ABMIL-GCN), which hierarchically aggregates context-aware features of instances into a global representation in a topology fashion to predict the slide labels and localize the region of lymph node metastasis in WSIs. ResultsWe verified the superiority of this method on the Camelyon16 dataset, and the results show that the average predicted ACC and AUC of the proposed model after flooding optimization can reach 90.89% and 0.9149, respectively. The average accuracy and ACC score are improved by more than 7% and 4% compared with the existing state-of-the-art algorithms. ConclusionsThe results demonstrate that context-aware GCN outperforms existing weakly supervised learning methods by introducing spatial correlations between the neighbor image patches, which also addresses the ‘accuracy-interpretability trade-off’ problem. The framework provides a novel paradigm for the clinical application of computer-aided diagnosis and intelligent systems.

ILGBMSH: an interpretable classification model for the shRNA target prediction with ensemble learning algorithm.

Short hairpin RNA (shRNA)-mediated gene silencing is an important technology to achieve RNA interference, in which the design of potent and reliable shRNA molecules plays a crucial role. However, efficient shRNA target selection through biological technology is expensive and time consuming. Hence, it is crucial to develop a more precise and efficient computational method to design potent and reliable shRNA molecules. In this work, we present an interpretable classification model for the shRNA target prediction using the Light Gradient Boosting Machine algorithm called ILGBMSH. Rather than utilizing only the shRNA sequence feature, we extracted 554 biological and deep learning features, which were not considered in previous shRNA prediction research. We evaluated the performance of our model compared with the state-of-the-art shRNA target prediction models. Besides, we investigated the feature explanation from the model's parameters and interpretable method called Shapley Additive Explanations, which provided us with biological insights from the model. We used independent shRNA experiment data from other resources to prove the predictive ability and robustness of our model. Finally, we used our model to design the miR30-shRNA sequences and conducted a gene knockdown experiment. The experimental result was perfectly in correspondence with our expectation with a Pearson's coefficient correlation of 0.985. In summary, the ILGBMSH model can achieve state-of-the-art shRNA prediction performance and give biological insights from the machine learning model parameters.

An Evolutionary Computation Approach for Twitter Bot Detection

Bot accounts are automated software programs that act as legitimate human profiles on social networks. Identifying these kinds of accounts is a challenging problem due to the high variety and heterogeneity that bot accounts exhibit. In this work, we use genetic algorithms and genetic programming to discover interpretable classification models for Twitter bot detection with competitive qualitative performance, high scalability, and good generalization capabilities. Specifically, we use a genetic programming method with a set of primitives that involves simple mathematical operators. This enables us to discover a human-readable detection algorithm that exhibits a detection accuracy close to the top state-of-the-art methods on the TwiBot-20 dataset while providing predictions that can be interpreted, and whose uncertainty can be easily measured. To the best of our knowledge, this work is the first attempt at adopting evolutionary computation techniques for detecting bot profiles on social media platforms.

Exploring the Suitability of Rule-Based Classification to Provide Interpretability in Outcome-Based Process Predictive Monitoring

The development of models for process outcome prediction using event logs has evolved in the literature with a clear focus on performance improvement. In this paper, we take a different perspective, focusing on obtaining interpretable predictive models for outcome prediction. We propose to use association rule-based classification, which results in inherently interpretable classification models. Although association rule mining has been used with event logs for process model approximation and anomaly detection in the past, its application to an outcome-based predictive model is novel. Moreover, we propose two ways of visualising the rules obtained to increase the interpretability of the model. First, the rules composing a model can be visualised globally. Second, given a running case on which a prediction is made, the rules influencing the prediction for that particular case can be visualised locally. The experimental results on real world event logs show that in most cases the performance of the rule-based classifier (RIPPER) is close to the one of traditional machine learning approaches. We also show the application of the global and local visualisation methods to real world event logs.

Quality control, classification, and authentication of Iranian rice varieties using FT-IR spectroscopy and sparse chemometric methods

The authentication and classification of rice varieties have attracted much attention in food industry, in recent years. In this study, FTIR spectroscopy and sparse chemometric methods were used for discriminating different classes of Iranian rice samples and detecting adulteration in high-quality products. The sparse version of the linear discriminant analysis (sLDA) was used for developing interpretable classification models and detecting adulteration in rice samples. More than 400 samples from eight different classes of the most favorable Iranian rice varieties were used for developing sparse models. The performance of the sLDA model was compared with common chemometric methods, and the results revealed the superiority of the sLDA in terms of interpretability and prediction accuracy. Moreover, sparse multivariate regression methods including least absolute shrinkage and selection operator (lasso), ridge, and elastic net were used for quantitative analysis and fast assessment of rice adulteration. The overall accuracy for sLDA was 0.967 and the ranges of coefficient of determination (R2) for lasso models were 0.879–0.948, for the test sets. The results showed that the sparse chemometric methods combined with FTIR spectroscopy can be successfully applied as robust and interpretable tools for classification and adulteration detection of rice samples in food industry.

Classification of Event-Related Potentials with Regularized Spatiotemporal LCMV Beamforming

The usability of EEG-based visual brain–computer interfaces (BCIs) based on event-related potentials (ERPs) benefits from reducing the calibration time before BCI operation. Linear decoding models, such as the spatiotemporal beamformer model, yield state-of-the-art accuracy. Although the training time of this model is generally low, it can require a substantial amount of training data to reach functional performance. Hence, BCI calibration sessions should be sufficiently long to provide enough training data. This work introduces two regularized estimators for the beamformer weights. The first estimator uses cross-validated L2-regularization. The second estimator exploits prior information about the structure of the EEG by assuming Kronecker–Toeplitz-structured covariance. The performances of these estimators are validated and compared with the original spatiotemporal beamformer and a Riemannian-geometry-based decoder using a BCI dataset with P300-paradigm recordings for 21 subjects. Our results show that the introduced estimators are well-conditioned in the presence of limited training data and improve ERP classification accuracy for unseen data. Additionally, we show that structured regularization results in lower training times and memory usage, and a more interpretable classification model.

An explainable XGBoost model improved by SMOTE-ENN technique for maize lodging detection based on multi-source unmanned aerial vehicle images

Remote sensing image is becoming an increasingly popular tool for crop lodging detection because it conveniently provides features for building machine learning models and predicting lodging. However, difficulties in interpreting machine learning models and their predictions limit the confidence of using remote sensing images to detect lodging. In addition, the lodging datasets used for modeling are difficult to balance under natural conditions. Designing a robust and interpretable classification model for the detection of lodging in an imbalanced distribution dataset poses a particularly difficult challenge. In this study, visible and multi-spectral images were collected with a UAV to extract relevant features from remote sensing images. In a preliminary step, Synthetic Minority Oversampling Technique (SMOTE) and Edited Nearest Neighbors (ENN) method were used to treat imbalanced datasets. The SMOTE-ENN-XGBoost model is proposed for the efficient identification of maize lodging at the plot scale. The SMOTE-ENN-XGBoost model achieved an F1-score of 0.930 and a recall of 0.899 on a testing set, suggesting that it can be used for modeling lodging detection. Additionally, the SHapley Additive exPlanations (SHAP) approach was employed to interpret the identification and prioritization of features that determine lodging classification and activity prediction. The results showed that canopy structure and textural features are relatively stable compared with spectral features, which are susceptible to the external environment when modeling is employed to detect lodging. This work also showed that canopy structural, spectral, and textural information should be considered simultaneously rather than separately when detecting crop lodging in a crop breeding program in order to prevent differences in expression controlled by the interaction between genotype and environment obscuring the change in a single feature before and after lodging. For practical applications of machine learning models in crop lodging detection, such insights are of critical relevance. Taken together, the results of this study encourage further applications of remote sensing techniques to build interpretable machine learning models.

Attributes regrouping by genetic algorithm in fuzzy inference systems

Fuzzy Rule-Based Classification Systems (FRBCS) are a very popular and useful tool due to their interpretable classification models based on linguistic variables. However, a high number of attributes leads to an explosion of the number of classification rules. In this paper, we are interested in ensemble methods applied to FRBCS. An attributes regrouping method based on association rules and called SIFRA was proposed to solve this problem. It determines the relevant groups of related attributes that will be treated separately by FRBCSs. We propose a Genetic Algorithm-based method to select final groups of related attributes determined by SIFRA. In our approach, the genetic algorithm is used to filter the groups of related attributes and conserve the best ones, while maintaining a good classification rate and a reduced number of rules. We lead experiments to compare our contribution with previous methods in the same context; the experimental results are satisfactory.

Cervical Cancer Prognosis and Diagnosis Using Electrical Impedance Spectroscopy.

Electrical impedance spectroscopy (EIS) has been used as an adjunct to colposcopy for cervical cancer diagnosis for many years, Currently, the template match method is employed for EIS measurements analysis, where the measured EIS spectra are compared with the templates generated from three-dimensional finite element (FE) models of cancerous and non-cancerous cervical tissue, and the matches between the measured EIS spectra and the templates are then used to derive a score that indicates the association strength of the measured EIS to the High-Grade Cervical Intraepithelial Neoplasia (HG CIN). These FE models can be viewed as the computational versions of the associated physical tissue models. In this paper, the problem is revisited with an objective to develop a new method for EIS data analysis that might reveal the relationship between the change in the tissue structure due to disease and the change in the measured spectrum. This could provide us with important information to understand the histopathological mechanism that underpins the EIS-based HG CIN diagnostic decision making and the prognostic value of EIS for cervical cancer diagnosis. A further objective is to develop an alternative EIS data processing method for HG CIN detection that does not rely on physical models of tissues so as to facilitate extending the EIS technique to new medical diagnostic applications where the template spectra are not available.An EIS data-driven method was developed in this paper to achieve the above objectives, where the EIS data analysis for cervical cancer diagnosis and prognosis were formulated as the classification problems and a Cole model-based spectrum curve fitting approach was proposed to extract features from EIS readings for classification. Machine learning techniques were then used to build classification models with the selected features for cervical cancer diagnosis and evaluation of the prognostic value of the measured EIS. The interpretable classification models were developed with real EIS data sets, which enable us to associate the changes in the observed EIS and the risk of being HG CIN or developing HG CIN with the changes in tissue structure due to disease. The developed classification models were used for HG CIN detection and evaluation of the prognostic value of EIS and the results demonstrated the effectiveness of the developed method. The method developed is of long-term benefit for EIS–based cervical cancer diagnosis and, in conjunction with standard colposcopy, there is the potential for the developed method to provide a more effective and efficient patient management strategy for clinic practice.

Robust Variable Selection with Optimality Guarantees for High-Dimensional Logistic Regression

High-dimensional classification studies have become widespread across various domains. The large dimensionality, coupled with the possible presence of data contamination, motivates the use of robust, sparse estimation methods to improve model interpretability and ensure the majority of observations agree with the underlying parametric model. In this study, we propose a robust and sparse estimator for logistic regression models, which simultaneously tackles the presence of outliers and/or irrelevant features. Specifically, we propose the use of L0-constraints and mixed-integer conic programming techniques to solve the underlying double combinatorial problem in a framework that allows one to pursue optimality guarantees. We use our proposal to investigate the main drivers of honey bee (Apis mellifera) loss through the annual winter loss survey data collected by the Pennsylvania State Beekeepers Association. Previous studies mainly focused on predictive performance, however our approach produces a more interpretable classification model and provides evidence for several outlying observations within the survey data. We compare our proposal with existing heuristic methods and non-robust procedures, demonstrating its effectiveness. In addition to the application to honey bee loss, we present a simulation study where our proposal outperforms other methods across most performance measures and settings.

MoMAC: Multi-objective optimization to combine multiple association rules into an interpretable classification

A crucial characteristic of machine learning models in various domains (such as medical diagnosis, financial analysis, or real-time process monitoring) is the interpretability. The interpretation supports humans in understanding the meaning behind every single prediction made by the machine, and enables the user to assess trustworthiness before acting on the predictions. This article presents our work in building an interpretable classification model based on association rule mining and multi-objective optimization. The classification model itself is a rule list, making a single prediction based on multiple rules. The rule list consists of If ... THEN statements that are understandable to humans. We choose these rules from a large set of pre-mined rules according to an interestingness measure which is formulated as a function of basic probabilities related to the rules. We learned the interestingness measure through multi-objective optimization, concentrating on two objectives: the classifier’s size in terms of number of rules and prediction accuracy. The model is called MoMAC, “Multi-Objective optimization to combine Multiple Association rules into an interpretable Classification”. The experimental results on benchmark datasets demonstrate that MoMAC outperforms other existing rule-based classification methods in terms of classification accuracy.

An extensive experimental evaluation of automated machine learning methods for recommending classification algorithms

This paper presents an experimental comparison among four automated machine learning (AutoML) methods for recommending the best classification algorithm for a given input dataset. Three of these methods are based on evolutionary algorithms (EAs), and the other is Auto-WEKA, a well-known AutoML method based on the combined algorithm selection and hyper-parameter optimisation (CASH) approach. The EA-based methods build classification algorithms from a single machine learning paradigm: either decision-tree induction, rule induction, or Bayesian network classification. Auto-WEKA combines algorithm selection and hyper-parameter optimisation to recommend classification algorithms from multiple paradigms. We performed controlled experiments where these four AutoML methods were given the same runtime limit for different values of this limit. In general, the difference in predictive accuracy of the three best AutoML methods was not statistically significant. However, the EA evolving decision-tree induction algorithms has the advantage of producing algorithms that generate interpretable classification models and that are more scalable to large datasets, by comparison with many algorithms from other learning paradigms that can be recommended by Auto-WEKA. We also observed that Auto-WEKA has shown meta-overfitting, a form of overfitting at the meta-learning level, rather than at the base-learning level.

Preventing the generation of inconsistent sets of crisp classification rules

In recent years, the interest in interpretable classification models has grown. One of the proposed ways to improve the interpretability of a classification model based on collections of crisp rules is to use sets (unordered collections) instead of lists (ordered collections). One of the problems associated with sets is that multiple rules may cover a single instance but predict different classes for it, thus requiring a conflict resolution strategy. In this work, we propose two algorithms capable of finding feature-space regions inside which any created rule would be consistent with the already existing rules, preventing inconsistencies from arising. Our algorithms, named CFSGS and CFSBE, do not generate classification rules nor classification models but are instead meant to enhance algorithms that do so, such as Learning Classifier Systems. We analyzed both algorithms from a theoretical perspective and conducted experiments with a proof of concept evolutionary algorithm that employs CFSBE. The experiments suggest that using CFSBE as an embedded tool does incur a computational overhead, but such cost is not prohibitive.