Matthews Correlation Research Articles

Recurrent neural network-based prediction of O-GlcNAcylation sites in mammalian proteins

O-GlcNAcylation has the potential to be an important target for therapeutics, but a motif or an algorithm to reliably predict O-GlcNAcylation sites is not available. Current predictive models are insufficient as they fail to generalize, and many are no longer available. This article constructs recurrent neural network models to predict O-GlcNAcylation sites based on protein sequences. Different datasets are evaluated separately and assessed in terms of strengths and issues. Within a given dataset, results are robust to changes in cross-validation and test data as determined by nested validation. The best model achieves an F1 score of 36% (more than 3.5-fold greater than the previous best model) and a Matthews Correlation Coefficient of 35% (more than 4.5-fold greater than the previous best model), and, for the F1 score, 7.6-fold higher than when not using any model. Shapley values are used to interpret the model’s predictions and provide biological insight into O-GlcNAcylation.

Parameter-efficient fine-tuning on large protein language models improves signal peptide prediction.

Signal peptides (SPs) play a crucial role in protein translocation in cells. The development of large protein language models (PLMs) and prompt-based learning provide a new opportunity for SP prediction, especially for the categories with limited annotated data. We present a parameter-efficient fine-tuning (PEFT) framework for SP prediction, PEFT-SP, to effectively utilize pretrained PLMs. We integrated low-rank adaptation (LoRA) into ESM-2 models to better leverage the protein sequence evolutionary knowledge of PLMs. Experiments show that PEFT-SP using LoRA enhances state-of-the-art results, leading to a maximum Matthews correlation coefficient (MCC) gain of 87.3% for SPs with small training samples and an overall MCC gain of 6.1%. Furthermore, we also employed two other PEFT methods, prompt tuning and adapter tuning, in ESM-2 for SP prediction. More elaborate experiments show that PEFT-SP using adapter tuning can also improve the state-of-the-art results by up to 28.1% MCC gain for SPs with small training samples and an overall MCC gain of 3.8%. LoRA requires fewer computing resources and less memory than the adapter tuning during the training stage, making it possible to adapt larger and more powerful protein models for SP prediction.

PLM_Sol: predicting protein solubility by benchmarking multiple protein language models with the updated Escherichia coli protein solubility dataset.

Protein solubility plays a crucial role in various biotechnological, industrial, and biomedical applications. With the reduction in sequencing and gene synthesis costs, the adoption of high-throughput experimental screening coupled with tailored bioinformatic prediction has witnessed a rapidly growing trend for the development of novel functional enzymes of interest (EOI). High protein solubility rates are essential in this process and accurate prediction of solubility is a challenging task. As deep learning technology continues to evolve, attention-based protein language models (PLMs) can extract intrinsic information from protein sequences to a greater extent. Leveraging these models along with the increasing availability of protein solubility data inferred from structural database like the Protein Data Bank holds great potential to enhance the prediction of protein solubility. In this study, we curated an Updated Escherichia coli protein Solubility DataSet (UESolDS) and employed a combination of multiple PLMs and classification layers to predict protein solubility. The resulting best-performing model, named Protein Language Model-based protein Solubility prediction model (PLM_Sol), demonstrated significant improvements over previous reported models, achieving a notable 6.4% increase in accuracy, 9.0% increase in F1_score, and 11.1% increase in Matthews correlation coefficient score on the independent test set. Moreover, additional evaluation utilizing our in-house synthesized protein resource as test data, encompassing diverse types of enzymes, also showcased the good performance of PLM_Sol. Overall, PLM_Sol exhibited consistent and promising performance across both independent test set and experimental set, thereby making it well suited for facilitating large-scale EOI studies. PLM_Sol is available as a standalone program and as an easy-to-use model at https://zenodo.org/doi/10.5281/zenodo.10675340.

ToxTeller: Predicting Peptide Toxicity Using Four Different Machine Learning Approaches.

Examining the toxicity of peptides is essential for therapeutic peptide-based drug design. Machine learning approaches are frequently used to develop highly accurate predictors for peptide toxicity prediction. In this paper, we present ToxTeller, which provides four predictors using logistic regression, support vector machines, random forests, and XGBoost, respectively. For prediction model development, we construct a data set of toxic and nontoxic peptides from SwissProt and ConoServer databases with existence evidence levels checked. We also fully utilize the protein annotation in SwissProt to collect more toxic peptides than using keyword search alone. From this data set, we construct an independent test data set that shares at most 40% sequence similarity within itself and with the training data set. From a quite comprehensive list of 28 feature combinations, we conduct 10-fold cross-validation on the training data set to determine the optimized feature combination for model development. ToxTeller's performance is evaluated and compared with existing predictors on the independent test data set. Since toxic peptides must be avoided for drug design, we analyze strategies for reducing false-negative predictions of toxic peptides and suggest selecting models by top sensitivity instead of the widely used Matthews correlation coefficient, and also suggest using a meta-predictor approach with multiple predictors.

Evaluation of underground hard rock mine pillar stability using gene expression programming and decision tree‐support vector machine models

AbstractAssessing the stability of pillars in underground mines (especially in deep underground mines) is a critical concern during both the design and the operational phases of a project. This study mainly focuses on developing two practical models to predict pillar stability status. For this purpose, two robust models were developed using a database including 236 case histories from seven underground hard rock mines, based on gene expression programming (GEP) and decision tree‐support vector machine (DT‐SVM) hybrid algorithms. The performance of the developed models was evaluated based on four common statistical criteria (sensitivity, specificity, Matthews correlation coefficient, and accuracy), receiver operating characteristic (ROC) curve, and testing data sets. The results showed that the GEP and DT‐SVM models performed exceptionally well in assessing pillar stability, showing a high level of accuracy. The DT‐SVM model, in particular, outperformed the GEP model (accuracy of 0.914, sensitivity of 0.842, specificity of 0.929, Matthews correlation coefficient of 0.767, and area under the ROC of 0.897 for the test data set). Furthermore, upon comparing the developed models with the previous ones, it was revealed that both models can effectively determine the condition of pillar stability with low uncertainty and acceptable accuracy. This suggests that these models could serve as dependable tools for project managers, aiding in the evaluation of pillar stability during the design and operational phases of mining projects, despite the inherent challenges in this domain.

Micro Target: MicroRNA Target Prediction and Validation with Experimentally Positive and Negative Examples

MicroRNAs (miRNAs) usually controls the gene by binding to complementary sites of 3’ untranslated region of its target genes. Numerous criteria-based and machine learning approaches are available in the literature to predict miRNA–mRNA interactions, but most of them struggle with either high false positive or false negative rates and also don’t show good validation with experimentally validated positive and negative examples. Here we present microTarget, a new computational approach for identifying miRNA target genes which are based on complementarity score, thermodynamic duplex stability and also independent of conservation of target sites in related genomes. In this article, we validated our algorithm using positive and negative data from the literature in various human tissues, and our method outperformed existing computational methods such as miRanda, RNA22, and PITA. Receiver operating characteristic curves (ROC) and Matthew's correlation coefficient (MCC) were calculated using experimentally validated data, and they reveal that microTarget greatly improves miRNA target prediction compared to the three algorithms employed individually. Additionally, an F-score analysis demonstrated that microTarget greatly enhances the relevance of the other techniques. Thus, microTarget is a useful tool for biologists looking for miRNA targets and integrating them into biological contexts.

Trainable pruned ternary quantization for medical signal classification models

The field of deep learning is renowned for its resource-intensive nature, hence improving its environmental impact is crucial. In this paper, we propose a novel model compression method to mitigate the energy demands of deep learning for a greener, and more sustainable AI landscape. Our approach relies on an asymmetric weakly-differentiable pruning function that leverages weight statistics to directly incorporate adaptable pruning into the quantization mechanism. This enables us to achieve higher compression rates globally while simultaneously reducing energy consumption and minimizing classification performance degradation. The efficacy of our approach was evaluated using three distinct models on three distinct datasets: cerebral emboli (HITS), epileptic seizure recognition (ESR), and MNIST. Our method demonstrated a superior balance between compression, energy consumption, and classification performance compared to other state-of-the-art extreme quantization methods, across all models and datasets. In fact, on the HITS dataset with a two-dimensional convolutional neural network, we achieved strong gains of 50.6%, 54.9%, 52.1% in compression rates (of the global model and the quantized layers only, respectively) and energy consumption, respectively, while improving the Matthews correlation coefficient by 2.5% compared to other approaches. The code is available at: https://github.com/yamilvindas/pTTQ.

PKalculator: A pK a predictor for C-H bonds.

Determining the pK a values of various C-H sites in organic molecules offers valuable insights for synthetic chemists in predicting reaction sites. As molecular complexity increases, this task becomes more challenging. This paper introduces pKalculator, a quantum chemistry (QM)-based workflow for automatic computations of C-H pK a values, which is used to generate a training dataset for a machine learning (ML) model. The QM workflow is benchmarked against 695 experimentally determined C-H pK a values in DMSO. The ML model is trained on a diverse dataset of 775 molecules with 3910 C-H sites. Our ML model predicts C-H pK a values with a mean absolute error (MAE) and a root mean squared error (RMSE) of 1.24 and 2.15 pK a units, respectively. Furthermore, we employ our model on 1043 pK a-dependent reactions (aldol, Claisen, and Michael) and successfully indicate the reaction sites with a Matthew's correlation coefficient (MCC) of 0.82.

To Feel the Spatial: Graph Neural Network-Based Method for Leakage Risk Assessment in Water Distribution Networks

As water distribution networks expand, evaluating pipeline network leakage risk has become increasingly crucial. Contrary to traditional evaluation methods, which are often hampered by subjective weight assignment, data scarcity, and high expenses, data-driven models provide advantages like autonomous weight learning, comprehensive coverage, and cost-efficiency. This study introduces a data-driven framework leveraging graph neural networks to assess leakage risk in water distribution networks. Employing geographic information system (GIS) data from a central Chinese city, encompassing pipeline network details and historical repair records, the model achieved superior performance compared to other data-driven approaches, evidenced by metrics such as precision, accuracy, recall, and the Matthews correlation coefficient. Further analysis of risk factors underscores the importance of factors like pipe age, material, prior failures, and length. This approach demonstrates robust predictive accuracy and offers significant reference value for leakage risk evaluation.

Wavelet Transform, Reconstructed Phase Space, and Deep Learning Neural Networks for EEG-Based Schizophrenia Detection.

This study proposes an innovative expert system that uses exclusively EEG signals to diagnose schizophrenia in its early stages. For diagnosing psychiatric/neurological disorders, electroencephalogram (EEG) testing is considered a financially viable, safe, and reliable alternative. Using the reconstructed phase space (RPS) and the continuous wavelet transform, the researchers maximized the differences between the EEG nonstationary signals of normal and schizophrenia individuals, which cannot be observed in the time, frequency, or time-frequency domains. This reveals significant information, highlighting more distinguishable features. Then, a deep learning network was trained to enhance the accuracy of the resulting image classification. The algorithm's efficacy was confirmed through three distinct methods: employing 70% of the dataset for training, 15% for validation, and the remaining 15% for testing. This was followed by a 5-fold cross-validation technique and a leave-one-out classification approach. Each method was iterated 100 times to ascertain the algorithm's robustness. The performance metrics derived from these tests - accuracy, precision, sensitivity, F1 score, Matthews correlation coefficient, and Kappa - indicated remarkable outcomes. The algorithm demonstrated steady performance across all evaluation strategies, underscoring its relevance and reliability. The outcomes validate the system's accuracy, precision, sensitivity, and robustness by showcasing its capability to autonomously differentiate individuals diagnosed with schizophrenia from those in a state of normal health.

Metabolism score and machine learning models for the prediction of esophageal squamous cell carcinoma progression.

The incomplete prediction of prognosis in esophageal squamous cell carcinoma (ESCC) patients is attributed to various therapeutic interventions and complex prognostic factors. Consequently, there is a pressing demand for enhanced predictive biomarkers that can facilitate clinical management and treatment decisions. This study recruited 491 ESCC patients who underwent surgical treatment at Huashan Hospital, Fudan University. We incorporated 14 blood metabolic indicators and identified independent prognostic indicators for overall survival through univariate and multivariate analyses. Subsequently, a metabolism score formula was established based on the biochemical markers. We constructed a nomogram and machine learning models utilizing the metabolism score and clinically significant prognostic features, followed by an evaluation of their predictive accuracy and performance. We identified alkaline phosphatase, free fatty acids, homocysteine, lactate dehydrogenase, and triglycerides as independent prognostic indicators for ESCC. Subsequently, based on these five indicators, we established a metabolism score that serves as an independent prognostic factor in ESCC patients. By utilizing this metabolism score in conjunction with clinical features, a nomogram can precisely predict the prognosis of ESCC patients, achieving an area under the curve (AUC) of 0.89. The random forest (RF) model showed superior predictive ability (AUC = 0.90, accuracy = 86%, Matthews correlation coefficient = 0.55). Finally, we used an RF model with optimal performance to establish an online predictive tool. The metabolism score developed in this study serves as an independent prognostic indicator for ESCC patients.

Machine learning models for classifying coffee fruits detachment force

AbstractThe maturation process of coffee (Coffea arabica) trees exhibits inherent variability, producing fruits at various physiological maturity stages. This variability affects the resistance between the fruit and its peduncle, posing a challenge in mechanized harvesting: non‐selective harvesting. A precise classification of coffee fruit detachment force is essential to address this challenge, ensuring coffee's quality and producer's profitability. This study assesses the efficacy of machine learning (ML) models in determining the detachment force across various coffee cultivars under drip‐irrigated and rainfed conditions. The dataset included detachment force measurements from 24 cultivars—13 drip‐irrigated and 11 rainfed—yielding 1152 data points. Variance analysis compared irrigation methods and three maturity stages: green, cherry, and dry. Detachment force was categorized into four classes based on the dataset's quartile distribution. The ML models utilized were random forest (RF), support vector machine (SVM), K‐nearest neighbors, and artificial neural networks. The SVM model was notably effective in classifying detachment force for rainfed cultivars, with a Matthews correlation coefficient (MCC) of 0.78. In contrast, the RF model was particularly adept for drip‐irrigated cultivars, with an MCC of 0.75. The highest classification accuracies were recorded for the extreme force classes I and IV, with precision values of 0.93 and 0.8, respectively, while classes II and III had lower precision at 0.57 and 0.69. Implementing these ML models for detachment force classification has been beneficial, improving decision‐making in mechanized harvesting systems.

Computational Machine Learning Analytics for Prediction of Water Quality

In terms of impacts on ecosystems, industry, people, and flora and fauna, water quality is paramount. Contamination and pollution have degraded water quality in recent decades. Predicting WQC and Water Quality Index (WQI) is the problem of this article; WQI is an important measure of water validity. This research use machine learning approaches to forecast WQI and WQC, and it does so by optimizing and tweaking the parameters of several machine learning models. Parameter optimization and tuning for four classification models and four regression models both make use of grid search, an essential tool in both contexts. To forecast WQC, classification models such as Random Forest (RF), Extreme Gradient Boosting (Xgboost), Gradient Boosting (GB), and Adaptive Boosting (Ada-Boost) are used. Predicting WQI is done using regression models such as K-nearest neighbour (KNN), decision tree (DT), support vector regression (SVR), and multi-layer perceptron (MLP). Data normalization and data imputation (mean imputation) were also executed as pretreatment steps to suit the data and make it convenient for any further processing. Seven characteristics and ninety-one cases make up the dataset used for this research. Five evaluation measures were calculated to evaluate the classification systems' effectiveness: accuracy, recall, precision, Matthews' Correlation Coefficient (MCC), and F1 score. A total of four evaluation metrics were calculated to measure the efficacy of the regression models: MAE, MedAE,MSE, and R2. The results of the testing showed that the GB model yielded the most accurate predictions of WQC values (99.50%), making it the top performer in terms of categorization. The experimental findings show that the MLP regressor model got a value of 99.8 percent R2 when predicting WQI values, making it the best performing model in regression.

Groundwater quality assessment using machine learning models: a comprehensive study on the industrial corridor of a semi-arid region.

Water plays a significant role in sustaining the lives of humans and other living organisms. Groundwater quality analysis has become inevitable, because of increased contamination of water resources and global warming. This study used machine learning (ML) models to predict the water quality index (WQI) and water quality classification (WQC). Forty groundwater samples were collected near the Ranipet industrial corridor, and the hydrogeochemistry and heavy metal contamination were analyzed. WQC prediction employed random forest (RF), gradient boosting (GB), decision tree (DT), and K-nearest neighbor (KNN) models, and WQI prediction used extreme gradient boosting (XGBoost), support vector regressor (SVR), RF, and multi-layer perceptron (MLP) models. The grid search method is used to evaluate the ML model by F1 score, accuracy, recall, precision, and Matthews correlation coefficient (MCC) for WQC and the coefficient of determination (R2), mean absolute error (MAE), mean square error (MSE), and median absolute percentage error (MAPE) for WQI. The WQI results indicate that the groundwater quality of the study area is very poor and unsuitable for drinking or irrigation purposes. The performance metrics of the RF model excelled in predicting both WQC (accuracy = 97%) and WQI (R2 = 91.0%), outperforming other models and emphasizing ML's superiority in groundwater quality assessment. The findings suggest that ML models perform well and yield better accuracy than conventional techniques used in groundwater quality assessment studies.

A SAR and QSAR study on 3CLpro inhibitors of SARS-CoV-2 using machine learning methods

ABSTRACT The 3C-like Proteinase (3CLpro) of novel coronaviruses is intricately linked to viral replication, making it a crucial target for antiviral agents. In this study, we employed two fingerprint descriptors (ECFP_4 and MACCS) to comprehensively characterize 889 compounds in our dataset. We constructed 24 classification models using machine learning algorithms, including Support Vector Machine (SVM), Random Forest (RF), extreme Gradient Boosting (XGBoost), and Deep Neural Networks (DNN). Among these models, the DNN- and ECFP_4-based Model 1D_2 achieved the most promising results, with a remarkable Matthews correlation coefficient (MCC) value of 0.796 in the 5-fold cross-validation and 0.722 on the test set. The application domains of the models were analysed using dSTD-PRO calculations. The collected 889 compounds were clustered by K-means algorithm, and the relationships between structural fragments and inhibitory activities of the highly active compounds were analysed for the 10 obtained subsets. In addition, based on 464 3CLpro inhibitors, 27 QSAR models were constructed using three machine learning algorithms with a minimum root mean square error (RMSE) of 0.509 on the test set. The applicability domains of the models and the structure-activity relationships responded from the descriptors were also analysed.

MuMCyp_Net: A multimodal neural network for the prediction of Cyp450 inhibition

The cytochrome P450 (CYP450) enzymes, heme-containing monooxygenases, are indispensable in metabolizing a broad range of drugs and other xenobiotics. Nonetheless, specific chemical entities (molecules) can inhibit these enzymes, resulting in undesirable drug-drug interactions, alterations in pharmacokinetics, and potentially compromising the efficacy and safety of drugs. Therefore, it is critical to identify molecules exhibiting inhibitory activities toward the five majors human CYP450 enzymes (1A2, 2C9, 2C19, 2D6, and 3A4) in the preliminary phases of drug development. This study presents a novel deep-learning model, MuMCyp_Net, for predicting CYP450 inhibition by small molecules. The architecture of MuMCyp_Net integrates a convolutional neural network (CNN) with an Attention mechanism, a bi-directional gated recurrent unit (biGRU) network, and a deep neural network (DNN) to efficiently process local chemical context information and global molecular properties of molecules. A total of 25,753 distinct compound molecules were utilized to train and evaluate the model’s performance. The proposed model demonstrates competitive performance as indicated by matthew’s correlation coefficients (MCC), ranging from 0.63 to 0.68, an accuracy rate between 0.82 and 0.90, and an AUC of 0.86 to 0.92. This novel approach shows considerable promise in accurately identifying CYP450 inhibitors, contributing to safer and more effective drug development. To further utilize the potential of the proposed model, a freely accessible web server tool for the virtual screening of molecules was developed, which will aid in identifying CYP450 inhibitors from non-inhibitors. The web-based tool can be accessed at https://mumcypnet.streamlit.app/.

Retinal Blood Vessels Segmentation With Improved SE‐UNet Model

ABSTRACTAccurate segmentation of retinal vessels is crucial for the early diagnosis and treatment of eye diseases, for example, diabetic retinopathy, glaucoma, and macular degeneration. Due to the intricate structure of retinal vessels, it is essential to extract their features with precision for the semantic segmentation of medical images. In this study, an improved deep learning neural network was developed with a focus on feature extraction based on the U‐Net structure. The enhanced U‐Net combines the architecture of convolutional neural networks (CNNs) with SE blocks (squeeze‐and‐excitation blocks) to adaptively extract image features after each U‐Net encoder's convolution. This approach aids in suppressing nonvascular regions and highlighting features for specific segmentation tasks. The proposed method was trained and tested on the DRIVECHASE_DB1 and STARE datasets. As a result, the proposed model had an algorithmic accuracy, sensitivity, specificity, Dice coefficient (Dc), and Matthews correlation coefficient (MCC) of 95.62/0.9853/0.9652, 0.7751/0.7976/0.7773, 0.9832/0.8567/0.9865, 82.53/87.23/83.42, and 0.7823/0.7987/0.8345, respectively, outperforming previous methods, including UNet++, attention U‐Net, and ResUNet. The experimental results demonstrated that the proposed method improved the retinal vessel segmentation performance.

Computer vision for eye diseases detection using pre‐trained deep learning techniques and raspberry Pi

AbstractEarly diagnosis of eye diseases is very important to prevent visual impairment and guide appropriate treatment methods. This paper presents a unique approach that can detect numerous eye diseases automatically. Initially, this approach used the pre‐trained ImageNet models that provides various pre‐trained models for training the acquired data. The existing data sets are composed of 645 data images acquired clinically, represented by two groups of subjects as healthy and others holding the proposed eye defect like cataracts, foreign bodies, glaucoma, subconjunctival haemorrhage, and viral conjunctivitis. Followed by comparisons of the pre‐trained model's coefficients and prediction performance. Later, the first‐class execution model is integrated within the Raspberry Pi staging and the real‐time digital camera detection. The evaluation process used the confusion matrix, model accuracy, precision factor, recall coefficient, F1 score, and the Matthews Correlation Coefficient (MCC). Resulting in the performance of these pre‐trained ImageNet models used in this study represented by 93% (InceptionResNetV2), 90% (MobileNet), 86% (Residual Network ResNet50), 85% (InceptionV3), 78% (Visual Geometry Group VGG19), and 72% (Neural Architecture Search Network NASNetMobile). The results show that the InceptionResNetV2 achieved the highest performance. This proposed approach shows its efficiency and strength by early detection of the subject's unhealthy eyes through real‐time monitoring in the field of ophthalmology.

ScLNet: A cornea with scleral lens OCT layers segmentation dataset and new multi-task model

ObjectiveTo develop deep learning methods with high accuracy for segmenting irregular corneas and detecting the tear fluid reservoir (TFR) boundary under the scleral lens. Additionally, this study aims to provide a publicly available cornea with scleral lens OCT dataset, including manually labeled layer masks for training and validation of segmentation algorithms. This study introduces ScLNet, a dataset comprising cornea with Scleral Lens (ScL) optical coherence tomography (OCT) images with layer annotations, and a multi-task network designed to achieve rapid, accurate, automated segmentation of scleral lens with regular and irregular corneas. MethodsWe created a dataset comprising 31,360 OCT images with scleral lens annotations. The network architecture includes an encoder with multi-scale input and a context coding layer, along with two decoders for specific tasks. The primary task focuses on predicting ScL, TFR, and cornea regions, while the auxiliary task, aimed at predicting the boundaries of ScL, TFR, and cornea, enhances feature extraction for the main task. Segmentation results were compared with state-of-the-art methods and evaluated using Dice similarity coefficient (DSC), intersection over union (IoU), Matthews correlation coefficient (MCC), Precision, and Hausdorff distance (HD). ResultsScLNet achieves 98.22 % DSC, 96.50 % IoU, 98.13 % MCC, 98.35 % Precision, and 3.6840 HD (in pixels) in segmenting ScL; 97.78 % DSC, 95.66 % IoU, 97.71 % MCC, 97.70 % Precision, and 3.7838 HD (in pixels) in segmenting TFR; and 99.22 % DSC, 98.45 % IoU, 99.15 % MCC, 99.14 % Precision, and 3.5355 HD (in pixels) in segmenting cornea. The layer interfaces recognized by ScLNet closely align with expert annotations, as evidenced by high IoU scores. Boundary metrics further confirm its effectiveness. ConclusionWe constructed a dataset of corneal OCT images with ScL wearing, which includes regular and irregular cornea patients. The proposed ScLNet achieves high accuracy in extracting ScL, TFR, and corneal layer masks and boundaries from OCT images of the dataset.

Developing the Benchmark: Establishing a Gold Standard for the Evaluation of AI Caries Diagnostics.

Background/Objectives: The aim of this study was to establish a histology-based gold standard for the evaluation of artificial intelligence (AI)-based caries detection systems on proximal surfaces in bitewing images. Methods: Extracted human teeth were used to simulate intraoral situations, including caries-free teeth, teeth with artificially created defects and teeth with natural proximal caries. All 153 simulations were radiographed from seven angles, resulting in 1071 in vitro bitewing images. Histological examination of the carious lesion depth was performed twice by an expert. A total of thirty examiners analyzed all the radiographs for caries. Results: We generated in vitro bitewing images to evaluate the performance of AI-based carious lesion detection against a histological gold standard. All examiners achieved a sensitivity of 0.565, a Matthews correlation coefficient (MCC) of 0.578 and an area under the curve (AUC) of 76.1. The histology receiver operating characteristic (ROC) curve significantly outperformed the examiners' ROC curve (p < 0.001). All examiners distinguished induced defects from true caries in 54.6% of cases and correctly classified 99.8% of all teeth. Expert caries classification of the histological images showed a high level of agreement (intraclass correlation coefficient (ICC) = 0.993). Examiner performance varied with caries depth (p ≤ 0.008), except between E2 and E1 lesions (p = 1), while central beam eccentricity, gender, occupation and experience had no significant influence (all p ≥ 0.411). Conclusions: This study successfully established an unbiased dataset to evaluate AI-based caries detection on bitewing surfaces and compare it to human judgement, providing a standardized assessment for fair comparison between AI technologies and helping dental professionals to select reliable diagnostic tools.