Latent Features Research Articles

Delving into CLIP latent space for Video Anomaly Recognition

We tackle the complex problem of detecting and recognising anomalies in surveillance videos at the frame level, utilising only video-level supervision. We introduce the novel method AnomalyCLIP, the first to combine Vision and Language Models (VLMs), such as CLIP, with multiple instance learning for joint video anomaly detection and classification. Our approach specifically involves manipulating the latent CLIP feature space to identify the normal event subspace, which in turn allows us to effectively learn text-driven directions for abnormal events. When anomalous frames are projected onto these directions, they exhibit a large feature magnitude if they belong to a particular class. We also leverage a computationally efficient Transformer architecture to model short- and long-term temporal dependencies between frames, ultimately producing the final anomaly score and class prediction probabilities. We compare AnomalyCLIP against state-of-the-art methods considering three major anomaly detection benchmarks, i.e. ShanghaiTech, UCF-Crime, and XD-Violence, and empirically show that it outperforms baselines in recognising video anomalies. Project website and code are available at https://lucazanella.github.io/AnomalyCLIP/.

A Plug-In Graph Neural Network to Boost Temporal Sensitivity in fMRI Analysis.

Learning-based methods offer performance leaps over traditional methods in classification analysis of high-dimensional functional MRI (fMRI) data. In this domain, deep-learning models that analyze functional connectivity (FC) features among brain regions have been particularly promising. However, many existing models receive as input temporally static FC features that summarize inter-regional interactions across an entire scan, reducing the temporal sensitivity of classifiers by limiting their ability to leverage information on dynamic FC features of brain activity. To improve the performance of baseline classification models without compromising efficiency, here we propose a novel plug-in based on a graph neural network, GraphCorr, to provide enhanced input features to baseline models. The proposed plug-in computes a set of latent FC features with enhanced temporal information while maintaining comparable dimensionality to static features. Taking brain regions as nodes and blood-oxygen-level-dependent (BOLD) signals as node inputs, GraphCorr leverages a node embedder module based on a transformer encoder to capture dynamic latent representations of BOLD signals. GraphCorr also leverages a lag filter module to account for delayed interactions across nodes by learning correlational features of windowed BOLD signals across time delays. These two feature groups are then fused via a message passing algorithm executed on the formulated graph. Comprehensive demonstrations on three public datasets indicate improved classification performance for several state-of-the-art graph and convolutional baseline models when they are augmented with GraphCorr.

Efficient and Effective One-Step Multiview Clustering.

Multiview clustering algorithms have attracted intensive attention and achieved superior performance in various fields recently. Despite the great success of multiview clustering methods in realistic applications, we observe that most of them are difficult to apply to large-scale datasets due to their cubic complexity. Moreover, they usually use a two-stage scheme to obtain the discrete clustering labels, which inevitably causes a suboptimal solution. In light of this, an efficient and effective one-step multiview clustering (E2OMVC) method is proposed to directly obtain clustering indicators with a small-time burden. Specifically, according to the anchor graphs, the smaller similarity graph of each view is constructed, from which the low-dimensional latent features are generated to form the latent partition representation. By introducing a label discretization mechanism, the binary indicator matrix can be directly obtained from the unified partition representation which is formed by fusing all latent partition representations from different views. In addition, by coupling the fusion of all latent information and the clustering task into a joint framework, the two processes can help each other and obtain a better clustering result. Extensive experimental results demonstrate that the proposed method can achieve comparable or better performance than the state-of-the-art methods. The demo code of this work is publicly available at https://github.com/WangJun2023/EEOMVC.

Metadata-guided feature disentanglement for functional genomics.

With the development of high-throughput technologies, genomics datasets rapidly grow in size, including functional genomics data. This has allowed the training of large Deep Learning (DL) models to predict epigenetic readouts, such as protein binding or histone modifications, from genome sequences. However, large dataset sizes come at a price of data consistency, often aggregating results from a large number of studies, conducted under varying experimental conditions. While data from large-scale consortia are useful as they allow studying the effects of different biological conditions, they can also contain unwanted biases from confounding experimental factors. Here, we introduce Metadata-guided Feature Disentanglement (MFD)-an approach that allows disentangling biologically relevant features from potential technical biases. MFD incorporates target metadata into model training, by conditioning weights of the model output layer on different experimental factors. It then separates the factors into disjoint groups and enforces independence of the corresponding feature subspaces with an adversarially learned penalty. We show that the metadata-driven disentanglement approach allows for better model introspection, by connecting latent features to experimental factors, without compromising, or even improving performance in downstream tasks, such as enhancer prediction, or genetic variant discovery. The code will be made available at https://github.com/HealthML/MFD.

Frequency-Enhanced Geometric-Constrained Reconstruction for Localizing Myocardial Infarction in 12-Lead Electrocardiograms.

Determining the location of myocardial infarction is crucial for clinical management and therapeutic stratagem. However, existing diagnostic tools either sacrifice ease of use or are limited by their spatial resolution. Addressing this, we aim to refine myocardial infarction localization via surface potential reconstruction of the ventricles in 12-lead electrocardiograms (ECG). A notable obstacle is the ill-posed nature of such reconstructions. To overcome this, we introduce the frequency-enhanced geometric-constrained iterative network (FGIN). FGIN begins by mining the latent features from ECG data across both time and frequency domains. Subsequently, it increases the data dimensionality of ECG and captures intricate features using convolutional layers. Finally, FGIN incorporates ventricular geometry as a constraint on surface potential distribution. It allocates variable weights to distinct edges. Experimental validation of FGIN confirms its efficacy over synthetic and clinical datasets. On the synthetic dataset, FGIN outperforms seven existing reconstruction methods, attaining the highest Pearson Correlation Coefficient of 0.8624, the lowest Root Mean Square Error of 0.1548, and the highest Structural Similarity Index Measure of 0.7988. On the clinical public dataset (2007 PhysioNet/Computers in Cardiology Challenge), FGIN achieves better localization results than other approaches, according to the clinical standard 17-segment model, achieving an average Segment Overlap of 87.2%. Clinical trials on 50 patients demonstrate FGIN's effectiveness, showing an average accuracy of 91.6% and an average Segment Overlap of 88.2%.

A conditional multi-label model to improve prediction of a rare outcome: An illustration predicting autism diagnosis

ObjectiveThis study aimed to develop a novel approach using routinely collected electronic health records (EHRs) data to improve the prediction of a rare event. We illustrated this using an example of improving early prediction of an autism diagnosis, given its low prevalence, by leveraging correlations between autism and other neurodevelopmental conditions (NDCs). MethodsTo achieve this, we introduced a conditional multi-label model by merging conditional learning and multi-label methodologies. The conditional learning approach breaks a hard task into more manageable pieces in each stage, and the multi-label approach utilizes information from related neurodevelopmental conditions to learn predictive latent features. The study involved forecasting autism diagnosis by age 5.5 years, utilizing data from the first 18 months of life, and the analysis of feature importance correlations to explore the alignment within the feature space across different conditions. ResultsUpon analysis of health records from 18,156 children, we are able to generate a model that predicts a future autism diagnosis with moderate performance (AUROC=0.76). The proposed conditional multi-label method significantly improves predictive performance with an AUROC of 0.80 (p < 0.001). Further examination shows that both the conditional and multi-label approach alone provided marginal lift to the model performance compared to a one-stage one-label approach. We also demonstrated the generalizability and applicability of this method using simulated data with high correlation between feature vectors for different labels. ConclusionOur findings underscore the effectiveness of the developed conditional multi-label model for early prediction of an autism diagnosis. The study introduces a versatile strategy applicable to prediction tasks involving limited target populations but sharing underlying features or etiology among related groups.

Classification of select functional dietary fiber ingredients based on quantitative properties and latent qualitative criteria.

Functional dietary fiber ingredient (FDFI) functionality can depend on the fibers' chemistry, composition, size, botanical origin, and microstructure. However, such claims have never been generalized for a broad range of fibers in one study before. To support these claims, 23 FDFIwere characterized based on 11 physicochemical, physical, and compositional property measurements: Water- and oil-holding capacity (WHC and OHC), water absorption and solubility indices (WAI and WSI), flour-swelling potential (FSP), particle size distribution (D10, D50, and D90 values), and soluble, insoluble, and total dietary fiber content. Multivariate statistical techniques were employed to partition fiber ingredients into functional categories based on these quantitative data, and scanning electron microscopy was used to examine the microstructure of the FDFI. Strong correlations (p<0.05) were found among many of the physicochemical properties measured, and five categories based on quantitative physicochemical functionality, size, and fiber composition were ultimately found. Distinct patterns emerged between these quantitative partitions and the latent microstructure features and botanical origins of the FDFI. These results can be combined into one intuitive summary of FDFI functionality based on the described quantitative and qualitative observations. Such summaries are useful for ingredient suppliers or product developers with limited resources to infer the general functionality, structure, and food applications utility of their materials based on a subset of the information provided here. PRACTICAL APPLICATION: The quantitative and qualitative relationships among a range of commercially available functional dietary fiber ingredients are documented. Industry may utilize this information to predict the general functionality of their ingredients based on a subset of the information provided here by assuming that the same relative relationships will exist. This cansave time during the ingredient screening process, either for product developers looking to optimize a formulation or for ingredient suppliers doing new ingredient applications testing.

Efficient and user-friendly visualization of neural relightable images for cultural heritage applications

We introduce an innovative multiresolution framework for encoding and interactively visualizing large relightable images using a neural reflectance model derived from a state-of-the-art technique. The framework is seamlessly integrated into a scalable multi-platform framework that supports adaptive streaming and exploration of multi-layered relightable models in web settings. To enhance efficiency, we optimized the neural model, simplified decoding, and implemented a custom WebGL shader specific to the task, eliminating the need for deep-learning library integration in the code. Additionally, we introduce an efficient level-of-detail management system supporting fine-grained adaptive rendering through on-the-fly resampling in latent feature space. The resulting viewer facilitates interactive neural relighting of large images. Its modular design allows the incorporation of functionalities for Cultural Heritage analysis, such as loading and simultaneous visualization of multiple relightable layers with arbitrary rotations.

Visual interpretability of image-based classification models by generative latent space disentanglement applied to in vitro fertilization

The success of deep learning in identifying complex patterns exceeding human intuition comes at the cost of interpretability. Non-linear entanglement of image features makes deep learning a “black box” lacking human meaningful explanations for the models’ decision. We present DISCOVER, a generative model designed to discover the underlying visual properties driving image-based classification models. DISCOVER learns disentangled latent representations, where each latent feature encodes a unique classification-driving visual property. This design enables “human-in-the-loop” interpretation by generating disentangled exaggerated counterfactual explanations. We apply DISCOVER to interpret classification of in vitro fertilization embryo morphology quality. We quantitatively and systematically confirm the interpretation of known embryo properties, discover properties without previous explicit measurements, and quantitatively determine and empirically verify the classification decision of specific embryo instances. We show that DISCOVER provides human-interpretable understanding of “black box” classification models, proposes hypotheses to decipher underlying biomedical mechanisms, and provides transparency for the classification of individual predictions.

Dynamic Soft Sensor Model for Endpoint Carbon Content and Temperature in BOF Steelmaking Based on Adaptive Feature Matching Variational Autoencoder

The key to endpoint control in basic oxygen furnace (BOF) steelmaking lies in accurately predicting the endpoint carbon content and temperature. However, BOF steelmaking data are complex and change distribution due to variations in raw material batches, process adjustments, and equipment conditions, leading to concept drift and affecting model performance. In order to resolve these problems, this paper proposes a dynamic soft sensor model based on an adaptive feature matching variational autoencoder (VAE-AFM). Firstly, this paper innovatively proposes an adaptive feature matching (AFM) method. This method utilizes the maximum mean discrepancy to calculate the values of the marginal and conditional distributions. Based on the discrepancy between these two values, a dynamic adjustment algorithm is designed to adaptively assign different weights to the two distributions. This approach dynamically and quantitatively evaluates and adjusts the relative importance of different distributions in the domain adaptation process, thereby enhancing the effectiveness of cross-domain data alignment. Secondly, a variational autoencoder (VAE) is employed to process the data, as the VAE model can capture the complex data structures and latent features in the steelmaking process. Finally, the features extracted by the VAE are processed with the adaptive feature matching method, thereby constructing the VAE-AFM dynamic soft sensor model. Experimental studies on actual BOF steelmaking data validate the efficacy of the offered approach, offering a reliable solution to the challenges of high complexity and concept drift in BOF steelmaking data.

Leveraging variational autoencoders and recurrent neural networks for demand forecasting in supply chain management: A case study

Accurate demand forecasting is key for companies to optimize inventory management and satisfy customer demand efficiently. This paper aims to Investigate on the application of generative AI models in demand forecasting. Two models were used: Long Short-Term Memory (LSTM) networks and Variational Autoencoder (VAE), and results were compared to select the optimal model in terms of performance and forecasting accuracy. The difference of actual and predicted demand values also ascertain LSTM’s ability to identify latent features and basic trends in the data. Further, some of the research works were focused on computational efficiency and scalability of the proposed methods for providing the guidelines to the companies for the implementation of the complicated techniques in demand forecasting. Based on these results, LSTM networks have a promising application in enhancing the demand forecasting and consequently helpful for the decision-making process regarding inventory control and other resource allocation.

Recognition of Forward Head Posture Through 3D Human Pose Estimation With a Graph Convolutional Network: Development and Feasibility Study.

Prolonged improper posture can lead to forward head posture (FHP), causing headaches, impaired respiratory function, and fatigue. This is especially relevant in sedentary scenarios, where individuals often maintain static postures for extended periods-a significant part of daily life for many. The development of a system capable of detecting FHP is crucial, as it would not only alert users to correct their posture but also serve the broader goal of contributing to public health by preventing the progression of chronic injuries associated with this condition. However, despite significant advancements in estimating human poses from standard 2D images, most computational pose models do not include measurements of the craniovertebral angle, which involves the C7 vertebra, crucial for diagnosing FHP. Accurate diagnosis of FHP typically requires dedicated devices, such as clinical postural assessments or specialized imaging equipment, but their use is impractical for continuous, real-time monitoring in everyday settings. Therefore, developing an accessible, efficient method for regular posture assessment that can be easily integrated into daily activities, providing real-time feedback, and promoting corrective action, is necessary. The system sequentially estimates 2D and 3D human anatomical key points from a provided 2D image, using the Detectron2D and VideoPose3D algorithms, respectively. It then uses a graph convolutional network (GCN), explicitly crafted to analyze the spatial configuration and alignment of the upper body's anatomical key points in 3D space. This GCN aims to implicitly learn the intricate relationship between the estimated 3D key points and the correct posture, specifically to identify FHP. The test accuracy was 78.27% when inputs included all joints corresponding to the upper body key points. The GCN model demonstrated slightly superior balanced performance across classes with an F1-score (macro) of 77.54%, compared to the baseline feedforward neural network (FFNN) model's 75.88%. Specifically, the GCN model showed a more balanced precision and recall between the classes, suggesting its potential for better generalization in FHP detection across diverse postures. Meanwhile, the baseline FFNN model demonstrates a higher precision for FHP cases but at the cost of lower recall, indicating that while it is more accurate in confirming FHP when detected, it misses a significant number of actual FHP instances. This assertion is further substantiated by the examination of the latent feature space using t-distributed stochastic neighbor embedding, where the GCN model presented an isotropic distribution, unlike the FFNN model, which showed an anisotropic distribution. Based on 2D image input using 3D human pose estimation joint inputs, it was found that it is possible to learn FHP-related features using the proposed GCN-based network to develop a posture correction system. We conclude the paper by addressing the limitations of our current system and proposing potential avenues for future work in this area.

A prediction method of diabetes comorbidity based on non-negative latent features

In this paper, we present a novel network-based approach, namely Inherently Non-negative Latent Feature Analysis for Diabetes Mellitus Comorbidity Detection (INDM), to enhance the detection and analysis of comorbidities associated with diabetes mellitus. Different from existing methods, INDM is the first computational approach that integrates comorbidity networks of the chronic disease spectrum with patient clinical characteristics. To perform the analytical tasks, the proposed INDM adopts the following core components. First, comorbidity networks representing patients diagnosed solely with hypertension and those with hypertension and diabetes are constructed, following the case-control design that establishes a 1:1 matching in age and gender between two cohorts. Subsequently, the disease set is modeled in the comorbidity network according to the relative risk methodology. This enables nodes and edges in the comorbidity network to represent disease interactions that are derived from the patient-disease bipartite graph. Second, a nonlinear loss function with the capability of inherently non-negative latent feature analysis followed by a comorbidity classifier is adopted to uncover the patterns indicating the diabetes comorbidity in the comorbidity network. The proposed INDM has been rigorously tested on actual diabetes comorbidity datasets. The notable results demonstrate that INDM exhibits superior detection accuracy. Furthermore, the topological structure discovered by the proposed INDM can provide a profound insight into hypertension comorbidity in both the case and control groups.

Diffusion Models-Based Purification for Common Corruptions on Robust 3D Object Detection.

LiDAR sensors have been shown to generate data with various common corruptions, which seriously affect their applications in 3D vision tasks, particularly object detection. At the same time, it has been demonstrated that traditional defense strategies, including adversarial training, are prone to suffering from gradient confusion during training. Moreover, they can only improve their robustness against specific types of data corruption. In this work, we propose LiDARPure, which leverages the powerful generation ability of diffusion models to purify corruption in the LiDAR scene data. By dividing the entire scene into voxels to facilitate the processes of diffusion and reverse diffusion, LiDARPure overcomes challenges induced from adversarial training, such as sparse point clouds in large-scale LiDAR data and gradient confusion. In addition, we utilize the latent geometric features of a scene as a condition to assist the generation of diffusion models. Detailed experiments show that LiDARPure can effectively purify 19 common types of LiDAR data corruption. Further evaluation results demonstrate that it can improve the average precision of 3D object detectors to an extent of 20% in the face of data corruption, much higher than existing defence strategies.

TripChain2RecDeepSurv: A novel framework to predict transit users’ lifecycle behavior status transitions for user management

Transit users’ lifecycle behavior pattern transition reflects the continuous and multi-phase changes in how frequently and regularly users utilize public transit over their lifetime. Predicting transit users’ lifecycle behavior pattern transition is vital for enhancing the efficiency and responsiveness of transportation systems. Thus, this study incorporates lifecycle analysis in predicting long-term sequential behavioral pattern transition processes to go beyond just examining user churning at a single point in time. Specifically, this study proposes the TripChain2RecDeepSurv, a novel model that pioneers the individual-level analysis of lifecycle behavior status transitions (LBST) within public transit systems. The TripChain2RecDeepSurv is composed of (1) the TripChain2Vec module for encoding transit users’ trip chains; (2) the self-attention Transformer module for exploring the latent features related to spatiotemporal patterns; (3) the recurrent deep survival analysis module for predicting LBSTs. We demonstrate TripChain2RecDeepSurv’s predictive performance for empirical analysis by employing Shenzhen Bus data. Our model achieves a 74.39% accuracy rate in churn determination and over 80% accuracy in status sequence identification on the churn path. In addition, our findings highlight the segmented nature of Kaplan-Meier curves and identify the optimal intervention time against the user churning process. Meanwhile, the proposed model provides individual-level heterogeneity analysis, which emphasizes the significance of customizing user engagement strategies, advocating for interventions that extend users’ engagement in patterns with high-frequency transit usage to curb the transition to less frequent travel usage.

An MRI multi-sequence feature imputation and fusion mutual-aid model based on sequence deletion for differentiation of high-grade from low-grade glioma

To evaluate the performance of magnetic resonance imaging (MRI) multi-sequence feature imputation and fusion mutual model based on sequence deletion in differentiating high-grade glioma (HGG) from low-grade glioma (LGG). We retrospectively collected multi-sequence MR images from 305 glioma patients, including 189 HGG patients and 116 LGG patients. The region of interest (ROI) of T1-weighted images (T1WI), T2-weighted images (T2WI), T2 fluid attenuated inversion recovery (T2_FLAIR) and post-contrast enhancement T1WI (CE_T1WI) were delineated to extract the radiomics features. A mutual-aid model of MRI multi-sequence feature imputation and fusion based on sequence deletion was used for imputation and fusion of the feature matrix with missing data. The discriminative ability of the model was evaluated using 5-fold cross-validation method and by assessing the accuracy, balanced accuracy, area under the ROC curve (AUC), specificity, and sensitivity. The proposed model was quantitatively compared with other non-holonomic multimodal classification models for discriminating HGG and LGG. Class separability experiments were performed on the latent features learned by the proposed feature imputation and fusion methods to observe the classification effect of the samples in twodimensional plane. Convergence experiments were used to verify the feasibility of the model. For differentiation of HGG from LGG with a missing rate of 10%, the proposed model achieved accuracy, balanced accuracy, AUC, specificity, and sensitivity of 0.777, 0.768, 0.826, 0.754 and 0.780, respectively. The fused latent features showed excellent performance in the class separability experiment, and the algorithm could be iterated to convergence with superior classification performance over other methods at the missing rates of 30% and 50%. The proposed model has excellent performance in classification task of HGG and LGG and outperforms other non-holonomic multimodal classification models, demonstrating its potential for efficient processing of non-holonomic multimodal data.

Continuous Dictionary of Nodes Model and Bilinear-Diffusion Representation Learning for Brain Disease Analysis.

Brain networks based on functional magnetic resonance imaging (fMRI) provide a crucial perspective for diagnosing brain diseases. Representation learning has recently attracted tremendous attention due to its strong representation capability, which can be naturally applied to brain disease analysis. However, traditional representation learning only considers direct and local node interactions in original brain networks, posing challenges in constructing higher-order brain networks to represent indirect and extensive node interactions. To address this problem, we propose the Continuous Dictionary of Nodes model and Bilinear-Diffusion (CDON-BD) network for brain disease analysis. The CDON model is innovatively used to learn the original brain network, with its encoder weights directly regarded as latent features. To fully integrate latent features, we further utilize Bilinear Pooling to construct higher-order brain networks. The Diffusion Module is designed to capture extensive node interactions in higher-order brain networks. Compared to state-of-the-art methods, CDON-BD demonstrates competitive classification performance on two real datasets. Moreover, the higher-order representations learned by our method reveal brain regions relevant to the diseases, contributing to a better understanding of the pathology of brain diseases.

Deep Learning with Pretrained Framework Unleashes the Power of Satellite-Based Global Fine-Mode Aerosol Retrieval.

Fine-mode aerosol optical depth (fAOD) is a vital proxy for the concentration of anthropogenic aerosols in the atmosphere. Currently, the limited data length and high uncertainty of the satellite-based data diminish the applicability of fAOD for climate research. Here, we propose a novel pretrained deep learning framework that can extract information underlying each satellite pixel and use it to create new latent features that can be employed for improving retrieval accuracy in regions without in situ data. With the proposed model, we developed a new global fAOD (at 0.5 μm) data from 2001 to 2020, resulting in a 10% improvement in the overall correlation coefficient (R) during site-based independent validation and a 15% enhancement in non-AERONET site areas validation. Over the past two decades, there has been a noticeable downward trend in global fAOD (-1.39 × 10-3/year). Compared to the general deep-learning model, our method reduces the global trend's previously overestimated magnitude by 7% per year. China has experienced the most significant decline (-5.07 × 10-3/year), which is 3 times greater than the global trend. Conversely, India has shown a significant increase (7.86 × 10-4/year). This study bridges the gap between sparse in situ observations and abundant satellite measurements, thereby improving predictive models for global patterns of fAOD and other climate factors.

A deep learning model for anti-inflammatory peptides identification based on deep variational autoencoder and contrastive learning

As a class of biologically active molecules with significant immunomodulatory and anti-inflammatory effects, anti-inflammatory peptides have important application value in the medical and biotechnology fields due to their unique biological functions. Research on the identification of anti-inflammatory peptides provides important theoretical foundations and practical value for a deeper understanding of the biological mechanisms of inflammation and immune regulation, as well as for the development of new drugs and biotechnological applications. Therefore, it is necessary to develop more advanced computational models for identifying anti-inflammatory peptides. In this study, we propose a deep learning model named DAC-AIPs based on variational autoencoder and contrastive learning for accurate identification of anti-inflammatory peptides. In the sequence encoding part, the incorporation of multi-hot encoding helps capture richer sequence information. The autoencoder, composed of convolutional layers and linear layers, can learn latent features and reconstruct features, with variational inference enhancing the representation capability of latent features. Additionally, the introduction of contrastive learning aims to improve the model's classification ability. Through cross-validation and independent dataset testing experiments, DAC-AIPs achieves superior performance compared to existing state-of-the-art models. In cross-validation, the classification accuracy of DAC-AIPs reached around 88%, which is 7% higher than previous models. Furthermore, various ablation experiments and interpretability experiments validate the effectiveness of DAC-AIPs. Finally, a user-friendly online predictor is designed to enhance the practicality of the model, and the server is freely accessible at http://dac-aips.online.

A Genomics-Driven Artificial Intelligence-Based Model Classifies Breast Invasive Lobular Carcinoma and Discovers CDH1 Inactivating Mechanisms.

Artificial intelligence (AI) systems can improve cancer diagnosis, yet their development often relies on subjective histologic features as ground truth for training. Herein, we developed an AI model applied to histologic whole-slide images using CDH1 biallelic mutations, pathognomonic for invasive lobular carcinoma (ILC) in breast neoplasms, as ground truth. The model accurately predicted CDH1 biallelic mutations (accuracy = 0.95) and diagnosed ILC (accuracy = 0.96). A total of 74% of samples classified by the AI model as having CDH1 biallelic mutations but lacking these alterations displayed alternative CDH1 inactivating mechanisms, including a deleterious CDH1 fusion gene and noncoding CDH1 genetic alterations. Analysis of internal and external validation cohorts demonstrated 0.95 and 0.89 accuracy for ILC diagnosis, respectively. The latent features of the AI model correlated with human-explainable histopathologic features. Taken together, this study reports the construction of an AI algorithm trained using a genetic rather than histologic ground truth that can robustly classify ILCs and uncover CDH1 inactivating mechanisms, providing the basis for orthogonal ground truth utilization for development of diagnostic AI models applied to whole-slide image. Significance: Genetic alterations linked to strong genotypic-phenotypic correlations can be utilized to develop AI systems applied to pathology that facilitate cancer diagnosis and biologic discoveries.