Multi-task Deep Neural Network Research Articles

Post-earthquake inspection of high-speed railway viaducts with multi-scale task interaction deep learning strategy

Automated computer vision-based inspections of railway infrastructures, such as component type, damaged status, and location, have been investigated actively by resorting to task-specific deep learning models. However, task-specific models that fulfill these separate inspection tasks encountered bottlenecks in improving inference accuracy, and bring huge computational costs. Multi-task deep learning, which can fulfill these inspection tasks concurrently, has yet to be fully investigated in the context of structural inspection. In this study, a multi-scale task interaction deep learning strategy is presented towards component recognition, damage segmentation, and depth estimation for a comprehensive post-earthquake inspection of high-speed railway viaducts. Three modules for multi-scale task interaction were proposed to modify the multi-task deep neural network, taking full advantage of task commonalities at multiple scales. The proposed method was validated with a large-scale image dataset of high-speed railway viaducts. Component recognition and depth estimation were incorporated to implement multi-task learning since they have higher pattern affinities at multiple scales. Results reported that mean Intersection over Unions of testing samples of component and damage tasks were 91.2% and 72.1%, RMSE of depth estimation was 1.54 m. Compared with single-task cases, training time, inference duration, and FLOPs of the multi-task model were reduced by 23%, 30%, 27% respectively. Results showed improvement in both inference accuracy and training efficiency, substantiating the superiority of the proposed strategy.

Super-resolution enhancement and segmentation for digital rock based on multi-task joint deep neural network

Reliable segmentation of pores and minerals from high-resolution (HR) digital rock data is the fundamental prerequisite for accurately characterizing the digital rock's physical properties. Limited by the complexity of the super-resolution (SR) issues and the user bias for segmentation, conventional SR enhancement and segmentation methods are generally brutal in satisfying the research demands. In recent years, with the introduction of deep learning technology into digital rock research, deep learning-based approaches for SR enhancement or segmentation methods have emerged and achieved significantly better effects than conventional approaches. Most deep learning-based digital rock SR enhancement and segmentation processing steps are currently separate. Still, preliminary studies have demonstrated that an end-to-end approach that integrates the two steps could achieve better performance. However, the training cost of 3D deep neural networks that can be directly applied to 3D digital rock processing is usually expensive. In contrast, 2D networks cannot be effectively applied to SR segmentation of 3D digital rocks. Here, we present an end-to-end SR segmentation framework for 3D digital rock based on a 2D multi-task joint deep neural network. The multi-task joint networks utilize a parallel architecture that integrates the SegNet for the segmentation task and the EDSR for the SR task. We also improved the loss function to address the issue of category imbalance and proposed an approach to utilize the 2D network for 3D digital rock processing. We demonstrate the effectiveness of the proposed network framework, the improved loss function, and the 3D digital rock processing strategies through ablation experiments on the high and low-resolution (LR) CT image datasets captured by imaging devices. The results show that the evaluation metrics and physical properties of the SR-segmented 2D\\3D digital rocks align with the HR-segmented results (ground truth). It indicates that the proposed framework can improve the performance of digital rock SR enhancement and segmentation and that it is essential to integrate deep learning frameworks into digital rock analysis.

Multi-microphone simultaneous speakers detection and localization of multi-sources for separation and noise reduction

This paper addresses the challenge of online blind speaker separation in a multi-microphone setting. The linearly constrained minimum variance (LCMV) beamformer is selected as the backbone of the separation algorithm due to its distortionless response and capacity to create a null towards interfering sources. A specific instance of the LCMV beamformer that considers acoustic propagation is implemented. In this variant, the relative transfer functions (RTFs) associated with each speaker of interest are utilized as the steering vectors of the beamformer. A control mechanism is devised to ensure robust estimation of the beamformer’s building blocks, comprising speaker activity detectors and direction of arrival (DOA) estimation branches. This control mechanism is implemented as a multi-task deep neural network (DNN). The primary task classifies each time frame based on speaker activity: no active speaker, single active speaker, or multiple active speakers. The secondary task is DOA estimation. It is implemented as a classification task, executed only for frames classified as single-speaker frames by the primary branch. The direction of the active speaker is classified into one of the multiple ranges of angles. These frames are also leveraged to estimate the RTFs using subspace estimation methods. A library of RTFs associated with these DOA ranges is then constructed, facilitating rapid acquisition of new speakers and efficient tracking of existing speakers. The proposed scheme is evaluated in both simulated and real-life recordings, encompassing static and dynamic scenarios. The benefits of the multi-task approach are showcased, and significant improvements are evident, even when the control mechanism is trained with simulated data and tested with real-life data. A comparison between the proposed scheme and the independent low-rank matrix analysis (ILRMA) algorithm reveals significant improvements in static scenarios. Furthermore, the tracking capabilities of the proposed scheme are highlighted in dynamic scenarios.

The Development and Application of KinomePro-DL: A Deep Learning Based Online Small Molecule Kinome Selectivity Profiling Prediction Platform.

Characterizing the kinome selectivity profiles of kinase inhibitors is essential in the early stages of novel small-molecule drug discovery. This characterization is critical for interpreting potential adverse events caused by off-target polypharmacology effects and provides unique pharmacological insights for drug repurposing development of existing kinase inhibitor drugs. However, experimental profiling of whole kinome selectivity is still time-consuming and resource-demanding. Here, we report a deep learning classification model using an in-house built data set of inhibitors against 191 well-representative kinases constructed based on a novel strategy by systematically cleaning and integrating six public data sets. This model, a multitask deep neural network, predicts the kinome selectivity profiles of compounds with novel structures. The model demonstrates excellent predictive performance, with auROC, prc-AUC, Accuracy, and Binary_cross_entropy of 0.95, 0.92, 0.90, and 0.37, respectively. It also performs well in a priori testing for inhibitors targeting different categories of proteins from internal compound collections, significantly improving over similar models on data sets from practical application scenarios. Integrated to subsequent machine learning-enhanced virtual screening workflow, novel CDK2 kinase inhibitors with potent kinase inhibitory activity and excellent kinome selectivity profiles are successfully identified. Additionally, we developed a free online web server, KinomePro-DL, to predict the kinome selectivity profiles and kinome-wide polypharmacology effects of small molecules (available on kinomepro-dl.pharmablock.com). Uniquely, our model allows users to quickly fine-tune it with their own training data sets, enhancing both prediction accuracy and robustness.

Prediction of human initial operation situation in confined space with a multi-task deep neural network

Most large manufacturing fields still rely on manual operations, requiring operators to enter the Confined Space (CS) to conduct assembly and maintenance tasks. The ergonomics analysis combined with the Digital Human Model (DHM) can provide valuable suggestions and guidance for product structure design and process planning. Due to the particularity of CS, the current algorithms applied to DHM operation planning are not suitable for CS operations. To address this challenge, we introduce a novel approach to predict the initial operation situation in CS. The Restriction State Expression Model for Bi-Directional Confined Space (RSEMfBDCS) is introduced to quantify the restricted degree, and a novel multi-task deep neural network, as the Human Operation Situation Prediction Network in CS (HOSPNiCS) is developed to predict the initial operation situation in CS, which includes operation reachability, the initial OPI, and the initial OPII of the operators. Training and Verifying by the data collected from the real operation cases, HOSPNiCS distinguishes whether the manual operation is reachable with an average accuracy of 0.996, and the average accuracy for initial OPI prediction can reach 0.957. For initial OPII prediction, HOSPNiCS has an average distance error of 0.036 m and an average angle error of 0.395°. Based on the proposed approach, the engineering designers can swiftly ascertain operation reachability and get the initial OPI and OPII of DHM with the help of the prediction results. In light of this, the human operation simulation would be more practical for future human factor analysis.

From processing to properties: Enhancing machine learning models with microstructural information in polymer nanocomposites

For polymers and their composites, processing conditions and the resultant microstructures are crucial in determining their properties. Traditional machine learning (ML) methods typically focus on establishing direct relationships between processing parameters and material properties, often overlooking the critical intermediate step of how processing influences microstructure, limiting the predictive accuracy. In this study, we introduce an approach that first establishes a detailed relationship between processing parameters and the resultant microstructure, and then uses transfer learning and feature fusion to integrate this relationship into the prediction of material properties. Using carbon black-reinforced rubber composites (CRC) as an example, we compared ML models in predicting mechanical properties from processing data. A multi-task deep neural network performed best achieving an R2 of 0.763 with only processing data as input. When incorporating transfer learning and feature fusion, the R2 improved to 0.852 and 0.878, respectively. Shapley explanation analysis validated our approach, highlighting the importance of integrating processing, microstructure, and properties in ML models. This method emphasizes the critical effect of comprehensively considering processing–(micro)structure–property (P–S–P) relationships, leading to more accurate predictions in polymer composite research.

Real-time estimation of the optimal coil placement in transcranial magnetic stimulation using multi-task deep learning

Transcranial magnetic stimulation (TMS) has emerged as a promising neuromodulation technique with both therapeutic and diagnostic applications. As accurate coil placement is known to be essential for focal stimulation, computational models have been established to help find the optimal coil positioning by maximizing electric fields at the cortical target. While these numerical simulations provide realistic and subject-specific field distributions, they are computationally demanding, precluding their use in real-time applications. In this paper, we developed a novel multi-task deep neural network which simultaneously predicts the optimal coil placement for a given cortical target as well as the associated TMS-induced electric field. Trained on large amounts of preceding numerical optimizations, the Attention U-Net-based neural surrogate provided accurate coil optimizations in only 35 ms, a fraction of time compared to the state-of-the-art numerical framework. The mean errors on the position estimates were below 2 mm, i.e., smaller than previously reported manual coil positioning errors. The predicted electric fields were also highly correlated (r> 0.97) with their numerical references. In addition to healthy subjects, we validated our approach also in glioblastoma patients. We first statistically underlined the importance of using realistic heterogeneous tumor conductivities instead of simply adopting values from the surrounding healthy tissue. Second, applying the trained neural surrogate to tumor patients yielded similar accurate positioning and electric field estimates as in healthy subjects. Our findings provide a promising framework for future real-time electric field-optimized TMS applications.

Predicting the translation efficiency of messenger RNA in mammalian cells.

The degree to which translational control is specified by mRNA sequence is poorly understood in mammalian cells. Here, we constructed and leveraged a compendium of 3,819 ribosomal profiling datasets, distilling them into a transcriptome-wide atlas of translation efficiency (TE) measurements encompassing >140 human and mouse cell types. We subsequently developed RiboNN, a multitask deep convolutional neural network, and classic machine learning models to predict TEs in hundreds of cell types from sequence-encoded mRNA features, achieving state-of-the-art performance (r=0.79 in human and r=0.78 in mouse for mean TE across cell types). While the majority of earlier models solely considered 5' UTR sequence, RiboNN integrates contributions from the full-length mRNA sequence, learning that the 5' UTR, CDS, and 3' UTR respectively possess ~67%, 31%, and 2% per-nucleotide information density in the specification of mammalian TEs. Interpretation of RiboNN revealed that the spatial positioning of low-level di- and tri-nucleotide features (i.e., including codons) largely explain model performance, capturing mechanistic principles such as how ribosomal processivity and tRNA abundance control translational output. RiboNN is predictive of the translational behavior of base-modified therapeutic RNA, and can explain evolutionary selection pressures in human 5' UTRs. Finally, it detects a common language governing mRNA regulatory control and highlights the interconnectedness of mRNA translation, stability, and localization in mammalian organisms.

Multitask Deep Neural Network With Knowledge-Guided Attention for Blind Image Quality Assessment

Multitask Deep Neural Network With Knowledge-Guided Attention for Blind Image Quality Assessment

Multi-task aquatic toxicity prediction model based on multi-level features fusion

IntroductionWith the escalating menace of organic compounds in environmental pollution imperiling the survival of aquatic organisms, the investigation of organic compound toxicity across diverse aquatic species assumes paramount significance for environmental protection. Understanding how different species respond to these compounds helps assess the potential ecological impact of pollution on aquatic ecosystems as a whole. Compared with traditional experimental methods, deep learning methods have higher accuracy in predicting aquatic toxicity, faster data processing speed and better generalization ability. ObjectivesThis article presents ATFPGT-multi, an advanced multi-task deep neural network prediction model for organic toxicity. MethodsThe model integrates molecular fingerprints and molecule graphs to characterize molecules, enabling the simultaneous prediction of acute toxicity for the same organic compound across four distinct fish species. Furthermore, to validate the advantages of multi-task learning, we independently construct prediction models, named ATFPGT-single, for each fish species. We employ cross-validation in our experiments to assess the performance and generalization ability of ATFPGT-multi. ResultsThe experimental results indicate, first, that ATFPGT-multi outperforms ATFPGT-single on four fish datasets with AUC improvements of 9.8%, 4%, 4.8%, and 8.2%, respectively, demonstrating the superiority of multi-task learning over single-task learning. Furthermore, in comparison with previous algorithms, ATFPGT-multi outperforms comparative methods, emphasizing that our approach exhibits higher accuracy and reliability in predicting aquatic toxicity. Moreover, ATFPGT-multi utilizes attention scores to identify molecular fragments associated with fish toxicity in organic molecules, as demonstrated by two organic molecule examples in the main text, demonstrating the interpretability of ATFPGT-multi. ConclusionIn summary, ATFPGT-multi provides important support and reference for the further development of aquatic toxicity assessment. All of codes and datasets are freely available online at https://github.com/zhaoqi106/ATFPGT-multi.

Streamlining neuroradiology workflow with AI for improved cerebrovascular structure monitoring

Radiological imaging to examine intracranial blood vessels is critical for preoperative planning and postoperative follow-up. Automated segmentation of cerebrovascular anatomy from Time-Of-Flight Magnetic Resonance Angiography (TOF-MRA) can provide radiologists with a more detailed and precise view of these vessels. This paper introduces a domain generalized artificial intelligence (AI) solution for volumetric monitoring of cerebrovascular structures from multi-center MRAs. Our approach utilizes a multi-task deep convolutional neural network (CNN) with a topology-aware loss function to learn voxel-wise segmentation of the cerebrovascular tree. We use Decorrelation Loss to achieve domain regularization for the encoder network and auxiliary tasks to provide additional regularization and enable the encoder to learn higher-level intermediate representations for improved performance. We compare our method to six state-of-the-art 3D vessel segmentation methods using retrospective TOF-MRA datasets from multiple private and public data sources scanned at six hospitals, with and without vascular pathologies. The proposed model achieved the best scores in all the qualitative performance measures. Furthermore, we have developed an AI-assisted Graphical User Interface (GUI) based on our research to assist radiologists in their daily work and establish a more efficient work process that saves time.

Robust returns ranking prediction and portfolio optimization for M6

The M6 competition aims to address challenging problems in stock returns ranking prediction and portfolio optimization. To tackle the volatility and low signal-to-noise ratio in the stock market, our team designs the overall solution from the perspective of robustness. On the task of returns ranking prediction, we present the MultiTask Deep Neural Network with Denoising Autoencoder Enhancement (MT-DNN-DAE), which incorporates the self-supervised learning of DAE and jointly optimizes the multi-task loss. Additionally, we propose Robust Feature Selection (RFS) to identify features with a high signal-to-noise ratio for DAE’s representation learning and construct a separate branch for important ID feature to prevent its information loss. Results show our solution is capable of accurately predicting returns ranking while maintaining generalization. On the task of portfolio optimization, a Differential Evolution algorithm is also presented to optimize asset allocation and maximize returns under risk constraints, demonstrating improved performance over traditional techniques. Taken together, these methods led to a 4th place global ranking in the M6 competition.

Classifying breast cancer subtypes on multi-omics data via sparse canonical correlation analysis and deep learning

BackgroundClassifying breast cancer subtypes is crucial for clinical diagnosis and treatment. However, the early symptoms of breast cancer may not be apparent. Rapid advances in high-throughput sequencing technology have led to generating large number of multi-omics biological data. Leveraging and integrating the available multi-omics data can effectively enhance the accuracy of identifying breast cancer subtypes. However, few efforts focus on identifying the associations of different omics data to predict the breast cancer subtypes.ResultsIn this paper, we propose a differential sparse canonical correlation analysis network (DSCCN) for classifying the breast cancer subtypes. DSCCN performs differential analysis on multi-omics expression data to identify differentially expressed (DE) genes and adopts sparse canonical correlation analysis (SCCA) to mine highly correlated features between multi-omics DE-genes. Meanwhile, DSCCN uses multi-task deep learning neural network separately to train the correlated DE-genes to predict breast cancer subtypes, which spontaneously tackle the data heterogeneity problem in integrating multi-omics data.ConclusionsThe experimental results show that by mining the associations among multi-omics data, DSCCN is more capable of accurately classifying breast cancer subtypes than the existing methods.

Abstract 7396: Development of deep neural networks that guide more individualized radiotherapy to the lung

Abstract Introduction: Radiotherapy is dependent on an expert-based model that impedes implementation of many potentially useful applications that could aid decision makers. This model is a barrier to a learning healthcare system due to noise introduced by decision-maker variation. Surmounting this barrier will require compelling clinical outcome data and highly useful and accurate information systems. We sought to develop new augmented-intelligence capabilities to better predict both high-dose lung irradiation treatment efficacy (i.e., local tumor control) and toxicity (i.e., radiation-induced pneumonitis). Methods: We curated one of the largest datasets of lung stereotactic body radiotherapy (SBRT) datasets globally (~1200 cases). We input pre-therapy lung computed tomography (CT) images using a bounding box approach that captures the gross tumor volume and adjacent normal lung into a multi-task deep neural network to generate an image fingerprint that predicts local failures after treatment. Separately, we input axial slices from CT scans of the lung into convolutional neural network (CNN) and trained using an outcome per slice method. We then aggregated intra-patient slices into a recurrent neural network (RNN) to estimate individual patient risk for pneumonitis. We validated our findings with the independent study cohort. Risk scores for local failure and pneumonitis were combined with clinical variables to assess the relative contribution of the deep learning-derived image risk scores. Results: A total of 1045 patients met our eligibility criteria. Radiation treatments in patients with high local failure risk scores failed at a significantly higher rate than in patients with low scores; 3-year cumulative incidence of local failure in the internal study cohort was 20·3% (16·0-24·9) in patients with high scores and 5·7% (95% CI 3·5-8·8) in patients with low scores (hazard ratio [HR]=3·64 [95% CI 2·19-6·05], p<0·0001). Models that included the local failure risk score and clinical variables predicted treatment failures with a concordance index (C-index) of 0·72 (95% CI 0·67-0·77), a significant improvement compared with classical radiomics (p<0·0001) or clinical variables (p<0·0001) alone. Radiation pneumonitis risk scores were highly predictive of toxicity, with an accuracy and CI of 0.78 (95% CI 0.72-0.85). A model that includes dosimetry variates (V20), improved the CI to 0.85 (95% CI 0.80-0.90). Conclusions: The image-based deep learning frameworks developed herein represent the first opportunity to use medical images to individualize radiotherapy dose and toxicity predictions. Our results signify a new roadmap for deep learning-guided predictions and treatment guidance in the image-replete and highly standardized discipline of radiation oncology. A phase 2 study assessing machine-derived dose predictions is currently enrolling patients. Citation Format: Mohamed Abazeed. Development of deep neural networks that guide more individualized radiotherapy to the lung [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 7396.

An uncertainty-based interpretable deep learning framework for predicting breast cancer outcome

BackgroundPredicting outcome of breast cancer is important for selecting appropriate treatments and prolonging the survival periods of patients. Recently, different deep learning-based methods have been carefully designed for cancer outcome prediction. However, the application of these methods is still challenged by interpretability. In this study, we proposed a novel multitask deep neural network called UISNet to predict the outcome of breast cancer. The UISNet is able to interpret the importance of features for the prediction model via an uncertainty-based integrated gradients algorithm. UISNet improved the prediction by introducing prior biological pathway knowledge and utilizing patient heterogeneity information.ResultsThe model was tested in seven public datasets of breast cancer, and showed better performance (average C-index = 0.691) than the state-of-the-art methods (average C-index = 0.650, ranged from 0.619 to 0.677). Importantly, the UISNet identified 20 genes as associated with breast cancer, among which 11 have been proven to be associated with breast cancer by previous studies, and others are novel findings of this study.ConclusionsOur proposed method is accurate and robust in predicting breast cancer outcomes, and it is an effective way to identify breast cancer-associated genes. The method codes are available at: https://github.com/chh171/UISNet.

Deep survival analysis model for incident clearance time prediction

Incident clearance time prediction is a key task for traffic incident management. A hazard-based duration model is a prevalent approach for predicting and analyzing the incident clearance time, which considers “duration dependence” of which the probability of an incident clearance ending depends on the time the clearance has lasted. However, the performance is limited due to its model assumptions for clearance time distribution, linear relationship, and the time-invariant effects of influential factors. This study proposes a deep survival analysis model that relaxes the assumptions of the hazard-based duration model while considering duration dependence based on a multi-task deep neural network (MTDNN). The MTDNN can consider the duration dependence when predicting incident clearance time by simultaneously estimating the survival function based on the concept of multi-task learning. The effects of influential factors on the prediction of MTDNN are also investigated using a post-analysis method. The proposed model is evaluated by its predictive performance and the estimated effects of influential factors using the freeway incident data collected in Korea from 2014 to 2019. These evaluations show that, compared to the baseline hazard-based duration model, the proposed MTDNN improves the predictive performance by 29.7% in terms of mean absolute percent error, and outperforms all statistical and machine learning models for both incident clearance time prediction and the survival function estimation. The analysis of the influential factors reveals that the hazard-based duration model and MTDNN had major influencing factors in common, but the impact of some factors is considerably different.

Abstract B082: Early risk stratification of ER+/HER2– breast cancer patients using digital pathology and multi-task, weakly-supervised deep learning

Abstract Breast cancer prognostication guides treatment decisions, with transcriptomic assays like Prosigna and Oncotype DX providing valuable risk of recurrence (ROR) scores in ER+/HER2– patients. However, these tests are costly and require sufficient biopsy tissue for accuracy. Pathology images are routinely available and could expedite rapid risk stratification at lower cost. We propose a multi-modal, multi-task deep neural network that learns transcriptomic ROR using digital pathology images and hormone receptor expression status. Because the incorporation of both pathology images and clinical data has proven beneficial in other deep learning models, we hypothesized that incorporation of this complementary information would improve ROR predictions compared to using pathology images alone. We modified the “clustering-constrained-attention multiple-instance learning” (CLAM) deep learning method to perform multi-task regression and infer continuous risk scores (instead of discrete risk categories). We evaluated (1) PAM50 ROR-P, calculated using a multivariate model based on intrinsic subtype centroids and a proliferation score, and (2) a 21-gene assay recurrence score (RS) (research version of Oncotype DX), calculated from component scores relating to proliferation, ER, HER2, and invasion. Using data from 899 patients in the Carolina Breast Cancer Study, a population-based study of diverse patients including 50% Black women and 50% women under age 50, we constructed training (n=714), validation (n=93), and test (n=92) sets. Genomic data were assayed by Nanostring, and digital pathology was based on H&E-stained whole slide images (WSIs). Models were assessed using Pearson correlation between measured and predicted ROR-P or RS. The model with the highest correlations on the test set used H&E WSIs and ER/PR/HER2 expression status as inputs and generated 10 independent outputs: ROR-P, four PAM50 centroid correlations (i.e., Basal, HER2-enriched, Luminal A, and Luminal B), RS, and four RS component scores. Prediction was strong when considering all participants in the test set (ROR-P, 0.78; RS, 0.83, N, 92) but was reduced among more clinically homogeneous ER+/HER2– patients (ROR-P, 0.55; RS, 0.49; N, 49). For RS, correlations were higher in models that included ER/PR/HER2 (Overall, 0.83; ER+/HER2–, 0.50) than in those that relied on pathology alone (Overall, 0.60, ER+/HER2–, 0.34). Future models will consider strategies to enhance performance in ER+/HER2– patients, such as oversampling of ER+/HER2– patients and use of loss functions designed to focus learning on mis-predicted examples. In summary, multi-task, tissue-based regression deep learning models recapitulate transcription-based risk assays with high correlations and, with optimization (especially integration of additional clinicopathologic data), hold promise for personalized treatment decisions from early, routinely-collected biopsy images. Citation Format: Jakub R Kaczmarzyk, Luke A Torre-Healy, Richard A Moffitt, Rajarsi Gupta, Alina M Hamilton, Tahsin M Kurc, Katherine A Hoadley, Melissa A Troester, Joel H Saltz. Early risk stratification of ER+/HER2– breast cancer patients using digital pathology and multi-task, weakly-supervised deep learning [abstract]. In: Proceedings of the AACR Special Conference in Cancer Research: Advances in Breast Cancer Research; 2023 Oct 19-22; San Diego, California. Philadelphia (PA): AACR; Cancer Res 2024;84(3 Suppl_1):Abstract nr B082.

Refined image quality assessment for color fundus photography based on deep learning.

Color fundus photography is widely used in clinical and screening settings for eye diseases. Poor image quality greatly affects the reliability of further evaluation and diagnosis. In this study, we developed an automated assessment module for color fundus photography image quality assessment using deep learning. A total of 55,931 color fundus photography images from multiple centers in Shanghai and the public database were collected and annotated as training, validation, and testing data sets. The pre-diagnosis image quality assessment module based on the multi-task deep neural network was designed. The detailed criterion of color fundus photography image quality including three subcategories with three levels of grading was applied to improve precision and objectivity. The auxiliary tasks such as the localization of the optic nerve head and macula, the classification of laterality, and the field of view were also included to assist the quality assessment. Finally, we validated our module internally and externally by evaluating the area under the receiver operating characteristic curve, sensitivity, specificity, accuracy, and quadratic weighted Kappa. The "Location" subcategory achieved area under the receiver operating characteristic curves of 0.991, 0.920, and 0.946 for the three grades, respectively. The "Clarity" subcategory achieved area under the receiver operating characteristic curves of 0.980, 0.917, and 0.954 for the three grades, respectively. The "Artifact" subcategory achieved area under the receiver operating characteristic curves of 0.976, 0.952, and 0.986 for the three grades, respectively. The accuracy and Kappa of overall quality reach 88.15% and 89.70%, respectively, on the internal set. These two indicators on the external set were 86.63% and 88.55%, respectively, which were very close to that of the internal set. This work showed that our deep module was able to evaluate the color fundus photography image quality using more detailed three subcategories with three grade criteria. The promising results on both internal and external validation indicated the strength and generalizability of our module.

AI-driven Automated Operational Modal Analysis of Bridges

The growing awareness on the daunting challenge posed by ageing civil infrastructure is fostering the increasingly frequent implementation of dense Structural Health Monitoring (SHM) systems. The Italian context represents a formidable example of this trend, especially after the tragic collapse of the Morandi Bridge in 2018 and the release of the National Guidelines on the Assessment and Management of the Risk Condition of Bridges and Viaducts in 2020. This has motivated the implementation of numerous monitoring systems all throughout the national territory, many of them involving heterogeneous and dense sensor networks. The implementation of such a vast number of sensors presents new challenges at the management and decision-making levels, particularly in the signal processing and feature extraction phases. This is particularly critical for vibration-based SHM techniques, in which classical operational modal analysis (OMA) techniques are proving inefficient for handling such large monitoring databases. As an attempt to alleviate these limitations, this contribution presents the development of an AI-driven OMA technique for rapid identification of bridges. The proposed methodology consists of a multi-task deep feedforward neural network capable of extracting the independent modal components from ambient vibration records. Trained in a supervised manner by a Second Order Blind Source Identification (SOBI) method, the developed AI model can extract both the real and complex independent modal components. The effectiveness of the presented approach is illustrated through of a real-world-bridge, the Méndez-Núñez Bridge in Granada, Spain, demonstrating great potential as a computationally efficient OMA technique for on-site edge modal identification of bridges.

A Multi-task Deep Feature Selection Method for Brain Imaging Genetics.

Using brain imaging quantitative traits (QTs) for identifying genetic risk factors is an important research topic in brain imaging genetics. Many efforts have been made for this task via building linear models between imaging QTs and genetic factors such as single nucleotide polymorphisms (SNPs). To the best of our knowledge, linear models could not fully uncover the complicated relationship due to the loci's elusive and diverse influences on imaging QTs. In this paper, we propose a novel multi-task deep feature selection (MTDFS) method for brain imaging genetics. MTDFS first builds a multi-task deep neural network to model the complicated associations between imaging QTs and SNPs. And then designs a multi-task one-to-one layer and imposes a combined penalty to identify SNPs that make significant contributions. MTDFS can not only extract the nonlinear relationship but also arms the deep neural network with feature selection. We compared MTDFS to multi-task linear regression (MTLR) and single-task DFS (DFS) methods on the real neuroimaging genetic data. The experimental results showed that MTDFS performed better than MTLR and DFS on the QT-SNP relationship identification and feature selection. Thus, MTDFS is powerful for identifying risk loci and could be a great supplement to brain imaging genetics.