Subset Of Images Research Articles

Association of lower extremity peripheral arterial disease with quantitative muscle features from computed tomography angiography

ObjectivesTo explore the association between lower extremity muscle features from CTA and peripheral arterial disease (PAD) severity using digital subtraction angiography (DSA) as reference standard.MethodsInformed consent was waived for this Institutional Review Board approved retrospective study. PAD patients were recruited from July 2016 to September 2020. Two radiologists evaluated PAD severity on DSA and CTA using runoff score. The patients were divided into two groups: mild PAD (DSA score ≤ 7) vs. severe PAD (DSA score > 7). After segmenting lower extremity muscles from CTA, 95 features were extracted for univariable analysis, logistic regression model (LRM) analysis, and sub-dataset analysis (PAD prediction based on only part of the images). AUC of CTA score and LRMs for PAD prediction were calculated. Features were analyzed using Student’s t test and chi-squared test. p < 0.05 was considered statistically significant.ResultsA total of 56 patients (69 ± 11 years; 38 men) with 56 lower legs were enrolled in this study. The lower leg muscles of mild PAD group (36 patients) showed higher CT values (44.6 vs. 39.5, p < 0.001) with smaller dispersion (35.6 vs. 41.0, p < 0.001) than the severe group (20 patients). The AUC of CTA score, LRM-I (constructed with muscle features), and LRM-II (constructed with muscle features and CTA score) for PAD severity prediction were 0.81, 0.84, and 0.89, respectively. The highest predictive performance was observed in the image subset of the middle and inferior segments of lower extremity (LRM-I, 0.83; LRM-II, 0.90).ConclusionsLower extremity muscle features are associated with PAD severity and can be used for PAD prediction.Critical relevance statementQuantitative image features of lower extremity muscles are associated with the degree of lower leg arterial stenosis/occlusion and can be a beneficial supplement to the current imaging methods of vascular stenosis evaluation for the prediction of peripheral arterial disease severity.Key points• Compared with severe PAD, lower leg muscles of mild PAD showed higher CT values (39.5 vs. 44.6, p < 0.001).• Models developed with muscle CT features had AUC = 0.89 for predicting PAD.• PAD severity prediction can be realized through the middle and inferior segment of images (AUC = 0.90).Graphical

A versatile, semi-automated image analysis workflow for time-lapse camera trap image classification

Camera traps are a powerful, practical, and non-invasive method used widely to monitor animal communities and evaluate management actions. However, camera trap arrays can generate thousands to millions of images that require significant time and effort to review. Computer vision has emerged as a tool to accelerate this image review process. We propose a multi-step, semi-automated workflow which takes advantage of site-specific and generalizable models to improve detections and consists of (1) automatically identifying and removing low-quality images in parallel with classification into animals, humans, vehicles, and empty, (2) automatically cropping objects from images and classifying them (rock, bait, empty, and species), and (3) manually inspecting a subset of images. We trained and evaluated this approach using 548,627 images from 46 cameras in two regions of the Arctic: “Finnmark” (Finnmark County, Norway) and “Yamal” (Yamalo-Nenets Autonomous District, Russia). The automated steps yield image classification accuracies of 92% and 90% for the Finnmark and Yamal sets, respectively, reducing the number of images that required manual inspection to 9.2% of the Finnmark set and 3.9% of the Yamal set. The amount of time invested in developing models would be offset by the time saved from automation after 960 thousand images have been processed. Researchers can modify this multi-step process to develop their own site-specific models and meet other needs for monitoring and surveying wildlife, balancing the acceptable levels of false negatives and positives.

Offloading the computational complexity of transfer learning with generic features.

Deep learning approaches are generally complex, requiring extensive computational resources and having high time complexity. Transfer learning is a state-of-the-art approach to reducing the requirements of high computational resources by using pre-trained models without compromising accuracy and performance. In conventional studies, pre-trained models are trained on datasets from different but similar domains with many domain-specific features. The computational requirements of transfer learning are directly dependent on the number of features that include the domain-specific and the generic features. This article investigates the prospects of reducing the computational requirements of the transfer learning models by discarding domain-specific features from a pre-trained model. The approach is applied to breast cancer detection using the dataset curated breast imaging subset of the digital database for screening mammography and various performance metrics such as precision, accuracy, recall, F1-score, and computational requirements. It is seen that discarding the domain-specific features to a specific limit provides significant performance improvements as well as minimizes the computational requirements in terms of training time (reduced by approx. 12%), processor utilization (reduced approx. 25%), and memory usage (reduced approx. 22%). The proposed transfer learning strategy increases accuracy (approx. 7%) and offloads computational complexity expeditiously.

NeRFail: Neural Radiance Fields-Based Multiview Adversarial Attack

Adversarial attacks, i.e., generating adversarial perturbations with a small magnitude to deceive deep neural networks, are important for investigating and improving model trustworthiness. Traditionally, the topic was scoped within 2D images without considering 3D multiview information. Benefiting from Neural Radiance Fields (NeRF), one can easily reconstruct a 3D scene with a Multi-Layer Perceptron (MLP) from given 2D views and synthesize photo-realistic renderings of novel vantages. This opens up a door to discussing the possibility of undertaking to attack multiview NeRF network with downstream tasks from different rendering angles, which we denote Neural Radiance Fiels-based multiview adversarial Attack (NeRFail). The goal is, given one scene and a subset of views, to deceive the recognition results of agnostic view angles as well as given views. To do so, we propose a transformation mapping from pixels to 3D points such that our attack generates multiview adversarial perturbations by attacking a subset of images with different views, intending to prevent the downstream classifier from correctly predicting images rendered by NeRF from other views. Experiments show that our multiview adversarial perturbations successfully obfuscate the downstream classifier at both known and unknown views. Notably, when retraining another NeRF on the perturbed training data, we show that the perturbation can be inherited and reproduced. The code can be found at https://github.com/jiang-wenxiang/NeRFail.

Tell Me What Is Good about This Property: Leveraging Reviews for Segment-Personalized Image Collection Summarization

Image collection summarization techniques aim to present a compact representation of an image gallery through a carefully selected subset of images that captures its semantic content. When it comes to web content, however, the ideal selection can vary based on the user's specific intentions and preferences. This is particularly relevant at Booking.com, where presenting properties and their visual summaries that align with users' expectations is crucial. To address this challenge, in this work, we consider user intentions in the summarization of property visuals by analyzing property reviews and extracting the most significant aspects mentioned by users. By incorporating the insights from reviews in our visual summaries, we enhance the summaries by presenting the relevant content to a user. Moreover, we achieve it without the need for costly annotations. Our experiments, including human perceptual studies, demonstrate the superiority of our cross-modal approach, which we coin as CrossSummarizer over the no-personalization and image-based clustering baselines.

Abstract 4877: Morphological detection of chemotherapy resistance in cancer cell lines using AI-based analysis

Abstract Chemoresistance can develop during the course of treatment of many cancers, which can limit the utility of standard therapeutics and lead to poor outcomes. The mechanism by which resistance arises is not well understood, and developing models to study resistance can be challenging. Resistance can spontaneously arise in cell lines, however, it is difficult to separate resistant from sensitive cells using traditional biomarkers, hampering their utility for studying mechanisms. Morphology presents another modality to examine resistance, independently of protein markers, and may solve a previously intractable problem of separating resistant from sensitive cells for further study. Here we sought to study the mechanism of chemoresistance in cell lines using multi-parameter morphology in combination with gene expression and other standard molecular approaches. We induced resistance to specific chemotherapeutic drugs in parent cancer cell lines from lung, ovarian, and melanoma cancers. IC-50s of each drug increased by ≥1 order of magnitude in the resistant cells. Whole exome sequencing and bulk RNAseq were performed to examine differentially expressed genes and pathways between the parent and resistant cell lines. Each cell line was analyzed using multi-dimensional morphology using the Deepcell platform, which extracts &amp;gt;100 morphological features from images of unlabeled single cells using artificial intelligence (AI), advanced imaging, and microfluidics. The morphology features represent both Deep Learning (DL)-derived features and human-interpretable morphometric features, such as size and shape. Unique morphotypes were observed differentiating each combination of parent and resistant cell lines, with notable changes in granularity distinguishing them. A subset of images was used to train a random forest classifier to predict the resistant state of each cell image, which performed with up to 85% accuracy on independent cell images. Strikingly, the combination of DL plus morphometric features outperformed the use of morphometric features alone, demonstrating the increased utility of combined analysis. The top gene pathways and top morphological features were examined to infer potential mechanisms underlying chemoresistance. This work demonstrates that morphology alone can be used to distinguish drug resistant vs. parent populations without the use of markers. Morphology analysis can be applied to understanding complex phenotypes, and future platform improvements will enable sorting of these resistant populations to better identify molecular pathways in both cell lines and in primary tumor samples. Citation Format: Andreja Jovic, Manisha Ray, Ryan Carelli, Kiran Saini, Tiffine Pham, Christian Corona, Stephane C. Boutet, Chassidy Johnson, Mahyar Salek, Maddison Masaeli, Matt Barnes, Cyril Y. Ramathal. Morphological detection of chemotherapy resistance in cancer cell lines using AI-based analysis [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4877.

Abstract 6083: Inter-observer agreement in ultrasound-detected steatotic liver in the Chile Biliary Longitudinal Study (BiLS)

Abstract Background: Metabolic-associated steatotic liver disease (MASLD) is a prevalent chronic liver condition with an increased risk of cancer-related mortality. In Latin America, MASLD prevalence is estimated at 40%, with Chile reporting rates of 47.5% in 2019. Early detection is crucial for MASLD control and management. Liver biopsy, the gold standard for diagnosis, is invasive and costly. Ultrasound (US) is a preferred alternative, yet its precision varies depending on the observer interpretation. Methods: We conducted a cross-sectional analysis of a subset of baseline US images from the Chile Biliary Longitudinal Study (Chile BiLS), a cohort of women aged 50-74 with gallstones in Chile. Of the 4,032 participants recruited from Cautín Province in the Araucanía region, a random sample of 10% of the images were selected for review by a radiologist from Baylor College of Medicine (Observer 1) and compared to the original readings generated at the baseline exam by a team of Chile BiLS ultrasound technicians. To assess reproducibility, 33 blinded duplicate images from the 10% sample were reanalyzed by Observer 1 and an additional Chile BiLS technician who was not part of the original baseline team (Observer 2). Inter- and intra-observer agreement was assessed using weighted kappa values. Images were categorized as none, mild, moderate/severe. Results: Of the 409 images reviewed, agreement between Observer 1 and original readings across all categories of steatosis was poor (kappa: 0.09; CI 95% 0.06;0.11, p &amp;lt;0.001). The 33 duplicates compared to the original readings showed slight agreement for Observer 1 (kappa: 0.11 CI 95% 0.02;0.20, p &amp;lt;0.05) and fair for Observer 2 (kappa: 0.28 (CI 95% 0.07 - 0.49, p &amp;lt;0.001). The intra-observer agreement for Observer 1 was moderate, with a kappa of 0.41 (CI 95% 0.09 - 0.74, p &amp;lt;0.05), and for Observer 2, it was substantial with a weighted kappa of 0.69 (CI 95% 0.5 - 0.9, p &amp;lt;0.001). Conclusion: The study highlights the variability in US interpretation of hepatic steatosis, emphasizing the importance of standardized training and methodology. US's sensitivity and specificity is operator dependence, which impacts the diagnostic interpretation. Our findings underscore the significance of inter and intra-observer comparisons for optimal diagnosis, especially when US is the primary diagnostic tool in MASLD assessment. Finally, our results emphasize the importance of group training. Observer 2, who underwent the same training as the original Chile BiLS ultrasound technicians and was consistently exposed to this technique as a diagnostic method, exhibited more consistent results. This reinforces the importance of collective training initiatives to enhance the reliability and accuracy of US in MASLD diagnosis. Citation Format: Maria Spencer Sandino, Vanessa Van De Wyngard, David Wynne, Maya Balakrishnan, Noldy Mardones, Ruth Pfeiffer, Allan Hildesheim, Catterina Ferreccio, Jill Koshiol. Inter-observer agreement in ultrasound-detected steatotic liver in the Chile Biliary Longitudinal Study (BiLS) [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6083.

Abstract 2486: Dissecting tumor-immune interaction in response and resistance to immune checkpoint blockade in metastatic melanoma

Abstract Background: Immune checkpoint blockade (ICB) has revolutionized the treatment and prognosis of various cancers, including melanoma. Nevertheless, intrinsic or acquired resistance is common, and the interactions between tumor-intrinsic features and the microenvironment underlying these resistances remain unclear for most patients. Methods: In this study, we developed a framework to analyze response and resistance, both intrinsic and acquired, via tumor-intrinsic programs, immune features, and tumor-stromal-immune interactions in a standardized, uniformly processed, and deeply clinically annotated cohort of metastatic melanoma patients (n=61) treated with ICB as part of the NCI funded human tumor atlas network (HTAN) initiative. From the same tumor samples (pre-treatment (n=33) and post-progression (n=28)), we conducted bulk whole-exome sequencing (WES), single-nucleus RNA sequencing, and for a subset of the samples high-resolution spatial imaging (including protein mIHC, CODEX, and transcriptomics MERFISH). Standardized processing and data pipelines allowed for integration of genomic, transcriptomic, and spatial features to elucidate shared tumor and microenvironmental states and their relationships with resistance. Results: Our preliminary analysis on a subset of 28 samples identified an enrichment of B cells, plasma cells, and T follicular helper-like cells in the tumor microenvironment (TME) of patients sensitive to treatment. The TME of non-responders was predominantly characterized by macrophage/monocyte cell populations. Post-treatment responder samples showed an enrichment of CD4+ T cells. Genomic heterogeneity, assessed through WES, was confirmed as a feature of intrinsic resistance across different ICB treatment settings, while ploidy was associated with treatment response in pre-treatment, ipilimumab-naïve patients treated with PD-1 inhibitors. Comparing the immune compartments of samples with high and low heterogeneity revealed NK cell enrichment in samples with a high proportion of subclonal mutations. Conclusions and Future Directions: Overall, our work provides a high-resolution understanding of the tumor-stromal-immune interaction in metastatic melanoma, shedding light on the factors contributing to therapy response and resistance, and contributing a repository of data from deeply clinically annotated, multimodally characterized and uniformly processed patient samples. Ongoing spatial analysis will further enhance our understanding of these compartments at a spatial level. The results of our study have the potential to guide more personalized and effective treatment strategies for metastatic melanoma in the future. Citation Format: Giuseppe Tarantino, Yiwen He, Priyanka Solanky, Aaron Thorner, Tyler Aprati, Ryan Sullivan, Emily Robitschek, Timothy Blosser, Xiaowei Zhuang, Xiaowei Zhuang, Elliot Boblitt, Allison Frangieh, Marta Holovatska, Aleigha Lawless, Michael Manos, Kathleen Pfaff, Karla Helvie, Tatyana Sharova, Dennie Frederick, James Liam Fahey, Diego Villamarin, Sami Farhi, Scott Rodig, Bruce Johnson, Alex K. Shalek, Eliezer Van Allen, Stephen Hodi, Genevieve M. Boland, David Liu. Dissecting tumor-immune interaction in response and resistance to immune checkpoint blockade in metastatic melanoma [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2486.

Detection of Early Stage Alzheimer's Disease in Gradient-Based MR Images Using Deep Learning Methods

Alzheimer's Disease (AD), a form of dementia prevalent in older age, is a neurodegenerative condition impacting brain nerve cells. Early-stage AD symptoms are often subtle, complicating timely diagnosis. Early detection allows for intervention, slowing disease progression and facilitating appropriate treatments. Deep learning methods, particularly gradient-based images, prove promising for early Alzheimer's detection in Magnetic Resonance (MR) imaging. Gradient-based images, highlighting details in low-intensity images and enhancing contrast, play a vital role in determining structures' location, shape, and size, notably in techniques like Magnetic Resonance Imaging (MRI). This study aims to boost model performance in early AD detection by applying the Gradient filter before training deep learning models on diverse-angle and constant-density brain MRI images. The dataset comprises three categories representing early-stage AD, including images of mild cognitive impairment and healthy individuals. Original and gradient-filtered image subsets were inputted into deep learning models. Results indicate superior performance of gradient-based images, with the Densenet201 deep learning model achieving the highest accuracy at 98.63%.

Detection of Ulcerative Colitis Lesions from Weakly Annotated Colonoscopy Videos Using Bounding Boxes

Ulcerative colitis is a chronic disease characterized by bleeding and ulcers in the colon. Disease severity assessment via colonoscopy videos is time-consuming and only focuses on the most severe lesions. Automated detection methods enable fine-grained assessment but depend on the training set quality. To suit the local clinical setup, an internal training dataset containing only rough bounding box annotations around lesions was utilized. Following previous works, we propose to use linear models in suitable color spaces to detect lesions. We introduce an efficient sampling scheme for exploring the set of linear classifiers and removing trivial models i.e., those showing zero false negative or positive ratios. Bounding boxes lead to exaggerated false detection ratios due to mislabeled pixels, especially in the corners, resulting in decreased model accuracy. Therefore, we propose to evaluate the model sensitivity on the annotation level instead of the pixel level. Our sampling strategy can eliminate up to 25% of trivial models. Despite the limited quality of annotations, the detectors achieved better performance in comparison with the state-of-the-art methods. When tested on a small subset of endoscopic images, the best models exhibit low variability. However, the inter-patient model performance was variable suggesting that appearance normalization is critical in this context.

Deep Learning for Automated Detection and Localization of Traumatic Abdominal Solid Organ Injuries on CT Scans.

Computed tomography (CT) is the most commonly used diagnostic modality for blunt abdominal trauma (BAT), significantly influencing management approaches. Deep learning models (DLMs) have shown great promise in enhancing various aspects of clinical practice. There is limited literature available on the use of DLMs specifically for trauma image evaluation. In this study, we developed a DLM aimed at detecting solid organ injuries to assist medical professionals in rapidly identifying life-threatening injuries. The study enrolled patients from a single trauma center who received abdominal CT scans between 2008 and 2017. Patients with spleen, liver, or kidney injury were categorized as the solid organ injury group, while others were considered negative cases. Only images acquired from the trauma center were enrolled. A subset of images acquired in the last year was designated as the test set, and the remaining images were utilized to train and validate the detection models. The performance of each model was assessed using metrics such as the area under the receiver operating characteristic curve (AUC), accuracy, sensitivity, specificity, positive predictive value, and negative predictive value based on the best Youden index operating point. The study developed the models using 1302 (87%) scans for training and tested them on 194 (13%) scans. The spleen injury model demonstrated an accuracy of 0.938 and a specificity of 0.952. The accuracy and specificity of the liver injury model were reported as 0.820 and 0.847, respectively. The kidney injury model showed an accuracy of 0.959 and a specificity of 0.989. We developed a DLM that can automate the detection of solid organ injuries by abdominal CT scans with acceptable diagnostic accuracy. It cannot replace the role of clinicians, but we can expect it to be a potential tool to accelerate the process of therapeutic decisions for trauma care.

A new approach for atmospheric turbulence removal using low-rank matrix factorization.

In this article, a novel method for removing atmospheric turbulence from a sequence of turbulent images and restoring a high-quality image is presented. Turbulence is modeled using two factors: the geometric transformation of pixel locations represents the distortion, and the varying pixel brightness represents spatiotemporal varying blur. The main framework of the proposed method involves the utilization of low-rank matrix factorization, which achieves the modeling of both the geometric transformation of pixels and the spatiotemporal varying blur through an iterative process. In the proposed method, the initial step involves the selection of a subset of images using the random sample consensus method. Subsequently, estimation of the mixture of Gaussian noise parameters takes place. Following this, a window is chosen around each pixel based on the entropy of the surrounding region. Within this window, the transformation matrix is locally estimated. Lastly, by considering both the noise and the estimated geometric transformations of the selected images, an estimation of a low-rank matrix is conducted. This estimation process leads to the production of a turbulence-free image. The experimental results were obtained from both real and simulated datasets. These results demonstrated the efficacy of the proposed method in mitigating substantial geometrical distortions. Furthermore, the method showcased the ability to improve spatiotemporal varying blur and effectively restore the details present in the original image.

Deep Learning in High-Resolution Anoscopy: Assessing the Impact of Staining and Therapeutic Manipulation on Automated Detection of Anal Cancer Precursors.

High-resolution anoscopy (HRA) is the gold standard for detecting anal squamous cell carcinoma (ASCC) precursors. Preliminary studies on the application of artificial intelligence (AI) models to this modality have revealed promising results. However, the impact of staining techniques and anal manipulation on the effectiveness of these algorithms has not been evaluated. We aimed to develop a deep learning system for automatic differentiation of high-grade squamous intraepithelial lesion vs low-grade squamous intraepithelial lesion in HRA images in different subsets of patients (nonstained, acetic acid, lugol, and after manipulation). A convolutional neural network was developed to detect and differentiate high-grade and low-grade anal squamous intraepithelial lesions based on 27,770 images from 103 HRA examinations performed in 88 patients. Subanalyses were performed to evaluate the algorithm's performance in subsets of images without staining, acetic acid, lugol, and after manipulation of the anal canal. The sensitivity, specificity, accuracy, positive and negative predictive values, and area under the curve were calculated. The convolutional neural network achieved an overall accuracy of 98.3%. The algorithm had a sensitivity and specificity of 97.4% and 99.2%, respectively. The accuracy of the algorithm for differentiating high-grade squamous intraepithelial lesion vs low-grade squamous intraepithelial lesion varied between 91.5% (postmanipulation) and 100% (lugol) for the categories at subanalysis. The area under the curve ranged between 0.95 and 1.00. The introduction of AI to HRA may provide an accurate detection and differentiation of ASCC precursors. Our algorithm showed excellent performance at different staining settings. This is extremely important because real-time AI models during HRA examinations can help guide local treatment or detect relapsing disease.

Reliability of ultrasound measurements of muscle thickness and echointensity in older adults with type 2 diabetes

Ultrasound is an emerging tool for muscle-based measurements. However, the analysis of hyperechoic images (high muscle fat infiltration, as in type 2 diabetes [T2D]) can be challenging. ObjectiveTo evaluate the reliability of ultrasound image analysis of muscle thickness and echointensity in older adults with T2D and non-diabetic controls by experienced and novice analysts. We hypothesized that reliability would be high for the experienced analysts, and lower, but acceptable, for a novice analyst. MethodsWe recruited three groups: 1) adults >60 years with T2D; 2) age- and sex-matched normoglycemic controls; and 3) healthy adults 18–35 years. All participants underwent ultrasound imaging of the abdomen and anterior thigh. Ultrasound images were de-identified, randomized, and separately analyzed by two expert analysts (>3 years experience each). A subset of images was also analyzed by a novice analyst with no prior experience. Inter- and intra-rater reliability was assessed using coefficients of variation (CVs), intraclass correlation coefficients (ICCs), and Bland-Altman analysis. ResultsFor the expert analysts, ICCs were >0.90 and CVs were <10 % for all measurements, regardless of participant group. For the novice analyst, all ICCs were >0.90, except for rectus abdominus thickness measurements in the T2D group (ICC: 0.659 [95%CI: -0.134, 0.911]). CVs were <10 % for all measurements, except for rectus abdominus thickness in the T2D group (CV: 12.7 %). Ultrasound image analysis of muscle thickness and echointensity by experienced analysts was highly reliable. ConclusionNovice and expert analysts produced comparable measurements, except for rectus abdominus thickness in images obtained from older adults with T2D.

MAMMOGRAM DETECTION USING MACHINE LEARNING

Mammogram detection plays a critical role in early breast cancer diagnosis, and leveraging machine learning techniques can significantly enhance diagnostic accuracy. This project addresses the classification and segmentation of mammogram images to identify abnormalities. We employed a Convolutional Neural Network (CNN) algorithm for binary classification to categorize mammogram images as either ‘Normal’ or ‘Malignant’. For region-based segmentation of detected abnormalities, we utilized an Attention-based optimized UNET algorithm. The preprocessing phase included resizing, normalization, and shuffling of the dataset to prepare it for training. Due to the large size of comprehensive mask image datasets (166 GB), which are impractical to download or process with standard systems, we used a smaller subset of images for training. Consequently, while the classification model performs with reasonable accuracy, the segmentation results may exhibit limited precision due to the constrained training data. The trained model can be applied to test images, where malignant predictions trigger segmentation of the affected regions using the UNET algorithm. This approach demonstrates the feasibility of integrating CNN and UNET for mammogram analysis, though future work will benefit from access to larger datasets for improved segmentation accuracy.

Deep learning empowered breast cancer diagnosis: Advancements in detection and classification.

Recent advancements in AI, driven by big data technologies, have reshaped various industries, with a strong focus on data-driven approaches. This has resulted in remarkable progress in fields like computer vision, e-commerce, cybersecurity, and healthcare, primarily fueled by the integration of machine learning and deep learning models. Notably, the intersection of oncology and computer science has given rise to Computer-Aided Diagnosis (CAD) systems, offering vital tools to aid medical professionals in tumor detection, classification, recurrence tracking, and prognosis prediction. Breast cancer, a significant global health concern, is particularly prevalent in Asia due to diverse factors like lifestyle, genetics, environmental exposures, and healthcare accessibility. Early detection through mammography screening is critical, but the accuracy of mammograms can vary due to factors like breast composition and tumor characteristics, leading to potential misdiagnoses. To address this, an innovative CAD system leveraging deep learning and computer vision techniques was introduced. This system enhances breast cancer diagnosis by independently identifying and categorizing breast lesions, segmenting mass lesions, and classifying them based on pathology. Thorough validation using the Curated Breast Imaging Subset of Digital Database for Screening Mammography (CBIS-DDSM) demonstrated the CAD system's exceptional performance, with a 99% success rate in detecting and classifying breast masses. While the accuracy of detection is 98.5%, when segmenting breast masses into separate groups for examination, the method's performance was approximately 95.39%. Upon completing all the analysis, the system's classification phase yielded an overall accuracy of 99.16% for classification. The potential for this integrated framework to outperform current deep learning techniques is proposed, despite potential challenges related to the high number of trainable parameters. Ultimately, this recommended framework offers valuable support to researchers and physicians in breast cancer diagnosis by harnessing cutting-edge AI and image processing technologies, extending recent advances in deep learning to the medical domain.

TOWARDS LARGE-SCALE BUILDING ATTRIBUTE MAPPING USING CROWDSOURCED IMAGES: SCENE TEXT RECOGNITION ON FLICKR AND PROBLEMS TO BE SOLVED

Abstract. Crowdsourced platforms provide huge amounts of street-view images that contain valuable building information. This work addresses the challenges in applying Scene Text Recognition (STR) in crowdsourced street-view images for building attribute mapping. We use Flickr images, particularly examining texts on building facades. A Berlin Flickr dataset is created, and pre-trained STR models are used for text detection and recognition. Manual checking on a subset of STR-recognized images demonstrates high accuracy. We examined the correlation between STR results and building functions, and analysed instances where texts were recognized on residential buildings but not on commercial ones. Further investigation revealed significant challenges associated with this task, including small text regions in street-view images, the absence of ground truth labels, and mismatches in buildings in Flickr images and building footprints in OpenStreetMap (OSM). To develop city-wide mapping beyond urban hotspot locations, we suggest differentiating the scenarios where STR proves effective while developing appropriate algorithms or bringing in additional data for handling other cases. Furthermore, interdisciplinary collaboration should be undertaken to understand the motivation behind building photography and labeling. The STR-on-Flickr results are publicly available at https://github.com/ya0-sun/STR-Berlin.

Comprehensive Analysis of Mammography Images Using Multi-Branch Attention Convolutional Neural Network

Breast cancer profoundly affects women’s lives; its early diagnosis and treatment increase patient survival chances. Mammography is a common screening method for breast cancer, and many methods have been proposed for automatic diagnosis. However, most of them focus on single-label classification and do not provide a comprehensive analysis concerning density, abnormality, and severity levels. We propose a method based on the multi-label classification of two-view mammography images to comprehensively diagnose a patient’s condition. It leverages the correlation between density type, lesion type, and states of lesions, which radiologists usually perform. It simultaneously classifies mammograms into the corresponding density, abnormality type, and severity level. It takes two-view mammograms (with craniocaudal and mediolateral oblique views) as input, analyzes them using ConvNeXt and the channel attention mechanism, and integrates the information from the two views. Finally, the fused information is passed to task-specific multi-branches, which learn task-specific representations and predict the relevant state. The system was trained, validated, and tested using two public domain benchmark datasets, INBreast and the Curated Breast Imaging Subset of DDSM (CBIS-DDSM), and achieved state-of-the-art results. The proposed computer-aided diagnosis (CAD) system provides a holistic observation of a patient’s condition. It gives the radiologists a comprehensive analysis of the mammograms to prepare a full report of the patient’s condition, thereby increasing the diagnostic precision.

Fusion of electronic health records and radiographic images for a multimodal deep learning prediction model of atypical femur fractures

Atypical femur fractures (AFF) represent a very rare type of fracture that can be difficult to discriminate radiologically from normal femur fractures (NFF). AFFs are associated with drugs that are administered to prevent osteoporosis-related fragility fractures, which are highly prevalent in the elderly population. Given that these fractures are rare and the radiologic changes are subtle currently only 7% of AFFs are correctly identified, which hinders adequate treatment for most patients with AFF. Deep learning models could be trained to classify automatically a fracture as AFF or NFF, thereby assisting radiologists in detecting these rare fractures. Historically, for this classification task, only imaging data have been used, using convolutional neural networks (CNN) or vision transformers applied to radiographs. However, to mimic situations in which all available data are used to arrive at a diagnosis, we adopted an approach of deep learning that is based on the integration of image data and tabular data (from electronic health records) for 159 patients with AFF and 914 patients with NFF. We hypothesized that the combinatorial data, compiled from all the radiology departments of 72 hospitals in Sweden and the Swedish National Patient Register, would improve classification accuracy, as compared to using only one modality. At the patient level, the area under the ROC curve (AUC) increased from 0.966 to 0.987 when using the integrated set of imaging data and seven pre-selected variables, as compared to only using imaging data. More importantly, the sensitivity increased from 0.796 to 0.903. We found a greater impact of data fusion when only a randomly selected subset of available images was used to make the image and tabular data more balanced for each patient. The AUC then increased from 0.949 to 0.984, and the sensitivity increased from 0.727 to 0.849.These AUC improvements are not large, mainly because of the already excellent performance of the CNN (AUC of 0.966) when only images are used. However, the improvement is clinically highly relevant considering the importance of accuracy in medical diagnostics. We expect an even greater effect when imaging data from a clinical workflow, comprising a more diverse set of diagnostic images, are used.

Boosting Diversity in Visual Search with Pareto Non-Dominated Re-Ranking

The field of visual search has gained significant attention recently, particularly in the context of web search engines and e-commerce product search platforms. However, the abundance of web images presents a challenge for modern image retrieval systems, as they need to find both relevant and diverse images that maximize users’ satisfaction. In response to this challenge, we propose a non-dominated visual diversity re-ranking (NDVDR) method based on the concept of Pareto optimality. To begin with, we employ a fast binary hashing method as a coarse-grained retrieval procedure. This allows us to efficiently obtain a subset of candidate images for subsequent re-ranking. Fed with this initial retrieved image results, the NDVDR performs a fine-grained re-ranking procedure for boosting both relevance and visual diversity among the top-ranked images. Recognizing the inherent conflict nature between the objectives of relevance and diversity, the re-ranking procedure is simulated as the analytical stage of a multi-criteria decision-making process, seeking the optimal tradeoff between the two conflicting objectives within the initial retrieved images. In particular, a non-dominated sorting mechanism is devised that produces Pareto non-dominated hierarchies among images based on the Pareto dominance relation. Additionally, two novel measures are introduced for the effective characterization of the relevance and diversity scores among different images. We conduct experiments on three popular real-world image datasets and compare our re-ranking method with several state-of-the-art image search re-ranking methods. The experimental results validate that our re-ranking approach guarantees retrieval accuracy while simultaneously boosting diversity among the top-ranked images.