3D Image Data Research Articles

BrainTACO: an explorable multi-scale multi-modal brain transcriptomic and connectivity data resource

Exploring the relationships between genes and brain circuitry can be accelerated by joint analysis of heterogeneous datasets from 3D imaging data, anatomical data, as well as brain networks at varying scales, resolutions, and modalities. Generating an integrated view, beyond the individual resources’ original purpose, requires the fusion of these data to a common space, and a visualization that bridges the gap across scales. However, despite ever expanding datasets, few platforms for integration and exploration of this heterogeneous data exist. To this end, we present the BrainTACO (Brain Transcriptomic And Connectivity Data) resource, a selection of heterogeneous, and multi-scale neurobiological data spatially mapped onto a common, hierarchical reference space, combined via a holistic data integration scheme. To access BrainTACO, we extended BrainTrawler, a web-based visual analytics framework for spatial neurobiological data, with comparative visualizations of multiple resources. This enables gene expression dissection of brain networks with, to the best of our knowledge, an unprecedented coverage and allows for the identification of potential genetic drivers of connectivity in both mice and humans that may contribute to the discovery of dysconnectivity phenotypes. Hence, BrainTACO reduces the need for time-consuming manual data aggregation often required for computational analyses in script-based toolboxes, and supports neuroscientists by directly leveraging the data instead of preparing it.

Design and Implementation of a Self-Supervised Algorithm for Vein Structural Patterns Analysis Using Advanced Unsupervised Techniques

Compared to other identity verification systems applications, vein patterns have the lowest potential for being used fraudulently. The present research examines the practicability of gathering vascular data from NIR images of veins. In this study, we propose a self-supervision learning algorithm that envisions an automated process to retrieve vascular patterns computationally using unsupervised approaches. This new self-learning algorithm sorts the vascular patterns into clusters and then uses 2D image data to recuperate the extracted vascular patterns linked to NIR templates. Our work incorporates multi-scale filtering followed by multi-scale feature extraction, recognition, identification, and matching. We design the ORC, GPO, and RDM algorithms with these inclusions and finally develop the vascular pattern mining model to visualize the computational retrieval of vascular patterns from NIR imageries. As a result, the developed self-supervised learning algorithm shows a 96.7% accuracy rate utilizing appropriate image quality assessment parameters. In our work, we also contend that we provide strategies that are both theoretically sound and practically efficient for concerns such as how many clusters should be used for specific tasks, which clustering technique should be used, how to set the threshold for single linkage algorithms, and how much data should be excluded as outliers. Consequently, we aim to circumvent Kleinberg’s impossibility while attaining significant clustering to develop a self-supervised learning algorithm using unsupervised methodologies.

Preparation of Retinal Samples for Volume Electron Microscopy.

Volume electron microscopy (Volume EM) has emerged as a powerful tool for visualizing the 3D structure of cells and tissues with nanometer-level precision. Within the retina, various types of neurons establish synaptic connections in the inner and outer plexiform layers. While conventional EM techniques have yielded valuable insights into retinal subcellular organelles, their limitation lies in providing 2D image data, which can hinder accurate measurements. For instance, quantifying the size of three distinct synaptic vesicle pools, crucial for synaptic transmission, is challenging in 2D. Volume EM offers a solution by providing large-scale, high-resolution 3D data. It is worth noting that sample preparation is a critical step in Volume EM, significantly impacting image clarity and contrast. In this context, we outline a sample preparation protocol for the 3D reconstruction of photoreceptor axon terminals in the retina. This protocol includes three key steps: retina dissection and fixation, sample embedding processes, and selection of the area of interest.

Virtual reassembling of 3D fragments for the data-driven analysis of fracture mechanisms in multi-component materials

This paper introduces a novel method for characterizing fracture mechanisms in multi-component materials using 3D image data gained by X-ray computed tomography (CT) measurements. In mineral liberation, the understanding of these mechanisms is crucial, particularly whether fractures occur along the boundaries of mineral phases (intergranular fracture) and/or within mineral phases (transgranular fracture). Conventional techniques for analyzing fracture mechanisms are focused on globally comparing the surface exposure of mineral phases extracted from image measurements before and after fracture. Instead, we present a virtual reassembling algorithm based on image registration techniques, which is applied to 3D data of multi-component materials before and after fracture in order to determine and characterize the individual fracture surfaces. This enables us to conduct a local quantitative analysis of fracture mechanisms by voxelwise comparing adjacent regions at fracture surfaces. A quantitative analysis of fracture mechanisms is especially important in the context of recycling processes. As primary deposits are decreasing worldwide and the focus is shifting on reducing wast, the interest in secondary raw materials has increased. However, lower-concentrated, but valuable materials are often overlooked in recycling routes designed for higher-concentration materials and dissipate. Therefore, efforts are being made to enrich valuable elements, such as lithium, as engineered artificial minerals through pyrometallurgical processes. The subsequent liberation through comminution processes, such as crushing, is essential for the recovery of valuable minerals. A better understanding of crushing processes, especially fracture mechanisms in slags, is crucial for the success of recycling. The reassembling algorithm presented in this paper is evaluated through a simulation study, followed by an application to a naturally occurring ore and a slag resulting from a recycling process.

Multiscale and multimodal imaging for three-dimensional vascular and histomorphological organ structure analysis of the pancreas

Exocrine and endocrine pancreas are interconnected anatomically and functionally, with vasculature facilitating bidirectional communication. Our understanding of this network remains limited, largely due to two-dimensional histology and missing combination with three-dimensional imaging. In this study, a multiscale 3D-imaging process was used to analyze a porcine pancreas. Clinical computed tomography, digital volume tomography, micro-computed tomography and Synchrotron-based propagation-based imaging were applied consecutively. Fields of view correlated inversely with attainable resolution from a whole organism level down to capillary structures with a voxel edge length of 2.0 µm. Segmented vascular networks from 3D-imaging data were correlated with tissue sections stained by immunohistochemistry and revealed highly vascularized regions to be intra-islet capillaries of islets of Langerhans. Generated 3D-datasets allowed for three-dimensional qualitative and quantitative organ and vessel structure analysis. Beyond this study, the method shows potential for application across a wide range of patho-morphology analyses and might possibly provide microstructural blueprints for biotissue engineering.

3D image processing technology based on interactive entertainment application in cultural and creative product design

3D printing technology has many advantages that traditional production models do not have, greatly accelerating the speed of product design and development. In order to achieve the application of machine learning based 3D image processing technology in cultural and creative product design, this article collects a large amount of 3D image data as training samples and uses machine learning algorithms to establish a training model. Considering the characteristics of the data and the processing objectives, these 3D image data are used as a training set and trained through machine learning algorithms to learn the features and patterns of the image. After the training is completed, use this model to process the new 3D image. By inputting new image data, the model automatically extracts image features and predicts and processes them based on previously learned patterns. Evaluate the advantages and disadvantages of machine learning based methods in processing complex 3D images by comparing experimental results. The experimental results show that machine learning based 3D image processing technology has high accuracy and efficiency in the design of cultural and creative products. Compared with traditional methods, this method can better process complex 3D images and provide better design effects.

2D and 3D compatible chaotic image encryption system based on checkers rules and shift register

The neighboring coordinates are strongly correlated in 3D images, but the encryption format of 3D images is not suitable for 2D image encryption. Thus, a novel compatible chaotic image encryption system based on checkers rules and shift register is proposed in this paper. Firstly, a discrete memristive hyperchaotic map(a discrete hyperchaotic map coupled with a memristor) is used to generate the chaotic sequence of the cryptosystem. The hyperchaotic map is analyzed through Phase diagrams, Lyapunov exponent spectrum(LEs), and bifurcation diagram. Secondly, the data format of a 3D image is converted to a 2D image data format. Then, the checkers rules are used to scramble the 2D image data format. Finally, the scrambled data and the chaotic sequence are converted to character types, and the scrambled data is further encrypted by the shift register principle. The 2D image and 3D image are tested and the results show that the compatibility encryption system has good performance, significantly reducing the correlation among the data.

Morphology of nanoporous glass: Stochastic 3D modeling, stereology and the influence of pore width

Excursion sets of Gaussian random fields are used to model the three-dimensional (3D) morphology of differently manufactured porous glasses (PGs), which vary with respect to their mean pore widths measured by mercury intrusion porosimetry. The stochastic 3D model is calibrated by means of volume fractions and two-point coverage probability functions estimated from tomographic image data. Model validation is performed by comparing model realizations and image data in terms of morphological descriptors which are not used for model fitting. For this purpose, we consider mean geodesic tortuosity and constrictivity of the pore space, quantifying the length of the shortest transportation paths and the strength of bottleneck effects, respectively. Additionally, a stereological approach for parameter estimation is presented, i.e., the 3D model is calibrated using merely two-dimensional (2D) cross-sections of the 3D image data. Doing so, on average, a comparable goodness of fit is achieved as well. The variance of the calibrated model parameters is discussed, which is estimated on the basis of randomly chosen, individual 2D cross-sections. Moreover, interpolating between the model parameters calibrated to differently manufactured glasses enables the predictive simulation of virtual but realistic PGs with mean pore widths that have not yet been manufactured. The predictive power is demonstrated by means of cross-validation. Using the presented approach, relationships between parameters of the manufacturing process and descriptors of the resulting morphology of PGs are quantified, which opens possibilities for an efficient optimization of the underlying manufacturing process. Published by the American Physical Society 2024

Shape variation in modern human upper premolars.

Morphological variation in modern human dentition is still an open field of study. The understanding of dental shape and metrics is relevant for the advancement of human biology and evolution and is thus of interest in the fields of dental anthropology, as well as human anatomy and medicine. Of concern is also the variation of the inner aspects of the crown which can be investigated using the tools and methods of virtual anthropology. In this study, we explored inter- and intra-population morphometric variation of modern humans' upper third and fourth premolars (P3s and P4s, respectively) considering both the inner and outer aspects of the crown, and discrete traits. We worked by means of geometric morphometrics on 3D image data from a geographically balanced sample of human populations from five continents, to analyse the shape of the dentinal crown, and the crown outline in 78 P3s and 76 P4s from 85 individuals. For the study of dental traits, we referred to the Arizona State University Dental Anthropology System integrated with more recent classification systems. The 3D shape variation of upper premolar crowns varied between short and mesio-distally broad, and tall and mesio-distally narrow. The observed shape variation was independent from the geographical origin of the populations, and resulted in extensive overlap. We noted a high pairwise correlation (r1 = 0.83) between upper P3s and P4s. We did not find any significant geographic differences in the analysed non-metric traits. Our outcomes thus suggest that geographical provenance does not play a determinant role in the shaping of the dental crown, whose genesis is under strict genetic control.

Modelling and protection of grotto temples based on iterative closest point and Poisson

As a non-renewable cultural heritage, grotto temples face many value threats and the crises of rapid extinction. Due to the unique environment, diverse volume and complex geometric structure of grotto temples, cultural relic protection faces huge challenges. Digitisation has become the most important and effective preventive protection method. This study proposes a comprehensive modeling method, which uses three-dimensional (3D) laser scanning technology to obtain the point cloud data of grotto ruins, close range photogrammetry to obtain multi-angle and high-resolution two-dimensional (2D) image data, and through the fusion modelling of the laser point cloud and image dense matching point cloud, a complete geometric model of grotto is obtained. The registration method of the improved algorithm is based on Euclidean distance residual. The performance is better, with residuals mainly distributed in the range of 0.02 to 0.03 mm. In terms of texture mapping, the accuracy of model mapping is adjusted by adjusting the constraints of control points. The gray values of 50 pixels around the control points show significant gradient changes in the X direction, while the changes in the Y direction are relatively small. These methods and technologies provide practical and effective means for the protection and preservation of cultural relics in grotto temples.

A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery.

Background: Three-dimensional reconstructions of state-of-the-art high-resolution imaging are progressively being used more for preprocedural assessment in thoracic surgery. It is a promising tool that aims to improve patient-specific treatment planning, for example, for minimally invasive or robotic-assisted lung resections. Increasingly available mixed-reality hardware based on video pass-through technology enables the projection of image data as a hologram onto the patient. We describe the novel method of real-time 3D surgical planning in a mixed-reality setting by presenting three representative cases utilizing volume rendering. Materials: A mixed-reality system was set up using a high-performance workstation running a video pass-through-based head-mounted display. Image data from computer tomography were imported and volume-rendered in real-time to be customized through live editing. The image-based hologram was projected onto the patient, highlighting the regions of interest. Results: Three oncological cases were selected to explore the potentials of the mixed-reality system. Two of them presented large tumor masses in the thoracic cavity, while a third case presented an unclear lesion of the chest wall. We aligned real-time rendered 3D holographic image data onto the patient allowing us to investigate the relationship between anatomical structures and their respective body position. Conclusions: The exploration of holographic overlay has proven to be promising in improving preprocedural surgical planning, particularly for complex oncological tasks in the thoracic surgical field. Further studies on outcome-related surgical planning and navigation should therefore be conducted. Ongoing technological progress of extended reality hardware and intelligent software features will most likely enhance applicability and the range of use in surgical fields within the near future.

Abstract 2336: A prototype for a machine-learning assisted image analysis tool for preclinical in vivo studies

Abstract The purpose of this study was to build a software tool to test the feasibility of machine-learning (ML) assisted analysis of in vivo 3D images in oncology research studies. Conventionally, in vivo image segmentations are performed by hand, and are a bottleneck for data analysis, taking time and introducing both inter and intra-user variability. Recently, advances in ML have shown promise in accelerating similar clinical image analysis tasks. Our objective was to work toward making this technology accessible to preclinical cancer researchers. 3D image data across multiple mouse models, organs, modalities, and imaging systems were used. ML segmentation models were trained with user-defined masks on images of mouse bladders (N=134), kidneys (N=131), livers (N=65), and lungs (N=26) from either ultrasound or micro-computed tomography (uCT) scanners. Models were developed using PyTorch/MONAI, hosted in an AWS cloud environment, and served to users via a custom extension for 3D Slicer (www.slicer.org). Performance was assessed as the difference between the human vs. ML organ volumes on a test set unseen by the models during training. Lastly, the functionality of on-the-fly segmentation improvement tools through iterative user feedback was assessed. Results demonstrated that generating segmentations with the prototype ML software took 2-6 seconds per image running on a modern GPU (NVIDIA RTX 3090), compared to 3-7 minutes if performed manually by an expert user (>30x improvement). The Dice scores between the ML generated and human segmentations ranged from 0.81 (liver ultrasound) to 0.84 (bladder ultrasound) but could be improved using ML-based on-the-fly mask improvement tools. Using data from this pilot study, we estimate that the threshold to achieve 90% accuracy on first-pass segmentations is between 200-300 training images, depending on the variance observed in the image data. In summary, the prototype software was able to decrease the manual analysis time across a variety of preclinical in vivo image modalities, which is broadly relevant to preclinical oncology research. The next steps for this project will be to extend the platform to additional orthotopic tumor models (e.g. pancreatic, liver, prostate, etc.) and deploy to the field to enable access to this toolset by preclinical imaging scientists across the world. Citation Format: Adam M. Aji, Thomas M. Kierski, Juan D. Rojas, Hannah Morilak, Olivia J. Kelada, Kathryn H. Gessner, Andrew S. Gdowski, Lucia Kim, William Y. Kim, Ryan C. Gessner, Tomek J. Czernuszewicz. A prototype for a machine-learning assisted image analysis tool for preclinical in vivo studies [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2336.

Abstract 76: Correlated multi-scale 3D optical microscopy of tumor tissues

Abstract Three-dimensional (3D) optical microscopy in combination with advanced tissue clearing methods permits the interrogation of intact large-sized tumor tissues. There are many optical microscopy methods (e.g. light-sheet and confocal microscopy) offering 3D tumor images at varying levels of spatial resolution (e.g. organ, tissue, cell, and molecule level). However, it is a challenge to correlatively integrate various resolution 3D images obtained by different optical microscopy of tumor tissue. In this project, we developed a correlated multi-scale 3D optical microscopy method that enables the acquisition of macroscopic to microscopic spatial information from a single tumor tissue in 3D. Firstly, we used light sheet microscopy to image a whole large-sized, cleared tumor tissue and analyzed the results to identify regions of interests (ROIs). After defining ROIs in the 3D whole tumor image, we added a UV-activatable visible dye, spiropyran, in the agarose gel surrounding the cleared tumor and exposed the sample to a UV light sheet horizontally at the height (z-axis) of the tumor where the ROIs present. The activated dye showed ‘purple’ colored lines in the agarose gel and we physically marked the gel along the visible colored lines. We further sectioned the tumor at the ROI positions using a vibratome after reversing tissue clearing process. After immunofluorescence staining and aqueous-based tissue clearing, confocal microscopy allowed the localization of various cell types in the ROIs of tumor macrosections. To validate the method for multi-scale evaluation of cancer immunotherapy, we tested it with control and DMXAA-treated mouse mammary tumor tissues. We successfully acquired the 3D images of whole tumors by capturing autofluorescence signals with a 633 nm excitation laser and its matched emission filter. Based on the autofluorescence tumor images, we defined two ROIs for each control and treated tumor and performed optical marking and physical sectioning of the tumors as described above. Next, we stained the macrosections (400 µm thick) with DAPI, CK8-DL488, CD45-DL550, and ER-TR7-DL680 to visualize cancer, immune, and stromal cells in the ROIs at cellular resolution. We quantitatively analyzed the correlated multi-scale 3D image data for assessment of the effects of DMXAA treatment on the tumor microenvironment including eradiation of cancer cells, immune infiltration, and remodeling of tumor stroma. In summary, this new microscopy method will provide comprehensive spatial information of tumor tissues and benefit a broad cancer research field such as the discovery of new tumor biomarkers and the development of effective cancer therapies. Citation Format: Jingtian Zheng, Steve Seung-Young Lee. Correlated multi-scale 3D optical microscopy of tumor tissues [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 76.

Abstract 6396: Ex vivo 3D micro tumor testing to assess chemotherapy and immunotherapy responses for NSCLC patients

Abstract Background: Systemic treatment options for NSCLC patients consists of chemo-, targeted or immunotherapy. Although predictive biomarkers can guide treatment decisions, response prediction for chemo- and immunotherapy is still a challenge. Recent studies showed the potential of functional testing to assess patient sensitivity to drugs in various cancers, but NSCLC is lacking. This study reports the feasibility of ex vivo 3D micro lung tumor testing with native tumor microenvironment (TME) to enable treatment sensitivity profiling for all drug classes in parallel. Methods: Fresh NSCLC tumor tissue was collected from pleural fluid stage IVa/b (N=24) or resected primary tumor for stage I-III disease (N=11). Tumor clusters were isolated and exposed to drugs, while preserving the TME. Thirteen therapies were included, including cisplatin, pembrolizumab, osimertinib, and positive control for immune sensitivity (SEA). Morphological features were extracted from high-throughput 3D imaging data. Chemo- and targeted therapy sensitivity was based on AUC comparison of fitted dose-response curves. Immunotherapy sensitivity was classified on percentage and statistical significance of tumor killing, immune cell proliferation and cytokine profiling. Results: The overall assay success rate for solid samples was 64% (7/11), with 100% (4/4) success after protocol optimization. For pleural fluids, the overall success rate was 33% (8/24), caused by many samples arriving without tumor content (13/24, 54%). Excluding these samples, a technical success rate of 73% (8/11) was obtained. The assay was robust and reproducible (mean Z’ = 0.38, CV = 16.1%). Results were delivered in two weeks and on average, eight treatments were tested in parallel per sample. Patient-specific response profiles were observed for chemo-, targeted and immunotherapy. General immunotherapy sensitivity (SEA) was measured in 50% (6/12) of the tissues. Distinct responses were observed for the different immunotherapies: 33% (4/12), 17% (2/12) and 8% (1/12) of the patients responded to pembrolizumab, atezolizumab and/or ipilimumab, respectively. Strong response to osimertinib was observed in a patient with EGFR exon 19 mutation (AUC = 0.26) versus resistance in a WT patient (AUC=0.92). Conclusion: Ex vivo 3D micro tumor testing using liquid and solid tissues is feasible for NSCLC patients with resected primary tumor or pleuritis carcinomatosa. Distinct patient sensitivity profiles for chemo-, targeted and immunotherapies were measured. A preliminary correlation of targeted therapy response with mutation status was found. Discussion: The ex vivo 3D micro tumor testing platform is proven predictive for platinum-based therapy. The current study established the methods to accurately measure ex vivo NSCLC responses to all drug classes to correlate with clinical measures and further validate the platform for guiding treatment decisions. Citation Format: Esmee Koedoot, Timothy J. Sijsenaar, Lieke J. Ceton, Dieudonné J. van der Meer, Thomas J. van Brakel, Peter van Zwam, Erica Geraedts, Willemijn Vader, Egbert F. Smit, Laurie C. Steinbusch, Christi M. Steendam. Ex vivo 3D micro tumor testing to assess chemotherapy and immunotherapy responses for NSCLC patients [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 6396.

Generating 3D images of material microstructures from a single 2D image: a denoising diffusion approach

Three-dimensional (3D) images provide a comprehensive view of material microstructures, enabling numerical simulations unachievable with two-dimensional (2D) imaging alone. However, obtaining these 3D images can be costly and constrained by resolution limitations. We introduce a novel method capable of generating large-scale 3D images of material microstructures, such as metal or rock, from a single 2D image. Our approach circumvents the need for 3D image data while offering a cost-effective, high-resolution alternative to existing imaging techniques. Our method combines a denoising diffusion probabilistic model with a generative adversarial network framework. To compensate for the lack of 3D training data, we implement chain sampling, a technique that utilizes the 3D intermediate outputs obtained by reversing the diffusion process. During the training phase, these intermediate outputs are guided by a 2D discriminator. This technique facilitates our method’s ability to gradually generate 3D images that accurately capture the geometric properties and statistical characteristics of the original 2D input. This study features a comparative analysis of the 3D images generated by our method, SliceGAN (the current state-of-the-art method), and actual 3D micro-CT images, spanning a diverse set of rock and metal types. The results shown an improvement of up to three times in the Frechet inception distance score, a typical metric for evaluating the performance of image generative models, and enhanced accuracy in derived properties compared to SliceGAN. The potential of our method to produce high-resolution and statistically representative 3D images paves the way for new applications in material characterization and analysis domains.

Ensemble dynamics and information flow deduction from whole-brain imaging data.

The recent advancements in large-scale activity imaging of neuronal ensembles offer valuable opportunities to comprehend the process involved in generating brain activity patterns and understanding how information is transmitted between neurons or neuronal ensembles. However, existing methodologies for extracting the underlying properties that generate overall dynamics are still limited. In this study, we applied previously unexplored methodologies to analyze time-lapse 3D imaging (4D imaging) data of head neurons of the nematode Caenorhabditis elegans. By combining time-delay embedding with the independent component analysis, we successfully decomposed whole-brain activities into a small number of component dynamics. Through the integration of results from multiple samples, we extracted common dynamics from neuronal activities that exhibit apparent divergence across different animals. Notably, while several components show common cooperativity across samples, some component pairs exhibited distinct relationships between individual samples. We further developed time series prediction models of synaptic communications. By combining dimension reduction using the general framework, gradient kernel dimension reduction, and probabilistic modeling, the overall relationships of neural activities were incorporated. By this approach, the stochastic but coordinated dynamics were reproduced in the simulated whole-brain neural network. We found that noise in the nervous system is crucial for generating realistic whole-brain dynamics. Furthermore, by evaluating synaptic interaction properties in the models, strong interactions within the core neural circuit, variable sensory transmission and importance of gap junctions were inferred. Virtual optogenetics can be also performed using the model. These analyses provide a solid foundation for understanding information flow in real neural networks.

Drop the shortcuts: image augmentation improves fairness and decreases AI detection of race and other demographics from medical images

It has been shown that AI models can learn race on medical images, leading to algorithmic bias. Our aim in this study was to enhance the fairness of medical image models by eliminating bias related to race, age, and sex. We hypothesise models may be learning demographics via shortcut learning and combat this using image augmentation. This study included 44,953 patients who identified as Asian, Black, or White (mean age, 60.68 years±18.21; 23,499 women) for a total of 194,359 chest X-rays (CXRs) from MIMIC-CXR database. The included CheXpert images comprised 45,095 patients (mean age 63.10 years±18.14; 20,437 women) for a total of 134,300 CXRs were used for external validation. We also collected 1195 3D brain magnetic resonance imaging (MRI) data from the ADNI database, which included 273 participants with an average age of 76.97 years±14.22, and 142 females. DL models were trained on either non-augmented or augmented images and assessed using disparity metrics. The features learned by the models were analysed using task transfer experiments and model visualisation techniques. In the detection of radiological findings, training a model using augmented CXR images was shown to reduce disparities in error rate among racial groups (-5.45%), age groups (-13.94%), and sex (-22.22%). For AD detection, the model trained with augmented MRI images was shown 53.11% and 31.01% reduction of disparities in error rate among age and sex groups, respectively. Image augmentation led to a reduction in the model's ability to identify demographic attributes and resulted in the model trained for clinical purposes incorporating fewer demographic features. The model trained using the augmented images was less likely to be influenced by demographic information in detecting image labels. These results demonstrate that the proposed augmentation scheme could enhance the fairness of interpretations by DL models when dealing with data from patients with different demographic backgrounds. National Science and Technology Council (Taiwan), National Institutes of Health.

Predicting carbonate rock dissolution using multi-scale residual neural networks with prior knowledge

The factors that affect carbonate dissolution are complex, and it is crucial to acquire efficient and accurate knowledge of carbonate dissolution characteristics for CO2 capture and storage (CCS) projects. In current research, the most precise outcomes can be achieved through experimental or simulation techniques, but they are frequently computationally demanding and time-consuming. Data-driven machine learning methods can efficiently perform regression prediction tasks. In this paper, we propose a multi-scale hierarchical regression model for the dissolution problem based on residual neural networks, incorporating prior knowledge. We have developed new equations for carbonate rock dissolution and have incorporated the factors that influence this process into 3D image data by introducing Dissolution Degree (Dd) parameters. This allows the image data to include both the pore space structure and dissolution information with physical meaning. We utilize the pore phase and matrix phase grayscale thresholds (Pt, Mt) to eliminate voxel noise in the characteristic maps obtained from the model. This ensures that the predicted characteristics of carbonate rock dissolution are consistent with practical physics knowledge. We selected a total of 5 core samples to test the model. Among them, three samples are from the test set, and the additional two core samples selected have strong and weak correlations with the training set samples, respectively. The predictions were evaluated using semantic segmentation evaluation parameters, porosity, geometrical and topological structure parameters, Péclet and Damköhler numbers, porous media flow field simulations, and absolute permeability. The results of both visual comparisons and quantitative analyses demonstrated a high degree of consistency between predicted and experimental results, and the trained multi-scale hierarchical regression residual neural network with prior knowledge (MSR-Net) demonstrates good accuracy and generalization ability. The results of this study demonstrate that model based on MSR-Net can be utilized to predict the dissolution properties of the pore space in the formation where the core was extracted, along with the related formation.

A Multiuser, Multisite, and Platform-Independent On-the-Cloud Framework for Interactive Immersion in Holographic XR

Background: The ever-growing extended reality (XR) technologies offer unique tools for the interactive visualization of images with a direct impact on many fields, from bioinformatics to medicine, as well as education and training. However, the accelerated integration of artificial intelligence (AI) into XR applications poses substantial computational processing demands. Additionally, the intricate technical challenges associated with multilocation and multiuser interactions limit the usability and expansion of XR applications. Methods: A cloud deployable framework (Holo-Cloud) as a virtual server on a public cloud platform was designed and tested. The Holo-Cloud hosts FI3D, an augmented reality (AR) platform that renders and visualizes medical 3D imaging data, e.g., MRI images, on AR head-mounted displays and handheld devices. Holo-Cloud aims to overcome challenges by providing on-demand computational resources for location-independent, synergetic, and interactive human-to-image data immersion. Results: We demonstrated that Holo-Cloud is easy to implement, platform-independent, reliable, and secure. Owing to its scalability, Holo-Cloud can immediately adapt to computational needs, delivering adequate processing power for the hosted AR platforms. Conclusion: Holo-Cloud shows the potential to become a standard platform to facilitate the application of interactive XR in medical diagnosis, bioinformatics, and training by providing a robust platform for XR applications.

Deep learning-enabled detection of hypoxic-ischemic encephalopathy after cardiac arrest in CT scans: a comparative study of 2D and 3D approaches.

To establish a deep learning model for the detection of hypoxic-ischemic encephalopathy (HIE) features on CT scans and to compare various networks to determine the best input data format. 168 head CT scans of patients after cardiac arrest were retrospectively identified and classified into two categories: 88 (52.4%) with radiological evidence of severe HIE and 80 (47.6%) without signs of HIE. These images were randomly divided into a training and a test set, and five deep learning models based on based on Densely Connected Convolutional Networks (DenseNet121) were trained and validated using different image input formats (2D and 3D images). All optimized stacked 2D and 3D networks could detect signs of HIE. The networks based on the data as 2D image data stacks provided the best results (S100: AUC: 94%, ACC: 79%, S50: AUC: 93%, ACC: 79%). We provide visual explainability data for the decision making of our AI model using Gradient-weighted Class Activation Mapping. Our proof-of-concept deep learning model can accurately identify signs of HIE on CT images. Comparing different 2D- and 3D-based approaches, most promising results were achieved by 2D image stack models. After further clinical validation, a deep learning model of HIE detection based on CT images could be implemented in clinical routine and thus aid clinicians in characterizing imaging data and predicting outcome.