Imaging Pipeline Research Articles

Pixel-wise exposure control for single-shot HDR imaging: A joint optimization approach

Dynamic range is one of the primary limitations that restricts digital image sensors from acquiring more visual information. Current high dynamic range (HDR) imaging techniques entail a trade-off between dynamic range and visual fidelity. In this work, we propose a HDR imaging method, termed PE-HDR, to achieve both a wide dynamic range and high visual fidelity without additional complex post-processing algorithms. Instead of merging a bracketed exposure sequence, the PE-HDR captures HDR images in a single shot using optical coded pixel-wise exposure control, enabling cost-effective and flexible HDR imaging. By incorporating a differentiable optical encoder and a neural network decoder, we jointly optimize the imaging pipeline from light irradiance to digital image signals, thereby refining the pixel-wise exposure control strategy and improving image fidelity. Both simulations and experiments demonstrate that the proposed method achieves a dynamic range of up to 120 dB and an excellent visual fidelity with spatial resolution of up to 2560 × 1600 pixels.

Innovative projection acquisition algorithm for optimizing portable LNDCT in oil and gas pipeline imaging

Fluid pipelines, commonly utilized in the oil industry, often face efficiency and reliability issues due to sediment buildup causing erosion, corrosion, and pipe wall thinning. Traditional assessment methods involve disruptive measures like cutting or creating holes and temporarily taking pipelines out of service. A non-destructive alternative, Limited-Number-Detector Computed Tomography (LNDCT), proves cost-effective and superior. Our proposed algorithm enhances data acquisition and projections using discrete detectors, employing Co-60 as a gamma-ray source and thallium-doped sodium iodide, NaI(Tl), detectors in an arc configuration. Monte Carlo simulations aligned closely with experimental data. Optimization involved adjusting the detector aperture angle based on a primary-to-scatter ratio of gamma-ray photons. We investigated the utility of various isotopes (Co-60, Cs-137, Am-241, Ir-192) to determine optimal projection signal amplitude. The algorithm generates a large sinogram matrix, and a filtered back-projection algorithm with a Hamming filter maximizes image quality while ensuring acceptable calculation volume and time. Using four phantoms, including pipelines filled to different scales, our study evaluates LNDCT configuration, performance, and validation. The results highlight its potential for efficiently evaluating sediment in pipelines, confirming the correctness and accuracy of our proposed algorithm.

MeXpose─A Modular Imaging Pipeline for the Quantitative Assessment of Cellular Metal Bioaccumulation

MeXpose─A Modular Imaging Pipeline for the Quantitative Assessment of Cellular Metal Bioaccumulation

Tumour or haemorrhage?: Differential diagnosis of an unknown mass within the brain of a budgerigar (Melopsittacus undulatus) using a novel imaging pipeline

AbstractWe report on a previously healthy zoo specimen of an adult budgerigar (Melopsittacus undulatus, obtained with permission from Southwick's Zoo) found deceased in its enclosure. To assess cause of death and ensure the absence of an infectious neoplasia, we used an integrated multiscale brain‐imaging workflow, previously only used on mammals. The specimen was imaged with microcomputed tomography before and after enhancing soft‐tissue contrast with diffusible iodine‐based contrast‐enhanced microcomputed tomography. Scans revealed an orbital blowout fracture and an unidentified large mass across majority of the diencephalon, striatum and midbrain caudal to the right orbit. After destaining, neural pathohistology confirmed the mass as a brain haemorrhage with no evidence of neoplasia or inflammation. We conclude that this specimen died of head trauma, likely from a head‐on collision within its enclosure. This multiscale imaging workflow (diffusible iodine‐based contrast‐enhanced microcomputed tomography followed by destaining and pathohistology) can improve our evaluation of differential diagnoses in avian specimens.

High-resolution hybrid micro-CT imaging pipeline for mouse brain region segmentation and volumetric morphometry.

Brain region segmentation and morphometry in humanized apolipoprotein E (APOE) mouse models with a human NOS2 background (HN) contribute to Alzheimer's disease (AD) research by demonstrating how various risk factors affect the brain. Photon-counting detector (PCD) micro-CT provides faster scan times than MRI, with superior contrast and spatial resolution to energy-integrating detector (EID) micro-CT. This paper presents a pipeline for mouse brain imaging, segmentation, and morphometry from PCD micro-CT. We used brains of 26 mice from 3 genotypes (APOE22HN, APOE33HN, APOE44HN). The pipeline included PCD and EID micro-CT scanning, hybrid (PCD and EID) iterative reconstruction, and brain region segmentation using the Small Animal Multivariate Brain Analysis (SAMBA) tool. We applied SAMBA to transfer brain region labels from our new PCD CT atlas to individual PCD brains via diffeomorphic registration. Region-based and voxel-based analyses were used for comparisons by genotype and sex. Together, PCD and EID scanning take ~5 hours to produce images with a voxel size of 22 μm, which is faster than MRI protocols for mouse brain morphometry with voxel size above 40 μm. Hybrid iterative reconstruction generates PCD images with minimal artifacts and higher spatial resolution and contrast than EID images. Our PCD atlas is qualitatively and quantitatively similar to the prior MRI atlas and successfully transfers labels to PCD brains in SAMBA. Male and female mice had significant volume differences in 26 regions, including parts of the entorhinal cortex and cingulate cortex. APOE22HN brains were larger than APOE44HN brains in clusters from the hippocampus, a region where atrophy is associated with AD. This work establishes a pipeline for mouse brain analysis using PCD CT, from staining to imaging and labeling brain images. Our results validate the effectiveness of the approach, setting a foundation for research on AD mouse models while reducing scanning durations.

Ductal carcinoma in situ develops within clonal fields of mutant cells in morphologically normal ducts.

Mutations are abundantly present in tissues of healthy individuals, including the breast epithelium. Yet it remains unknown whether mutant cells directly induce lesion formation or first spread, leading to a field of mutant cells that is predisposed towards lesion formation. To study the clonal and spatial relationships between morphologically normal breast epithelium adjacent to pre-cancerous lesions, we developed a three-dimensional (3D) imaging pipeline combined with spatially resolved genomics on archival, formalin-fixed breast tissue with the non-obligate breast cancer precursor ductal carcinoma in situ (DCIS). Using this 3D image-guided characterization method, we built high-resolution spatial maps of DNA copy number aberration (CNA) profiles within the DCIS lesion and the surrounding normal mammary ducts. We show that the local heterogeneity within a DCIS lesion is limited. However, by mapping the CNA profiles back onto the 3D reconstructed ductal subtree, we find that in eight out of 16 cases the healthy epithelium adjacent to the DCIS lesions has overlapping structural variations with the CNA profile of the DCIS. Together, our study indicates that pre-malignant breast transformations frequently develop within mutant clonal fields of morphologically normal-looking ducts. © 2024 The Authors. The Journal of Pathology published by John Wiley & Sons Ltd on behalf of The Pathological Society of Great Britain and Ireland.

Efficient Image Details Preservation of Image Processing Pipeline Based on Two-Stage Tone Mapping

Converting a camera’s RAW image to an RGB format for human perception involves utilizing an imaging pipeline, and a series of processing modules. Existing modules often result in varying degrees of original information loss, which can render the reverse imaging pipeline unable to recover the original RAW image information. To this end, this paper proposes a new, almost reversible image imaging pipeline. Thus, RGB images and RAW images can be effectively converted between each other. Considering the impact of original information loss, this paper introduces a two-stage tone mapping operation (TMO). In the first stage, the RAW image with a linear response is transformed into an RGB color image. In the second stage, color scale mapping corrects the dynamic range of the image suitable for human perception through linear stretching, and reduces the loss of sensitive information to the human eye during the integer process. effectively preserving the original image’s dynamic information. The DCRAW imaging pipeline addresses the problem of high light overflow by directly highlighting cuts. The proposed imaging pipeline constructs an independent highlight processing module, and preserves the highlighted information of the image. The experimental results demonstrate that the two-stage tone mapping operation embedded in the imaging processing pipeline provided in this article ensures that the image output is suitable for human visual system (HVS) perception and retains more original image information.

PyDesigner v1.0: A Pythonic Implementation of the DESIGNER Pipeline for Diffusion Magnetic Resonance Imaging.

PyDesigner is a Python-based software package based on the original Diffusion parameter EStImation with Gibbs and NoisE Removal (DESIGNER) pipeline (Dv1) for dMRI preprocessing and tensor estimation. This software is openly provided for non-commercial research and may not be used for clinical care. PyDesigner combines tools from FSL and MRtrix3 to perform denoising, Gibbs ringing correction, eddy current motion correction, brain masking, image smoothing, and Rician bias correction to optimize the estimation of multiple diffusion measures. It can be used across platforms on Windows, Mac, and Linux to accurately derive commonly used metrics from DKI, DTI, WMTI, FBI, and FBWM datasets as well as tractography ODFs and .fib files. It is also file-format agnostic, accepting inputs in the form of .nii, .nii.gz, .mif, and dicom format. User-friendly and easy to install, this software also outputs quality control metrics illustrating signal-to-noise ratio graphs, outlier voxels, and head motion to evaluate data integrity. Additionally, this dMRI processing pipeline supports multiple echo-time dataset processing and features pipeline customization, allowing the user to specify which processes are employed and which outputs are produced to meet a variety of user needs.

Shifting to machine supervision: annotation-efficient semi and self-supervised learning for automatic medical image segmentation and classification

Advancements in clinical treatment are increasingly constrained by the limitations of supervised learning techniques, which depend heavily on large volumes of annotated data. The annotation process is not only costly but also demands substantial time from clinical specialists. Addressing this issue, we introduce the S4MI (Self-Supervision and Semi-Supervision for Medical Imaging) pipeline, a novel approach that leverages advancements in self-supervised and semi-supervised learning. These techniques engage in auxiliary tasks that do not require labeling, thus simplifying the scaling of machine supervision compared to fully-supervised methods. Our study benchmarks these techniques on three distinct medical imaging datasets to evaluate their effectiveness in classification and segmentation tasks. Notably, we observed that self-supervised learning significantly surpassed the performance of supervised methods in the classification of all evaluated datasets. Remarkably, the semi-supervised approach demonstrated superior outcomes in segmentation, outperforming fully-supervised methods while using 50% fewer labels across all datasets. In line with our commitment to contributing to the scientific community, we have made the S4MI code openly accessible, allowing for broader application and further development of these methods. The code can be accessed at https://github.com/pranavsinghps1/S4MI.

Open-source data pipeline for street-view images: A case study on community mobility during COVID-19 pandemic.

Street View Images (SVI) are a common source of valuable data for researchers. Researchers have used SVI data for estimating pedestrian volumes, demographic surveillance, and to better understand built and natural environments in cityscapes. However, the most common source of publicly available SVI data is Google Street View. Google Street View images are collected infrequently, making temporal analysis challenging, especially in low population density areas. Our main contribution is the development of an open-source data pipeline for processing 360-degree video recorded from a car-mounted camera. The video data is used to generate SVIs, which then can be used as an input for longitudinal analysis. We demonstrate the use of the pipeline by collecting an SVI dataset over a 38-month longitudinal survey of Seattle, WA, USA during the COVID-19 pandemic. The output of our pipeline is validated through statistical analyses of pedestrian traffic in the images. We confirm known results in the literature and provide new insights into outdoor pedestrian traffic patterns. This study demonstrates the feasibility and value of collecting and using SVI for research purposes beyond what is possible with currently available SVI data. Our methods and dataset represent a first of its kind longitudinal collection and application of SVI data for research purposes. Limitations and future improvements to the data pipeline and case study are also discussed.

An Empirical Study On Imaging Sensitivity Of SKA1-LOW Based on Point Source Sensitivity

The Square Kilometre Array (SKA) radio telescope will achieve unparalleled sensitivity and angular resolution, substantially progressing our research into the formation and development of the early universe. Sensitivity is one of the most critical telescope metrics, often used to determine the observing mode and observing time of a radio source. It is also used to determine imaging parameters to balance imaging sensitivity and spatial resolution during data processing. In this paper, to meet the needs of continuum imaging pipeline data processing for the SKA1-LOW telescope, we derive equations for point-source-sensitivity (PSS) calculations under multichannel, multipolarization, discrete frequency sampling, discrete time sampling, and different weighting schemes. After validating the correctness of the equation, we improved and refined the sensitivity calculation application in the Radio Astronomy Simulation, Calibration, and Imaging Library, and further calculated the variation of the PSS versus angular resolution for the full-scale SKA1-LOW with different weighting schemes, and provided SKA1-LOW broadband image performance as functions of angular scale for four frequencies. These results can provide not only a theoretical basis for the construction and commissioning of the SKA1-LOW telescope but also guidance for imaging processing in future scientific data processing.

Color matching in the wild

We present a method that, given two different views of the same scene taken by two cameras with unknown settings and internal parameters, corrects the colors of one of the images making it look as if it was captured under the other camera settings. Our method is able to deal with any standard non-linear encoded images (gamma-corrected, logarithmic-encoded, or any other) without requiring any previous knowledge of the encoding.To this end, our method makes use of two important observations. First, the camera imaging pipeline from RAW to sRGB can be well approximated by considering just a per-pixel shading and a color transformation matrix, and second, for correcting the images we only need to estimate a single matrix –that will contain information from both of the original images– and an approximation of the shading term (that emulates the non-linearity).Our proposed method is fast and the results have no spurious artifacts. The method outperforms the state-of-the-art when compared with other methods that do not require knowledge of the encoding used. It is also able to compete with –and even surpass in some cases– methods that consider information about image encoding.

Compressed-SDR to HDR Video Reconstruction.

The new generation of organic light emitting diode display is designed to enable the high dynamic range (HDR), going beyond the standard dynamic range (SDR) supported by the traditional display devices. However, a large quantity of videos are still of SDR format. Further, most pre-existing videos are compressed at varying degrees for minimizing the storage and traffic flow demands. To enable movie-going experience on new generation devices, converting the compressed SDR videos to the HDR format (i.e., compressed-SDR to HDR conversion) is in great demands. The key challenge with this new problem is how to solve the intrinsic many-to-many mapping issue. However, without constraining the solution space or simply imitating the inverse camera imaging pipeline in stages, existing SDR-to-HDR methods can not formulate the HDR video generation process explicitly. Besides, they ignore the fact that videos are often compressed. To address these challenges, in this work we propose a novel imaging knowledge-inspired parallel networks (termed as KPNet) for compressed-SDR to HDR (CSDR-to-HDR) video reconstruction. KPNet has two key designs: Knowledge-Inspired Block (KIB) and Information Fusion Module (IFM). Concretely, mathematically formulated using some priors with compressed videos, our conversion from a CSDR-to-HDR video reconstruction is conceptually divided into four synergistic parts: reducing compression artifacts, recovering missing details, adjusting imaging parameters, and reducing image noise. We approximate this process by a compact KIB. To capture richer details, we learn HDR representations with a set of KIBs connected in parallel and fused with the IFM. Extensive evaluations show that our KPNet achieves superior performance over the state-of-the-art methods.

Context-Specific Stress Causes Compartmentalized SARM1 Activation and Local Degeneration in Cortical Neurons.

Sterile alpha and TIR motif containing 1 (SARM1) is an inducible NADase that localizes to mitochondria throughout neurons and senses metabolic changes that occur after injury. Minimal proteomic changes are observed upon either SARM1 depletion or activation, suggesting that SARM1 does not exert broad effects on neuronal protein homeostasis. However, whether SARM1 activation occurs throughout the neuron in response to injury and cell stress remains largely unknown. Using a semiautomated imaging pipeline and a custom-built deep learning scoring algorithm, we studied degeneration in both mixed-sex mouse primary cortical neurons and male human-induced pluripotent stem cell-derived cortical neurons in response to a number of different stressors. We show that SARM1 activation is differentially restricted to specific neuronal compartments depending on the stressor. Cortical neurons undergo SARM1-dependent axon degeneration after mechanical transection, and SARM1 activation is limited to the axonal compartment distal to the injury site. However, global SARM1 activation following vacor treatment causes both cell body and axon degeneration. Context-specific stressors, such as microtubule dysfunction and mitochondrial stress, induce axonal SARM1 activation leading to SARM1-dependent axon degeneration and SARM1-independent cell body death. Our data reveal that compartment-specific SARM1-mediated death signaling is dependent on the type of injury and cellular stressor.

Continuous distribution of cancer cells in the cell cycle unveiled by AI-segmented imaging of 37,000 HeLa FUCCI cells

Classification of live or fixed cells based on their unlabeled microscopic images would be a powerful tool for cell biology and pathology. For such software, the first step is the generation of a ground truth database that can be used for training and testing AI classification algorithms. The Application of cells expressing fluorescent reporter proteins allows the building of ground truth datasets in a straightforward way. In this study, we present an automated imaging pipeline utilizing the Cellpose algorithm for the precise cell segmentation and measurement of fluorescent cellular intensities across multiple channels. We analyzed the cell cycle of HeLa–FUCCI cells expressing fluorescent red and green reporter proteins at various levels depending on the cell cycle state. To build the dataset, 37,000 fixed cells were automatically scanned using a standard motorized microscope, capturing phase contrast and fluorescent red/green images. The fluorescent pixel intensity of each cell was integrated to calculate the total fluorescence of cells based on cell segmentation in the phase contrast channel. It resulted in a precise intensity value for each cell in both channels. Furthermore, we conducted a comparative analysis of Cellpose 1.0 and Cellpose 2.0 in cell segmentation performance. Cellpose 2.0 demonstrated notable improvements, achieving a significantly reduced false positive rate of 2.7 % and 1.4 % false negative. The cellular fluorescence was visualized in a 2D plot (map) based on the red and green intensities of the FUCCI construct revealing the continuous distribution of cells in the cell cycle. This 2D map enables the selection and potential isolation of single cells in a specific phase. In the corresponding heatmap, two clusters appeared representing cells in the red and green states. Our pipeline allows the high-throughput and accurate measurement of cellular fluorescence providing extensive statistical information on thousands of cells with potential applications in developmental and cancer biology. Furthermore, our method can be used to build ground truth datasets automatically for training and testing AI cell classification. Our automated pipeline can be used to analyze thousands of cells within 2 h after putting the sample onto the microscope.

Simulated host galaxy analogues of high-z quasars observed with JWST

ABSTRACT The hosts of two low-luminosity high-z quasars, J2255+0251 and J2236+0032, were recently detected using JWST’s NIRCam instrument. These represent the first high-z quasar host galaxy stellar detections and open a new window into studying high-z quasars. We examine the implications of the measured properties of J2255+0251 and J2236+0032 within the context of the hydrodynamic simulation BlueTides at z = 6.5. We find that these observed quasars fall on the BlueTides stellar to black hole mass relation and have similar luminosities to the brightest simulated quasars. We predict their star formation rates, estimating approximately $10^{2-3}$${\rm M}_{\odot }\, \rm yr^{-1}$ for both quasar hosts. J2255+0251 and J2236+0032’s host galaxy radii also fall within estimates of the radii of the simulated host galaxies of similar luminosity quasars. We generate mock JWST NIRCam images of analogues to the observed quasars within BlueTides and perform a point source removal to illustrate both a qualitative and quantitative comparison of the measured and simulated radii and magnitudes. The quasar subtraction works well for similar luminosity quasars, and the recovered host images are consistent with what was observed for J2255+0251 and J2236+0032, further supporting the success of those observations. We also use our mock imaging pipeline to make predictions for the detection of J2255+0251 and J2236+0032’s hosts in upcoming JWST observations. We anticipate that the simulation analogues of future high-z quasar host discoveries will allow us to make accurate predictions of their properties beyond the capabilities of JWST.

Illuminating the complete ß-cell mass of the human pancreas- signifying a new view on the islets of Langerhans

Pancreatic islets of Langerhans play a pivotal role in regulating blood glucose homeostasis, but critical information regarding their mass, distribution and composition is lacking within a whole organ context. Here, we apply a 3D imaging pipeline to generate a complete account of the insulin-producing islets throughout the human pancreas at a microscopic resolution and within a maintained spatial 3D context. These data show that human islets are far more heterogenous than previously accounted for with regards to their size distribution and cellular make up. By deep tissue 3D imaging, this in-depth study demonstrates that 50% of the human insulin-expressing islets are virtually devoid of glucagon-producing α-cells, an observation with significant implications for both experimental and clinical research.

CiftiStorm pipeline: facilitating reproducible EEG/MEG source connectomics.

We present CiftiStorm, an electrophysiological source imaging (ESI) pipeline incorporating recently developed methods to improve forward and inverse solutions. The CiftiStorm pipeline produces Human Connectome Project (HCP) and megconnectome-compliant outputs from dataset inputs with varying degrees of spatial resolution. The input data can range from low-sensor-density electroencephalogram (EEG) or magnetoencephalogram (MEG) recordings without structural magnetic resonance imaging (sMRI) to high-density EEG/MEG recordings with an HCP multimodal sMRI compliant protocol. CiftiStorm introduces a numerical quality control of the lead field and geometrical corrections to the head and source models for forward modeling. For the inverse modeling, we present a Bayesian estimation of the cross-spectrum of sources based on multiple priors. We facilitate ESI in the T1w/FSAverage32k high-resolution space obtained from individual sMRI. We validate this feature by comparing CiftiStorm outputs for EEG and MRI data from the Cuban Human Brain Mapping Project (CHBMP) acquired with technologies a decade before the HCP MEG and MRI standardized dataset.

HiFAST: An Hi data calibration and imaging pipeline for FAST

HiFAST: An Hi data calibration and imaging pipeline for FAST

Low-frequency Radio Interferometric Array Imaging Pipeline Optimization Based on Distributed Execution Framework

The Square Kilometre Array (SKA) project is an international collaboration to build the world’s largest radio telescope, whose sensitivity and measurement speed will be an order of magnitude higher than those of all current radio telescopes. Radio continuum survey is one of the main observation mode of the SKA, and the establishment of a standard map of the survey area based on continuum imaging will provide an important foundation for subsequent astronomical science. The GaLactic and Extragalactic All-sky Murchison Widefield Array survey eXtended (GLEAM-X) is a project of the SKA pilot telescope Murchison Widefield Array (MWA) in 2018—2020. GLEAM-X is a new radio continuum survey project to be carried out with the MWA Phase II expansion array in 2018—2020. The experience of optimizing the imaging pipeline based on the distributed execution framework will help to solve the problem of massive data processing. In this paper, we describe the process steps of GLEAM-X imaging pipeline, integrate and improve it, and realize parallel processing of multiple pipelines on the China SKA Regional Centre Prototype (CSRC-P), and verify the deployment and test the correctness of the imaging pipeline system using GLEAM-X observation data. The GLEAM-X observations were used to validate the deployment of the imaging pipeline system and test its correctness. Then, to optimize the pipelines and improve the processing efficiency, the Data Activated Liu Graph Engine (DALiuGE) was used to integrate the imaging pipelines into the DALiuGE execution framework to automate the distributed parallel processing of the pipelines. Performance tests and results’ analysis show that the optimized imaging pipeline based on the DALiuGE execution framework has better performance, more flexible adaptability, and scalability than the traditional parallel approach, and can support future large-scale continuum imaging experiments during the first phase of SKA commissioning.