Imaging Conditions Research Articles

Clinical boundary conditions for propagation-based X-ray phase contrast imaging: from bio-sample models targeting to clinical applications.

Typical propagation-based X-ray phase contrast imaging (PB-PCI) experiments using polyenergetic sources are tested in very ideal conditions: low-energy spectrum (mainly characteristic X-rays), small thickness and homogeneous materials considered weakly absorbing objects, large object-to-detector distance, long exposure times and non-clinical detector. Explore PB-PCI features using boundary conditions imposed by a low power polychromatic X-ray source (X-ray spectrum without characteristic X-rays), thick and heterogenous materials and a small area imaging detector with high low-detection radiation threshold, elements commonly found in a clinical scenario. A PB-PCI setup implemented using a microfocus X-ray source and a dental imaging detector was characterized in terms of different spectra and geometric parameters on the acquired images. Test phantoms containing fibers and homogeneous materials with close attenuation characteristics and animal bone and mixed soft tissues (bio-sample models) were analyzed. Contrast to Noise Ratio (CNR), system spatial resolution and Kerma values were obtained for all images. Phase contrast images showed CNR up to 15% higher than conventional contact images. Moreover, it is better seen when large magnifications (>3) and object-to-detector distances (>13 cm) were used. The influence of the spectrum was not appreciable due to the low efficiency of the detector (thin scintillator screen) at high energies. Despite the clinical boundary condition used in this work, regarding the X-ray spectrum, thick samples, and detection system, it was possible to acquire phase contrast images of biological samples.

Reconstruction method for gamma-ray coded-aperture imaging based on mask and anti-mask functions

Gamma-ray coded-aperture imaging technology plays an important role in nuclear security, decommissioning of nuclear facilities, and nuclear medicine diagnosis. However, under near-field imaging condition, artifacts in the reconstructed image can interfere with identifying the shape and position of the radioactive source. In this paper, a gamma-ray coded-aperture imaging method based on mask and anti-mask functions was proposed to suppress imaging artifacts and speed up the acquisition of low-noise reconstructed images. Through simulation, the effects of the number of iterations and the thickness of the coded-aperture collimator on the imaging quality were studied, and the range of the optimal correction factor in the method was determined. Imaging experiments were conducted using a compact coded-aperture gamma camera based on CdZnTe detector to verify the applicability of the optimal correction factor range. The limitations of the proposed method were analyzed through complex-shaped source imaging simulations and multi-source imaging experiments. This method has an insufficient suppression effect on random artifacts and requires further improvement in imaging irregular radioactive sources. However, it has good imaging performance for single-point source and multi-point sources, effectively reducing regular cross-shaped and stripe-like artifacts. In the non-uniform radioactive background, it can eliminate a part of artifacts, significantly improving imaging quality. Therefore, this method has potential applications in complex radioactive environments.

Near-Infrared Spontaneously Blinking Fluorophores for Live Cell Super-Resolution Imaging with Minimized Phototoxicity.

Single-molecule localization microscopy (SMLM) requires high-intensity laser irradiation, typically exceeding kW/cm2, to yield a sufficient photon count. However, this intense visible light exposure incurs substantial cellular toxicity, hindering its use in living cells. Here, we developed a class of near-infrared (NIR) spontaneously blinking fluorophores for SMLM. These NIR fluorophores are a combination of rhodamine spirolactams and merocyanine derivatives, where the rhodamine spirolactam component converts between a bright and dark state based on pH-dependent spirocyclization and merocyanine derivatives shift the excitation wavelength into the infrared. Single-molecule characterizations demonstrated their potential for SMLM. At a moderate power density of 3.93 kW/cm2, these probes exhibit duty cycle as low as 0.18% and an emission rate as high as 26,700 photons/s. Phototoxicity assessment under single-molecule imaging conditions reveals that NIR illumination (721 nm) minimizes harm to living cells. Employing these NIR fluorophores, we successfully captured time-lapse super-resolution tracking of mitochondria at a Fourier ring correlation (FRC) resolution of 69.4 nm and reconstructed the ultrastructures of endoplasmic reticulum (ER) in living cells.

Improving real-time apple fruit detection: Multi-modal data and depth fusion with non-targeted background removal

In automated fruit detection, RGB-Depth (RGB-D) images aid the detection model with additional depth information to enhance detection accuracy. However, outdoor depth images are usually of low quality, which limits the quality of depth data. In this study, an approach/technique for real-time apple fruit detection in a high-density orchard environment by using multi-modal data is presented. Non-targeted background removal using the depth fusion (NBR-DF) method was developed to reduce the high noise condition of depth images. The noise occurred due to the uncontrolled lighting condition and holes with incomplete depth information in the depth images. NBR-DF technique follows three primary steps: pre-processing of depth images (point cloud generation), target object extraction, and background removal. The NBR-DF method serves as a pipeline to pre-process multi-modal data to enhance features of depth images by filling holes to eliminate noise generated by depth holes. Further, the NBR-DF implemented with the YOLOv5 enhances the detection accuracy in dense orchard conditions by using multi-modal information as input. An attention-based depth fusion module that adaptively fuses the multi-modal features was developed. The integration of the depth-attention matrix involved pooling operations and sigmoid normalization, both of which are efficient methods for summarizing and normalizing depth information. The fusion module improves the identification of multiscale objects and strengthens the network's resistance to noise. The network then detects the fruit position using multiscale information from the RGB-D images in highly complex orchard environments. The detection results were compared and validated with other methods using different input modals and fusion strategies. The results showed that the detection accuracy using the NBR-DF approach achieved an average precision rate of 0.964 in real time. The performance comparison with other state-of-the-art methods and the model generalization study also establish that the present advanced depth-fusion attention mechanism and effective preprocessing steps in NBR-DF-YOLOv5 significantly surpass those in performance. In conclusion, the developed NBR-DF technique showed the potential to improve real-time apple fruit detection using multi-modal information.

Musical harmonies and its relationship with emotional processing: An ERP study in young adults

Music has been used to express and communicate emotional states through its different dimensions such as tone, rhythm, melody, and harmony. Consonant harmonies consistently are rated as pleasant whereas dissonant are considered unpleasant. The aim of this study was to explore the effect of consonant and dissonant musical harmonies used as prime on the emotional classification of images, as indexed by event-related potentials. Thirty volunteers (ages 21–27, 50 % women) were presented with a task consisting of 4 musical intervals in the C major scale, divided into consonant and dissonant harmonies, followed by 180 positive, negative, or neutral images from the International Affective Picture System (IAPS). Participants had to rate the images as pleasant or unpleasant. We found a bias effect on negative images rated as positive when preceded by a consonant musical interval. A N200 component, non-sensible to the valence of the images, was found. On the other hand, a significant difference was found in the amplitude of the P300 component, with a greater amplitude in the consonant-positive images condition compared to the dissonant-positive images. Lastly, a late positivity component around 500–700 ms was found in both negative conditions dissonant and consonant, but with a larger amplitude for the consonant condition when followed by a negative image. These results indicate that additionally to the P300 processing the relevance of the stimulus there are processes like recognition memory involved. As part of the novelty effect this late positive activity may also be related to the emotional content integration of the relevant stimulus.

Large-Scale Screening of Pharmaceutical Compounds to Explore the Application Space of On-Tissue MALDI and MALDI-2 Mass Spectrometry.

The successful application of matrix-assisted laser desorption ionization mass spectrometry imaging (MALDI-MSI) in pharmaceutical research is strongly dependent on the detection of the drug of interest at physiologically relevant concentrations. Here we explored how insufficient sensitivity due to low ionization efficiency and/or the interaction of the drug molecule with the local biochemical environment of the tissue can be mitigated for many compound classes using the recently introduced MALDI-MSI coupled with laser-induced postionization, known as MALDI-2-MSI. Leveraging a MALDI-MSI screen of about 1,200 medicines/drug-like compounds from a broad range of medicinal application areas, we demonstrate a significant improvement in drug detection and the degree of sensitivity uplift by using MALDI-2 versus traditional MALDI. Our evaluation was made under simulated imaging conditions using liver homogenate sections as substrate, onto which the compounds were spotted to mimic biological conditions to the first order. To enable an evaluable detection by both MALDI and MALDI-2 for the majority of employed compounds, we spotted 1 μL of a 10 mM solution using a spotting robot and performed our experiments with a Bruker timsTOF fleX MALDI-2 instrument in both positive and negative ion modes. Specifically, we demonstrate using a large cohort of drug-like compounds that ∼60% of the tested compounds showed a more than 10-fold increase in signal intensity and ∼16% showed a more than 100-fold increase upon use of MALDI-2 postionization. Such increases in sensitivity could help advance pharmaceutical MALDI-MSI applications toward the single-cell level.

Robust Quantification of Live-Cell Single-Molecule Tracking Data for Fluorophores with Different Photophysical Properties.

High-speed single-molecule tracking in live cells is becoming an increasingly popular method for quantifying the spatiotemporal behavior of proteins in vivo. The method provides a wealth of quantitative information, but users need to be aware of biases that can skew estimates of molecular mobilities. The range of suitable fluorophores for live-cell single-molecule imaging has grown substantially over the past few years, but it remains unclear to what extent differences in photophysical properties introduce biases. Here, we tested two fluorophores with entirely different photophysical properties, one that photoswitches frequently between bright and dark states (TMR) and one that shows exceptional photostability without photoswitching (JFX650). We used a fusion of the Escherichia coli DNA repair enzyme MutS to the HaloTag and optimized sample preparation and imaging conditions for both types of fluorophore. We then assessed the reliability of two common data analysis algorithms, mean-square displacement (MSD) analysis and Hidden Markov Modeling (HMM), to estimate the diffusion coefficients and fractions of MutS molecules in different states of motion. We introduce a simple approach that removes discrepancies in the data analyses and show that both algorithms yield consistent results, regardless of the fluorophore used. Nevertheless, each dye has its own strengths and weaknesses, with TMR being more suitable for sampling the diffusive behavior of many molecules, while JFX650 enables prolonged observation of only a few molecules per cell. These characterizations and recommendations should help to standardize measurements for increased reproducibility and comparability across studies.

Elastic time-reverse imaging of backscattered surface-wave data

Imaging shallow near-vertical or short-wavelength structure using seismic data is an important goal in near-surface geophysical applications. Numerous seismic analysis approaches use backscattered analysis of surface waves for these purposes; however, there is a need for new associated frameworks that can take advantage of these events to generate interpretable images in the depth-migrated domain. By drawing a connection with elastic time-reverse imaging (E-TRI) approaches developed for microseismic event analysis, an efficient elastic migration approach is developed that uses two adjoint wavefields defined by partitioned outward-propagating transmission and inward-propagating scattered surface-wave data. Subsurface images are generated by applying the energy-norm imaging condition that correlates temporally and spatially colocated energy throughout the two time-reversed adjoint wavefield domains. Synthetic tests highlight the ability of the E-TRI framework to image shallow (quasi)vertical subsurface discontinuities. Results from a field test using near-surface seismic data acquired in the Majes region of Arequipa, Peru, demonstrate the ability of the E-TRI framework to highlight short-wavelength structure identified in a parallel elastic full-waveform inversion analysis. The results suggest that the E-TRI technique should be applicable in a range of near-surface geotechnical and environmental applications.

Transient enhancement of stimulus-evoked activity in neocortex during sensory learning.

Synaptic potentiation has been linked to learning in sensory cortex, but the connection between this potentiation and increased sensory-evoked neural activity is not clear. Here, we used longitudinal in vivo Ca2+ imaging in the barrel cortex of awake mice to test the hypothesis that increased excitatory synaptic strength during the learning of a whisker-dependent sensory-association task would be correlated with enhanced stimulus-evoked firing. To isolate stimulus-evoked responses from dynamic, task-related activity, imaging was performed outside of the training context. Although prior studies indicate that multiwhisker stimuli drive robust subthreshold activity, we observed sparse activation of L2/3 pyramidal (Pyr) neurons in both control and trained mice. Despite evidence for excitatory synaptic strengthening at thalamocortical and intracortical synapses in this brain area at the onset of learning-indeed, under our imaging conditions thalamocortical axons were robustly activated-we observed that L2/3 Pyr neurons in somatosensory (barrel) cortex displayed only modest increases in stimulus-evoked activity that were concentrated at the onset of training. Activity renormalized over longer training periods. In contrast, when stimuli and rewards were uncoupled in a pseudotraining paradigm, stimulus-evoked activity in L2/3 Pyr neurons was significantly suppressed. These findings indicate that sensory-association training but not sensory stimulation without coupled rewards may briefly enhance sensory-evoked activity, a phenomenon that might help link sensory input to behavioral outcomes at the onset of learning.

Large-scale deep tissue voltage imaging with targeted-illumination confocal microscopy.

Voltage imaging with cellular specificity has been made possible by advances in genetically encoded voltage indicators. However, the kilohertz rates required for voltage imaging lead to weak signals. Moreover, out-of-focus fluorescence and tissue scattering produce background that both undermines the signal-to-noise ratio and induces crosstalk between cells, making reliable in vivo imaging in densely labeled tissue highly challenging. We describe a microscope that combines the distinct advantages of targeted illumination and confocal gating while also maximizing signal detection efficiency. The resulting benefits in signal-to-noise ratio and crosstalk reduction are quantified experimentally and theoretically. Our microscope provides a versatile solution for enabling high-fidelity in vivo voltage imaging at large scales and penetration depths, which we demonstrate across a wide range of imaging conditions and different genetically encoded voltage indicator classes.

Improved Denoising of Cryo-Electron Microscopy Micrographs with Simulation-Aware Pretraining.

Cryo-electron microscopy (cryo-EM) has emerged as a potent technique for determining the structure and functionality of biological macromolecules. However, limited by the physical imaging conditions, such as low electron beam dose, micrographs in cryo-EM typically contend with an extremely low signal-to-noise ratio (SNR), impeding the efficiency and efficacy of subsequent analyses. Therefore, there is a growing demand for an efficient denoising algorithm designed for cryo-EM micrographs, aiming to enhance the quality of macromolecular analysis. However, owing to the absence of a comprehensive and well-defined dataset with ground truth images, supervised image denoising methods exhibit limited generalization when applied to experimental micrographs. To tackle this challenge, we introduce a simulation-aware image denoising (SaID) pretrained model designed to enhance the SNR of cryo-EM micrographs where the training is solely based on an accurately simulated dataset. First, we propose a parameter calibration algorithm for simulated dataset generation, aiming to align simulation parameters with those of experimental micrographs. Second, leveraging the accurately simulated dataset, we propose to train a deep general denoising model that can well generalize to real experimental cryo-EM micrographs. Comprehensive experimental results demonstrate that our pretrained denoising model achieves excellent denoising performance on experimental cryo-EM micrographs, significantly streamlining downstream analysis.

Integrated photonic encoder for low power and high-speed image processing.

Modern lens designs are capable of resolving greater than 10 gigapixels, while advances in camera frame-rate and hyperspectral imaging have made data acquisition rates of Terapixel/second a real possibility. The main bottlenecks preventing such high data-rate systems are power consumption and data storage. In this work, we show that analog photonic encoders could address this challenge, enabling high-speed image compression using orders-of-magnitude lower power than digital electronics. Our approach relies on a silicon-photonics front-end to compress raw image data, foregoing energy-intensive image conditioning and reducing data storage requirements. The compression scheme uses a passive disordered photonic structure to perform kernel-type random projections of the raw image data with minimal power consumption and low latency. A back-end neural network can then reconstruct the original images with structural similarity exceeding 90%. This scheme has the potential to process data streams exceeding Terapixel/second using less than 100 fJ/pixel, providing a path to ultra-high-resolution data and image acquisition systems.

Toward drift-free high-throughput nanoscopy through adaptive intersection maximization.

Single-molecule localization microscopy (SMLM) often suffers from suboptimal resolution due to imperfect drift correction. Existing marker-free drift correction algorithms often struggle to reliably track high-frequency drift and lack the computational efficiency to manage large, high-throughput localization datasets. We present an adaptive intersection maximization-based method (AIM) that leverages the entire dataset's information content to minimize drift correction errors, particularly addressing high-frequency drift, thereby enhancing the resolution of existing SMLM systems. We demonstrate that AIM can robustly and efficiently achieve an angstrom-level tracking precision for high-throughput SMLM datasets under various imaging conditions, resulting in an optimal resolution in simulated and biological experimental datasets. We offer AIM as one simple, model-free software for instant resolution enhancement with standard CPU devices.

Testing the generalizability and effectiveness of deep learning models among clinics: sperm detection as a pilot study

BackgroundDeep learning has been increasingly investigated for assisting clinical in vitro fertilization (IVF). The first technical step in many tasks is to visually detect and locate sperm, oocytes, and embryos in images. For clinical deployment of such deep learning models, different clinics use different image acquisition hardware and different sample preprocessing protocols, raising the concern over whether the reported accuracy of a deep learning model by one clinic could be reproduced in another clinic. Here we aim to investigate the effect of each imaging factor on the generalizability of object detection models, using sperm analysis as a pilot example.MethodsAblation studies were performed using state-of-the-art models for detecting human sperm to quantitatively assess how model precision (false-positive detection) and recall (missed detection) were affected by imaging magnification, imaging mode, and sample preprocessing protocols. The results led to the hypothesis that the richness of image acquisition conditions in a training dataset deterministically affects model generalizability. The hypothesis was tested by first enriching the training dataset with a wide range of imaging conditions, then validated through internal blind tests on new samples and external multi-center clinical validations.ResultsAblation experiments revealed that removing subsets of data from the training dataset significantly reduced model precision. Removing raw sample images from the training dataset caused the largest drop in model precision, whereas removing 20x images caused the largest drop in model recall. by incorporating different imaging and sample preprocessing conditions into a rich training dataset, the model achieved an intraclass correlation coefficient (ICC) of 0.97 (95% CI: 0.94-0.99) for precision, and an ICC of 0.97 (95% CI: 0.93-0.99) for recall. Multi-center clinical validation showed no significant differences in model precision or recall across different clinics and applications.ConclusionsThe results validated the hypothesis that the richness of data in the training dataset is a key factor impacting model generalizability. These findings highlight the importance of diversity in a training dataset for model evaluation and suggest that future deep learning models in andrology and reproductive medicine should incorporate comprehensive feature sets for enhanced generalizability across clinics.

Automated segmentation of large image datasets using artificial intelligence for microstructure characterisation and damage analysis

Many properties of commonly used materials are driven by their microstructure, which can be influenced by the composition and manufacturing processes. To optimise future materials, understanding the microstructure is critically important. Here, we present two novel approaches based on artificial intelligence that allow the segmentation of the phases of a microstructure for which simple numerical approaches, such as thresholding, are not applicable: One is based on the nnU-Net neural network, and the other on generative adversarial networks (GAN).Using scanning electron microscopy images collected from large areas (∼1 mm2) of dual-phase steels as a case study, we demonstrate how both methods effectively segment intricate microstructural details, including martensite, ferrite, and damage sites, for subsequent analysis.Either method shows substantial generalizability across a range of image sizes and conditions, including heat-treated microstructures with different phase configurations. The nnU-Net excels in mapping large image areas. Conversely, the GAN-based method performs reliably on smaller images, providing greater step-by-step control and flexibility over the segmentation process.This study highlights the benefits of segmented microstructural data for various purposes, such as calculating phase fractions, modelling material behaviour through finite element simulation, and conducting geometrical analyses of damage sites and the local properties of their surrounding microstructure.

Synthetic Data for Video Surveillance Applications of Computer Vision: A Review

In recent years, there has been a growing interest in synthetic data for several computer vision applications, such as automotive, detection and tracking, surveillance, medical image analysis and robotics. Early use of synthetic data was aimed at performing controlled experiments under the analysis by synthesis approach. Currently, synthetic data are mainly used for training computer vision models, especially deep learning ones, to address well-known issues of real data, such as manual annotation effort, data imbalance and bias, and privacy-related restrictions. In this work, we survey the use of synthetic training data focusing on applications related to video surveillance, whose relevance has rapidly increased in the past few years due to their connection to security: crowd counting, object and pedestrian detection and tracking, behaviour analysis, person re-identification and face recognition. Synthetic training data are even more interesting in this kind of application, to address further, specific issues arising, e.g., from typically unconstrained image or video acquisition conditions and cross-scene application scenarios. We categorise and discuss the existing methods for creating synthetic data, analyse the synthetic data sets proposed in the literature for each of the considered applications, and provide an overview of their effectiveness as training data. We finally discuss whether and to what extent the existing synthetic data sets mitigate the issues of real data, highlight existing open issues, and suggest future research directions in this field.

Subtraction of liposome signals in cryo-EM structural determination of protein–liposome complexes

Abstract Reconstituting membrane proteins in liposomes and determining their structure is a common method for determining membrane protein structures using single-particle cryo-electron microscopy (cryo-EM). However, the strong signal of liposomes under cryo-EM imaging conditions often interferes with the structural determination of the embedded membrane proteins. Here, we propose a liposome signal subtraction method based on single-particle two-dimensional (2D) classification average images, aimed at enhancing the reconstruction resolution of membrane proteins. We analyzed the signal distribution characteristics of liposomes and proteins within the 2D classification average images of protein–liposome complexes in the frequency domain. Based on this analysis, we designed a method to subtract the liposome signals from the original particle images. After the subtraction, the accuracy of single-particle three-dimensional (3D) alignment was improved, enhancing the resolution of the final 3D reconstruction. We demonstrated this method using a PIEZO1-proteoliposome dataset by improving the resolution of the PIEZO1 protein.

Zero-shot learning enables instant denoising and super-resolution in optical fluorescence microscopy.

Computational super-resolution methods, including conventional analytical algorithms and deep learning models, have substantially improved optical microscopy. Among them, supervised deep neural networks have demonstrated outstanding performance, however, demanding abundant high-quality training data, which are laborious and even impractical to acquire due to the high dynamics of living cells. Here, we develop zero-shot deconvolution networks (ZS-DeconvNet) that instantly enhance the resolution of microscope images by more than 1.5-fold over the diffraction limit with 10-fold lower fluorescence than ordinary super-resolution imaging conditions, in an unsupervised manner without the need for either ground truths or additional data acquisition. We demonstrate the versatile applicability of ZS-DeconvNet on multiple imaging modalities, including total internal reflection fluorescence microscopy, three-dimensional wide-field microscopy, confocal microscopy, two-photon microscopy, lattice light-sheet microscopy, and multimodal structured illumination microscopy, which enables multi-color, long-term, super-resolution 2D/3D imaging of subcellular bioprocesses from mitotic single cells to multicellular embryos of mouse and C. elegans.

Photoswitching Reagent for Super-Resolution Fluorescence Microscopy.

Single-molecule localization microscopy (SMLM) has revolutionized optical microscopy by exceeding the diffraction limit and revealing previously unattainable nanoscale details of cellular structures and molecular dynamics. This super-resolution imaging capability relies on fluorophore photoswitching, which is crucial for optimizing the imaging conditions and accurately determining the fluorophore positions. To understand the general on and off photoswitching mechanisms of single dye molecules, various photoswitching reagents were evaluated. Systematic measurement of the single-molecule-level fluorescence on and off rates (kon and koff) in the presence of various photoswitching reagents and theoretical calculation of the structure of the photoswitching reagent-fluorophore pair indicated that the switch-off mechanism is mainly determined by the nucleophilicity of the photoswitching reagent, and the switch-on mechanism is a two-photon-induced dissociation process, which is related to the power of the illuminating laser and bond dissociation energy of this pair. This study contributes to a broader understanding of the molecular photoswitching mechanism in SMLM imaging and provides a basis for designing improved photoswitching reagents with potential applications extending to materials science and chemistry.

Photoswitching Reagent for Super‐Resolution Fluorescence Microscopy

AbstractSingle‐molecule localization microscopy (SMLM) has revolutionized optical microscopy by exceeding the diffraction limit and revealing previously unattainable nanoscale details of cellular structures and molecular dynamics. This super‐resolution imaging capability relies on fluorophore photoswitching, which is crucial for optimizing the imaging conditions and accurately determining the fluorophore positions. To understand the general on and off photoswitching mechanisms of single dye molecules, various photoswitching reagents were evaluated. Systematic measurement of the single‐molecule‐level fluorescence on and off rates (kon and koff) in the presence of various photoswitching reagents and theoretical calculation of the structure of the photoswitching reagent‐fluorophore pair indicated that the switch‐off mechanism is mainly determined by the nucleophilicity of the photoswitching reagent, and the switch‐on mechanism is a two‐photon‐induced dissociation process, which is related to the power of the illuminating laser and bond dissociation energy of this pair. This study contributes to a broader understanding of the molecular photoswitching mechanism in SMLM imaging and provides a basis for designing improved photoswitching reagents with potential applications extending to materials science and chemistry.