Real-time Imaging Research Articles

Application value of surgical navigation system based on deep learning and mixed reality for guiding puncture in percutaneous nephrolithotomy: a retrospective study.

This study was conducted to investigate the clinical value of a navigation system based on deep learning and mixed reality for the treatment of kidney stones with percutaneous nephrolithotomy (PNL), and to improve its theoretical basis for the treatment of kidney stones. The data of 136 patients with kidney stones from October 2021 to December 2023 were retrospectively analyzed. All patients underwent PNL, and were categorized into a control group (Group 1) and a surgical navigation group (Group 2) according to puncture positioning method. Preoperative computed tomography (CT) was performed in both groups. In group 1, procedures were performed under standard ultrasound guidance. PNL was performed with navigation system fused with ultrasound to guide percutaneous puncture in group 2. The baseline information and procedural characteristics of both groups were compared. PNL was successfully performed in both groups. No significant difference was found in the baseline date between the two groups. In group 2, real-time ultrasound images could be accurately matched with CT images with the aid of navigation system. The success rate of single puncture, puncture time, and decrease in hemoglobin were significantly improved in group 2 compared to group 1. (p < 0.05). The application of navigation system based on deep learning and mixed reality in PNL for kidney stones allows for real-time intraoperative navigation, with acceptable accuracy and safety. Most importantly, this technique is easily mastered, particularly by novice surgeons in the field of PNL.

Online self-evaluation of fMRI-based neurofeedback performance.

This study explores the subjective evaluation of supplementary motor area (SMA) regulation performance in a real-time functional magnetic resonance imaging neurofeedback (fMRI-NF) task. In fMRI-NF, people learn how to self-regulate their brain activity by performing mental actions to achieve a certain target level (TL) of blood-oxygen-level-dependent (BOLD) activation. Here, we studied two types of self-evaluation: performance predictions and perceived confidence in the prediction judgement. Participants completed three sessions of SMA regulation in a 7 T fMRI scanner, performing a mental drawing task. During each trial, they modulated their imagery strategy to achieve one of two different levels of SMA activation and reported a performance prediction and their confidence in the prediction before receiving delayed BOLD-activation feedback. Results show that participants' performance predictions improved with learning throughout the three sessions, and that these improvements were not driven exclusively by their knowledge of previous performance. Confidence reports on the other hand showed no change throughout training and did not correlate with better and worse predictions. In addition to shedding light on mechanisms of internal self-evaluation during neurofeedback training, these results also point to a dissociation between predictions of performance and confidence reports in the presence of feedback. This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Inducing representational change in the hippocampus through real-time neurofeedback.

When you perceive or remember something, other related things come to mind, affecting how these competing items are subsequently perceived and remembered. Such behavioural consequences are believed to result from changes in the overlap of neural representations of these items, especially in the hippocampus. According to multiple theories, hippocampal overlap should increase (integration) when there is high coactivation between cortical representations. However, prior studies used indirect proxies for coactivation by manipulating stimulus similarity or task demands. Here, we induce coactivation in visual cortex more directly using closed-loop neurofeedback from real-time functional magnetic resonance imaging (fMRI). While viewing one object, participants were rewarded for activating the representation of another object as strongly as possible. Across multiple real-time fMRI sessions, participants succeeded in using this neurofeedback to increase coactivation. Compared with a baseline of untrained objects, this protocol led to memory integration in behaviour and the brain: the trained objects became harder for participants to discriminate behaviourally in a categorical perception task and harder to discriminate neurally from patterns of fMRI activity in their hippocampus as a result of losing unique features. These findings demonstrate that neurofeedback can be used to alter and combine memories.This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Nonstoichiometry Promoted Solventless Recrystallization of a Thick and Compact CsPbBr3 Film for Real-Time Dynamic X-Ray Imaging.

Inorganic CsPbBr3 perovskite emerges as a promising material for the development of next-generation X-ray detectors. However, the formation of a high-quality thick film of CsPbBr3 has been challenging due to the low solubility of its precursor and its high melting point. To address this limitation, a nonstoichiometry approach is taken that allows lower-temperature crystallization of the target perovskite under the solventless condition. This approach capitalizes on the presence of excess volatile PbBr2 within the CsPbBr3 film, which induces melting point depression and promotes recrystallization of CsPbBr3 at a temperature much lower than its melting point concomitant with the escape of PbBr2. Consequently, thick and compact films of CsPbBr3 are formed with grains ten times larger than those in the pristine films. The resulting X-ray detector exhibits a remarkable sensitivity of 4.2 × 104 µC Gyair -1 cm-2 and a low detection limit of 136 nGyair s-1, along with exceptional operational stability. Notably, the CsPbBr3-based flat-panel detector achieves a high resolution of 0.65 lp pix-1 and the first demonstration of real-time dynamic X-ray imaging for perovskite-based devices.

Regulation of craving for real-time fMRI neurofeedback based on individual classification.

In previous real-time functional magnetic resonance imaging neurofeedback (rtfMRI-NF) studies on smoking craving, the focus has been on within-region activity or between-region connectivity, neglecting the potential predictive utility of broader network activity. Moreover, there is debate over the use and relative predictive power of individual-specific and group-level classifiers. This study aims to further advance rtfMRI-NF for substance use disorders by using whole-brain rtfMRI-NF to assess smoking craving-related brain patterns, evaluate the performance of group-level or individual-level classification (n = 31) and evaluate the performance of an optimized classifier across repeated NF runs. Using real-time individual-level classifiers derived from whole-brain support vector machines, we found that classification accuracy between crave and no-crave conditions and between repeated NF runs increased across repeated runs at both individual and group levels. In addition, individual-level accuracy was significantly greater than group-level accuracy, highlighting the potential increased utility of an individually trained whole-brain classifier for volitional control over brain patterns to regulate smoking craving. This study provides evidence supporting the feasibility of using whole-brain rtfMRI-NF to modulate smoking craving-related brain responses and the potential for learning individual strategies through optimization across repeated feedback runs. This article is part of the theme issue 'Neurofeedback: new territories and neurocognitive mechanisms of endogenous neuromodulation'.

Learning infrared degradations for coherent visible image fusion in the undecimated dual-tree complex wavelet domain

Traditional infrared (IR) and visible (VIS) image fusion methods demand identical resolution levels for source images, which can be problematic due to the inherent low-resolution nature of IR imagery. In this paper, we introduce an innovative image fusion approach that harmonizes resolution across IR-VIS source images, leading to the generation of fused images with higher resolution. We employ a convolutional neural network model to recover the real-time image degradations in IR data, with a particular focus on super-resolution through the multi degradation resolution enhancement network (MDREN). We adopt undecimated dual-tree complex wavelet transform (UDT-CWT) in our fusion process due to its near shift invariance and better directionality capabilities. This results in coherent information of the fused images with minimized noise and loss. Experiments employing five image quality assessment measures are used to compare the proposed method to nine state-of-the-art approaches and show its efficacy.

Ultrasensitive dim-light neuromorphic vision sensing via momentum-conserved reconfigurable van der Waals heterostructure

Reconfigurable phototransistors featuring bipolar photoresponses are favorable for manipulating high-performance neuromorphic vision sensory. Here, we present a momentum-conserved reconfigurable phototransistor based on the van der Waals heterojunction between methylammonium lead iodide perovskite and two-dimensional Bi2O2Se semiconductor, which exhibits a synergistic interplay of interband hot-carrier transitions and reconfigurable heterointerface band alignments, eventually achieving the ultrahigh bipolar optoelectronic performances with the photoresponsivity of 6×107 AW−1, accompanied by the specific detectivity of 5.2×1011 Jones, and the dynamic range of 110 dB. Moreover, A 3×3 heterotransistor array is fabricated to perform in-sensor analog multiply-accumulate operations even under the challenging dim illumination of 0.1 μWcm−2 that comparable to natural moonlight. The reconfigurable heterotransistor array can be further adopted to enhance the traffic-light detection under dim-light conditions. Our advancement in momentum-conserved reconfigurable heterotransistor signifies a leap forward in real-time, energy-efficient, and low-light image processing for neuromorphic vision sensors.

An adaptive parameter decoupling algorithm-based image reconstruction model (ADAIR) for rapid golden-angle radial DCE-MRI

Objective. The acceleration of magnetic resonance imaging (MRI) acquisition is crucial for both clinical and research applications, particularly in dynamic MRI. Existing compressed sensing (CS) methods, despite being effective for fast imaging, face limitations such as the need for incoherent sampling and residual noise, which restrict their practical use for rapid MRI.Approach. To overcome these challenges, we propose a novel image reconstruction framework that integrates the MRI physical model with a flexible, self-adjusting, decoupling data-driven model. We validated this method through experiments using both simulated andin vivodynamic contrast-enhanced MRI datasets.Main results. The experimental results demonstrate that the proposed framework achieves high spatial and temporal resolution reconstructions. Additionally, when compared to state-of-the-art image reconstruction approaches, our method significantly enhances acceleration capabilities, enabling sparse and rapid imaging with high resolution.Significance. Our proposed framework offers a promising solution for real-time imaging and image-guided radiation therapy applications by providing superior performance in achieving high spatial and temporal resolution reconstructions, thus addressing the limitations of existing CS schemes.

An effectual underwater image enhancement framework using adaptive trans-resunet ++ with attention mechanism

ABSTRACT The intricacy of the underwater setting makes it difficult for optical lenses to capture clear underwater photos without haze and colour distortion. Some studies use domain adaptation and transfer learning to address this issue, they aim to reduce the latent mismatch between composition and real-world data, making the space of latent data difficult to read and impractical to control. The background light is a crucial component of the decaying paradigm that directly impacts how well images are enhanced. Thus, to improve the quality of the images over the underwater, new deep-learning techniques are being designed in this paper. Here, the Adaptive Trans-ResUnet++ with Attention Mechanism-based model performs the real-time underwater image enhancement process. In addition, a novel Random Enhanced Artificial Gorilla Troops Optimizer algorithm model is used for optimising the parameters over the given model to further enhance the given model’s performance. A diverse quantitative and qualitative validation is also carried out to learn the enhancement of underwater image quality. The enhanced underwater image may be also useful in the underwater object detection process. Thus, the enhanced images obtained from the developed model are compared with the existing techniques to confirm the efficacy of the suggested underwater image enhancement process.

NIR-II Fluorescent Nanotheranostics with a Switchable Irradiation Mode for Immunogenic Sonodynamic Therapy.

Nanotheranostics, which integrate diagnostic and therapeutic functionalities, offer significant potential for tumor treatment. However, current nanotheranostic systems typically involve multiple molecules, each providing a singular diagnostic or therapeutic function, leading to challenges such as complex structural composition, poor targeting efficiency, lack of spatiotemporal control, and dependence on a single therapeutic modality. This study introduces NPRBOXA, a nanoparticle functionalized with surface-bound cRGD for targeted delivery to αvβ3/αvβ5 receptors on tumor cells, achieving theranostic integration by sequentially switching its irradiation modes. Under 808nm laser irradiation, NPRBOXA emits NIR-II fluorescence, which aids in identifying the nanoparticle's location and fluorescence intensity, thereby determining the optimal treatment window. Following this, the irradiation mode switches to ultrasound irradiation at the optimal treatment window. Ultrasound irradiation induces NPRBOXA to generate reactive oxygen species, promoting the reduction of OXA-IV to OXA-II, which in turn triggers immunogenic cell death. This mechanism enables a combination of sonodynamic therapy, chemotherapy, and immunotherapy for tumor treatment. The versatile design of NPRBOXA holds promise for advancing precision oncology through enhanced therapeutic efficacy and real-time imaging guidance.

Real-time acousto-optic imaging using high-peak-power long-pulsed illumination.

Acousto-optic imaging (AOI) of absorbing objects embedded in highly scattering media remains challenging, since the detectable signal suitable for image reconstruction is weak. To increase the detected signals to a level required by live biological applications, we designed a high-peak-power quasi-continuous laser source based on the coherent combination of two pulsed amplifiers, delivering 100-µs-long pulses with a 9 W peak power at a 100 Hz repetition rate while maintaining an average power below 100 mW; jointly used with a digital holographic detection that maximizes the amount of collected signal, we demonstrate for the first time to our knowledge the optical imaging of 2-cm-thick highly scattering media (µs' ∼ 10 cm-1) at near-video frame rate (0.2 Hz) using the Fourier transform-AOI.

Real-Time 3D Imaging and Inhibition Analysis of Human Serum Amyloid A Aggregations Using Quantum Dots.

Serum amyloid A (SAA) is one of the most important precursor amyloid proteins discovered during the study of amyloidosis, but its underlying aggregation mechanism has not yet been well elucidated. Since SAA aggregation is a key step in the pathogenesis of AA amyloidosis, amyloid inhibitors can be used as a tool to study its pathogenesis. Previously, we reported a novel microliter-scale high-throughput screening (MSHTS) system for screening amyloid β (Aβ) aggregation inhibitors based on quantum dot (QD) fluorescence imaging technology. In this study, we report the aggregation of human SAA (hSAA) in phosphate-buffered saline, in which we successfully visualized hSAA aggregation by QD using fluorescence microscopy and confocal microscopy. Two-dimensional and three-dimensional image analyses showed that most aggregations were observed at 40 μM hSAA, which was the optimal aggregation concentration in vitro. The accuracy of this finding was verified by a Thioflavin T assay. The transmission electron microscopy results showed that QD uniformly bound to hSAA aggregation. hSAA aggregation inhibitory activity was also evaluated by rosmarinic acid (RA). The results showed that RA, which is a compound with high inhibitory activity against Aβ aggregation, also exhibited high inhibitory activity against 40 μM hSAA. These results indicate that the MSHTS system is an effective tool for visualizing hSAA aggregation and for screening highly active inhibitors.

An automatic tracking method to measure the mandibula movement during real time MRI

Mandibular movement is complex and individual due to variations in the temporomandibular joint (TMJ). Consequently, patient-centered dentistry should incorporate patients’ specific anatomy and condylar function in treatment planning. Real-time magnetic resonance imaging (rt-MRI) visualizes relevant structures and tracks mandibular movement. However, current assessments rely on qualitative observations or time-consuming manual tracking, lacking reliability. This study developed an automatic tracking algorithm for mandibular movement in rt-MRI using least mean square registration (LMS) and compared it to manual tracking (MT) during mouth opening. Ten participants with skeletal class I underwent rt-MRI (10 frames/s). The same operator tracked the condylar pathway for the two methods, setting 2000 landmarks (2 landmarks x100 frames x10 participants) for MT and 210 landmarks (3 landmarks x7 frames x10 participants) for LMS. Time required, superimposition error, and the distance between tracked condylar pathways were compared between methods. LMS tracking was 76% faster and showed significantly better superimposition (0.0289 ± 0.0058) than MT (0.059 ± 0.0145) (p = 0.002). During one-third of the movement, the pathways tracked by both methods were more than 1 mm and 1° apart. These findings highlight the benefits of automatic condylar movement tracking in rt-MRI, laying the groundwork for more objective and quantitative observation of TMJ function.

INTRAOPERATIVE MRI FOR BRAIN TUMOURS: A 15-YEAR RETROSPECTIVE REVIEW AT A SINGLE AUSTRALIAN INSTITUTION

Abstract AIMS Maximal safe resection is the objective of most neuro-oncological operations. The extent of resection can be maximized using imaging updated intraoperatively, including intraoperative MRI (iMRI) theatre, which may translate to improved overall survival. This study aimed to present our 15-year iMRI theatre experience and review its utility. METHOD A retrospective cohort analysis was conducted to include all intracranial tumour operations performed in the iMRI theatre (2007–2022). Cases were classified according to their histopathology: low- and high-grade gliomas (LGG and HGG respectively), meningiomas, pituitary adenomas, and metastases. Demographic, clinical, and histological data was extracted from SurgiNet and analyzed by using the WHO 2021 classification. The impact of intraoperative MRI on operative outcomes was reviewed. RESULTS A total of 478 neuro-oncology cases were identified, consisting of 194 pituitary adenomas (n= 69 cases via microscope), 162 HGG, 108 LGG (n= 7 cases for biopsy), 10 cerebral metastases, and 4 meningiomas. Intraoperative MRI theatre utilization averaged 41 cases per year throughout our 15-year experience. The median age was 47 years old (range 35-59 yo.) with a slight predominance in the female population (n= 298, 52%). We identified 85 (17.8%) re-do cases, which include: pituitary adenoma (n= 15, 17.6%), glioma (n= 68, 80%), and intracranial metastasis (n= 2, 2.4%). Of the 10 intracranial metastases, melanoma consisted of 5, lung carcinoma of 3, and renal cell- and colorectal-carcinoma represented 1 case, respectively. Intraoperative MRI confirmed complete resection in all meningioma cases. CONCLUSION In this series, iMRI has been utilised mostly for pituitary adenoma resection and gliomas. It provides near realtime intraoperative navigation and imaging to maximise the extent of resection, prevent injury to important structures and improve patient outcomes in tumour resections.

Enhancing precision targeting of ovarian cancer tumor cells in vivo through extracellular vesicle engineering.

Extracellular vesicles (EVs) function as natural mediators of intercellular communication, secreted by cells to facilitate cell-cell signaling. Due to their low toxicity, immunogenicity, biodegradability, and potential to encapsulate therapeutic drugs, EVs hold significant therapeutic promise. Nevertheless, their limited targeting ability often diminishes their therapeutic impact. Therefore, enhancing EVs by incorporating targeting units onto their membranes could bolster their targeting capabilities, enabling them to accumulate in specific cells and tissues. In this study, we engineered EVs to fuse ephrin-B2 with the EV membrane protein LAMP2b. This modification aimed to direct the engineered EVs toward the ephrin-B4 receptor expressed on the surface of ovarian cancer cells. The engineered EVs retained their inherent properties, including size, expression of EV membrane proteins, and morphology, upon isolation. In vitro experiments using real-time imaging revealed that EVs engineered with the ephrin-B2 ligand exhibited substantial internalization and uptake by ovarian cancer cells, in stark contrast to native EVs. In vivo, the engineered EVs carrying the ephrin-B2 ligand effectively targeted ovarian cancer cells, surpassing the targeting efficiency of control EVs. This innovative approach establishes a novel targeting system, enhancing the uptake of EVs by ovarian cancer cells. Our findings underscore the potential of using EVs to target cancer cells, thereby enhancing the effectiveness of anti-cancer therapies while minimizing off-target effects and toxicity in normal cells and organs.

Inclusive and Accurate Clinical Diagnostics Using Intelligent Computation and Smartphone Imaging.

Smartphone-based colorimetry has been widely applied in clinical analysis, although significant challenges remain in its practical implementation, including the need to consider biases introduced by the ambient imaging environment, which limit its potential within a clinical decision pathway. In addition, most commercial devices demonstrate variability introduced by manufacturer-to-manufacturer differences. Here, we undertake a systematic characterization of the potential imaging interferences that lead to this limited performance in conventional smartphones and, in doing so, provide a comprehensive new understanding of smartphone color imaging. Through derivation of a strongly correlated parameter for sample quantification, we enable real-time imaging, which for the first time, takes the first steps to turning the mobile phone camera into an analytical instrument - irrespective of model, software, and the operating systems used. We demonstrate clinical applicability through the imaging of patients' skin, enabling rapid and convenient diagnosis of cyanosis and measurement of local oxygen concentration to a level that unlocks clinical decision-making for monitoring cardiovascular disease and anemia. Importantly, we show that our solution also accounts for the differences in individuals' skin tones as measured across the Fitzpatrick scale, overcoming potential clinically significant errors in current optical oximetry.

Hierarchical Nanostructures as Acoustically Manipulatable Multifunctional Agents in Dynamic Fluid Flow.

Acoustic waves provide a biocompatible and deep-tissue-penetrating tool suitable for contactless manipulation in in vivo environments. Despite the prevalence of dynamic fluids within the body, previous studies have primarily focused on static fluids, and manipulatable agents in dynamic fluids are limited to gaseous core-shell particles. However, these gas-filled particles face challenges in fast-flow manipulation, complex setups, design versatility, and practical medical imaging, underscoring the need for effective alternatives. In this study, flower-like hierarchical nanostructures (HNS) into microparticles (MPs) are incorporated, and demonstrated that various materials fabricated as HNS-MPs exhibit effective and reproducible acoustic trapping within high-velocity fluid flows. Through simulations, it is validated that the HNS-MPs are drawn to the focal point by acoustic streaming and form a trap through secondary acoustic streaming at the tips of the nanosheets comprising the HNS-MPs. Furthermore, the wide range of materials and modification options for HNS, combined with their high surface area and biocompatibility, enable them to serve as acoustically manipulatable multimodal imaging contrast agents and microrobots. They can perform intravascular multi-trap maneuvering with real-time imaging, purification of wastewater flow, and highly-loaded drug delivery. Given the diverse HNS materials developed to date, this study extends their applications to acoustofluidic and biomedical fields.

Designer bright and fast CsPbBr3 perovskite nanocrystal scintillators for high-speed X-ray imaging

Bright and fast scintillators are highly crucial for high-speed X-ray imaging in the medical diagnostic radiology including angiography and cardiac computed tomography. The CsPbBr3 nanocrystal scintillator featuring a nanosecond radioluminescence decay time is a promising candidate. However, it suffers from a substantial photon self-absorption limiting the light output, and being bright and fast simultaneously is difficult. Here we design and in-situ synthesize multi-site ZnS(Ag)-CsPbBr3 heterostructures to modulate the bright and fast features of scintillators. We find external energy from ZnS(Ag) can effectively transfer to CsPbBr3 based on the non-radiative Förster resonance energy transfer, resulting in a light yield of 40,000 photons MeV−1. By combing a radioluminescence decay time of 36 ns and a spatial resolution of 30 lp mm−1, the scintillator enables high-speed X-ray imaging at 200 frames per second. This study showcases the structure design is significant for obtaining bright and fast perovskite scintillators for the real-time X-ray imaging.

MONITTR allows real-time imaging of transcription and endogenous proteins in C. elegans.

Maximizing cell survival under stress requires rapid and transient adjustments of RNA and protein synthesis. However, capturing these dynamic changes at both single-cell level and across an organism has been challenging. Here, we developed a system named MONITTR (MS2-embedded mCherry-based monitoring of transcription) for real-time simultaneous measurement of nascent transcripts and endogenous protein levels in C. elegans. Utilizing this system, we monitored the transcriptional bursting of fasting-induced genes and found that the epidermis responds to fasting by modulating the proportion of actively transcribing nuclei and transcriptional kinetics of individual alleles. Additionally, our findings revealed the essential roles of the transcription factors NHR-49 and HLH-30 in governing the transcriptional kinetics of fasting-induced genes under fasting. Furthermore, we tracked transcriptional dynamics during heat-shock response and ER unfolded protein response and observed rapid changes in the level of nascent transcripts under stress conditions. Collectively, our study provides a foundation for quantitatively investigating how animals spatiotemporally modulate transcription in various physiological and pathological conditions.

Real-Time Imaging of Calcium Dynamics in Human Sperm After Precise Single-Cell Stimulation.

Calcium signaling is a critical regulator of sperm activation and function during the processes of capacitation and fertilization. Here, we describe a combined method for calcium imaging of single, live human sperm in response to stimuli administered with a precisely targeted delivery technique. This protocol is an adaptation of techniques developed for studies of murine sperm [1, 2], and enables real-time monitoring of human sperm calcium dynamics with high spatiotemporal resolution and concurrent detection of acrosome exocytosis (AE), a functional endpoint of sperm capacitation and requirement for physiological fertilization.The described imaging technique provides a valuable tool for exploration of calcium regulation in human sperm, which is essential to answer important questions and knowledge gaps regarding the link between calcium dynamics, AE, and fertilization. The versatility of this technique can be amplified through use of various indicator dyes or integration with pharmacological strategies such as pre-treating sperm with inhibitors or activators targeting specific receptors, channels, or intracellular signaling pathways of interest. Beyond fundamental inquiries into sperm physiology, this method can also be applied to assess the impact of potential contraceptive compounds on calcium signaling, AE, and membrane integrity.