Multichannel Image Research Articles

OPTICAM triple-channel astronomical image acquisition control software and external triggering synchronization system

We present the development of software to control the multichannel image acquisition process and the design of an external timing module (ETM) for the OPTICAM system (Castro et al., 2019). The ETM is used to generate high-speed synchronized pulses for OPTICAM’s three different sCMOS cameras and is based on an ESP32-S2 micro-controller. The graphical user interface (GUI) of this instrument is also described, which allows users to enter the configuration parameters of the observations and execute actions such as starting and stopping image acquisition sequences or initializing cameras. 3 mini PCs are in charge of reading each one of the cameras, respectively. The software and hardware tools presented here have been successfully implemented at the Observatorio Astronómico Nacional (OAN-SPM), Mexico. By analyzing the recorded image sequences, it is possible to perform high-precision multi-channel photometric measurements of astrophysical objects that exhibit high temporal variability, revealing information about periodicities, inter-band delays, and quasi-periodic oscillations.

Hyperspectral Imaging with Active Illumination: A Theoretical Study on the Use of Incandescent Lamp and Variable Filament Temperature

In this study, we introduce a novel hyperspectral imaging approach that leverages variable filament temperature incandescent lamps for active illumination, coupled with multi-channel image acquisition, and provide a comprehensive characterization of the approach. Our methodology simulates the imaging process, encompassing spectral illumination ranging from 400 to 700 nm at varying filament temperatures, multi-channel image capture, and hyperspectral image reconstruction. We present an algorithm for spectrum reconstruction, addressing the inherent challenges of this ill-posed inverse problem. Through a rigorous sensitivity analysis, we assess the impact of various acquisition parameters on the accuracy of reconstructed spectra, including noise levels, temperature steps, filament temperature range, illumination spectral uncertainties, spectral step sizes in reconstructed spectra, and the number of detected spectral channels. Our simulation results demonstrate the successful reconstruction of most spectra, with Root Mean Squared Errors (RMSE) below 5%, reaching as low as 0.1% for specific cases such as black color. Notably, illumination spectrum accuracy emerges as a critical factor influencing reconstruction quality, with flat spectra exhibiting higher accuracy than complex ones. Ultimately, our study establishes the theoretical grounds of this innovative hyperspectral approach and identifies optimal acquisition parameters, setting the stage for future practical implementations.

Multimodal Deep Learning for Robust Road Attribute Detection

Automatic inference of missing road attributes (e.g., road type and speed limit) for enriching digital maps has attracted significant research attention in recent years. A number of machine learning-based approaches have been proposed to detect road attributes from GPS traces, dash-cam videos, or satellite images. However, existing solutions mostly focus on a single modality without modeling the correlations among multiple data sources. To bridge this gap, we present a multimodal road attribute detection method, which improves the robustness by performing pixel-level fusion of crowdsourced GPS traces and satellite images. A GPS trace is usually given by a sequence of location, bearing, and speed. To align it with satellite imagery in the spatial domain, we render GPS traces into a sequence of multi-channel images that simultaneously capture the global distribution of the GPS points, the local distribution of vehicles’ moving directions and speeds, and their temporal changes over time, at each pixel. Unlike previous GPS-based road feature extraction methods, our proposed GPS rendering does not require map matching in the data preprocessing step. Moreover, our multimodal solution addresses single-modal challenges such as occlusions in satellite images and data sparsity in GPS traces by learning the pixel-wise correspondences among different data sources. On top of this, we observe that geographic objects and their attributes in the map are not isolated but correlated with each other. Thus, if a road is partially labeled, then the existing information can be of great help on inferring the missing attributes. To fully use the existing information, we extend our model and discuss the possibilities for further performance improvement when partially labeled map data is available. Extensive experiments have been conducted on two real-world datasets in Singapore and Jakarta. Compared with previous work, our method is able to improve the detection accuracy on road attributes by a large margin.

A Robust Memristor-Enhanced Polynomial Hyper-Chaotic Map and Its Multi-Channel Image Encryption Application.

Nowadays, the utilization of memristors to enhance the dynamical properties of chaotic systems has become a popular research topic. In this paper, we present the design of a novel 2D memristor-enhanced polynomial hyper-chaotic map (2D-MPHM) by utilizing the cross-coupling of two TiO2 memristors. The dynamical properties of the 2D-MPHM were investigated using Lyapunov exponents, bifurcation diagrams, and trajectory diagrams. Additionally, Kolmogorov entropy and sample entropy were also employed to evaluate the complexity of the 2D-MPHM. Numerical analysis has demonstrated the superiority of the 2D-MPHM. Subsequently, the proposed 2D-MPHM was applied to a multi-channel image encryption algorithm (MIEA-MPHM) whose excellent security was demonstrated by key space, key sensitivity, plaintext sensitivity, information entropy, pixel distribution, correlation analysis, and robustness analysis. Finally, the encryption efficiency of the MIEA-MPHM was evaluated via numerous encryption efficiency tests. These tests demonstrate that the MIEA-MPHM not only possesses excellent security but also offers significant efficiency advantages, boasting an average encryption rate of up to 87.2798 Mbps.

Detection of cell-free histones in the cerebrospinal fluid of pediatric central nervous system malignancies by imaging flow cytometry.

Introduction: Pediatric brain tumours (PBT) are one of the most common malignancies during childhood, with variable severity according to the location and histological type. Certain types of gliomas, such a glioblastoma and diffuse intrinsic pontine glioma (DIPG), have a much higher mortality than ependymoma and medulloblastoma. Early detection of PBT is essential for diagnosis and therapeutic interventions. Liquid biopsies have been demonstrated using cerebrospinal fluid (CSF), mostly restricted to cell free DNA, which display limitations of quantity and integrity. In this pilot study, we sought to demonstrate the detectability and robustness of cell free histones in the CSF. Methods: We collected CSF samples from a pilot cohort of 8 children with brain tumours including DIPG, medulloblastoma, glioblastoma, ependymoma and others. As controls, we collected CSF samples from nine children with unrelated blood malignancies and without brain tumours. We applied a multichannel flow imaging approach on ImageStream(X) to image indiviual histone or histone complexes on different channels. Results: Single histones (H2A, macroH2A1.1, macroH2A1.2 H2B, H3, H4 and histone H3 bearing the H3K27M mutation), and histone complexes are specifically detectable in the CSF of PBT patients. H2A and its variants macroH2A1.1/macroH2A1/2 displayed the strongest signal and abundance, together with disease associated H3K27M. In contrast, mostly H4 is detectable in the CSF of pediatric patients with blood malignancies. Discussion: In conclusion, free histones and histone complexes are detectable with a strong signal in the CSF of children affected by brain tumours, using ImageStream(X) technology and may provide additive diagnostic and predictive information.

Lifting-based variational multiclass segmentation algorithm: Design, convergence analysis, and implementation with applications in medical imaging

We propose, analyze and realize a variational multiclass segmentation scheme that partitions a given image into multiple regions exhibiting specific properties. Our method determines multiple functions that encode the segmentation regions by minimizing an energy functional combining information from different channels. Multichannel image data can be obtained by lifting the image into a higher dimensional feature space using specific multichannel filtering or may already be provided by the imaging modality under consideration, such as an RGB image or multimodal medical data. Experimental results show that the proposed method performs well in various scenarios. In particular, promising results are presented for two medical applications involving classification of brain abscess and tumor growth, respectively. As main theoretical contributions, we prove the existence of global minimizers of the proposed energy functional and show its stability and convergence with respect to noisy inputs. In particular, these results also apply to the special case of binary segmentation, and these results are also novel in this particular situation.

Encoding Hole-Particle Information in the Multi-Channel MolOrbImage for Machine-Learned Excited-State Energies of Large Photofunctional Materials.

We present a novel class of one-electron multi-channel molecular orbital images (MolOrbImages) designed for the prediction of excited-state energetics in conjunction with the state-of-the-art VGG-type machine-learning architecture. By representing hole and particle states in the excitation process as channels of MolOrbImages, the revised VGG model achieves excellent prediction accuracy for both low-lying singlet and triplet states, with mean absolute errors (MAEs) of <0.08 and <0.1 eV for QM9 molecules and large photofunctional materials with up to 560 atoms, respectively. Remarkably, the model demonstrates exceptional performance (MAE < 1 kcal/mol) for the T1 state of QM9 molecules, making it a non-system-specific model that approaches chemical accuracy. The general rules attained, for instance, the improved performance with well-defined MO energies and the reduced overfitting concern via the inclusion of physically insightful hole-particle information, provide invaluable guidelines for the further design of orbital-based descriptors targeting molecular excited states.

Design and simulation of multilayer coatings for a multi-channel Wolter-like x-ray imager with large field of view and high resolution.

X-ray diagnostics are key instruments for understanding the physics behind inertial confinement fusion experiments. We report on the multilayer design optimization for the Toroidal X-ray Imager (TXI), a hard x-rays microscope instrument designed by Commissariat à l'énergie atomique (CEA) and Laboratoire Charles Fabry (LCF) to be installed on the National Ignition Facility. TXI includes six channels designed for three different energy bands centered on 8.7, 13, and 17.5keV. Each channel is made up of two toroidal mirrors arranged in a Wolter-like configuration. The required field of view is 800 × 400 µm2, and the resolution should be better than 5 µm. In addition, we seek to estimate the spatial distribution of the temperature, which requires no spectral overlap of the different energy bands and a good spectral homogeneity of the image produced. The development of the multilayer coatings was performed in a two-step method. First, the coatings were optimized to obtain proper energy bands. Then, an x-ray tracing code was used to calculate the integrated optical response of each channel and adjust the response of the mirror to fulfill the requirements. To fulfill all the specifications, we propose an original design using a combination of two aperiodic coatings, one with a narrow bandwidth and the other one with a larger bandwidth.

A study on the construction of multi-channel GPR plan view images for underground object detection

A study on the construction of multi-channel GPR plan view images for underground object detection

Self-confocal NIR-II fluorescence microscopy for multifunctional in vivo imaging

Fluorescence imaging in the second near-infrared window (NIR-II, 900–1880[Formula: see text]nm) with less scattering background in biological tissues has been combined with the confocal microscopic system for achieving deep in vivo imaging with high spatial resolution. However, the traditional NIR-II fluorescence confocal microscope with separate excitation focus and detection pinhole makes it possess low confocal efficiency, as well as difficultly to adjust. Two types of upgraded NIR-II fluorescence confocal microscopes, sharing the same pinhole by excitation and emission focus, leading to higher confocal efficiency, are built in this work. One type is fiber-pinhole-based confocal microscope applicable to CW laser excitation. It is constructed for fluorescence intensity imaging with large depth, high stabilization and low cost, which could replace multiphoton fluorescence microscopy in some applications (e.g., cerebrovascular and hepatocellular imaging). The other type is air-pinhole-based confocal microscope applicable to femtosecond (fs) laser excitation. It can be employed not only for NIR-II fluorescence intensity imaging, but also for multi-channel fluorescence lifetime imaging to recognize different structures with similar fluorescence spectrum. Moreover, it can be facilely combined with multiphoton fluorescence microscopy. A single fs pulsed laser is utilized to achieve up-conversion (visible multiphoton fluorescence) and down-conversion (NIR-II one-photon fluorescence) excitation simultaneously, extending imaging spectral channels, and thus facilitates multi-structure and multi-functional observation.

Four-framed X-ray imaging crystal spectrometer for time-resolved laser plasma diagnostics

X-ray imaging crystal spectrometers with high spectral and spatial resolutions are the important diagnostic tools for acquiring high-quality plasma information in the study of laser–matter interactions. In this paper we propose a new multi-channel imaging crystal spectrometer with coupling X-ray framing camera to record the dynamic evolution of a laser-induced plasma: a four-channel imaging mica crystal spectrometer with a spatial resolution of 40–60 μm for diagnosing the aluminum (Al) plasma targets was developed. Calibration experiments revealed that the spectral resolution is higher than 1700 at Al Kα3 characteristic lines (1496.4eV). Further, we established an aiming method based on the positioning ball and two monitoring CCDs. The optical design, mechanical design, and experimental measurements of the four-channel imaging mica crystal spectrometer are discussed.

An edge-preserving total nuclear variation minimization algorithm in EPR image reconstruction

ObjectiveCurrently, slow scanning speed has become a technical bottleneck limiting the development of electron paramagnetic resonance imaging (EPRI), a novel oxygen imaging modality. Image reconstruction from sparse-view projections is an important imaging configuration to achieve fast scanning in EPRI. However, EPR images reconstructed from sparse-view projections using traditional analytic algorithms suffer from sparse artifacts, causing severe image degradation. In this work, we aim to use an optimization-based approach to achieve high-precision sparse reconstruction for EPRI. Methods. Inspired by the success of total nuclear variation (TVN) for denoising multi-channel spectral CT images and adaptive-weighted total variation (awTV) for improving the edge-preserving performance of TV, we proposed an edge-preserving total nuclear variation (EPTVN) minimization algorithm for sparse reconstruction in EPRI. The EPTVN combines the advantage of TVN in modelling structural similarity and low-rank prior between multi-channel images for denoising with the advantage of awTV for edge-preserving, which can better protect the edge structure of images while suppressing sparse artifacts. Results. We designed two numerical phantoms with different properties, a piecewise-constant phantom and a gradient phantom, and a physical phantom, and then performed inverse crime, simulation study and real-data study. Experimental results show that the EPTVN minimization algorithm can reconstruct more accurate EPR images from sparse-view projections for both the numerical and physical phantoms. Significance. The proposed method can effectively suppress sparse artifacts and protect edge structures of reconstructed images, thus achieving high-precision sparse-view reconstruction for EPRI. The insights of this work can also be applied to multi-channel image denoising tasks.

SiGra: single-cell spatial elucidation through an image-augmented graph transformer

Recent advances in high-throughput molecular imaging have pushed spatial transcriptomics technologies to subcellular resolution, which surpasses the limitations of both single-cell RNA-seq and array-based spatial profiling. The multichannel immunohistochemistry images in such data provide rich information on the cell types, functions, and morphologies of cellular compartments. In this work, we developed a method, single-cell spatial elucidation through image-augmented Graph transformer (SiGra), to leverage such imaging information for revealing spatial domains and enhancing substantially sparse and noisy transcriptomics data. SiGra applies hybrid graph transformers over a single-cell spatial graph. SiGra outperforms state-of-the-art methods on both single-cell and spot-level spatial transcriptomics data from complex tissues. The inclusion of immunohistochemistry images improves the model performance by 37% (95% CI: 27–50%). SiGra improves the characterization of intratumor heterogeneity and intercellular communication and recovers the known microscopic anatomy. Overall, SiGra effectively integrates different spatial modality data to gain deep insights into spatial cellular ecosystems.

Deep learning-based pectoralis muscle volume segmentation method from chest computed tomography image using sagittal range detection and axial slice-based segmentation.

The pectoralis muscle is an important indicator of respiratory muscle function and has been linked to various parenchymal biomarkers, such as airflow limitation severity and diffusing capacity for carbon monoxide, which are widely used in diagnosing parenchymal diseases, including asthma and chronic obstructive pulmonary disease. Pectoralis muscle segmentation is a method for measuring muscle volume and mass for various applications. The segmentation method is based on deep-learning techniques that combine a muscle area detection model and a segmentation model. The training dataset for the detection model comprised multichannel images of patients, whereas the segmentation model was trained on 7,796 cases of the computed tomography (CT) image dataset of 1,841 patients. The dataset was expanded incrementally through an active learning process. The performance of the model was evaluated by comparing the segmentation results with manual annotations by radiologists and the volumetric differences between the CT image datasets of the same patients. The results indicated that the machine learning model is promising in segmenting the pectoralis major muscle, with good agreement between the automatic segmentation and manual annotations by radiologists. The training accuracy and loss values of the validation set were 0.9954 and 0.0725, respectively, and for segmentation, the loss value was 0.0579. This study shows the potential clinical usefulness of the machine learning model for pectoralis major muscle segmentation as a quantitative biomarker for various parenchymal and muscular diseases.

Binlets: Data fusion-aware denoising enables accurate and unbiased quantification of multichannel signals

As monitoring multiple signals becomes more cost-effective, combining them through a data fusion-aware denoising method can produce a more robust estimation of the underlying process. Here, we present a method based on the Haar wavelet transform that trades off resolution against accuracy based on statistical significance. By taking advantage of correlations between channels, it offers a superior performance compared to denoising each channel separately. It outperforms standard wavelet methods when the magnitude of interest in the data-fusion process involves a non-linear transformation or reduction of a multichannel signal. We demonstrate its efficacy by benchmarking our method against standard wavelet thresholding for synthetic single and multichannel time series, and a multichannel two-dimensional image. The method has a simple interpretation as an adaptive binning of the signal, and neither requires training data nor specialized hardware to run fast. In addition, a reference Python implementation is available on GitHub and PyPI, making it simple to integrate into any analysis pipeline.

Delivery of Small-Molecule Drugs and Protein Drugs by Injectable Acid-Responsive Self-Assembled COF Hydrogels for Combinatorial Lung Cancer Treatment.

Covalent organic frameworks (COFs) have revealed enormous application prospects for cancer therapeutics recently, but their assembly systems face considerable challenges, such as the codelivery of hydrophobic and hydrophilic protein drugs with different physicochemical properties for in vivo delivery and release, as well as endosomal/lysosomal escape of protein drugs. To address these issues, we leveraged the high specific surface area, lipotropism, and structural tunability of boronate ester-linked COFs (COF-1) for the construction of advanced drug delivery systems. We first encapsulated the small-molecule drug doxorubicin (DOX) into a lipophilic COF (COF-1@DOX) and immobilized the functional protein drug ribonuclease A (RNase A) on the surface of the COF (RNase A-COF-1@DOX). We then created a novel composite delivery system (RNase A-COF-1@DOX gel) by cross-linking an albumin-oxygenated hydrogel (gel) network into the pores of COFs, allowing targeted codelivery of protein and small-molecule drugs in vivo. Using in-living body and multichannel fluorescence imaging, we analyzed the in vivo codelivery of protein and small-molecule drugs in a Lewis lung carcinoma (LLC) model. Finally, we applied the RNase A-COF-1@DOX gel to treat lung cancer in mice. This study paves an avenue for constructing COF-based drug delivery systems for lung cancer treatment and holds the potential to be extended to other types of cancer for more effective and targeted therapeutic treatments.

Research on Landslide Detection in SAR Images Based on Multi-channel Change Detection Fusion

The landslides have resulted in the grave loss of human life and economic damage. Currently, emergency investigations predominantly employ manual field investigations and remote sensing interpretation. Traditional methods for assessing landslides are time-consuming and ineffective for widespread and numerous landslides. In response, many scholars have successfully employed optical remote sensing imagery to extract landslides. Nonetheless, landslide regions are typically accompanied by copious rainfall along with inclement weather conditions, such as cloudy or overcast skies, which limits the observation capabilities of optical remote sensing methods. At this juncture, the advantages of SAR systems that can penetrate clouds, rain, and fog, and work around the clock under any weather conditions are highlighted. This paper proposes a framework using multi-channel SAR images to extract landslides. The framework consists of fused multi-channel change detection images and traditional detection methods, resulting in effective landslide detection. The proposed method was verified in an experiment using landslide data triggered by an earthquake in Milin County, Tibet, on November 17, 2017.

Machine Learning Application of Two-Dimensional Fracture Properties Estimation

Fractures are substantial contributors to solute transport sedimentary systems that form pathways. The pathway formed in a fracture has two physical parameters, there are mean aperture and surface roughness. Mean aperture is the thickness of the pathway that the fluid will pass through, and surface roughness is the roughness of the fracture pathway. The two physical parameters of the fracture are important to determine since they affect the permeability value in petroleum reservoir analysis. We developed a machine learning algorithm based on the Convolutional Neural Network (CNN) to predict those two parameters. Furthermore, image processing analysis is performed to generate the datasets. The results show that the CNN algorithm shows good agreement with the reference results. In addition, the algorithms showed efficient performance in terms of computational time. CNN is a type of deep neural designed to perform analysis on multi-channel images that can classify fracture geometry. The best model was determined using a benchmark dataset with a CNN model provided by Keras. The results of experiments conducted on fracture geometry images show that the machine learning model created is able to predict the mean aperture and surface roughness values.

Machine learning prediction of tortuosity in digital rock

Physical rock property measurement is an important stage in energy exploration, both for hydrocarbons and geothermal sources. The value of physical rock properties can provide information about reservoir quality, and one of these properties is tortuosity. Tortuosity is an intrinsic property of porous materials that describes the level of complexity of the porous arrangement when a fluid passes through it. Conventionally, tortuosity values are measured through laboratory analysis and numerical simulation, but these measurements can take a long time. An alternative method for measuring tortuosity is using machine learning with a convolutional neural network (CNN). A CNN is a type of deep neural network designed to analyze multi-channel images and has been applied successfully to classification and non-linear regression problems. By training a CNN on a dataset of digital rock samples that have been simulated using numerical computation to obtain their tortuosity values, it is possible to demonstrate that CNNs can accurately predict the tortuosity of digital rock. The result is that the CNN model can predict tortuosity values with the Xception model being the most accurate with the lowest RMSE value of 0.90962.

Single-cell multimodal imaging uncovers energy conversion pathways in biohybrids.

Microbe-semiconductor biohybrids, which integrate microbial enzymatic synthesis with the light-harvesting capabilities of inorganic semiconductors, have emerged as promising solar-to-chemical conversion systems. Improving the electron transport at the nano-bio interface and inside cells is important for boosting conversion efficiencies, yet the underlying mechanism is challenging to study by bulk measurements owing to the heterogeneities of both constituents. Here we develop a generalizable, quantitative multimodal microscopy platform that combines multi-channel optical imaging and photocurrent mapping to probe such biohybrids down to single- to sub-cell/particle levels. We uncover and differentiate the critical roles of different hydrogenases in the lithoautotrophic bacterium Ralstonia eutropha for bioplastic formation, discover this bacterium's surprisingly large nanoampere-level electron-uptake capability, and dissect the cross-membrane electron-transport pathways. This imaging platform, and the associated analytical framework, can uncover electron-transport mechanisms in various types of biohybrid, and potentially offers a means to use and engineer R. eutropha for efficient chemical production coupled with photocatalytic materials.