Source Images Research Articles

Multi-Dimensional Medical Image Fusion With Complex Sparse Representation.

In the field of medical imaging, the fusion of data from diverse modalities plays a pivotal role in advancing our understanding of pathological conditions. Sparse representation (SR), a robust signal modeling technique, has demonstrated noteworthy success in multi-dimensional (MD) medical image fusion. However, a fundamental limitation appearing in existing SR models is their lack of directionality, restricting their efficacy in extracting anatomical details from different imaging modalities. To tackle this issue, we propose a novel directional SR model, termed complex sparse representation (ComSR), specifically designed for medical image fusion. ComSR independently represents MD signals over directional dictionaries along specific directions, allowing precise analysis of intricate details of MD signals. Besides, current studies in medical image fusion mostly concentrate on addressing either 2D or 3D fusion problems. This work bridges this gap by proposing a MD medical image fusion method based on ComSR, presenting a unified framework for both 2D and 3D fusion tasks. Experimental results across six multi-modal medical image fusion tasks, involving 93 pairs of 2D source images and 20 pairs of 3D source images, substantiate the superiority of our proposed method over 11 state-of-the-art 2D fusion methods and 4 representative 3D fusion methods, in terms of both visual quality and objective evaluation. The source code of our fusion method is available at https://github.com/Imagefusions/imagefusions/tree/main.

BP3M: Bayesian Positions, Parallaxes, and Proper Motions Derived from the Hubble Space Telescope and Gaia Data

We present a hierarchical Bayesian pipeline, BP3M, that measures positions, parallaxes, and proper motions (PMs) for cross-matched sources between Hubble Space Telescope (HST) images and Gaia—even for sparse fields (N * < 10 per image)—expanding from the recent GaiaHub tool. This technique uses Gaia-measured astrometry as priors to predict the locations of sources in HST images, and is therefore able to put the HST images onto a global reference frame without the use of background galaxies/QSOs. Testing our publicly available code in the Fornax and Draco dwarf spheroidal galaxies, we measure PMs that are a median of 8–13 times more precise than Gaia DR3 alone for 20.5 < G < 21 mag. We are able to explore the effect of observation strategies on BP3M astrometry using synthetic data, finding an optimal strategy to improve parallax and position precision at no cost to the PM uncertainty. Using 1619 HST images in the sparse COSMOS field (median nine Gaia sources per HST image), we measure BP3M PMs for 2640 unique sources in the 16 < G < 21.5 mag range, 25% of which have no Gaia PMs; the median BP3M PM uncertainty for 20.25 < G < 20.75 mag sources is 0.44 mas yr−1 compared to 1.03 mas yr−1 from Gaia, while the median BP3M PM uncertainty for sources without Gaia-measured PMs (20.75 < G < 21.5 mag) is 1.16 mas yr−1. The statistics that underpin the BP3M pipeline are a generalized way of combining position measurements from different images, epochs, and telescopes, which allows information to be shared between surveys and archives to achieve higher astrometric precision than that from each catalog alone.

DLP-Fusion: Depth of Field, Light Source, and Polarization Fusion Toward Intelligent Optical Imaging for Complex Scenes

DLP-Fusion: Depth of Field, Light Source, and Polarization Fusion Toward Intelligent Optical Imaging for Complex Scenes

INav-based, Automated Coronary Magnetic Resonance Angiography for the Detection of Coronary Artery Stenosis (iNav-AUTO CMRA)

BackgroundCoronary computed tomography angiography (CCTA) is recommended as the first line diagnostic imaging modality in low to intermediate risk individuals suspected of stable coronary artery disease (CAD). However, CCTA exposes patients to ionising radiation and potentially nephrotoxic contrast agents. Invasive coronary angiography (ICA) is the gold-standard investigation to guide coronary revascularisation strategy, however, invasive procedures incur an inherent risk to the patient. Coronary magnetic resonance angiography (Coronary MRA) avoids these issues. Nevertheless, clinical implementation is currently limited due to extended scanning durations, inconsistent image quality, and consequent lack of diagnostic accuracy. Several technical Coronary MRA innovations including advanced respiratory motion correction with 100% scan efficiency (no data rejection), fast image acquisition with motion-corrected undersampled image reconstruction and deep-learning (DL)-based automated planning have been implemented and now await clinical validation in multi-centre trials. MethodsThe objective of the iNav-AUTO CMRA prospective multi-centre study is to evaluate the diagnostic accuracy of a newly developed, state-of-the-art, standardised, and automated Coronary MRA framework compared to CCTA in 230 patients undergoing clinical investigation for CAD. The study protocol mandates the administration of oral beta-blockers to decrease heart rate to below 60bpm and the use of sublingual nitroglycerine spray to induce vasodilation. Additionally, the study incorporates the utilisation of standardised postprocessing with sliding-thin-slab multiplanar reformatting, in combination with evaluation of the source images, to optimize the visualisation of coronary artery stenosis. DiscussionIf proven effective, Coronary MRA could provide a non-invasive, needle-free, yet also clinically viable, alternative to CCTA. Trial RegistrationThis study is registered at clinicaltrials.gov (NCT05473117).

Interactive Feature Embedding for Infrared and Visible Image Fusion.

General deep learning-based methods for infrared and visible image fusion rely on the unsupervised mechanism for vital information retention by utilizing elaborately designed loss functions. However, the unsupervised mechanism depends on a well-designed loss function, which cannot guarantee that all vital information of source images is sufficiently extracted. In this work, we propose a novel interactive feature embedding in a self-supervised learning framework for infrared and visible image fusion, attempting to overcome the issue of vital information degradation. With the help of a self-supervised learning framework, hierarchical representations of source images can be efficiently extracted. In particular, interactive feature embedding models are tactfully designed to build a bridge between self-supervised learning and infrared and visible image fusion learning, achieving vital information retention. Qualitative and quantitative evaluations exhibit that the proposed method performs favorably against state-of-the-art methods.

Optical image authentication and encryption scheme with computational ghost imaging

Most of the previous image encryption schemes overlook certifying the source of the encrypted images, thus the receiver may be misled by messages coming from unauthorized users or wrong sources. To simultaneously accomplish image encryption and source authentication of the encrypted images, an optical image authentication and encryption scheme is proposed by integrating computational ghost imaging (CGI) with quick response (QR) code. Firstly, the plaintext image is converted into a QR code and encrypted by a scrambling-diffusion structure. Then, the authentication image is enciphered by a CGI system and a diffusion structure. The intermediate resulting images are marked and embedded into the random matrices to withstand the differential attack. Moreover, the source of the encrypted images can be identified by the reconstructed authentication image and the nonlinear correlation coefficient (NCC) result. The authentication is considered successful if a sharp peak appears in the center of the NCC result. Numerical simulations show that the proposed scheme can not only withstand different attacks, but also avoid being misled by misinformation during the decryption process.

HDCCT: Hybrid Densely Connected CNN and Transformer for Infrared and Visible Image Fusion

Multi-modal image fusion is a methodology that combines image features from multiple types of sensors, effectively improving the quality and content of fused images. However, most existing deep learning fusion methods need to integrate global or local features, restricting the representation of feature information. To address this issue, a hybrid densely connected CNN and transformer (HDCCT) fusion framework is proposed. In the proposed HDCCT framework, the network of the CNN-based blocks obtain the local structure of the input data, and the transformer-based blocks obtain the global structure of the original data, significantly improving the feature representation. In the fused image, the proposed encoder–decoder architecture is designed for both the CNN and transformer blocks to reduce feature loss while preserving the characterization of all-level features. In addition, the cross-coupled framework facilitates the flow of feature structures, retains the uniqueness of information, and makes the transform model long-range dependencies based on the local features already extracted by the CNN. Meanwhile, to retain the information in the source images, the hybrid structural similarity (SSIM) and mean square error (MSE) loss functions are introduced. The qualitative and quantitative comparisons of grayscale images with infrared and visible image fusion indicate that the suggested method outperforms related works.

SeACPFusion: An Adaptive Fusion Network for Infrared and Visible Images based on brightness perception

Generating a single fused image that highlights important targets and preserves textural details is the aim of fusing visible and infrared images. The majority of deep learning-based fusion algorithms now in use can produce decent fusion outcomes; however, the modeling process still lacks consideration of the different amounts of information in different scenes or regions. Thus, we propose in this research SeACPFusion, a luminance-aware adaptive fusion network for infrared and visible images, which adaptively preserves the intensity information of the noticeable targets of the source images with the texture information of the background in an optimal ratio. Specifically, we design pixel-level luminance loss (PBL) to direct the fusion model’s training in real-time, and PBL retains the optimal intensity information according to the pixel luminance ratio of different source images. In addition, we designed the Channel Transformer (CTF) to consider the relationship between different attributes from the point of view of the feature channel and to focus on the key information by using the self-focusing mechanism to achieve the goal of adaptive fusion. Our extensive tests on the MSRS, RoadScene, and TNO datasets demonstrate that SeACPFusion surpasses nine representative deep learning methods on six objective metrics and achieves the best visual results in scenes such as overexposure or underexposure. In addition, the relatively efficient operation and fewer model parameters make our algorithm promising as a preprocessing module for downstream complicated vision tasks.

Infrared and visible image fusion based on semi-global weighted least squares and guided edge-aware filters

Most of the existing image fusion methods are dedicated to extracting the respective private features in the source image to reconstruct to get a fused image that contains rich information. However, its neglect of the enhancement of texture details in the source image as well as the preservation of background structures, which makes the fusion result look less impressive. To this end, an infrared and visible image fusion method based on a semi-global weighted least squares (SGWLS) method and guided edge-aware filter (GEAF) is proposed. First, a SGWLS method is applied to decompose the source image into base and detail layers containing different scale information. Then, the detail layer are texture-enhanced by using the scale coefficients. Due to the lack of edge features in the original base layer, for this reason, a guided edge-aware filter is designed, in which the enhanced detail layer was used as the guidance image as a way to perform a quadratic feature enhancement fusion for the base layer in order to retain richer gradient information. Finally, reconstruction of the fused detail and base layers by using inverse transformation to obtain the fusion result. Compared with nine state-of-the-art fusion algorithms, a large number of experimental results demonstrate the superior performance of the proposed method in highlighting texture details and background features, as well as the obvious advantages in SF and AG metrics.

Research on fractional wavelet transform combined with parameter adaptive PCNN for infrared and visible image fusion algorithm

Infrared and visible image fusion holds significant value across various fields due to its ability to provide complementary information. However, existing fusion algorithms often suffer from susceptibility to external physical conditions of source images, leading to inconsistent fusion quality and limited adaptive capability. In this paper, we propose a fractional wavelet transform fusion algorithm for infrared and visible images. This algorithm uses fractional wavelet domain image decomposition to more accurately locate the image structures of different scales. Initially, we perform image decomposition using a non-subsampled shearlet transform (NSST) with multi-scale and multi-directional decomposition. Subsequently, to effectively fuse detailed information, we employ discrete fractional wavelet transform (DFRWT) to decompose low-frequency subbands while selecting an appropriate p-value. High-frequency subbands are fused using a parameterized adaptive pulse-coupled neural network (PCNN) model. To validate the superiority of our algorithm, we conduct visual and quantitative comparative analyses against several existing algorithms. The results confirm that our proposed algorithm exhibits robustness against external physical conditions and surpasses these existing algorithms, making it highly applicable for infrared–visible fusion tasks.

Design of a High-Frame-Rate and Large-Grayscale Simulation Projection System Based on Digital Micromirror Devices

With the increasing acquisition speed of image sensors, it has become necessary to provide image sources with higher frame rates and grayscale in order to meet testing requirements. In the field of semi-physical simulation projection, digital micromirror devices are often chosen for their high resolution, uniformity, response speed, and energy concentration. In this study, we utilized digital micromirror devices to construct a high-frame-rate and large-grayscale simulation projection system. To achieve our experiment goals, we employed two digital micromirror devices. One DMD was used to modulate the light intensity of the light source, while the other generated images with different bit planes. By projecting the target images onto the image sensor, we were able to achieve a frame rate of 1611 hz for the projected 12-bit image. This system meets the requirements for our experiment design and ensures the accurate testing of image sensors.

Modeling the Long-Term Variability in the Surfaces of Three Lakes in Morocco with Limited Remote Sensing Image Sources

Satellite imagery has become a widespread resource for modeling variability in lake surfaces. However, the extended monitoring of a lake’s perimeter faces significant challenges due to atmospheric obstacles that cannot be rectified. Due to the atmosphere’s everchanging opacity, only half of the acquired satellite images have reliable qualitative accuracy making it possible to identify a lake’s contour. Consequently, approximately 50% of the monthly lake outline values can be determined using remote sensing methods, leaving the remaining 50% unknown. This situation is applicable to three lakes in Morocco (Abakhan, Ouiouan, and Tiglmanine), the subjects of the current research for the period between 1984 and 2022. What can we do if, during a period of time in which we monitored the evolution of the surface of a lake by satellite means, we obtain only about 50% of the possible images? Shall we just settle for this and stop the analysis? Although it may be challenging to believe, the present study introduces two statistical methods for interpolating and validating the monthly values of the lake outline: the iterative ratio method based on the autocorrelation of the monthly water balance and the Kalman filter. We estimated the reconstruction errors of the missing values and validated the methodology using an inverse philosophy, reconstructing the initial data from the table of the simulation results. Given that the difference between the initial values and the reconstructed initial values resembles white noise or an AR (1) process with a low coefficient, we deemed the methodological approach acceptable. Several comparison criteria between the two interpolation methods were employed, yet determining the more appropriate one remains challenging. Based on our surface reconstruction method, Lake Abakhan, with an average area of 22 hectares, experienced significant fluctuations, ranging from a maximum of 34 hectares in 2010 to a minimum of 0.8 hectares in 2022. Lake Ouiouan, with an average area of 14 hectares, displayed much lower variation, with a maximum of 17 hectares in 2020 and a minimum of 6.5 hectares in 1988. Lake Tiglmanine showed a pattern similar to that of Lake Abakhan but with less pronounced fluctuations. With an average area of 6.1 hectares, its maximum was 9.2 hectares in 2011 and its minimum was 4.1 hectares in 1984.

Liver Cancer Diagnosis: Enhanced Deep Maxout Model with Improved Feature Set

This work proposed a liver cancer classification scheme that includes Preprocessing, Feature extraction, and classification stages. The source images are pre-processed using Gaussian filtering. For segmentation, this work proposes a LUV transformation-based adaptive thresholding-based segmentation process. After the segmentation, certain features are extracted that include multi-texon based features, Improved Local Ternary Pattern (LTP-based features), and GLCM features during this phase. In the Classification phase, an improved Deep Maxout model is proposed for liver cancer detection. The adopted scheme is evaluated over other schemes based on various metrics. While the learning rate is 60%, an improved deep maxout model achieved a higher F-measure value (0.94) for classifying liver cancer; however, the previous method like Support Vector Machine (SVM), Random Forest (RF), Recurrent Neural Network (RNN), Long Short Term Memory (LSTM), K-Nearest Neighbor (KNN), Deep maxout, Convolutional Neural Network (CNN), and DL model holds less F-measure value. An improved deep maxout model achieved minimal False Positive Rate (FPR), and False Negative Rate (FNR) values with the best outcomes compared to other existing models for liver cancer classification.

Dual-initiation ruptures in the 2024 Noto earthquake encircling a fault asperity at a swarm edge.

To reveal the connections between the 2024 moment magnitude (Mw) 7.5 Noto earthquake in Japan and the seismicity swarms that preceded it, we investigated its rupture process through near-source waveform analysis and source imaging techniques, combining seismic and geodetic datasets. We found notable complexity in the initial rupture stages. A strong fault asperity, which remained unbroken in preceding seismic swarms, slowed down the rupture. Then, a second rupture initiated at the opposite edge of the asperity, and the asperity succumbed to double-pincer rupture fronts. The failure of this high-stress drop asperity drove the earthquake into a large-scale event. Our observations help unravel the crucial role of fault asperities in controlling swarm migration and rupture propagation and underscore the need for detailed seismological and interdisciplinary studies to assess seismic risk in swarm-prone regions.

XDL-ESI: Electrophysiological Sources Imaging via explainable deep learning framework with validation on simultaneous EEG and iEEG

Electroencephalography (EEG) or Magnetoencephalography (MEG) source imaging aims to estimate the underlying activated brain sources to explain the observed EEG/MEG recordings. Solving the inverse problem of EEG/MEG Source Imaging (ESI) is challenging due to its ill-posed nature. To achieve a unique solution, it is essential to apply sophisticated regularization constraints to restrict the solution space. Traditionally, the design of regularization terms is based on assumptions about the spatiotemporal structure of the underlying source dynamics. In this paper, we propose a novel paradigm for ESI via an Explainable Deep Learning framework, termed as XDL-ESI, which connects the iterative optimization algorithm with deep learning architecture by unfolding the iterative updates with neural network modules. The proposed framework has the advantages of (1) establishing a data-driven approach to model the source solution structure instead of using hand-crafted regularization terms; (2) improving the robustness of source solutions by introducing a topological loss that leverages the geometric spatial information applying varying penalties on distinct localization errors; (3) improving the reconstruction efficiency and interpretability as it inherits the advantages from both the iterative optimization algorithms (interpretability) and deep learning approaches (function approximation). The proposed XDL-ESI framework provides an efficient, accurate, and interpretable paradigm to solve the ESI inverse problem with satisfactory performance in both simulated data and real clinical data. Specially, this approach is further validated using simultaneous EEG and intracranial EEG (iEEG).

Saliency-based video summarization for face anti-spoofing

With the growing availability of databases for face presentation attack detection, researchers are increasingly focusing on video-based face anti-spoofing methods that involve hundreds to thousands of images for training the models. However, there is currently no clear consensus on the optimal number of frames in a video to improve face spoofing detection. Inspired by the visual saliency theory, we present a video summarization method for face anti-spoofing detection that aims to enhance the performance and efficiency of deep learning models by leveraging visual saliency. In particular, saliency information is extracted from the differences between the Laplacian and Wiener filter outputs of the source images, enabling the identification of the most visually salient regions within each frame. Subsequently, the source images are decomposed into base and detail images, enhancing the representation of the most important information. Weighting maps are then computed based on the saliency information, indicating the importance of each pixel in the image. By linearly combining the base and detail images using the weighting maps, the method fuses the source images to create a single representative image that summarizes the entire video. The key contribution of the proposed method lies in demonstrating how visual saliency can be used as a data-centric approach to improve the performance and efficiency for face presentation attack detection. By focusing on the most salient images or regions within the images, a more representative and diverse training set can be created, potentially leading to more effective models. To validate the method’s effectiveness, a simple CNN–RNN deep learning architecture was used, and the experimental results showcased state-of-the-art performance on four challenging face anti-spoofing datasets.

MADMFuse: A multi-attribute diffusion model to fuse infrared and visible images

In the field of deep learning vision, infrared and visible image fusion (IVIF) has received significant attention due to its ability to enhance scene comprehension by combining the complementary features of both types of images. Existing methods based on generative models are either unstable in training, such as generative adversarial network, or only have a single denoising object, such as diffusion models, resulting in poor fused results that cannot fully contain multi-modal features. To solve these problems, we propose a multi-attribute diffusion model to fuse infrared and visible images, termed MADMFuse. Specifically, to preserve the salient information in infrared images and the texture details in visible images, we have designed a forward diffusion process with shared noise for multiple independent attributes, ensuring that the denoising network learns complementary features of different attributes simultaneously during training. Subsequently, our reverse process with multi-attribute as conditions employs the denoising network that iteratively reparameterizes noise, gradually adjusting and achieving fine image fusion. Furthermore, to address the issue of feature degradation resulting from solely focusing on a single denoising object, we derive a multiple diffusion object loss function based on pixel-level fusion target, the fidelity and luminance term in this loss function aim to guide the fused results towards visual similarity with the source image while preserving its brightness. Extensive experiments indicate that MADMFuse is more effective than other state-of-the-art image fusion methods.

Radio U-Net: a convolutional neural network to detect diffuse radio sources in galaxy clusters and beyond

ABSTRACT The forthcoming generation of radio telescope arrays promises significant advancements in sensitivity and resolution, enabling the identification and characterization of many new faint and diffuse radio sources. Conventional manual cataloguing methodologies are anticipated to be insufficient to exploit the capabilities of new radio surveys. Radio interferometric images of diffuse sources present a challenge for image segmentation tasks due to noise, artifacts, and embedded radio sources. In response to these challenges, we introduce Radio U-Net, a fully convolutional neural network based on the U-Net architecture. Radio U-Net is designed to detect faint and extended sources in radio surveys, such as radio haloes, relics, and cosmic web filaments. Radio U-Net was trained on synthetic radio observations built upon cosmological simulations and then tested on a sample of galaxy clusters, where the detection of cluster diffuse radio sources relied on customized data reduction and visual inspection of Low-Frequency Array Two metre Sky Survey (LoTSS) data. The 83 per cent of clusters exhibiting diffuse radio emission were accurately identified, and the segmentation successfully recovered the morphology of the sources even in low-quality images. In a test sample comprising 246 galaxy clusters, we achieved a 73 per cent accuracy rate in distinguishing between clusters with and without diffuse radio emission. Our results establish the applicability of Radio U-Net to extensive radio survey data sets, probing its efficiency on cutting-edge high-performance computing systems. This approach represents an advancement in optimizing the exploitation of forthcoming large radio surveys for scientific exploration.

High-Resolution Identification of Sound Sources Based on Sparse Bayesian Learning with Grid Adaptive Split Refinement

Sound source identification technology based on a microphone array has many application scenarios. The compressive beamforming method has attracted much attention due to its high accuracy and high-resolution performance. However, for the far-field measurement problem of large microphone arrays, existing methods based on fixed grids have the defect of basis mismatch. Due to the large number of grid points representing potential sound source locations, the identification accuracy of traditional grid adjustment methods also needs to be improved. To solve this problem, this paper proposes a sound source identification method based on adaptive grid splitting and refinement. First, the initial source locations are obtained through a sparse Bayesian learning framework. Then, higher-weight candidate grids are retained, and local regions near them are split and updated. During the iteration process, Green’s function and the source strength obtained in the previous iteration are multiplied to get the sound pressure matrix. The robust principal component analysis model of the Gaussian mixture separates and replaces the sound pressure matrix with a low-rank matrix. The actual sound source locations are gradually approximated through the dynamically adjusted sound pressure low-rank matrix and optimized grid transfer matrix. The performance of the method is verified through numerical simulations. In addition, experiments on a standard aircraft model are conducted in a wind tunnel and speakers are installed on the model, proving that the proposed method can achieve fast, high-precision imaging of low-frequency sound sources in an extensive dynamic range at long distances.

Cross-Domain Object Detection through Consistent and Contrastive Teacher with Fourier Transform

The teacher–student framework has been employed in unsupervised domain adaptation, which transfers knowledge learned from a labeled source domain to an unlabeled target domain. However, this framework suffers from two serious challenges: the domain gap, causing performance degradation, and noisy teacher pseudo-labels, which tend to mislead students. In this paper, we propose a Consistent and Contrastive Teacher with Fourier Transform (CCTF) method to address these challenges for high-performance cross-domain object detection. To mitigate the negative impact of domain shifts, we use the Fourier transform to exchange the low-frequency components of the source and target domain images, replacing the source domain inputs with the transformed image, thereby reducing domain gaps. In addition, we encourage the localization and classification branches of the teacher to make consistent predictions to minimize the noise in the generated pseudo-labels. Finally, contrastive learning is employed to resist the impact of residual noise in pseudo-labels. After extensive experiments, we show that our method achieves the best performance. For example, our model outperforms previous methods by 3.0% on FoggyCityscapes.