Normalized Cross Correlation Research Articles

Performance evaluation of image co-registration methods in photoacoustic mesoscopy of the vasculature

Objective:The formation of functional vasculature in solid tumours enables delivery of oxygen and nutrients, and is vital for effective treatment with chemotherapeutic agents. Longitudinal characterisation of vascular networks can be enabled using mesoscopic photoacoustic imaging, but requires accurate image co-registration to precisely assess local changes across disease development or in response to therapy. Co-registration in photoacoustic imaging is challenging due to the complex nature of the generated signal, including the sparsity of data, artefacts related to the illumination/detection geometry, scan-to-scan technical variability, and biological variability, such as transient changes in perfusion. To better inform the choice of co-registration algorithms, we compared five open-source methods, in physiological and pathological tissues, with the aim of aligning evolving vascular networks in tumours imaged over growth at different time-points.Approach:Co-registration techniques were applied to 3D vascular images acquired with photoacoustic mesoscopy from murine ears and breast cancer patient-derived xenografts, at a fixed time-point and longitudinally. Images were pre-processed and segmented using an unsupervised generative adversarial network. To compare co-registration quality in different settings, pairs of fixed and moving intensity images and/or segmentations were fed into five methods split into the following categories: affine intensity-based using 1)mutual information (MI) or 2)normalised cross-correlation (NCC) as optimisation metrics, affine shape-based using 3)NCC applied to distance-transformed segmentations or 4)iterative closest point algorithm, and deformable weakly supervised deep learning-based using 5)LocalNet co-registration. Percent-changes in Dice coefficients, surface distances, MI, structural similarity index measure and target registration errors were evaluated.Main results:Co-registration using MI or NCC provided similar alignment performance, better than shape-based methods. LocalNet provided accurate co-registration of substructures by optimising subfield deformation throughout the volumes, outperforming other methods, especially in the longitudinal breast cancer xenograft dataset by minimising target registration errors.Significance:We showed the feasibility of co-registering repeatedly or longitudinally imaged vascular networks in photoacoustic mesoscopy, taking a step towards longitudinal quantitative characterisation of these complex structures. These tools open new outlooks for monitoring tumour angiogenesis at the meso-scale and for quantifying treatment-induced co-localised alterations in the vasculature.

Using a patient-specific diffusion model to generate CBCT-based synthetic CTs for CBCT-guided adaptive radiotherapy.

Cone beam computed tomography (CBCT) can be used to evaluate the inter-fraction anatomical changes during the entire course for image-guided radiotherapy (IGRT). However, CBCT artifacts from various sources restrict the full application of CBCT-guided adaptive radiation therapy (ART). Inter-fraction anatomical changes during ART, including variations in tumor size and normal tissue anatomy, can affect radiation therapy (RT) efficacy. Acquiring high-quality CBCT images that accurately capture patient- and fraction-specific (PFS) anatomical changes is crucial for successful IGRT. To enhance CBCT image quality, we proposed PFS lung diffusion models (PFS-LDMs). The proposed PFS models use a pre-trained general lung diffusion model (GLDM) as a baseline, which is trained on historical deformed CBCT (dCBCT)-planning CT (pCT) paired data. For a given patient, a new PFS model is fine-tuned on a CBCT-deformed pCT (dpCT) pair after each fraction to learn the PFS knowledge for generating personalized synthetic CT (sCT) with quality comparable to pCT or dpCT. The learned PFS knowledge is the specific mapping relationships, including personal inter-fraction anatomical changes between personalized CBCT-dpCT pairs. The PFS-LDMs were evaluated on an institutional lung cancer dataset, quantified by mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and structural similarity index measure (SSIM) metrics. We also compared our PFS-LDMs with a mainstream GAN-based model, demonstrating that our PFS fine-tuning strategy could be applied to existing generative models. Our models showed remarkable improvements across all four evaluation metrics. The proposed PFS-LDMs outperformed the GLDM, demonstrating the effectiveness of our proposed fine-tuning strategy. The PFS model fine-tuned with CBCT images from four prior fractions, reduced the MAE from 103.95 to 15.96 Hounsfield units (HU), and increased the mean PSNR, NCC, and SSIM from 25.36dB to 33.57dB, 0.77 to 0.98, and 0.75 to 0.97, respectively. Applying our PFS fine-tuning strategy to a Cycle GAN model also showed improvements, with all four fine-tuned PFS Cycle GAN (PFS-CG) models outperforming the general Cycle GAN model. Overall, our proposed PFS fine-tuning strategy improved CBCT image quality compared to both the pre-correction and non-fine-tuned general models, with our proposed PFS-LDMs yielding better performance than the GAN-based model across all metrics. Our proposed PFS-LDMs significantly improve CBCT image quality with increased HU accuracy and fewer artifacts, thus better capturing inter-fraction anatomical changes. This lays the groundwork for enabling CBCT-based ART, which could enhance clinical efficiency and achieve personalized high-precision treatment by accounting for inter-fraction anatomical changes.

Research on Signal Noise Reduction and Leakage Localization in Urban Water Supply Pipelines Based on Northern Goshawk Optimization.

In order to enhance the accuracy and adaptability of urban water supply pipeline leak localization, based on the Northern Goshawk Optimization, a novel joint denoising method is proposed in this paper to reduce noise in negative pressure wave signals caused by leaks. Firstly, the Northern Goshawk Optimization optimizes the decomposition levels and penalty factors of Variational Mode Decomposition, and obtains their optimal combination. Subsequently, the optimized parameters are used to decompose the pressure signals into modal components, and the effective components and noise components are distinguished according to the correlation coefficients. Then, an optimized wavelet thresholding method is applied to the selected effective components for secondary denoising. Finally, the signal components that have been denoised twice are reconstructed with the effective signal components, and the denoised negative pressure wave signals are obtained. Simulation experiments demonstrate that compared to wavelet transforms and Empirical Mode Decomposition, our method achieves the highest signal-to-noise ratio improvement of 12.23 dB and normalized cross correlation of 0.991. It effectively preserves useful leak information in the signal while suppressing noise, laying a solid foundation for improving leak localization accuracy. After several leak simulation tests on the leakage simulation test platform, the test results verify the effectiveness of the proposed method. The minimum relative error of the leakage localization is 0.29%, and an average relative error is 1.64%, achieving accurate leakage localization.

Pseudo‐CT synthesis in adaptive radiotherapy based on a stacked coarse‐to‐fine model: Combing diffusion process and spatial‐frequency convolutions

AbstractBackgroundCone beam computed tomography (CBCT) provides critical anatomical information for adaptive radiotherapy (ART), especially for tumors in the pelvic region that undergo significant deformation. However, CBCT suffers from inaccurate Hounsfield Unit (HU) values and lower soft tissue contrast. These issues affect the accuracy of pelvic treatment plans and implementation of the treatment, hence requiring correction.PurposeA novel stacked coarse‐to‐fine model combining Denoising Diffusion Probabilistic Model (DDPM) and spatial‐frequency domain convolution modules is proposed to enhance the imaging quality of CBCT images.MethodsThe enhancement of low‐quality CBCT images is divided into two stages. In the coarse stage, the improved DDPM with U‐ConvNeXt architecture is used to complete the denoising task of CBCT images. In the fine stage, the deep convolutional network model jointly constructed by fast Fourier and dilated convolution modules is used to further enhance the image quality in local details and global imaging. Finally, the accurate pseudo‐CT (pCT) images consistent with the size of the original data are obtained. Two hundred fifty paired CBCT‐CT images from cervical and rectal cancer, combined with 200 public dataset cases, were used collectively for training, validation, and testing.ResultsTo evaluate the anatomical consistency between pCT and real CT, we have used the mean(std) of structure similarity index measure (SSIM), peak signal to noise ratio (PSNR), and normalized cross‐correlation (NCC). The numerical results for the above three metrics comparing the pCT synthesized by the proposed model against real CT for cervical cancer cases were 87.14% (2.91%), 34.02 dB (1.35 dB), and 88.01% (1.82%), respectively. For rectal cancer cases, the corresponding results were 86.06% (2.70%), 33.50 dB (1.41 dB), and 87.44% (1.95%). The paired t‐test analysis between the proposed model and the comparative models (ResUnet, CycleGAN, DDPM, and DDIM) for these metrics revealed statistically significant differences (p < 0.05). The visual results also showed that the anatomical structures between the real CT and the pCT synthesized by the proposed model were closer. For the dosimetric verification, mean absolute error of dosimetry (MAEdoes) values for the maximum dose (Dmax), the minimum dose (Dmin), and the mean dose (Dmean) in the planning target volume (PTV) were analyzed, with results presented as mean (lower quartile, upper quartile). The experimental results show that the values of the above three dosimetry indexes (Dmin, Dmax, and Dmean) for the pCT images synthesized by the proposed model were 0.90% (0.48%, 1.29%), 0.82% (0.47%, 1.17%), and 0.57% (0.44%, 0.67%). Compared with 10 cases of the original CBCT image by Mann–Whitney test (p < 0.05), it also proved that pCT can significantly improve the accuracy of HU values for the dose calculation.ConclusionThe pCT synthesized by the proposed model outperforms the comparative models in numerical accuracy and visualization, promising for ART of pelvic cancers.

Musculoskeletal models determine the effect of a soft active exosuit on muscle activations and forces during lifting and lowering tasks

Exosuits have the potential to mitigate musculoskeletal stress and prevent back injuries during industrial tasks. This study aimed to 1) validate the implementation of a soft active exosuit into a musculoskeletal model of the spine by comparing model predicted muscle activations versus corresponding surface EMG measurements, and 2) evaluate the effect of the exosuit on peak back and hip muscle forces. Fourteen healthy participants performed squat and stoop lift and lower tasks with boxes of 6 and 10 kg, with and without wearing a 2.7 kg soft active exosuit. Participant-specific musculoskeletal models, which included the exosuit, were created in OpenSim. Model validation focused on the back and hip extensors, where temporal agreement between EMG and model estimated muscle activity was generally strong to excellent (average cross-correlation coefficients ranging from 0.84 to 0.98). Root mean square errors of muscle activity (0.05–0.10) were similar with and without the exosuit, and compared well to prior model validation studies without the exosuit (average root mean square errors ranging from 0.05 to 0.19). In terms of performance, the exosuit reduced the estimated peak erector spinae forces during lifting and lowering phases across all lifting tasks but reduced peak hip extensor muscles forces only in a squat lift task of 10 kg. These reductions in total peak muscle forces were approximately 1.7–4.2 times greater than the corresponding exosuit assistance force, which were 146 ± 19 N and 102 ± 14 N at the times of peak erector spinae forces in lifting and lowering, respectively. Overall, the results support the hypothesis that exosuits reduce soft tissue loading, and thereby potentially reduce fatigue and injury risk during manual materials handling tasks. Incorporating exosuits into musculoskeletal models is a valid approach to understand the impact of exosuit assistance on muscle activity and forces.

Unveiling complex fault geometry and driving mechanisms: insights from a refined data processing and multiplet analysis of the 2010 Beni-Ilmane seismic sequence (NE Algeria)

SUMMARY This paper offers a comprehensive re-analysis of the Beni-Ilmane 2010 seismic sequence, using a data set that is 100 per cent larger than previous studies. This unprecedented sequence in Algeria features three main shocks with magnitudes Mw 5.4, 5.1 and 5.1. Our approach involves high-precision relocation, which includes the development of a new 1-D minimum velocity model, followed by a double-difference (DD) procedure and hierarchical clustering. We determined the focal mechanisms (FMs) for 128 key events and identified 21 multiplet groups using an average cross-correlation threshold of 0.8. Our analysis offers new insights into fault geometry and addresses ongoing debates, by proposing a seismotectonic model that reveals the activation of 14 fault segments during the sequence, in contrast to previous oversimplified models that suggested two or three faults. The computed stress field from the inversion of 128 FMs aligns with a tectonic loading force due to the convergence of the African and Eurasian plates. These findings highlight the complexity of the fault network in the study area and shed light on the role of strike-slip faults in shaping the thrust belt. We found a strong link between multiplet groups and fluid movement along the fault network. Analysis of the temporal history of these multiplet groups provides new insights into fluid dynamics timescales, with an estimated hydraulic diffusivity (D) of 0.36 m2 s−1 suggesting a fluid pressure diffusion process. The observed expansion of the aftershock area with the logarithm of time and the existence of repeating earthquakes indicates, for the first time, an aseismic slip mechanism that adds an additional layer to the driven processes. In conclusion, our results suggest that the underlying mechanisms governing the BI-2010 seismic sequence involve a complex interplay of tectonic loading, coseismic stress transfer, fluid dynamics and aseismic slip transients. We attempt to correlate our findings with various studies linking the structure, mechanics and fluid flow properties of fault zones and fault systems. The activation of smaller fault segments potentially averted a larger quake, resulting in three moderate main shocks and numerous aftershocks. This work not only enrich our understanding of seismic phenomena but also provides useful insights for seismic hazard assessment and risk mitigation strategies.

Digital Watermarking Using Hybrid Grasshopper Optimization Algorithm and Genetic Algorithm (HGOAGA)

The advancement of computer technology has raised significant issues with digital content piracy and copyright law. A popular method of protecting copyright and related uses is digital watermarking. Various algorithms have been developed to address the need for invisible performance and robustness in digital watermarking schemes. Here, we proposed a novel evolutionary algorithm, the hybrid Grass Hopper Optimization algorithm, and the Genetic algorithm (HGOAGA) for optimizing multiple scaling factors in the digital watermarking scheme in the frequency domain of hybrid Discrete Wavelet Transformation (DWT) and Singular Value Decomposition (SVD) method (hybrid DWT-SVD). The subcomponents of the image are determined by calculating the DWT of the cover image. The problem is determining the best scaling factor for watermarking after converting the subcomponent to the frequency domain using SVD. In HGOAGA, an optimal solution of multiple scaling factors is found after several iterations, starting with a set of randomly generated solutions. The advantages of GOA and GA are combined in the HGOAGA to balance exploration and exploitation functionalities. Furthermore, HGOAGA can converge quickly and escape local optima well. Some standard images were used in the MATLAB environment to test the proposed algorithm. The evaluation of the experiment was carried out using various metrics such as the Structural Similarity Index (SSIM), the Normalized Cross-Correlation (NCC), and the Peak Signal-to-Noise Ratio (PSNR). The experimental results of the tests showed a PSNR value of 51db for the proposed method compared to existing methods, and they are best suited to solving conflict problems between robustness and quality.

Spatiotemporal correlation enhanced real-time 4D-CBCT imaging using convolutional LSTM networks.

To enhance the accuracy of real-time four-dimensional cone beam CT (4D-CBCT) imaging by incorporating spatiotemporal correlation from the sequential projection image into the single projection-based 4D-CBCT estimation process. We first derived 4D deformation vector fields (DVFs) from patient 4D-CT. Principal component analysis (PCA) was then employed to extract distinctive feature labels for each DVF, focusing on the first three PCA coefficients. To simulate a wide range of respiratory motion, we expanded the motion amplitude and used random sampling to generate approximately 900 sets of PCA labels. These labels were used to produce 900 simulated 4D-DVFs, which in turn deformed the 0% phase 4D-CT to obtain 900 CBCT volumes with continuous motion amplitudes. Following this, the forward projection was performed at one angle to get all of the digital reconstructed radiographs (DRRs). These DRRs and the PCA labels were used as the training data set. To capture the spatiotemporal correlation in the projections, we propose to use the convolutional LSTM (ConvLSTM) network for PCA coefficient estimation. For network testing, when several online CBCT projections (with different motion amplitudes that cover the full respiration range) are acquired and sent into the network, the corresponding 4D-PCA coefficients will be obtained and finally lead to a full online 4D-CBCT prediction. A phantom experiment is first performed with the XCAT phantom; then, a pilot clinical evaluation is further conducted. Results on the XCAT phantom and the patient data show that the proposed approach outperformed other networks in terms of visual inspection and quantitative metrics. For the XCAT phantom experiment, ConvLSTM achieves the highest quantification accuracy with MAPE(Mean Absolute Percentage Error), PSNR (Peak Signal-to-Noise Ratio), and RMSE(Root Mean Squared Error) of 0.0459, 64.6742, and 0.0011, respectively. For the patient pilot clinical experiment, ConvLSTM also achieves the best quantification accuracy with that of 0.0934, 63.7294, and 0.0019, respectively. The quantification evaluation labels that we used are 1) the Mean Absolute Error (MAE), 2) the Normalized Cross Correlation (NCC), 3)the Structural Similarity Index Measurement(SSIM), 4)the Peak Signal-to-Noise Ratio (PSNR), 5)the Root Mean Squared Error(RMSE), and 6) the Absolute Percentage Error (MAPE). The spatiotemporal correlation-based respiration motion modeling supplied a potential solution for accurate real-time 4D-CBCT reconstruction.

CBCT-based synthetic CT image generation using a diffusion model for CBCT-guided lung radiotherapy.

Although cone beam computed tomography (CBCT) has lower resolution compared to planning CTs (pCT), its lower dose, higher high-contrast resolution, and shorter scanning time support its widespread use in clinical applications, especially in ensuring accurate patient positioning during the image-guided radiation therapy (IGRT) process. While CBCT is critical to IGRT, CBCT image quality can be compromised by severe stripe and scattering artifacts. Tumor movement secondary to respiratory motion also decreases CBCT resolution. In order to improve the image quality of CBCT, we propose a Lung Diffusion Model (L-DM) framework. Our proposed algorithm is based on a conditional diffusion model trained on pCT and deformed CBCT (dCBCT) image pairs to synthesize lung CT images from dCBCT images and benefit CBCT-based radiotherapy. dCBCT images were used as the constraint for the L-DM. The image quality and Hounsfield unit (HU) values of the synthetic CTs (sCT) images generated by the proposed L-DM were compared to three selected mainstream generation models. We verified our model in both an institutional lung cancer dataset and a selected public dataset. Our L-DM showed significant improvement in the four metrics of mean absolute error (MAE), peak signal-to-noise ratio (PSNR), normalized cross-correlation (NCC), and structural similarity index measure (SSIM). In our institutional dataset, our proposed L-DM decreased the MAE from 101.47 to 37.87HU and increased the PSNR from 24.97 to 29.89 dB, the NCC from 0.81 to 0.97, and the SSIM from 0.80 to 0.93. In the public dataset, our proposed L-DM decreased the MAE from 173.65 to 58.95HU, while increasing the PSNR, NCC, and SSIM from 13.07 to 24.05 dB, 0.68 to 0.94, and 0.41 to 0.88, respectively. The proposed L-DM significantly improved sCT image quality compared to the pre-correction CBCT andthree mainstream generative models. Our model can benefit CBCT-based IGRT and other potential clinical applications as it increases the HU accuracy and decreases the artifacts from input CBCT images.

Adaptive complementary neighboring sub-aperture beamforming for thermoacoustic imaging.

When applied to thermoacoustic imaging (TAI), the delay-and-sum (DAS) algorithm produces strong sidelobes due to its disadvantages of uniform aperture weighting. As a result, the quality of TAI images recovered by DAS is often severely degraded by strong non-coherent clutter, which restricts the development and application of TAI. To address this issue, we propose an adaptive complementary neighboring sub-aperture (NSA) beamforming algorithm for TAI. In NSA, we introduce a coordinate system transformation when calculating the normalized cross-correlation (NCC) matrix. This approach enables the computation of the NCC coefficient within the specified kernel without complex coordinate calculations. We first conducted the numerical simulation experiment to validate NSA using a tree branch phantom. In addition, we also conducted phantom (five sauce tubes), ex vivo (ablation needle in ex vivo porcine liver), and in vivo (human arm) TAI experiments using our TAI system with a center frequency of 3GHz. In the numerical simulation experiment, the structural similarity index (SSIM) value for NSA is increased from 0.37828 for DAS to 0.75492. In the point target phantom TAI experiment, the generalized contrast-to-noise ratio (gCNR) value for NSA is increased from 0.936 for DAS to 0.962. The experimental results show that NSA can recover clearer thermoacoustic images compared to DAS. In the ex vivo TAI experiment, the full width at half maxima (FWHM) of an ablation needle (diameter=1.5mm) for coherence factor (CF) weighted DAS and NSA are 0.9 and 1.3mm, respectively. Furthermore, in the in vivo TAI experiment, CF reduces the signals within the arm compared to NSA. Therefore, compared with CF, NSA can maintain the integrity of target information in TAI while effectively suppressing non-coherent background clutter. NSA can effectively reduce non-coherent background noise while ensuring the completeness of the target information. So, NSA offers the potential to provide high-quality thermoacoustic images and further advance their clinical application.

Phase-specific augmented reality guidance for microscopic cataract surgery using spatiotemporal fusion network

Phacoemulsification cataract surgery (PCS) is a routine procedure conducted using a surgical microscope, heavily reliant on the skill of the ophthalmologist. While existing PCS guidance systems extract valuable information from surgical microscopic videos to enhance intraoperative proficiency, they suffer from non-phase-specific guidance, leading to redundant visual information. In this study, our major contribution is the development of a novel phase-specific augmented reality (AR) guidance system, which offers tailored AR information corresponding to the recognized surgical phase. Leveraging the inherent quasi-standardized nature of PCS procedures, we propose a two-stage spatiotemporal fusion network for surgical microscopic video recognition. In the first stage, we implement a multi-task learning structure to segment the surgical limbus region and extract limbus region-focused spatial feature for each frame. In the second stage, we propose a spatiotemporal fusion transformer (STFT) network to model local fine-grained and global temporal relationships, and fuse the extracted spatial features to recognize the current surgical phase. Additionally, we collaborate closely with ophthalmologists to design AR visual cues by utilizing techniques such as limbus ellipse fitting and regional restricted normal cross-correlation rotation computation. We evaluated the network on publicly available and in-house datasets, with comparison results demonstrating its superior performance compared to related works. Ablation results further validated the effectiveness of the limbus region-focused spatial feature extractor and the fusion of temporal features. Furthermore, the developed system was evaluated in a clinical setup, with results indicating acceptable accuracy and real-time performance, underscoring its potential for clinical applications.

A Low-Complexity Beamformer for Ultrasound Imaging Based on Sub-Beamformer and Multi-Apodization With Cross-Correlation

ObjectiveThis paper proposes an ultrasound imaging algorithm based on sub-beamformer and multi-apodization with cross-correlation (SUB-MAX), aiming to achieve high resolution close to the minimum variance (MV) beamforming with low complexity and to enhance image contrast while maintaining background quality. MethodsThe output of two (N/2)-element DAS beamformers with asymmetric phase centers is subtracted, resulting in a large drop in the main-lobe amplitude, while the sidelobe maintains a relatively high amplitude level. Inspired by this characteristic, the coefficients with opposite trends compared with the subtracted output are obtained and fused with the normalized cross-correlation (NCC) weighting matrix acquired by using multi-pair received apodization, the proposed SUB-MAX obtains a new weighting matrix to weight the output of the DAS beamformer. ResultsFor ats_wire point targets, the average full-width at half-maximum (FWHM) of SUB-MAX compared with DAS, DMAS, CF, and MAX decreases by 52.7%, 43.5%, 33.3%, and 52.7%, respectively. For geabr_0 cysts, the average contrast ratio (CR) of SUB-MAX compared with DAS, MV, DMAS, and CF increases by 57.7%, 86.8%, 2.5%, and 14.4%, respectively. Experiments on rat_tumor dataset also indicate that SUB-MAX has a superior comprehensive imaging performance. ConclusionThe experimental results indicate that the superior comprehensive imaging performance of the proposed SUB-MAX is expected to be suitable for real-time imaging systems due to its non-reliance on covariance matrix inversion.

Coded speckle target: Design, analysis and applications

In this paper, a novel encoding feature based on speckle patterns. It comprises three components: a pre-positioning ring, localization speckle area, and encoding speckle area. The pre-positioning ring identifies potential regions of interest. Shape function of the localization speckle area can be estimated by formulating the elliptical equation of the pre-positioning ring. Next, digital image correlation method is employed to achieve precise positioning and refine the shape function. Through the high-precision shape function, the encoded speckle area can be determined. Encoding technique utilizes a reference encoding scheme, in which each coded value region either same or is inverse to the reference region. Zero-mean normalized cross correlation relative to the reference region can be used to obtain a highly robust decoding value. Experimental results demonstrate that, despite challenges such as noise, defocus, variations in lighting intensity, and lighting uneven, the coded speckle target maintains high robustness and precise positioning accuracy. Moreover, the structure from motion and displacement measurement experiments further verifies its high accuracy in practical applications.

AI-driven generalized polynomial transformation models for unsupervised fundus image registration.

We introduce a novel AI-driven approach to unsupervised fundus image registration utilizing our Generalized Polynomial Transformation (GPT) model. Through the GPT, we establish a foundational model capable of simulating diverse polynomial transformations, trained on a large synthetic dataset to encompass a broad range of transformation scenarios. Additionally, our hybrid pre-processing strategy aims to streamline the learning process by offering model-focused input. We evaluated our model's effectiveness on the publicly available AREDS dataset by using standard metrics such as image-level and parameter-level analyzes. Linear regression analysis reveals an average Pearson correlation coefficient (R) of 0.9876 across all quadratic transformation parameters. Image-level evaluation, comprising qualitative and quantitative analyzes, showcases significant improvements in Structural Similarity Index (SSIM) and Normalized Cross Correlation (NCC) scores, indicating its robust performance. Notably, precise matching of the optic disc and vessel locations with minimal global distortion are observed. These findings underscore the potential of GPT-based approaches in image registration methodologies, promising advancements in diagnosis, treatment planning, and disease monitoring in ophthalmology and beyond.

A Robust Tie-Points Matching Method with Regional Feature Representation for Synthetic Aperture Radar Images

The precise tie-points (TPs) on synthetic aperture radar (SAR) images are a critical cornerstone in the global digital elevation model (DEM) and digital ortho map (DOM) production process. While there are abundant studies on SAR TPs matching, improvement opportunities persist in large areas. The correspondences have pixel-level errors during geocoding, which result in misalignment between global products. Consequently, this paper proposed a robust method for SAR images TPs matching, which consists of three key steps: (1) interest point extraction based on the dynamic Harris area entropy (DHAE) grid; (2) adaptive determination of template size; (3) normalized cross correlation (NCC) template matching. DHAE is a regional texture information grid based on the SAR-Harris map, and it is achieved through dynamic block division. Generating the DHAE grid over SAR images enables the extraction of interest points that have regional feature representation and distribution uniformity. A variable-size matching template is adaptively determined based on DHAE to enhance template quality while maintaining computational efficiency. Subsequently, the NCC algorithm is employed to find subpixel-precise correspondences. The proposed method is applied on TPs matching in 57 Terra-SAR images, which cover a large geographical area. Furthermore, the overlapping area is partitioned into five segments according to different coverage types. The experimental results demonstrate that the proposed method outperforms other template matching methods. For all coverage types, the proposed method exhibits high-precision sub-pixel results that reach up to 38.64% in terms of the relative positioning error (RPE), particularly in texture-weak and large areas.

Deep learning decryption approach for asymmetric computer-generated holography (CGH) cryptosystem.

Deep-learning-based optical image decryption has attracted attention due to its remarkable advantages of keyless managements. Here, a high-fidelity deep learning (DL) decryption strategy is proposed, aiming for the asymmetric DRPE-based CGH cryptosystem, which is combined with phase truncation technique and chaotic iris phase masks. First, a mass of ciphertext and plaintext image pairs are generated to create a dataset. Then, a deep neural network, namely ACGHC-Net (network for the asymmetric DRPE-based CGH cryptosystem), is designed and trained in a supervised learning manner. After the model training and tuning, the ACGHC-Net can quickly and accurately decrypt the ciphertext images. The average cross-correlation coefficient (CC) of the decrypted images achieves 0.998, the average structural similarity (SSIM) 0.895, and the average peak signal-to-noise ratio (PSNR) 31.090 dB. Furthermore, we conducted anti-noise and anti-clipping analysis on the ACGHC-Net. The results prove that the proposed ACGHC-Net can successfully decrypt the encrypted complex grayscale images, and has good anti-noise and anti-cropping robustness for the asymmetric DRPE-based CGH cryptosystem. The proposed method will be expected to further boost keyless decryption in image encryption systems.

Enhancing voxel-based dosimetry accuracy with an unsupervised deep learning approach for hybrid medical image registration.

Deformable registration is required to generate a time-integrated activity (TIA) map which is essential for voxel-based dosimetry. The conventional iterative registration algorithm using anatomical images (e.g., computed tomography (CT)) could result in registration errors in functional images (e.g., single photon emission computed tomography (SPECT) or positron emission tomography (PET)). Various deep learning-based registration tools have been proposed, but studies specifically focused on the registration of serial hybrid images were not found. In this study, we introduce CoRX-NET, a novel unsupervised deep learning network designed for deformable registration of hybrid medical images. The CoRX-NET structure is based on the Swin-transformer (ST), allowing for the representation of complex spatial connections in images. Its self-attention mechanism aids in the effective exchange and integration of information across diverse image regions. To augment the amalgamation of SPECT and CT features, cross-stitch layers have been integrated into the network. Two different 177 Lu DOTATATE SPECT/CT datasets were acquired at different medical centers. 22 sets from Seoul National University and 14 sets from Sunway Medical Centre are used for training/internal validation and external validation respectively. The CoRX-NET architecture builds upon the ST, enabling the modeling of intricate spatial relationships within images. To further enhance the fusion of SPECT and CT features, cross-stitch layers have been incorporated within the network. The network takes a pair of SPECT/CT images (e.g., fixed and moving images) and generates a deformed SPECT/CT image. The performance of the network was compared with Elastix and TransMorph using L1 loss and structural similarity index measure (SSIM) of CT, SSIM of normalized SPECT, and local normalized cross correlation (LNCC) of SPECT as metrics. The voxel-wise root mean square errors (RMSE) of TIA were compared among the different methods. The ablation study revealed that cross-stitch layers improved SPECT/CT registration performance. The cross-stitch layers notably enhance SSIM (internal validation: 0.9614vs. 0.9653, external validation: 0.9159vs. 0.9189) and LNCC of normalized SPECT images (internal validation: 0.7512vs. 0.7670, external validation: 0.8027vs. 0.8027). CoRX-NET with the cross-stitch layer achieved superior performance metrics compared to Elastix and TransMorph, except for CT SSIM in the external dataset. When qualitatively analyzed for both internal and external validation cases, CoRX-NET consistently demonstrated superior SPECT registration results. In addition, CoRX-NET accomplished SPECT/CT image registration in less than 6s, whereas Elastix required approximately 50s using the same PC's CPU. When employing CoRX-NET, it was observed that the voxel-wise RMSE values for TIA were approximately 27% lower for the kidney and 33% lower for the tumor, compared to when Elastix was used. This study represents a major advancement in achieving precise SPECT/CT registration using an unsupervised deep learning network. It outperforms conventional methods like Elastix and TransMorph, reducing uncertainties in TIA maps for more accurate dose assessments.

Machine learning techniques for estimation of Pc5 geomagnetic pulsations observed at geostationary orbits during solar cycle 23

Pc5 geomagnetic pulsations can accelerate electrons in the radiation belts, which can pose adverse threats to both astronauts and satellites in space. The estimation of Pc5 waves in space is crucial to radiation belt dynamics studies and will help mitigate these challenges. Here, we explore the advantages of the Feed-forward Neural Network (FFNN) and Random Forest (RF) algorithm for effective estimation of Pc5 geomagnetic pulsations observed in space at geostationary orbit during solar cycle 23. The dataset used in this study is the vector magnetic field measurements retrieved from the Geostationary Operational Environmental Satellite-10 (GOES-10) and the solar wind parameters: Bz and Vx component of the solar wind in the Geocentric Solar Ecliptic (GSE) coordinate system, proton density, flow pressure, and plasma beta obtained from the OMNI Web database during part of solar cycle 23. Pc5 geomagnetic pulsations were extracted from the toroidal component of the magnetic field time series using a bandpass Butterworth filter. The continuous wavelet transform (CWT) was utilized to study the characteristics of the extracted wave in the time-frequency domain for its validation. The validated Pc5 events were used as the target in the model's development, with the solar wind parameters as the inputs. In addition to the solar wind parameters, we included an attribute of the magnetic field time series as an input variable in the model. The dataset is carefully divided to ensure effective training and testing of the models. Finally, we trained both models using the same inputs and targets and explored their estimation abilities. The model was tested during the maximum, descending, and minimum phases of solar cycle 23. Both the FFNN and RF models have a similar estimation, with average cross-correlation score (R) values of 0.74 and 0.73 and corresponding average root mean squared error (RMSEs) of 0.16 nT and 0.67 nT, respectively. The model was deployed to investigate the response of Pc5 waves during three storm days in each testing year. The machine learning (ML) model outputs showed good coherence with the observed Pc5 waves. To validate the models, we studied the correlation between the estimated Pc5 events with the Kp index, and a good correlation was seen to exist between both events. This validates the good performance of the developed models. This work will aid in the study of radiation belt dynamics and the construction of electron depletion regions in the radiation belt.

Deformable motion compensation in interventional cone-beam CT with a context-aware learned autofocus metric.

Interventional Cone-Beam CT (CBCT) offers 3D visualization of soft-tissue and vascular anatomy, enabling 3D guidance of abdominal interventions. However, its long acquisition time makes CBCT susceptible to patient motion. Image-based autofocus offers a suitable platform for compensation of deformable motion in CBCT, but it relies on handcrafted motion metrics based on first-order image properties and that lack awareness of the underlying anatomy. This work proposes a data-driven approach to motion quantification via a learned, context-aware, deformable metric, , that quantifies the amount of motion degradation as well as the realism of the structural anatomical content in the image. The proposed was modeled as a deep convolutional neural network (CNN) trained to recreate a reference-based structural similarity metric-visual information fidelity (VIF). The deep CNN acted on motion-corrupted images, providing an estimation of the spatial VIF map that would be obtained against a motion-free reference, capturing motion distortion, and anatomic plausibility. The deep CNN featured a multi-branch architecture with a high-resolution branch for estimation of voxel-wise VIF on a small volume of interest. A second contextual, low-resolution branch provided features associated to anatomical context for disentanglement of motion effects and anatomical appearance. The deep CNN was trained on paired motion-free and motion-corrupted data obtained with a high-fidelity forward projection model for a protocol involving 120kV and 9.90mGy. The performance of was evaluated via metrics of correlation with ground truth and with the underlying deformable motion field in simulated data with deformable motion fields with amplitude ranging from 5 to 20mm and frequency from 2.4 up to 4 cycles/scan. Robustness to variation in tissue contrast and noise levels was assessed in simulation studies with varying beam energy (90-120kV) and dose (1.19-39.59mGy). Further validation was obtained on experimental studies with a deformable phantom. Final validation was obtained via integration of on an autofocus compensation framework, applied to motion compensation on experimental datasets and evaluated via metric of spatial resolution on soft-tissue boundaries and sharpness of contrast-enhanced vascularity. The magnitude and spatial map of showed consistent and high correlation levels with the ground truth in both simulation and real data, yielding average normalized cross correlation (NCC) values of 0.95 and 0.88, respectively. Similarly, achieved good correlation values with the underlying motion field, with average NCC of 0.90. In experimental phantom studies, properly reflects the change in motion amplitudes and frequencies: voxel-wise averaging of the local across the full reconstructed volume yielded an average value of 0.69 for the case with mild motion (2mm, 12 cycles/scan) and 0.29 for the case with severe motion (12mm, 6 cycles/scan). Autofocus motion compensation using resulted in noticeable mitigation of motion artifacts and improved spatial resolution of soft tissue and high-contrast structures, resulting in reduction of edge spread function width of 8.78% and 9.20%, respectively. Motion compensation also increased the conspicuity of contrast-enhanced vascularity, reflected in an increase of 9.64% in vessel sharpness. The proposed , featuring a novel context-aware architecture, demonstrated its capacity as a reference-free surrogate of structural similarity to quantify motion-induced degradation of image quality and anatomical plausibility of image content. The validation studies showed robust performance across motion patterns, x-ray techniques, and anatomical instances. The proposed anatomy- and context-aware metric poses a powerful alternative to conventional motion estimation metrics, and a step forward for application of deep autofocus motion compensation for guidance in clinical interventional procedures.

Multi-apodization with cross-correlation combined with generalized sidelobe canceller applied to ultrasound imaging.

Beamforming is vital for medical ultrasound imaging systems. The generalized sidelobe canceller (GSC) beamforming can improve the image quality of lateral resolution, but its performance improvement in contrast and robustness is limited. This paper proposes an improved generalized sidelobe canceller algorithm based on multi-apodization with cross-correlation (MAXB-IGSC), which aims to improve the contrast and robustness of ultrasound imaging while maintaining the high image resolution and background speckle quality of GSC. The proposed MAXB-IGSC uses multiple pairs of complementary received apodization functions to process the echo data individually to obtain multiple pairs of data sets. The average of their normalized cross-correlation coefficients is then calculated and utilized to determine the adaptive subarray length of the GSC covariance matrix and weights the output of the improved GSC. The MAXB-IGSC improves the contrast ratio (CR) by 171.18% in anechoic cyst simulation and by 91.23%/130.97%/171.76% in geabr_0 (a dataset from the University of Michigan) experiment compared with GSC, respectively. Furthermore, MAXB-IGSC exhibits significant noise immunity, which greatly improves the robustness of the imaging. The technology also maintains the brightness and uniformity of the background speckle. The proposed MAXB-IGSC has potential for obtaining high-quality ultrasound images in clinical applications.