Multi-scale Structural Similarity Research Articles

Generalized 3D registration algorithm for enhancing retinal optical coherence tomography images.

Optical coherence tomography (OCT) has emerged as the standard of care for diagnosing and monitoring the treatment of various ocular disorders due to its noninvasive nature and in vivo volumetric acquisition capability. Despite its widespread applications in ophthalmology, motion artifacts remain a challenge in OCT imaging, adversely impacting image quality. While several multivolume registration algorithms have been developed to address this issue, they are often designed to cater to one specific OCT system or acquisition protocol. We aim to generate an OCT volume free of motion artifacts using a system-agnostic registration algorithm that is independent of system specifications or protocol. We developed a B-scan registration algorithm that removes motion and corrects for both translational eye movements and rotational angle differences between volumes. Tests were carried out on various datasets obtained from two different types of custom-built OCT systems and one commercially available system to determine the reliability of the proposed algorithm. Additionally, different system specifications were used, with variations in axial resolution, lateral resolution, signal-to-noise ratio, and real-time motion tracking. The accuracy of this method has further been evaluated through mean squared error (MSE) and multiscale structural similarity index measure (MS-SSIM). The results demonstrate improvements in the overall contrast of the images, facilitating detailed visualization of retinal vasculatures in both superficial and deep vasculature plexus. Finer features of the inner and outer retina, such as photoreceptors and other pathology-specific features, are discernible after multivolume registration and averaging. Quantitative analyses affirm that increasing the number of averaged registered volumes will decrease MSE and increase MS-SSIM as compared to the reference volume. The multivolume registered data obtained from this algorithm offers significantly improved visualization of the retinal microvascular network as well as retinal morphological features. Furthermore, we have validated that the versatility of our methodology extends beyond specific OCT modalities, thereby enhancing the clinical utility of OCT for the diagnosis and monitoring of ocular pathologies.

Vehicle detection algorithm for foggy based on improved AOD-Net

To strengthen the safety monitoring of foggy road traffic and maintain the safety of vehicle driving on foggy roads, image dehazing algorithms are used to improve the clarity of road images detected in foggy environments, thereby improving the detection ability and monitoring efficiency of intelligent transportation systems for vehicle targets. Due to the low accuracy of vehicle detection and serious problem of missed detections in haze environments, this paper proposes an improved All-in-One Dehazing Network (AOD-Net) algorithm for detecting foggy vehicles, which adds batch normalization (BN) layers after each layer of convolution in AOD-Net, accelerating the convergence of the model and controlling overfitting. To enhance image detail information, an effective pyramid-shaped PSA attention module is embedded to extract richer feature information, enrich model representation, and improve the loss function to a multi-scale structural similarity (MS-SSIM) + L1 mixed loss function, thereby improving the quality, brightness, and contrast of dehazing images. Compared with current image dehazing algorithms, the dehazing quality of our algorithm is superior to other dehazing algorithms, such as dark channel prior (DCP), Dehaze-Net, and Fusion Feature Attention Network (FFA-Net). Compared with AOD-Net, the improved algorithm has increased the peak signal-to-noise ratio by 3.23 dB. At the same time, after the improved AOD-Net image dehazing processing, YOLOv7 object detection was performed and experimentally validated on a real foggy dataset. The results showed that compared with the previous method, it had better recognition performance in foggy detection and recognition, and higher detection accuracy for vehicles.

Enhancing unsupervised rock CT image super-resolution with non-local attention

Traditional low-resolution rock core images fail to capture the details and features of rocks, limiting the accurate analysis of rock properties. Therefore, researchers have proposed supervised and unsupervised digital rock super-resolution reconstruction methods. Supervised methods require paired data and manual annotation, while unsupervised methods generate high-resolution images through self-learning that are suitable for real-world scenarios with a lack of paired data. However, the current unsupervised methods still face challenges in terms of consistency and feature capturing. To address this problem, a novel unsupervised single-image super-resolution model called NL-CycleGAN is proposed. This model employs a non-local attention-guided CycleGAN framework to effectively capture low-level pixel variations between unpaired source and target images while preserving the overall tone. To evaluate the performance of NL-CycleGAN, we conduct both quantitative and qualitative tests using the DeepRock-SR dataset. In terms of quantitative evaluation, our method achieved the lowest perceptual loss (LPIPS) metric compared to existing methods. Additionally, our model obtained the highest performance in peak signal-to-noise ratio (PSNR), and multi-scale structural similarity index measure (MS-SSIM). In qualitative testing, the comparison images clearly show that our model accurately captures the intricate details of pores in carbonate and sandstone, as well as the complex crack patterns in coal. In conclusion, our model contributes to improving rock resolution and provides more accurate image processing techniques for rock property analysis and geological research.

Unsupervised MRI motion artifact disentanglement: introducing MAUDGAN.

This study developed an unsupervised motion artifact reduction method for MRI images of patients with brain tumors. The proposed novel design uses multi-parametric multicenter contrast-enhanced T1W (ceT1W) and T2-FLAIR MRI images.&#xD;Approach: The proposed framework included two generators, two discriminators, and two feature extractor networks. A 3-fold cross-validation was used to train and fine-tune the hyperparameters of the proposed model using 230 brain MRI images with tumors, which were then tested on 148 patients' in-vivo datasets. An ablation was performed to evaluate the model's compartments. Our model was compared with Pix2pix and CycleGAN. Six evaluation metrics were reported, including normalized mean squared error (NMSE), structural similarity index (SSIM), multi-scale-SSIM (MS-SSIM), peak signal-to-noise ratio (PSNR), visual information fidelity (VIF), and multi-scale gradient magnitude similarity deviation (MS-GMSD). Artifact reduction and consistency of tumor regions, image contrast, and sharpness were evaluated by three evaluators using Likert scales and compared with ANOVA and Tukey's HSD tests. &#xD;Main results: On average, our method outperforms comparative models to remove heavy motion artifacts with the lowest NMSE (18.34±5.07%) and MS-GMSD (0.07±0.03) for heavy motion artifact level. Additionally, our method creates motion-free images with the highest SSIM (0.93±0.04), PSNR (30.63±4.96), and VIF (0.45±0.05) values, along with comparable MS-SSIM (0.96±0.31). Similarly, our method outperformed comparative models in removing in-vivo motion artifacts for different distortion levels except for MS- SSIM and VIF, which have comparable performance with CycleGAN. Moreover, our method had a consistent performance for different artifact levels. For the heavy level of motion artifacts, our method got the highest Likert scores of 2.82±0.52, 1.88±0.71, and 1.02±0.14 (p-values<<0.0001) for our method, CycleGAN, and Pix2pix respectively. Similar trends were also found for other motion artifact levels.&#xD;Significance: Our proposed unsupervised method was demonstrated to reduce motion artifacts from the ceT1W brain images under a multi-parametric framework.

Medical steganography: Enhanced security and image quality, and new S-Q assessment

Medical diagnosing systems generate sensitive information involving the patient’s privacy. Steganography supports securely embedding secrecy, guaranteeing that an adversary cannot distinguish cover images and stego images. However, directly applying natural-image steganography in medicine significantly affects efficacy and security. Until now, medical steganography is still in its infancy, mostly solving simple data-hiding policies, remaining barren in enhancing security.To fill this gap, we propose an end-to-end steganographic framework targeting protecting medical images efficiently. Our framework integrates medical-image adaptive steganography with a generative adversarial network named MedSteGAN. As for medical images without rich semantics and textures, we observe the heightened importance of involving both security and image quality. We shed light on designing a “security-quality (S-Q) assessment”, attaining comprehensive evaluation for all aspects. Following the S-Q assessment, we improve the overall practicality of medical steganography by (i) redesigning an effective U-shape generator network for removing the dependence on complex modules, (ii) improving feature extraction capability via convolutions with various kernel sizes and avoiding the checkerboard effect through upsampling layers, (iii) optimizing the loss function via the multi-scale structural similarity and high-frequency components. Extensive experiments demonstrate that MedSteGAN outperforms state-of-the-art steganographic schemes, showing the best overall results in the security-quality domain.

Noise reduction in low-dose positron emission tomography with adaptive parameter estimation in sinogram domain

Noise reduction in low-dose positron emission tomography (PET) is a well-researched topic aimed at reducing patient radiation doses and improving diagnosis. Software-based noise reduction mainly improves the contrast between regions by reducing the variation of the acquired image. However, it should be performed under appropriate parameters to reduce discrimination. We propose a method that derives optimal noise-reduction parameters using the multi-scale structural similarity index measure and visual information fidelity, which are metrics for image quality assessment. Simulation and experimental studies demonstrated the viability of the proposed algorithm. The contrast-to-noise ratio value of the denoised reconstruction slice, which was used as the optimal parameter, increased approximately three times compared to that of the low-dose slice while preserving the resolution. The results indicate that the proposed method successfully predicted the parameters according to the noise-reduction algorithm and PET system conditions in the sinogram domain. The proposed algorithm should help prevent misdiagnosis and provide standardized medical images for clinical application by performing appropriate noise reduction.

Perceptual Quality-Oriented Rate Allocation via Distillation from End-to-End Image Compression

Mainstream image/video coding standards, exemplified by the state-of-the-art H.266/VVC, AVS3, and AV1, follow the block-based hybrid coding framework. Due to the block-based framework, encoders designed for these standards are easily optimized for peak signal-to-noise ratio (PSNR) but have difficulties optimizing for the metrics more aligned to perceptual quality, e.g., multi-scale structural similarity (MS-SSIM), since these metrics cannot be accurately evaluated at the small block level. We address this problem by leveraging inspiration from the end-to-end image compression built on deep networks, which is easily optimized through network training for any metric as long as the metric is differentiable. We compared the trained models using the same network structure but different metrics and observed that the models allocate rates in different ratios. We then propose a distillation method to obtain the rate allocation rule from end-to-end image compression models with different metrics and to utilize such a rule in the block-based encoders. We implement the proposed method on the VVC reference software—VTM and the AVS3 reference software—HPM, focusing on intraframe coding. Experimental results show that the proposed method on top of VTM achieves more than 10% BD-rate reduction than the anchor when evaluated with MS-SSIM or LPIPS, which leads to concrete perceptual quality improvement.

Patient-derived PixelPrint phantoms for evaluating clinical imaging performance of a deep learning CT reconstruction algorithm.

Objective&#xD;Deep learning reconstruction (DLR) algorithms exhibit object-dependent resolution and noise performance. Thus, traditional geometric CT phantoms cannot fully capture the clinical imaging performance of DLR. This study uses a patient-derived 3D-printed PixelPrint lung phantom to evaluate a commercial DLR algorithm across a wide range of radiation dose levels.&#xD;&#xD;Method&#xD;The lung phantom used in this study is based on a patient chest CT scan containing ground glass opacities and was fabricated using PixelPrint 3D-printing technology. The phantom was placed inside two different size extension rings to mimic a small- and medium-sized patient and was scanned on a conventional CT scanner at exposures between 0.5 and 20 mGy. Each scan was reconstructed using filtered back projection (FBP), iterative reconstruction, and DLR at five levels of denoising. Image noise, contrast to noise ratio (CNR), root mean squared error (RMSE), structural similarity index (SSIM), and multi-scale SSIM (MS SSIM) were calculated for each image.&#xD;&#xD;Results&#xD;DLR demonstrated superior performance compared to FBP and iterative reconstruction for all measured metrics in both phantom sizes, with better performance for more aggressive denoising levels. DLR was estimated to reduce dose by 25-83% in the small phantom and by 50-83% in the medium phantom without decreasing image quality for any of the metrics measured in this study. These dose reduction estimates are more conservative compared to the estimates obtained when only considering noise and CNR. &#xD;&#xD;Conclusion&#xD;DLR has the capability of producing diagnostic image quality at up to 83% lower radiation dose, which can improve the clinical utility and viability of lower dose CT scans. Furthermore, the PixelPrint phantom used in this study offers an improved testing environment with more realistic tissue structures compared to traditional CT phantoms, allowing for structure-based image quality evaluation beyond noise and contrast-based assessments.

Full-dose whole-body PET synthesis from low-dose PET using high-efficiency denoising diffusion probabilistic model: PET consistency model.

Positron Emission Tomography (PET) has been a commonly used imaging modality in broad clinical applications. One of the most important tradeoffs in PET imaging is between image quality and radiation dose: high image quality comes with high radiation exposure. Improving image quality is desirable for all clinical applications while minimizing radiation exposure is needed to reduce risk to patients. We introduce PET Consistency Model (PET-CM), an efficient diffusion-based method for generating high-quality full-dose PET images from low-dose PET images. It employs a two-step process, adding Gaussian noise to full-dose PET images in the forward diffusion, and then denoising them using a PET Shifted-window Vision Transformer (PET-VIT) network in the reverse diffusion. The PET-VIT network learns a consistency function that enables direct denoising of Gaussian noise into clean full-dose PET images. PET-CM achieves state-of-the-art image quality while requiring significantly less computation time than other methods. Evaluation with normalized mean absolute error (NMAE), peak signal-to-noise ratio (PSNR), multi-scale structure similarity index (SSIM), normalized cross-correlation (NCC), and clinical evaluation including Human Ranking Score (HRS) and Standardized Uptake Value (SUV) Error analysis shows its superiority in synthesizing full-dose PET images from low-dose inputs. In experiments comparing eighth-dose to full-dose images, PET-CM demonstrated impressive performance with NMAE of 1.278±0.122%, PSNR of 33.783±0.824dB, SSIM of 0.964±0.009, NCC of 0.968±0.011, HRS of 4.543, and SUV Error of 0.255±0.318%, with an average generation time of 62 s per patient. This is a significant improvement compared to the state-of-the-art diffusion-based model with PET-CM reaching this result 12× faster. Similarly, in the quarter-dose to full-dose image experiments, PET-CM delivered competitive outcomes, achieving an NMAE of 0.973±0.066%, PSNR of 36.172±0.801dB, SSIM of 0.984±0.004, NCC of 0.990±0.005, HRS of 4.428, and SUV Error of 0.151±0.192% using the same generation process, which underlining its high quantitative and clinical precision in both denoising scenario. We propose PET-CM, the first efficient diffusion-model-based method, for estimating full-dose PET images from low-dose images. PET-CM provides comparable quality to the state-of-the-art diffusion model with higher efficiency. By utilizing this approach, it becomes possible to maintain high-quality PET images suitable for clinical use while mitigating the risks associated with radiation. The code is availble at https://github.com/shaoyanpan/Full-dose-Whole-body-PET-Synthesis-from-Low-dose-PET-Using-Consistency-Model.

Breaking the limitations with sparse inputs by variational frameworks (BLIss) in terahertz super-resolution 3D reconstruction

Data acquisition, image processing, and image quality are the long-lasting issues for terahertz (THz) 3D reconstructed imaging. Existing methods are primarily designed for 2D scenarios, given the challenges associated with obtaining super-resolution (SR) data and the absence of an efficient SR 3D reconstruction framework in conventional computed tomography (CT). Here, we demonstrate BLIss, a new approach for THz SR 3D reconstruction with sparse 2D data input. BLIss seamlessly integrates conventional CT techniques and variational framework with the core of the adapted Euler-Elastica-based model. The quantitative 3D image evaluation metrics, including the standard deviation of Gaussian, mean curvatures, and the multi-scale structural similarity index measure (MS-SSIM), validate the superior smoothness and fidelity achieved with our variational framework approach compared with conventional THz CT modal. Beyond its contributions to advancing THz SR 3D reconstruction, BLIss demonstrates potential applicability in other imaging modalities, such as X-ray and MRI. This suggests extensive impacts on the broader field of imaging applications.

Gan-based data augmentation to improve breast ultrasound and mammography mass classification

Data imbalance is a common problem in breast cancer diagnosis, to address this challenge, the research explores the use of Generative Adversarial Networks (GANs) to generate synthetic medical data. Various GAN methods, including Wasserstein GAN with Gradient Penalty (WGAN-GP), Cycle GAN, Conditional GAN, and Spectral Normalization GAN (SNGAN), were tested for data augmentation in breast regions of interest (ROIs) using mammography and ultrasound databases. The study employed real, synthetic, and hybrid ROIs (128x128 pixels) to train a Resnet network for classifying as benign (B) or malignant (M) classes. The quality and diversity of the synthetic data were assessed using several metrics: Fréchet Inception Distance (FID), Kernel Inception Distance (KID), Structural Similarity Index (SSIM), Multi-Scale SSIM (MS-SSIM), Blind Reference Image Spatial Quality Evaluator (BRISQUE), Naturalness Image Quality Evaluator (NIQE), and Perception-based Image Quality Evaluator (PIQE).Results revealed that the SNGAN model (FID = 52.89) was most effective for augmenting mammography data, while CGAN (FID = 116.03) excelled with ultrasound data. Cycle GAN and WGAN-GP, though demonstrating lower KID values, did not perform better than SNGAN and CGAN. The lower average MS-SSIM values suggested that SNGAN and CGAN produced a high diversity of synthetic images. However, lower SSIM, BRISQUE, NIQE, and PIQE values indicated poor quality in both real and synthetic images. Classification results showed high accuracy without data augmentation in both US (93.1 %B/94.9 %M) and mammography (80.9 %B/76.9 %M). The research concludes that preprocessing and characterizing ROIs by abnormality type is crucial to generate diverse synthetic data and improve accuracy in the classification process using combined GANs and CNN models.

The Accuracy and Precision of Memory for Natural Scenes: A Walk in the Park.

It is challenging to quantify the accuracy and precision of scene memory because it is unclear what 'space' scenes occupy (how can we quantify error when misremembering a natural scene?). To address this, we exploited the ecologically valid, metric space in which scenes occur and are represented: routes. In a delayed estimation task, participants briefly saw a target scene drawn from a video of an outdoor 'route loop', then used a continuous report wheel of the route to pinpoint the scene. Accuracy was high and unbiased, indicating there was no net boundary extension/contraction. Interestingly, precision was higher for routes that were more self-similar (as characterized by the half-life, in meters, of a route's Multiscale Structural Similarity index), consistent with previous work finding a 'similarity advantage' where memory precision is regulated according to task demands. Overall, scenes were remembered to within a few meters of their actual location.

Storing Images in DNA via base128 Encoding.

Current DNA storage schemes lack flexibility and consistency in processing highly redundant and correlated image data, resulting in low sequence stability and image reconstruction rates. Therefore, according to the characteristics of image storage, this paper proposes storing images in DNA via base128 encoding (DNA-base128). In the data writing stage, data segmentation and probability statistics are carried out, and then, the data block frequency and constraint encoding set are associated with achieving encoding. When the image needs to be recovered, DNA-base128 completes internal error correction by threshold setting and drift comparison. Compared with representative work, the DNA-base128 encoding results show that the undesired motifs were reduced by 71.2-90.7% and that the local guanine-cytosine content variance was reduced by 3 times, indicating that DNA-base128 can store images more stably. In addition, the structural similarity index (SSIM) and multiscale structural similarity (MS-SSIM) of image reconstruction using DNA-base128 were improved by 19-102 and 6.6-20.3%, respectively. In summary, DNA-base128 provides image encoding with internal error correction and provides a potential solution for DNA image storage. The data and code are available at the GitHub repository: https://github.com/123456wk/DNA_base128.

Technical note: Minimizing CIED artifacts on a 0.35 T MRI-Linac using deep learning.

Artifacts from implantable cardioverter defibrillators (ICDs) are a challenge to magnetic resonance imaging (MRI)-guided radiotherapy (MRgRT). This study tested an unsupervised generative adversarial network to mitigate ICD artifacts in balanced steady-state free precession (bSSFP) cine MRIs and improve image quality and tracking performance for MRgRT. Fourteen healthy volunteers (Group A) were scanned on a 0.35 T MRI-Linac with and without an MR conditional ICD taped to their left pectoral to simulate an implanted ICD. bSSFP MRI data from 12 of the volunteers were used to train a CycleGAN model to reduce ICD artifacts. The data from the remaining two volunteers were used for testing. In addition, the dataset was reorganized three times using a Leave-One-Out scheme. Tracking metrics [Dice similarity coefficient (DSC), target registration error (TRE), and 95 percentile Hausdorff distance (95% HD)] were evaluated for whole-heart contours. Image quality metrics [normalized root mean square error (nRMSE), peak signal-to-noise ratio (PSNR), and multiscale structural similarity (MS-SSIM) scores] were evaluated. The technique was also tested qualitatively on three additional ICD datasets (Group B) including a patient with an implanted ICD. For the whole-heart contour with CycleGAN reconstruction: 1) Mean DSC rose from 0.910 to 0.935; 2) Mean TRE dropped from 4.488 to 2.877mm; and 3) Mean 95% HD dropped from 10.236 to 7.700mm. For the whole-body slice with CycleGAN reconstruction: 1) Mean nRMSE dropped from 0.644 to 0.420; 2) Mean MS-SSIM rose from 0.779 to 0.819; and 3) Mean PSNR rose from 18.744 to 22.368. The three Group B datasets evaluated qualitatively displayed a reduction in ICD artifacts in the heart. CycleGAN-generated reconstructions significantly improved both tracking and image quality metrics when used to mitigate artifacts from ICDs.

High-resolution MRI synthesis using a data-driven framework with denoising diffusion probabilistic modeling

Objective. High-resolution magnetic resonance imaging (MRI) can enhance lesion diagnosis, prognosis, and delineation. However, gradient power and hardware limitations prohibit recording thin slices or sub-1 mm resolution. Furthermore, long scan time is not clinically acceptable. Conventional high-resolution images generated using statistical or analytical methods include the limitation of capturing complex, high-dimensional image data with intricate patterns and structures. This study aims to harness cutting-edge diffusion probabilistic deep learning techniques to create a framework for generating high-resolution MRI from low-resolution counterparts, improving the uncertainty of denoising diffusion probabilistic models (DDPM). Approach. DDPM includes two processes. The forward process employs a Markov chain to systematically introduce Gaussian noise to low-resolution MRI images. In the reverse process, a U-Net model is trained to denoise the forward process images and produce high-resolution images conditioned on the features of their low-resolution counterparts. The proposed framework was demonstrated using T2-weighted MRI images from institutional prostate patients and brain patients collected in the Brain Tumor Segmentation Challenge 2020 (BraTS2020). Main results. For the prostate dataset, the bicubic interpolation model (Bicubic), conditional generative-adversarial network (CGAN), and our proposed DDPM framework improved the noise quality measure from low-resolution images by 4.4%, 5.7%, and 12.8%, respectively. Our method enhanced the signal-to-noise ratios by 11.7%, surpassing Bicubic (9.8%) and CGAN (8.1%). In the BraTS2020 dataset, the proposed framework and Bicubic enhanced peak signal-to-noise ratio from resolution-degraded images by 9.1% and 5.8%. The multi-scale structural similarity indexes were 0.970 ± 0.019, 0.968 ± 0.022, and 0.967 ± 0.023 for the proposed method, CGAN, and Bicubic, respectively. Significance. This study explores a deep learning-based diffusion probabilistic framework for improving MR image resolution. Such a framework can be used to improve clinical workflow by obtaining high-resolution images without penalty of the long scan time. Future investigation will likely focus on prospectively testing the efficacy of this framework with different clinical indications.

Cross-scale Vision Transformer for crowd localization

Crowd localization can provide the positions of individuals and the total number of people, which has great application value for security monitoring and public management, meanwhile it meets the challenges of lighting, occlusion and perspective effect. In recent times, Transformer has been applied in crowd localization to overcome these challenges. Yet such kind of methods only consider to integrate the multi-scale information once, which results in incomplete multi-scale information fusion. In this paper, we propose a novel Transformer network named Cross-scale Vision Transformer (CsViT) for crowd localization, which simultaneously fuses multi-scale information during both the encoder and decoder stages and meanwhile building the long-range context dependencies on the combined feature maps. To this end, we design the multi-scale encoder to fuse the feature maps of multiple scales at corresponding positions so as to obtain the combined feature maps, and meanwhile design the multi-scale decoder to integrate the tokens at multiple scales when modeling the long-range context dependencies. Furthermore, we propose Multi-scale SSIM (MsSSIM) loss to adaptively compute head regions and optimize the similarity at multiple scales. Specifically, we set the adaptive windows with different scales for each head and compute the loss values within these windows so as to enhance the accuracy of the predicted distance transform map. We perform comprehensive experiments on five public datasets, and the results obtained validate the effectiveness of our method.

QoE-Based Performance Comparison of AVC, HEVC, and VP9 on Mobile Devices with Additional Influencing Factors

While current video quality assessment research predominantly revolves around resolutions of 4 K and beyond, targeted at ultra high-definition (UHD) displays, effective video quality for mobile video streaming remains primarily within the range of 480 p to 1080 p. In this study, we conducted a comparative analysis of the quality of experience (QoE) for widely implemented video codecs on mobile devices, specifically Advanced Video Coding (AVC), its successor High-Efficiency Video Coding (HEVC), and Google’s VP9. Our choice of 720 p video sequences from a newly developed database, all with identical bitrates, aimed to maintain a manageable subjective assessment duration, capped at 35–40 min. To mimic real-time network conditions, we generated stimuli by streaming original video clips over a controlled emulated setup, subjecting them to eight different packet-loss scenarios. We evaluated the quality and structural similarity of the distorted video clips using objective metrics, including the Video Quality Metric (VQM), Peak Signal-to-Noise Ratio (PSNR), Video Multi-Method Assessment Fusion (VMAF), and Multi-Scale Structural Similarity Index (MS-SSIM). Subsequently, we collected subjective ratings through a custom mobile application developed for Android devices. Our findings revealed that VMAF accurately represented the degradation in video quality compared to other metrics. Moreover, in most cases, HEVC exhibited an advantage over both AVC and VP9 under low packet-loss scenarios. However, it is noteworthy that in our test cases, AVC outperformed HEVC and VP9 in scenarios with high packet loss, based on both subjective and objective assessments. Our observations further indicate that user preferences for the presented content contributed to video quality ratings, emphasizing the importance of additional factors that influence the perceived video quality of end users.

Infrared and visible image fusion method based on full convolutional network (FCN)

In the IP sector, the combination of visible image fusion (VIF) with infrared (IR) gives a more comprehensive and accurate description of a target image. To get over the problems of detail and energy loss during the fusion process caused by current deep learning fusion approaches, it is proposed to use a fusion strategy of IR and visible pictures based on full convolutional network (FCN) applying transfer learning. FCN model can take any size of the input and generate constant size of the output with desired rules. Through effective inference and learning procedure, the ability of features extraction and energy conservation can be enhanced a lot. Experimental results demonstrate that the suggested method succeeds in improving IF quality over the other two comparable methods by preserving high light intensity and retrieving detail information. This also confirms its dominance across five different objective assessment indices: mutual information (MI), entropy (EN), edge-based similarity measure (Qabf), sum of correlations of differences (SCD), and multi-scale structural similarity for image (MS-SSIM).

A Multi-Stage Progressive Network with Feature Transmission and Fusion for Marine Snow Removal.

Improving underwater image quality is crucial for marine detection applications. However, in the marine environment, captured images are often affected by various degradation factors due to the complexity of underwater conditions. In addition to common color distortions, marine snow noise in underwater images is also a significant issue. The backscatter of artificial light on marine snow generates specks in images, thereby affecting image quality, scene perception, and subsequently impacting downstream tasks such as target detection and segmentation. Addressing the issues caused by marine snow noise, we have designed a new network structure. In this work, a novel skip-connection structure called a dual channel multi-scale feature transmitter (DCMFT) is implemented to reduce information loss during downsampling in the feature encoding and decoding section. Additionally, in the feature transfer process for each stage, iterative attentional feature fusion (iAFF) modules are inserted to fully utilize marine snow features extracted at different stages. Finally, to further optimize the network's performance, we incorporate the multi-scale structural similarity index (MS-SSIM) into the loss function to ensure more effective convergence during training. Through experiments conducted on the Marine Snow Removal Benchmark (MSRB) dataset with an augmented sample size, our method has achieved significant results. The experimental results demonstrate that our approach excels in removing marine snow noise, with a peak signal-to-noise ratio reaching 38.9251 dB, significantly outperforming existing methods.

Pseudo-medical image-guided technology based on 'CBCT-only' mode in esophageal cancer radiotherapy

Purpose To minimize the various errors introduced by image-guided radiotherapy (IGRT) in the application of esophageal cancer treatment, this study proposes a novel technique based on the 'CBCT-only' mode of pseudo-medical image guidance.Methods The framework of this technology consists of two pseudo-medical image synthesis models in the CBCT→CT and the CT→PET direction. The former utilizes a dual-domain parallel deep learning model called AWM-PNet, which incorporates attention waning mechanisms. This model effectively suppresses artifacts in CBCT images in both the sinogram and spatial domains while efficiently capturing important image features and contextual information. The latter leverages tumor location and shape information provided by clinical experts. It introduces a PRAM-GAN model based on a prior region aware mechanism to establish a non-linear mapping relationship between CT and PET image domains. As a result, it enables the generation of pseudo-PET images that meet the clinical requirements for radiotherapy.Results The NRMSE and multi-scale SSIM (MS-SSIM) were utilized to evaluate the test set, and the results were presented as median values with lower quartile and upper quartile ranges. For the AWM-PNet model, the NRMSE and MS-SSIM values were 0.0218 (0.0143, 0.0255) and 0.9325 (0.9141, 0.9410), respectively. The PRAM-GAN model produced NRMSE and MS-SSIM values of 0.0404 (0.0356, 0.0476) and 0.9154 (0.8971, 0.9294), respectively. Statistical analysis revealed significant differences (p < 0.05) between these models and others. The numerical results of dose metrics, including D98 %, Dmean, and D2 %, validated the accuracy of HU values in the pseudo-CT images synthesized by the AWM-PNet. Furthermore, the Dice coefficient results confirmed statistically significant differences (p < 0.05) in GTV delineation between the pseudo-PET images synthesized using the PRAM-GAN model and other compared methods.Conclusion The AWM-PNet and PRAM-GAN models have the capability to generate accurate pseudo-CT and pseudo-PET images, respectively. The pseudo-image-guided technique based on the 'CBCT-only' mode shows promising prospects for application in esophageal cancer radiotherapy.