Subjective Visual Evaluation Research Articles

Dark Channel Based Multiframe Super-Resolution Reconstruction

Multiframe super-resolution (MFSR) can obtain a high-resolution image from a set of low-resolution images. The performance of super-resolution is affected by the image prior information. The current super-resolution algorithms typically use total variation prior and its improved version, restoring the image edges well. However, it will produce artifacts and stair effects in the smooth region of the image. Therefore, we propose a dark channel-based MFSR algorithm to achieve edge-preserving and noise-suppressing. Firstly, the total variation prior is used to ensure the edge-preserving ability of the algorithm. Secondly, the dark channel prior is added to suppress artifacts and stair effects. Finally, the weights of the prior terms are iteratively adapted to obtain the final high-resolution image. Experiments show that the proposed algorithm can achieve a better result in objective and subjective visual evaluations.

Underwater Image Restoration and Enhancement Based on a Fusion Algorithm With Color Balance, Contrast Optimization, and Histogram Stretching

A fusion algorithm is proposed for the restoration and enhancement of underwater images. Color balance, contrast optimization and histogram stretching are carried out. To alleviate the effect of color shift in an underwater image, the scalar values of R, G, B channels are renewed so that the distributions of the three channels in histogram are similar. Instead of refining the transmittance in dark channel prior based restoration, an optimized contrast algorithm is employed by which the optimal transmittance is determined. To further improve the brightness and contrast of underwater images, a histogram stretching algorithm based on the red channel is given. To verify the effectiveness of the proposed fusion algorithm, experimental underwater images are treated. Results show that the quality of underwater images is improved significantly, both in term of subjective visual effect and objective evaluation. The proposed underwater image processing strategy is also compared with some popular techniques. Comparison results indicate the advantage of the proposed strategy over others.

Multimodal Medical Image Fusion Based on Weighted Local Energy Matching Measurement and Improved Spatial Frequency

Multimodal medical image fusion (MMIF) technology can effectively improve the efficiency and accuracy of doctors in clinical diagnosis and treatment by combining different modes of medical images into a medical image with rich information. To achieve an optimal balance between computational loss and fusion quality, this article presents a new MMIF method based on weighted local energy matching measurement (WLEMM) and improved spatial frequency (SF). First, the latent low-rank representation (LatLRR) is employed to learn a decomposition matrix that is applied to decompose a source image into its base part and saliency part. Then, in order to adaptively select a fusion strategy for the base part, we propose a novel fusion rule, namely WLEMM, which is constructed by calculating the matching degree between the corresponding pixels of the base parts. Meanwhile, after considering the characteristics of the main diagonal SF and the secondary diagonal SF, a fusion strategy based on improved SF and L2 norm is designed to merge saliency parts. Finally, the fused image is obtained by combining the fused base part and saliency part. Various experiments on multiple groups of medical images demonstrate that the proposed method has superior performance compared with the state-of-the-art fusion methods, in subjective visual and objective index evaluations. Furthermore, the research work of this article is helpful for further detection and segmentation of medical images.

Multi-focus image fusion algorithm based on supervised learning for fully convolutional neural network

Abstract To improve the quality of multi-focus image fusion in photography applications, a multi-focus image fusion algorithm based on supervised learning for fully convolutional network is proposed. The aim of this algorithm is to make the neural network learn the complementary relationship between different focusing areas of source images, which is to select different focusing positions of the source images to synthesize a global clear image. In this algorithm, focusing images are constructed as training data. Dense connection and 1 × 1 convolution are used in the network to improve the understanding ability and efficiency of the network. The result of experiment shows that the proposed algorithm is superior to other contrast algorithms in both subjective visual evaluation and objective evaluation, and the quality of image fusion is significantly improved. Code is available at https://github.com/littlebaba/SF_MFIF.

Subjective Visual Vertical Evaluation by a Smartphone-based Test-Taking the Phone Out of the Bucket.

To measure the subjective visual vertical (SVV) in patients suffering from peripheral vestibular disorders versus controls, using a smartphone-based test designed to simulate the bucket test, in order to validate it as an available tool for the clinician. Prospective cohort study. Academic tertiary medical center. Forty-five adult patients were recruited to the study, 25 had vestibular disorders, and 20 did not (controls). All patients underwent conventional bucket-SVV (b-SVV) and smartphone-based SVV (s-SVV) testing. Correlation and agreement of b-SVV and s-SVV scores in patients with peripheral vestibular disorders compared to controls. SVV score in the vestibular disorders group was significantly higher compared to controls in both testing methods (p < 0.001 for b-SVV and p = 0.01 for s-SVV, effect size d = 1.7 for both testing methods). Intragroup correlation was excellent within the study group. Spearman's rank correlation coefficient between b-SVV and s-SVV was 0.902 (p = 0.01). Agreement measurements suggested a greater sensitivity for the b-SVV test, showing a mean difference of 1.088 degree (SD ± 1.77); directionality, however, was preserved. The smartphone-based test is a valid, simple, and efficient in-office screening tool for assisting in the diagnosis of vestibular disorders.

Radiographic assessment of carpal conformation in the horse: Technique development and validation of the consistency of measurements.

Carpal conformation is often considered as a contributory factor to performance and lameness in the horse; however, few attempts have been made to objectively measure radiographic variations of carpal conformation in horses due to insufficient measurable carpal parameters. This pilot study used carpal radiographic images acquired from 10 cadaveric equine forelimbs transected at the antebrachial midshaft from 7 adult horses (7.2±2.6years), positioned at 'zero lateromedial' (ZLM) and 'zero dorsopalmar' (ZDP) views, to investigate the anatomy of the equine carpus and develop parameters that could be objectively used to assess carpal conformation in horses. Dorsal carpal angle (DCA: 176.61±0.66º), distal radial slope carpal angle (DRSCA: 145.59±2.19º), intermediate carpal bone proximal tuberosity-radial angle (CiPxTRA: 115.69±3.15º) and third carpal bone palmar facet angle (C3PalFCA: 84.43±1.13º) were all developed from the ZLM view while medial carpal angle (MCA: 183.34±1.02º), disto-dorsal slope angle of the third carpal bone (C3DDSA: 8.27±0.92º) and width ratio of distal radius to proximal metacarpus (WDR:WPM=1.13±0.03) were 3 of the 10 parameters developed from the ZDP view. Easy to identify and measurable parameters will help to provide quantitative assessment of carpal conformation in the horse with potential of eliminating subjective observational variation errors between clinicians. These newly developed parameters will be useful in further studies to measure variations in the conformation of the equine carpus in live horses and comparison between subjective visual assessment and objective radiographic evaluation methods.

Multifocus Image Fusion Method Based on Convolutional Deep Belief Network

Multifocus image fusion is a technique that can integrate the focus information of different source images into a single composite image. At present, most fusion methods still suffer from problems such as block artifacts, artificial edges, halo effects, ringing effects, and contrast reduction. To address these problems, a novel multifocus image fusion method based on a convolutional deep belief network is proposed in this paper. The convolutional operator can effectively extract the focus information of source images, and the focused features extracted from source images can effectively distinguish focused windows from defocused windows. After multiple rounds of training, the convolutional deep belief network model can establish an effective mapping between source images and a score map, which is essential to generate an accurate focus map. Then, the focus map is further modified using binary segmentation and small region filtering, and the final decision map for fusion is obtained. Finally, according to the weights provided by the final decision map, a final fusion image will be formed by fusing multiple source images. The experimental results show that the proposed method is superior to other existing fusion methods in terms of subjective visual effects and objective quantitative evaluation. © 2020 Institute of Electrical Engineers of Japan. Published by Wiley Periodicals LLC.

Multi-scale fractal residual network for image super-resolution

Recent studies have shown that the use of deep convolutional neural networks (CNNs) can improve the performance of single image super-resolution reconstruction (SISR) methods. However, the existing CNN-based SISR model ignores the multi-scale features and shallow and deep features of the image, resulting in relatively low image reconstruction performance. To address these issues, this paper proposes a new multi-scale fractal residual network (MSFRN) for image super-resolution. On the basis of residual learning, a multi-scale fractal residual block (MSFRB) is designed. This block uses convolution kernels of different sizes to extract image multi-scale features and uses multiple paths to extract and fuse image features of different depths. Then, the shallow features extracted at the shallow feature extraction stage and the local features output by all MSFRBs are used to perform global hierarchical feature fusion. Finally, through sub-pixel convolution, the fused global features are used to reconstruct high-resolution images from low-resolution images. The experimental results on the five standard benchmark datasets show that MSFRN improved subjective visual effects and objective image quality evaluation indicators, and is superior to other state-of-the-art SISR methods.

Improving medical image fusion method using fuzzy entropy and nonsubsampling contourlet transform

AbstractMany types of medical images must be fused, as single‐modality medical images can only provide limited information due to the imaging principles and the complexity of human organ structures. In this paper, a multimodal medical image fusion method that combines the advantages of nonsubsampling contourlet transform (NSCT) and fuzzy entropy is proposed to provide a basis for clinical diagnosis and improve the accuracy of target recognition and the quality of fused images. An image is initially decomposed into low‐ and high‐frequency subbands through NSCT. The corresponding fusion rules are adopted in accordance with the different characteristics of the low‐ and high‐frequency components. The membership degree of low‐frequency coefficients is calculated. The fuzzy entropy is also computed and subsequently used to guide the fusion of coefficients to preserve image details. High‐frequency components are fused by maximizing the regional energy. The final fused image is obtained by inverse transformation. Experimental results show that the proposed method achieves good fusion effect based on the subjective visual effect and objective evaluation criteria. This method can also obtain high average gradient, SD, and edge preservation and effectively retain the details of the fused image. The results of the proposed algorithm can provide effective reference for doctors to assess patient condition.

Simulation-Based Optimisation for Urban Environmental Noise Coupled with Auditory and Visual Subjective Evaluation

Environmental noise has a major negative impact, causing interference for people to communicate, rest, sleep, and general annoyance. In order to control urban environmental noise, simulation-based optimisation for urban environmental noise is conducted. On the basis of the current acoustic environment of Zhaohui Campus of Zhejiang University of Technology, three optimisation plans that include noise barrier optimisation, greenbelt optimisation, and a combination optimisation of noise barrier and greenbelt are proposed. These scenarios are simulated using NoiseSystem software to obtain their ability to reduce noise and determine auditory perception by using different control measures. To determine the visual perception of the optimisation plan, 100 people of different age groups were randomly selected, and each person scored the noise barrier and the greenbelt in the simulated scenarios. Results of the comprehensive evaluation of auditory and visual perception show that noise barriers can provide better auditory feelings, but greenbelts can provide a better auditory and visual subjective experience. Therefore, in addition to the application of noise control measures, their visual design must be fully considered. Moreover, the visual perception of plants is stronger than that of others.

Infrared and Visible Image Fusion with Hybrid Image Filtering

Image fusion is an important technique aiming to generate a composite image from multiple images of the same scene. Infrared and visible images can provide the same scene information from different aspects, which is useful for target recognition. But the existing fusion methods cannot well preserve the thermal radiation and appearance information simultaneously. Thus, we propose an infrared and visible image fusion method by hybrid image filtering. We represent the fusion problem with a divide and conquer strategy. A Gaussian filter is used to decompose the source images into base layers and detail layers. An improved co-occurrence filter fuses the detail layers for preserving the thermal radiation of the source images. A guided filter fuses the base layers for retaining the background appearance information of the source images. Superposition of the fused base layer and fused detail layer generates the final fusion image. Subjective visual and objective quantitative evaluations comparing with other fusion algorithms demonstrate the better performance of the proposed method.

Medical image fusion using the PCNN based on IQPSO in NSST domain

In this study, an improved quantum-behaved particle swarm optimisation based pulse-coupled neural network (IQPSO-PCNN) is proposed in the non-subsampled shearlet transform (NSST) domain for medical image fusion. First, NSST tool is used to decompose the source image into low-frequency and high-frequency subbands. Then, for low-frequency subbands, the fusion rules of two different functions are presented, which simultaneously addresses two key issues of energy preservation and detail extraction. For high-frequency subbands, unlike conventional PCNN-based methods, parameters are manually set based on experience, and the decomposed high-frequency subbands share a set of parameters. The IQPSO-PCNN model can obtain the optimal parameters for each high-frequency subband adaptively according to its own information. Finally, the fused low-frequency subband and high-frequency subbands are inversely transformed by NSST to acquire the final fused image. The proposed algorithm uses >90 pairs of images with four different modalities. In addition, fusion experiments are performed on different sequences of the three modes. The experimental results demonstrate that the proposed method is superior to existing state-of-art methods in subjective visual performance and objective evaluation.

FPGA Acceleration of Color Interpolation Algorithm Based on Gradient and Color Difference

To quickly and efficiently achieve a high-quality color image on the embedded platform, designed and used FPGA to complete Bayer pattern image Color interpolation. In this paper, we proposed a color interpolation algorithm based on image gradient direction and color difference. The algorithm calculates the G component according to the gradient value and color difference at the pixel to be interpolated, and then calculates the missing R component and B component based on the G component interpolation. Finally, the pixels of the image boundary area are interpolated. Tested on the Kodak dataset, the experimental results show that the subjective visual quality and objective image quality evaluation of the algorithm is superior to the conventional algorithm. On the other hand, and reduce problems such as image aliasing and edge blurring. The proposed algorithm has been applied to the actual Xilinx’s FPGA platform, a real-time speed of 120 MHz, enabling real-time image color Interpolation.

Indocyanine green fluorescence angiography to evaluate anastomotic perfusion in colorectal surgery.

The aim of this study was to evaluate the efficacy of indocyanine green (ICG) fluorescence angiography with respect to the anastomotic leakage rate for patients undergoing colorectal operations. This prospective cohort involved patients who underwent colorectal surgery between August 2018 and September 2019. ICG was injected after colonic transection. Vascular perfusion was observed by ICG fluorescence system before completing anastomosis. Data was compared with those by subjective visual evaluation. The primary outcome was anastomotic leakage rate within 30days from surgery. A total of 131 patients were enrolled, of which ICG was injected in 63 of them. Demographic data were similar between the two groups. There were two (3.23%) and three (4.35%) anastomotic leaks in the ICG and non-ICG group respectively (p = 1.000). Change of resection plane occurred in one patient in the ICG group. There was no ICG related toxicity or adverse events. ICG fluorescent imaging is a feasible and safe tool to assess colonic vascularisation for patients undergoing colorectal surgery. However, it did not significantly lower the anastomotic leakage rate. ICG should not be routinely used in colorectal surgery before an available large scale randomised controlled trial to prove any clinical benefits.

Diagnostic Drama. Use of ICDAS II and Fluorescence-Based Intraoral Camera in Early Occlusal Caries Detection: A Clinical Study.

Background: Early diagnosis of occlusal caries is of paramount importance for a minimally invasive approach in dentistry. The aim of the present in vivo clinical prospective study was to compare the diagnostic outcomes of visual subjective evaluation between the International Caries Detection and Assessment System (ICDAS-II) and an intraoral fluorescence-based camera (VistaCam iX Proof, Dürr Dental, Bietigheim-Bissingen, Germany) for the detection of pits and fissures in early caries lesions of posterior teeth. Methods: The study included 1011 posterior teeth in 255 patients aged 13–20 years (mean age 16 ± 2.2 years). Two blinded operators evaluated all the occlusal surfaces and the first assigned an ICDAS-II code, while the second assessed the VistaCam score: sound enamel (score 0–1.2); initial enamel decay (score 1.2–1.5); dentine caries (score 1.5–3). Results: Some 283 (28%) of the assessed teeth were ICDAS-II code 0; 334 (33%) code 1; 189 (18.7%) code 2; 176 (17.4%) code 3; and 29 (2.9%) code 4. The level of agreement between the two procedures was expressed by using Cohen’s and Fleiss’ kappa statistics and performing McNemar’s test. VistaCam assessed in 513 (50.7%) sound enamel; in 292 (28.9%) initial enamel decay; and in 206 (20.4%) dentine caries. Conclusions: This comparative study showed a poor agreement between the two diagnostic methods, especially between ICDAS-II 0, 1 and 2 codes and fluorescence assessments.

Generative Adversarial Network for Image Super-Resolution Combining Texture Loss

Objective: Super-resolution reconstruction is an increasingly important area in computer vision. To alleviate the problems that super-resolution reconstruction models based on generative adversarial networks are difficult to train and contain artifacts in reconstruction results, we propose a novel and improved algorithm. Methods: This paper presented TSRGAN (Super-Resolution Generative Adversarial Networks Combining Texture Loss) model which was also based on generative adversarial networks. We redefined the generator network and discriminator network. Firstly, on the network structure, residual dense blocks without excess batch normalization layers were used to form generator network. Visual Geometry Group (VGG)19 network was adopted as the basic framework of discriminator network. Secondly, in the loss function, the weighting of the four loss functions of texture loss, perceptual loss, adversarial loss and content loss was used as the objective function of generator. Texture loss was proposed to encourage local information matching. Perceptual loss was enhanced by employing the features before activation layer to calculate. Adversarial loss was optimized based on WGAN-GP (Wasserstein GAN with Gradient Penalty) theory. Content loss was used to ensure the accuracy of low-frequency information. During the optimization process, the target image information was reconstructed from different angles of high and low frequencies. Results: The experimental results showed that our method made the average Peak Signal to Noise Ratio of reconstructed images reach 27.99 dB and the average Structural Similarity Index reach 0.778 without losing too much speed, which was superior to other comparison algorithms in objective evaluation index. What is more, TSRGAN significantly improved subjective visual evaluations such as brightness information and texture details. We found that it could generate images with more realistic textures and more accurate brightness, which were more in line with human visual evaluation. Conclusions: Our improvements to the network structure could reduce the model’s calculation amount and stabilize the training direction. In addition, the loss function we present for generator could provide stronger supervision for restoring realistic textures and achieving brightness consistency. Experimental results prove the effectiveness and superiority of TSRGAN algorithm.

Novel fusion method for SAR and optical images based on non-subsampled shearlet transform

ABSTRACTDue to the different imaging modes of SAR images and optical images, traditional image fusion methods are no longer suitable for the fusion of the two types of images. Fused images often have problems of spectral distortion and excessive introduction of noise. This study proposes an improved SAR-optical images fusion algorithm based on non-subsampled shearlet transform (NSST). NSST decomposition is first performed on the two types of images. Then, in the low-frequency sub-band of the decomposition image, a weighted average fusion rule using the coefficient of variation according to the different imaging characteristics of SAR images and optical images is proposed to avoid spectral distortion. In the high-frequency sub-band of the decomposition image, the effect of SAR image noise on the fused image is removed by setting the coherence coefficient threshold. The subjective visual assessment and objective index evaluation on the experimental results both show that the fusion results using the proposed algorithm are significantly improved. The proposed algorithm smoothly fuses the detailed information of the SAR image into the optical image without the excessive introduction of noise while maintaining the spectral information of the optical image. Meanwhile, the proposed algorithm has a relatively simpler mathematical structure compared to the algorithm based on sparse representation, thus reducing the operating time and manual work.

Multi-focus image fusion method using energy of Laplacian and a deep neural network.

Multi-focus image fusion consists in the integration of the focus regions of multiple source images into a single image. At present, there are still some common problems in image fusion methods, such as block artifacts, artificial edges, halo effects, and contrast reduction. To address these problems, a novel, to the best of our knowledge, multi-focus image fusion method using energy of Laplacian and a deep neural network (DNN) is proposed in this paper. The DNN is composed of multiple denoising autoencoders and a classifier. The Laplacian energy operator can effectively extract the focus information of source images, and the trained DNN model can establish a valid mapping relationship between source images and a focus map according to the extracted focus information. First, the Laplacian energy operator is used to perform focus measurement for two source images to obtain the corresponding focus information maps. Then, the sliding window technology is used to sequentially obtain the windows from the corresponding focus information map, and all of the windows are fed back to the trained DNN model to obtain a focus map. After binary segmentation and small region filtering, a final decision map with good consistency is obtained. Finally, according to the weights provided by the final decision map, multiple source images are fused to obtain a final fusion image. Experimental results demonstrate that the proposed fusion method is superior to other existing ones in terms of subjective visual effects and objective quantitative evaluation.

Single Image Super-Resolution: Depthwise Separable Convolution Super-Resolution Generative Adversarial Network

The super-resolution generative adversarial network (SRGAN) is a seminal work that is capable of generating realistic textures during single image super-resolution. However, the hallucinated details are often accompanied by unpleasant artifacts. To further enhance the visual quality, we propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The method is based on depthwise separable convolution super-resolution generative adversarial network (DSCSRGAN). A new depthwise separable convolution dense block (DSC Dense Block) was designed for the generator network, which improved the ability to represent and extract image features, while greatly reducing the total amount of parameters. For the discriminator network, the batch normalization (BN) layer was discarded, and the problem of artifacts was reduced. A frequency energy similarity loss function was designed to constrain the generator network to generate better super-resolution images. Experiments on several different datasets showed that the peak signal-to-noise ratio (PSNR) was improved by more than 3 dB, structural similarity index (SSIM) was increased by 16%, and the total parameter was reduced to 42.8% compared with the original model. Combining various objective indicators and subjective visual evaluation, the algorithm was shown to generate richer image details, clearer texture, and lower complexity.

Ultra‐low‐dose chest computed tomography without anesthesia in the assessment of pediatric pulmonary diseases

ObjectiveTo evaluate the feasibility of using ultra‐low‐dose computed tomography of the chest with iterative reconstruction without anesthesia for assessment of pulmonary diseases in children. MethodsThis prospective study enrolled 86 consecutive pediatric patients (ranging from 1 month to 18 years) that underwent ultra‐low‐dose computed tomography due to suspicion of pulmonary diseases, without anesthesia and contrast. Parameters used were: 80kVp; 15–30mA; acquisition time, 0.5s; and pitch, 1.375. The adaptive statistical iterative reconstruction technique was used. Subjective visual evaluation and quantitative assessment of image quality were done using a 5‐point scale in 12 different structures of the chest. ResultsMean age was 66 months (interquartile range, 16–147). Final diagnosis was performed in all exams, and 44 (51.2%) were diagnosed with cystic fibrosis, 27 (31.4%) with bronchiolitis obliterans, and 15 (17.4%) with congenital pulmonary airways malformations. Diagnostic quality was achieved in 98.9%, of which 82.6% were considered excellent and 16.3% were slightly blurred but did not interfere with image evaluation. Only one case (1.2%) presented moderate blurring that slightly compromised the image, and previous examinations demonstrated findings compatible with bronchiolitis obliterans. Mean effective radiation dose was 0.39±0.15mSv. Percentages of images with motion artifacts were 0.3% for cystic fibrosis, 1.3% for bronchiolitis obliterans, and 1.1% for congenital pulmonary airways malformations. ConclusionChest ultra‐low‐dose computed tomography without sedation or anesthesia delivering a sub‐millisievert dose can provide image quality to allow identification of common pulmonary anatomy and diseases.