Natural Image Quality Evaluator Research Articles

Effects of total variation regularization noise reduction algorithm in improved K-edge log-subtraction X-ray images with photon-counting cadmium telluride detectors

X-ray systems with photon-counting cadmium telluride (CdTe) detectors can achieve greatly improved images by using the K-edge log-subtraction (KELS) imaging method. This paper discusses methods for acquiring KELS images with photon-counting CdTe detectors and applying the modeled total variation (TV) regularization noise reduction algorithm to the obtained images. Monte Carlo simulation using the Geant4 Application for Tomographic Emission platform was used to model the system. To demonstrate the usefulness of the proposed TV algorithm, we investigated the normalized noise power spectrum, contrast to noise ratio, and no reference-based assessment parameter using a natural image quality evaluator. The results demonstrate that the proposed TV noise reduction regularization algorithm can better preserve image details than the conventional denoising methods in all the evaluation parameters.

Fast Single Image Defogging With Robust Sky Detection

Haze is a source of unreliability for computer vision applications in outdoor scenarios, and it is usually caused by atmospheric conditions. The Dark Channel Prior (DCP) has shown remarkable results in image defogging with three main limitations: 1) high time-consumption, 2) artifact generation, and 3) sky-region over-saturation. Therefore, current work has focused on improving processing time without losing restoration quality and avoiding image artifacts during image defogging. Hence in this research, a novel methodology based on depth approximations through DCP, local Shannon entropy, and Fast Guided Filter is proposed for reducing artifacts and improving image recovery on sky regions with low computation time. The proposed-method performance is assessed using more than 500 images from three datasets: Hybrid Subjective Testing Set from Realistic Single Image Dehazing (HSTS-RESIDE), the Synthetic Objective Testing Set from RESIDE (SOTS-RESIDE) and the HazeRD. Experimental results demonstrate that the proposed approach has an outstanding performance over state-of-the-art methods in reviewed literature, which is validated qualitatively and quantitatively through Peak Signal-to-Noise Ratio (PSNR), Naturalness Image Quality Evaluator (NIQE) and Structural SIMilarity (SSIM) index on retrieved images, considering different visual ranges, under distinct illumination and contrast conditions. Analyzing images with various resolutions, the method proposed in this work shows the lowest processing time under similar software and hardware conditions.

Image Super-Resolution Reconstruction Using Generative Adversarial Networks Based on Wide-Channel Activation

In recent years, residual learning has shown excellent performance on convolutional neural network (CNN)-based single-image super-resolution (SISR) tasks. However, CNN-based SISR approaches have focused mainly on the design of deep architectures, and the rectified linear units (ReLUs) used in these networks hinder shallow-to-deep information transfer. As a result, these methods are unable to utilize some shallow information, and improving model performance is difficult. To solve the above issues, this paper proposes an image SR reconstruction method based on a generative adversarial network with a residual dense architecture. First, before ReLU activation, the number of feature channels is expanded by a factor of 6~9 using a 1 × 1 convolutional layer, which improves the utilization of shallow information. Next, the original discriminator is replaced with a relativistic average discriminator, thereby improving the authenticity of the discriminative network. Finally, preactivation features are used to improve the perceptual loss, thus providing stronger monitoring for brightness consistency and texture restoration. Experimental results show that the proposed algorithm improves the utilization of shallow information in a deep network. Structural similarity (SSIM) index evaluations show that the overall utilization of shallow information is increased by 105.52%. In addition, the average runtime is 0.42 sec/frame, nearly 3.6 times faster than those of traditional methods. Moreover, the recovered images have an average natural image quality evaluator value of 3.4 and high perceptual quality, showing that the proposed method is suitable for image reconstruction applications in fields such as agriculture and medicine.

Multi‐exposure image fusion technique using multi‐resolution blending

This study proposes a multi-exposure image fusion (MEF) technique that takes multi-exposure input images and produces a high-quality output image without any artefacts. The proposed technique consists of four steps. In the first step, three quality measures (contrast, saturation, and well exposedness) are measured. Secondly, the colour dissimilarity approach is used to detect moving objects. Thirdly, the authors calculate the weight maps using three quality measures (contrast, saturation, and well exposedness) and colour dissimilarity feature. Finally, the fused image is generated using the multi-resolution blending utilising pyramid decomposition. The vital advantage of the proposed technique is that it blends the multi-exposure images very well and avoids the seams effectively. The method can be used for consumer cameras as the presented technique is quite fast. Experimental results prove that the proposed method produces high dynamic range images without ghost artefacts. Furthermore, the comparison, objectively and subjectively, of the proposed technique in terms of mutual information, MEF structural similarity index, and natural image quality evaluator, with state-of-the-art techniques, shows significant improvement of the proposed scheme over existing techniques.

Two‐scale image decomposition based image fusion using structure tensor

AbstractThe PET/SPECT image generates functional information of the human brain. In the field of multimodal medical image fusion, the decomposition scheme of PET/SPECT image in pseudo‐color is thought to be negligible. In this article, a two‐step model for PET/SPECT image decomposition is proposed to achieve contrast enhancement that inherently captures gradient to distinguish individual edge information from detail information. First, sharp structure is preserved using structure tensor on the intensity component of PET/SPECT image. Second, sharp structure in gray is transformed back into the image in pseudo‐color by generalized intensity‐hue‐saturation, one of the color space methods. The combination of structure tensor and generalized intensity‐hue‐saturation based fusion technique can preserve not only more intensity information but also functional information content. Finally, we demonstrate the effectiveness of our proposed decomposition method in the context of PET/SPECT image and then make a comparison of fusion result by two‐scale image fusion method in terms of the full‐reference objective metric structural similarity and no‐reference objective metric natural image quality evaluator.

Image Statistic Models Characterize Well Log Image Quality

Assessing the image quality of well logs is essential to ensure the accuracy of their digitization and subsequent processing. Currently, the suitability of well logs for information retrieval is solely determined on the basis of subjective judgments of their image quality by human experts. The success of natural scene statistics (NSS)-based models that are used to conduct no-reference (NR) quality assessment of photographic images motivates us to try to exploit them to characterize the quality of nonphotographic images, such as well logs. Accordingly, we develop a scheme to characterize the quality of a well log as “acceptable” or “unacceptable” for subsequent processing based on the natural image quality evaluator (NIQE), a successful NR image quality assessment model based on the NSS. Our experimental results show that the objective quality scores thus obtained can be reliably used to eliminate well logs of inferior quality from the processing pipeline, which can serve as a beneficial step to reduce the human hours spent in examining well logs and to improve the rate of information retrieval as well as the accuracy of retrieved information. Source code for the trained well log image quality predictor is available at https://github.com/Somdyuti2/Well_log_IQA .

Triangular Fuzzy Membership-Contrast Limited Adaptive Histogram Equalization (TFM-CLAHE) for Enhancement of Multimodal Biometric Images

The research work proposes a novel triangular fuzzy membership (TFM) function-based contrast limited adaptive histogram equalization (CLAHE) for biometric image enhancement. Biometric images have wide applications in the areas of verification and authentication systems. For accurate identification and verification, pre-processing of captured biometric images becomes essential. When the region of interest is smaller than the original image, a variation of histogram equalization called adaptive histogram equalization (AHE) is used. AHE enhances contrast of images by considering local regions. Along with local contrast, noise in those regions also get amplified by using AHE. This amplification of noise can be resolved by applying a contrast limited AHE (CLAHE) which limits the contrast in the enhanced local regions by clipping the histogram at a pre-fixed limit. CLAHE yields good results by limiting the contrast and enhancing local regions, but it is image invariant since it uses pre-determined clip limit for limiting contrast. The proposed research work TFM-CLAHE puts forward the idea of image variant, automatic clip value determinant algorithm for enhancement. The algorithm employs triangular fuzzy membership function to determine clip-limit and limits contrast by clipping the histogram at the computed clip-level. TFM function computes the clipping parameter by considering intensities of pixels. The computed fuzzy clip-limit overrides the pre-defined limit. Consequently, the clipping parameter varies according to the image under consideration and yields better enhancement results. The proposed work is experimented on multimodal biometric images acquired from Chinese Academy of Science, Institute of Automation Iris, Face and Fingerprint databases. TFM-CLAHE computes appropriate clipping limit for each of these heterogenous images. The results of the proposed work are evaluated on the grounds of images’ average information content, mean square error, peak signal noise ratio, natural image quality evaluator, no-reference free energy based robust metric, blind image quality measure of enhanced images and no reference quality metric for contrast distortion. The results show good enhancement and these are compared with existing conventional image enhancement techniques.

ARFurniture: Augmented Reality Interior Decoration Style Colorization

Augmented Reality (AR) can seamlessly create an illusion of virtual elements blended into the real world scene, which is one of the most fascinating human-machine interaction technologies. AR has been utilized in a variety of real-life applications including immersive collaborative gaming, fashion appreciation, interior design, and assistive devices for individuals with vision impairments. This paper contributes a real-time AR application, ARFurniture, which will allow the users to envision furniture-ofinterests in different colors and different styles, all from their smart devices. The core software architecture consists of deep-learning based semantic segmentation and fast-speed color transformation. Our software architecture allows the user prompt the system to colorize the style of the furniture-of-interest within the scene on their mobile devices, and has been successfully deployed on mobile devices. In addition, using eye gaze as a pointing indicator, a head-mounted user-centric augmented reality based indoor decoration style colorization concept is discussed. Related algorithms, system design, and simulation results for ARFurniture are presented. Furthermore, a no-reference image quality measure, Naturalness Image Quality Evaluator (NIQE), was utilized to evaluate the immersiveness and naturalness of ARFuniture. The results demonstrate that ARFurniture has game-changing value to enhance user experience in indoor decoration.

Abnormality Detection in Musculoskeletal Radiographs Using Capsule Network

To treat the diseases or injuries of the joints, bones, muscles, and spine in both adult and pediatric imaging the musculoskeletal radiographs bring a significant depth of expertise. Abnormality detection in the musculoskeletal study is backbreaking as more than 1.7 billion people are affected by musculoskeletal condition (BMU, 2017). Hence if we want to create enough opportunity to treat a maximum amount of patients, machine learning and deep learning can play a crucial role. CNN is an excellent deep learning method for image classification and other computer vision tasks. But CNN has exhibited some serious limitations when the images are rotated and deformed. Hence capsule network architecture is introduced in this paper for musculoskeletal radiographs abnormality detection and this capsnet architecture has shown very promising features that can help to vanquish the limitations of CNN. In addition, this capsule network has scored 10% higher kappa score than 169 layer densenet using less training data in the case of musculoskeletal radiographs abnormality detection. This feature of capsule network can help to use deep learning in such cases where an aggregate of a large amount of data is not possible. For image quality investigation, blind image spatial quality evaluator (BRISQUE) and naturalness image quality evaluator (NIQE) scores are measured and it is found that when the pixel size of the resized images are more close to the pixel size of the original images, we get a better approximation. Hence in the case of musculoskeletal radiographs abnormality detection, our method outperforms state-of-the-art method using a less amount of training data.

An objective and subjective quality assessment study of passive gaming video streaming

SummaryPassive gaming video‐streaming applications have recently gained much attention as evident with the rising popularity of many Over The Top (OTT) providers such as Twitch.tv and YouTube Gaming. For the continued success of such services, it is imperative that the user Quality of Experience (QoE) remains high, which is usually assessed using subjective and objective video quality assessment methods. Recent years have seen tremendous advancement in the field of objective video quality assessment (VQA) metrics, with the development of models that can predict the quality of the videos streamed over the Internet. A study on the performance of objective VQA on gaming videos, which are artificial and synthetic and have different streaming requirements than traditionally streamed videos, is still missing. Towards this end, we present in this paper an objective and subjective quality assessment study on gaming videos considering passive streaming applications. Subjective ratings are obtained for 90 stimuli generated by encoding six different video games in multiple resolution‐bitrate pairs. Objective quality performance evaluation considering eight widely used VQA metrics is performed using the subjective test results and on a data set of 24 reference videos and 576 compressed sequences obtained by encoding them in 24 resolution‐bitrate pairs. Our results indicate that Video Multimethod Assessment Fusion (VMAF) predicts subjective video quality ratings the best, while Naturalness Image Quality Evaluator (NIQE) turns out to be a promising alternative as a no‐reference metric in some scenarios.

Endoscopic image enhancement with noise suppression.

Stereoscopic endoscopes have been used increasingly in minimally invasive surgery to visualise the organ surface and manipulate various surgical tools. However, insufficient and irregular light sources become major challenges for endoscopic surgery. Not only do these conditions hinder image processing algorithms, sometimes surgical tools are barely visible when operating within low-light regions. In addition, low-light regions have low signal-to-noise ratio and metrication artefacts due to quantisation errors. As a result, present image enhancement methods usually suffer from heavy noise amplification in low-light regions. In this Letter, the authors propose an effective method for endoscopic image enhancement by identifying different illumination regions and designing the enhancement design criteria for desired image quality. Compared with existing image enhancement methods, the proposed method is able to enhance the low-light region while preventing noise amplification during image enhancement process. The proposed method is tested with 200 images acquired by endoscopic surgeries. Computed results show that the proposed algorithm can outperform state-of-the-art algorithms for image enhancement, in terms of naturalness image quality evaluator and illumination index.

A maximum likelihood filter using non-local information for despeckling of ultrasound images

This work presents a new ultrasound image despeckling method based on the maximum likelihood principle that effectively exploits non-local information for estimating noise-free pixels. First, a new maximum likelihood filter is proposed which uses neighborhood information to despeckle images. For this purpose, the general speckle model is used in the log-likelihood function and despeckled pixels are obtained by maximizing this function. Second, the proposed filter is developed to use non-local information such that the distribution of each noisy pixel is weighted according to the statistical distance between the patch of the noisy pixel and that of the target pixel. Because it is optimally designed for ultrasound images, the Pearson distance is used to measure the statistical distance between the patches. A series of experiments are conducted on three different ultrasound images and one synthetic image. Subjective evaluations show that the proposed method is able to preserve edges and structural details of the image and objective evaluations using equivalent number of looks, natural image quality evaluator, peak signal-to-noise ratio, mean preservation, and structural similarity confirm that the proposed method can achieve superior performance.

Image Dehazing Using Residual-Based Deep CNN

There is a series of image degradation in the image acquired in haze and other weather. The single image dehazing is a challenging and ill-posed problem. Using deep neural network methods, it solves the drawbacks of manually designing haze-related features. This paper proposes a dehazing algorithm using residual-based deep CNN. The network model is divided into two phases: in the first stage, a haze image is input, and the transmission map is estimated by network; in the second stage, the ratio of foggy image and transmission map is used as input, and the residual network is used to remove haze. It avoids the estimation of atmospheric light and improves the efficiency of dehazing. To train the proposed network, we use the NYU2 depth dataset as the training set. In the full-reference metric peak signal to noise ratio, structural similarity, and feature similarity and no-reference metric Spatial–Spectral Entropy-based Quality, Blind/Referenceless Image Spatial Quality Evaluator, and Natural Image Quality Evaluator aspect, the experimental results confirm the efficiency and robustness of the proposed method.

Nonlocal total variation based on symmetric Kullback-Leibler divergence for the ultrasound image despeckling

BackgroundUltrasound imaging is safer than other imaging modalities, because it is noninvasive and nonradiative. Speckle noise degrades the quality of ultrasound images and has negative effects on visual perception and diagnostic operations.MethodsIn this paper, a nonlocal total variation (NLTV) method for ultrasonic speckle reduction is proposed. A spatiogram similarity measurement is introduced for the similarity calculation between image patches. It is based on symmetric Kullback-Leibler (KL) divergence and signal-dependent speckle model for log-compressed ultrasound images. Each patch is regarded as a spatiogram, and the spatial distribution of each bin of the spatiogram is regarded as a weighted Gamma distribution. The similarity between the corresponding bins of the two spatiograms is computed by the symmetric KL divergence. The Split-Bregman fast algorithm is then used to solve the adapted NLTV object function. Kolmogorov-Smirnov (KS) test is performed on synthetic noisy images and real ultrasound images.ResultsWe validate our method on synthetic noisy images and clinical ultrasound images. Three measures are adopted for the quantitative evaluation of the despeckling performance: the signal-to-noise ratio (SNR), structural similarity index (SSIM), and natural image quality evaluator (NIQE). For synthetic noisy images, when the noise level increases, the proposed algorithm achieves slightly higher SNRS than that of the other two algorithms, and the SSIMS yielded by the proposed algorithm is obviously higher than that of the other two algorithms. For liver, IVUS and 3DUS images, the NIQE values are 8.25, 6.42 and 9.01, all of which are higher than that of the other two algorithms.ConclusionsThe results of the experiments over synthetic and real ultrasound images demonstrate that the proposed method outperforms current state-of-the-art despeckling methods with respect to speckle reduction and tissue texture preservation.

Bi-histogram modification method for non-uniform illumination and low-contrast images

Researchers face non-uniform illumination and low-contrast image challenges during the image-processing stage. A new contrast enhancement method is proposed in this paper to address these challenges. The proposed method first separates the dark and bright regions of an image. Then, these regions are enhanced using two new enhancers, namely, dark and bright. Modified clipped histogram equalization is then applied for contrast enhancement. Finally, the details of the image are added back into the illumination-corrected and contrast-enhanced image for the final output image. Visually, the proposed method successfully produces better images with more uniform illumination and better contrast than the state-of-the-art methods. This claim is supported by quantitative analysis that shows that the proposed method produces the best average measure of enhancement, natural image quality evaluator, and entropy values of 797 test images compared with other state-of-the-art methods.

Multi-frame example-based super-resolution using locally directional self-similarity

This paper presents a multi-frame superresolution approach to reconstruct a high-resolution image from several low-resolution video frames. The proposed algorithm consists of three steps: i) definition of a local search region for the optimal patch using motion vectors, ii) adaptive selection of the optimum patch based on lowresolution image degradation model, and iii) combination of the optimum patch and reconstructed image. As a result, the proposed algorithm can remove interpolation artifacts using directionally adaptive patch selection based on the lowresolution image degradation model. Moreover, superresolved images without distortion between consecutive frames can be generated. The proposed method provides a significantly improved super-resolution performance over existing methods in the sense of both subjective and objective measures including peak-to-peak signal-to-noise ratio (PSNR), structural similarity measure (SSIM), and naturalness image quality evaluator (NIQE). The proposed multi-frame super-resolution algorithm is designed for realtime video processing hardware by reducing the search region for optimal patches, and suitable for consumer imaging devices including ultra-high-definition (UHD) digital televisions, surveillance systems, and medical imaging systems for image restoration and enhancement.

Contrast enhancement using real coded genetic algorithm based modified histogram equalization for gray scale images

ABSTRACTHistogram equalization is a well‐known technique used for contrast enhancement. The global HE usually results in excessive contrast enhancement because of lack of control on the level of contrast enhancement. A new technique named modified histogram equalization using real coded genetic algorithm (MHERCGA) is aimed to sweep over this drawback. The primary aim of this paper is to obtain an enhanced method which keeps the original brightness. This method incorporates a provision to have a control over the level of contrast enhancement and applicable for all types of image including low contrast MRI brain images. The basic idea of this technique is to partition the input image histogram into two subhistograms based on a threshold which is obtained using Otsu's optimality principle. Then, bicriteria optimization problem is formulated to satisfy the aforementioned requirements. The subhistograms are modified by selecting optimal contrast enhancement parameters. Finally, the union of the modified subhistograms produce a contrast enhanced and details preserved output image. While developing an optimization problem, real coded genetic algorithm is applied to determine the optimal value of contrast enhancement parameters. This mechanism enhances the contrast of the input image better than the existing contemporary HE methods. The quality of the enhanced brain image indicates that the image obtained after this method can be useful for efficient detection of brain cancer in further process like segmentation, classification, etc. The performance of the proposed method is well supported by the contrast enhancement quantitative metrics such as discrete entropy and natural image quality evaluator. © 2015 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 25, 24–32, 2015

Multipeak Mean Based Optimized Histogram Modification Framework Using Swarm Intelligence for Image Contrast Enhancement

A novel approach, Multipeak mean based optimized histogram modification framework (MMOHM) is introduced for the purpose of enhancing the contrast as well as preserving essential details for any given gray scale and colour images. The basic idea of this technique is the calculation of multiple peaks (local maxima) from the original histogram. The mean value of multiple peaks is computed and the input image’s histogram is segmented into two subhistograms based on this multipeak mean (mmean) value. Then, a bicriteria optimization problem is formulated and the subhistograms are modified by selecting optimal contrast enhancement parameters. While formulating the enhancement parameters, particle swarm optimization is employed to find optimal values of them. Finally, the union of the modified subhistograms produces a contrast enhanced and details preserved output image. This mechanism enhances the contrast of the input image better than the existing contemporary HE methods. The performance of the proposed method is well supported by the contrast enhancement quantitative metrics such as discrete entropy, natural image quality evaluator, and absolute mean brightness error.

Using the Natural Scenes’ Edges for Assessing Image Quality Blindly and Efficiently

Two real blind/no-reference (NR) image quality assessment (IQA) algorithms in the spatial domain are developed. To measure image quality, the introduced approach uses an unprecedented concept for gathering a set of novel features based on edges of natural scenes. The enhanced sensitivity of the human eye to the information carried by edge and contour of an image supports this claim. The effectiveness of the proposed technique in quantifying image quality has been studied. The gathered features are formed using both Weibull distribution statistics and two sharpness functions to devise two separate NR IQA algorithms. The presented algorithms do not need training on databases of human judgments or even prior knowledge about expected distortions, so they are real NR IQA algorithms. In contrast to the most general no-reference IQA, the model used for this study is generic and has been created in such a way that it is not specified to any particular distortion type. When testing the proposed algorithms on LIVE database, experiments show that they correlate well with subjective opinion scores. They also show that the introduced methods significantly outperform the popular full-reference peak signal-to-noise ratio (PSNR) and the structural similarity (SSIM) methods. Besides they outperform the recently developed NR natural image quality evaluator (NIQE) model.

Entirely Blind Image Quality Assessment Estimator

Prior knowledge about anticipated distortions and their corresponding human opinion scores is needed in the most general purpose no-reference image quality assessment algorithms. When creating the model, all distortion types may not be existed. Predicting the quality of distorted images in practical no-reference image quality assessment algorithms is devised without prior knowledge about images or their distortions. In this study, a blind/no-reference opinion and distortion unaware image quality assessment algorithm based on natural scenes is developed. The proposed approach uses a set of novel features to measure image quality in a spatial domain. The extracted features which are from the scenes gist are formed using Weibull distribution statistics. When testing the proposed algorithm on LIVE database, experiments show that it correlates well with subjective opinion scores. They also show that the proposed algorithm significantly outperforms the popular full-reference peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) methods. Not only do the results reasonably well compete with the recently developed natural image quality evaluator (NIQE) model, but also outperform it.