CSIQ Databases Research Articles

Feature rectification and enhancement for no-reference image quality assessment

The assessment of image is a critical task in computer vision, particularly in a no-referenced context, which presents numerous challenges. Though Deep neural networks have exhibited excellent performance on no-reference quality assessment (NR-IQA), the distortion-aware NR-IQA methods are limited to specific distortion type and struggle to handle realistic distortion scenarios. In this work, we propose a feature rectification and enhancement convolutional neural network namely FREIQA for NR-IQA. Our approach targets to fully utilize the multi-stage semantic features to produce a well fused and rectified score vector for effective image quality regression. Specifically, the multi-stage semantic features extracted by a pre-trained backbone would be rectified with a multi-level channel attention module before integrating to a global score vector. The fused score vector then will be gradually optimized through the perception feature provided by the multi-stage semantic feature, and finally sent to quality regression network to obtain the image quality score. Experimental results on public benchmarks including LIVE, TID2013, CSIQ, LIVEC, KADID-10k, KonIQ-10k and Waterloo exploration database show that our FREIQA achieve the state-of-the-art performance.

A systematic approach for enhancement of homogeneous background images using structural information

Image enhancement is an indispensable pre-processing step for several image processing applications. Mainly, histogram equalization is one of the widespread techniques used by various researchers to improve the image quality by expanding the pixel values to fill the entire dynamic grayscale. It results in the visual artifact, structural information loss near edges due to the information loss (due to many-to-one mapping), and alteration in average luminance to a higher value. This paper proposes an enhancement algorithm based on structural information for homogeneous background images. The intensities are divided into two segments using the median value to preserve the average luminance. Unlike traditional techniques, this algorithm incorporates the spatial locations in the equalization process instead of the number of intensity values occurrences. The occurrences of each intensity concerning their spatial locations are combined using Rènyi entropy to enumerate a discrete function. An adaptive clipping limit is applied to the discrete function to control the enhancement rate. Then histogram equalization is performed on each segment separately, and the equalized segments are integrated to produce an enhanced image. The algorithm’s effectiveness is validated by evaluating the proposed method on CEED, CSIQ, LOL, and TID2013 databases. Experimental results reveal that the proposed method improves the contrast while preserving structural information, detail information, and average luminance. They are quantified by the high value of contrast improvement index, structural similarity index, and discrete entropy, and low value of average mean brightness error values of the proposed method when compared with the methods available in the literature, including deep learning architectures.

Unifying Dual-Attention and Siamese Transformer Network for Full-Reference Image Quality Assessment

Image Quality Assessment (IQA) is a critical task of computer vision. Most Full-Reference (FR) IQA methods have limitation in the accurate prediction of perceptual qualities of the traditional distorted images and the Generative Adversarial Networks (GANs) based distorted images. To address this issue, we propose a novel method by Unifying Dual-Attention and Siamese Transformer Network (UniDASTN) for FR-IQA. An important contribution is the spatial attention module composed of a Siamese Transformer Network and a feature fusion block. It can focus on significant regions and effectively maps the perceptual differences between the reference and distorted images to a latent distance for distortion evaluation. Another contribution is the dual-attention strategy that exploits channel attention and spatial attention to aggregate features for enhancing distortion sensitivity. In addition, a novel loss function is designed by jointly exploiting Mean Square Error (MSE), bidirectional Kullback–Leibler divergence, and rank order of quality scores. The designed loss function can offer stable training and thus enables the proposed UniDASTN to effectively learn visual perceptual image quality. Extensive experiments on standard IQA databases are conducted to validate the effectiveness of the proposed UniDASTN. The IQA results demonstrate that the proposed UniDASTN outperforms some state-of-the-art FR-IQA methods on the LIVE, CSIQ, TID2013, and PIPAL databases.

No-Reference Image Quality Assessment via Multibranch Convolutional Neural Networks

No-reference image quality assessment (NR-IQA) aims to evaluate image quality without using the original reference images. Since the early NR-IQA methods based on distortion types were only applicable to specific distortion scenarios, and lack of practicality, it is challenging to designing a universal NR-IQA method. In this article, a multibranch convolutional neural network (MB-CNN) based NR-IQA method is proposed, which includes a spatial-domain feature extractor, a gradient-domain feature extractor, and a weight mechanism. The spatial-domain feature extractor aims to extract the distortion features from the spatial domain. The gradient-domain feature extractor is used to guide the spatial-domain feature extractor to pay more attention to the distortions of the structure information. Particularly, the spatial-domain feature extractor uses the hierarchical feature merge module to realize multiscale feature representation, and the gradient-domain feature extractor uses pyramidal convolution to extract the multiscale structure information of the distorted image. In addition, a position vector is proposed to build the weight mechanism by considering the position relationships between patches and its entire image for improving the final prediction performance. We conduct the experiments on five representative databases: LIVE, TID2013, CSIQ, LIVE MD and Waterloo Exploration Database, and the experimental results show that the proposed NR-IQA method achieves the state-of-the-art performance, which demonstrate the effectiveness of our proposed NR-IQA method. The code ofthe proposed MB-CNN will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/NUIST-Videocoding/MB-CNN</uri> .

No-Reference Framework for Image Quality Assessment

The technological advances in computer and communication devices have been an important factor in the appearance for the applications of visual communication. Image quality assessment is one of the pillars for these applications. It is a measure to assess the quality of images. There are many degradations in image quality that occur during the reproduction and transmission of the image. This paper aims to introduce a framework of No-Reference Image Quality Assessment. The framework provides a general estimation for three types of image distortions which are sharpness, blackness, and noisiness. Based on this framework, experiments have been conducted by the use of two datasets (LIVE database, CSIQ database). The results of the experiments have shown that we have introduced a more precise framework of No-Reference Image Quality Assessment.

Channel Recombination and Projection Network for Blind Image Quality Measurement

In this paper, a novel channel recombination and projection network-based no-reference image quality assessment (NR-IQA) is proposed, which is termed as CRPNet. The proposed CRPNet is composed of four parts, a feature extractor, a channel recombination strategy, a saliency-guided selective projection, and a channel score weighting. The feature extractor is first utilized to learn patch-level features from patches to address the lack of training samples. The channel recombination strategy is proposed to obtain image-level features by recombining patch-level features in channel dimension, which can solve the mismatch problem between the image score and the patch-level quality. To further increase the image quality prediction accuracy, the saliency-guided selective projection is designed, in which the mapping ratios of fully connected layers are calculated by the saliency priority of patches. Moreover, a channel score weighting is introduced to enhance the image visual quality representation. Experimental results on six image quality assessment datasets, e.g., LIVE, TID2013, CSIQ, LIVE MD, LIVE CH and Waterloo Exploration Database, show that the proposed CRPNet outperforms the state-of-the-art NR-IQA methods, and achieves the superior performance. Code is available at https: //github.com/zhaob10/CRPNet.

Precise No-Reference Image Quality Evaluation Based on Distortion Identification

The difficulty of no-reference image quality assessment (NR IQA) often lies in the lack of knowledge about the distortion in the image, which makes quality assessment blind and thus inefficient. To tackle such issue, in this article, we propose a novel scheme for precise NR IQA, which includes two successive steps, i.e., distortion identification and targeted quality evaluation. In the first step, we employ the well-known Inception-ResNet-v2 neural network to train a classifier that classifies the possible distortion in the image into the four most common distortion types, i.e., Gaussian white noise (WN), Gaussian blur (GB), jpeg compression (JPEG), and jpeg2000 compression (JP2K). Specifically, the deep neural network is trained on the large-scale Waterloo Exploration database, which ensures the robustness and high performance of distortion classification. In the second step, after determining the distortion type of the image, we then design a specific approach to quantify the image distortion level, which can estimate the image quality specially and more precisely. Extensive experiments performed on LIVE, TID2013, CSIQ, and Waterloo Exploration databases demonstrate that (1) the accuracy of our distortion classification is higher than that of the state-of-the-art distortion classification methods, and (2) the proposed NR IQA method outperforms the state-of-the-art NR IQA methods in quantifying the image quality.

Image quality assessment based on the image contents visual perception

Image quality assessment (IQA) is widely used in image transmission, processing, and storage, which has important application value. To obtain an IQA model with the excellent performance, combining with the image content features and human visual system characteristics, based on the contrast definition in visual psychology, an IQA method and its mathematical model are proposed. In this method, first based on the contrast definition, combining with the visual characteristics, the contrast between the distorted image and the reference image is used to describe the difference between them; further, based on the difference, a definition method for image quality is proposed. Second, based on the gray and gradient co-occurrence matrix, a concept, which is called as the image gray-gradient expectation (GGE), is proposed, and its calculation method is also illustrated. And based on the GGE values and local contrast of image, a method for describing the image content and its visual perception is proposed. Finally, based on the image content features and the proposed definition method of image quality, an IQA method and its mathematical model are proposed. Further, they were tested using the 142 reference images and 7220 distorted images in the LIVE, CSIQ, TID2008, TID2013, CIDIQ, and IVC databases. And the results were compared with those of seven existing typical IQA models in terms of accuracy, complexity, and generalization performance. These experimental results show that, in the six databases, the IQA accuracy Pearson linear correlation coefficient of the proposed model can all be more than 0.7892 and reach 0.9638 highest, whose comprehensive efficiency is better than ones of the seven existing IQA models. These analysis and comparison show that the proposed model is an excellent IQA model on performance.

No‐reference image quality assessment based on multiscale feature representation

The no-reference image quality assessment (NR-IQA) method can evaluate the distortions in an image without the reference image. However, due to the diversity of the image contents and distortion types, it is hard for the existing NR-IQA algorithms to obtain competitive performance on both synthetically and authentically distorted images. To address the problem, a multiscale feature representation-based NR-IQA method that performs well for both synthetic and authentic distortions is proposed. This model consists of two parts: The feature extraction part and the feature fusion part. First, part of the Res2Net-50 network is chosen as the feature extraction part due to its high ability in increasing the range of receptive fields. Then, the feature fusion part consisting of a novel residual block and two fully connected layers is designed to fuse the extracted features and realize the quality score mapping. After a series of stepwise optimization experiments, the most competitive network architecture consisting of the feature extraction part and the feature fusion part is obtained. Comprehensive experiments on the LIVE, TID2013, CSIQ, KADID-10k, KonIQ-10K, and LIVE challenge databases demonstrate that the proposed method can work powerfully on both the synthetic and authentic distortions and also has a strong generalization ability.

No-reference image quality assessment for contrast-changed images via a semi-supervised robust PCA model

Contrast plays an important role in human perception of image quality. In this paper, we propose a metric for no-reference quality assessment of contrast-changed images by using a novel semi-supervised robust PCA, which can realize feature selection and denoising simultaneously, guided by the available supervisory information. To select features adaptively, the information-oriented features (e.g. entropy and natural scene statistics) and appearance-oriented features (e.g. colorfulness) are adopted. The proposed model is formulated as a constraint optimization problem, which is further casted to a convex problem and solved via augmented Lagrangian multiplier method. Extensive experimental results on CCID2014, CSIQ, SIQAD and TID2013 databases show that the proposed semi-supervised image quality metric based on robust PCA (SIQMR) provides a more accurate prediction than other metrics on the human perception of contrast variations.

Contrast-distorted image quality assessment based on curvelet domain features

Contrast is one of the most popular forms of distortion. Recently, the existing image quality assessment algorithms (IQAs) works focusing on distorted images by compression, noise and blurring. Reduced-reference image quality metric for contrast-changed images (RIQMC) and no reference-image quality assessment (NR-IQA) for contrast-distorted images (NR-IQA-CDI) have been created for CDI. NR-IQA-CDI showed poor performance in two out of three image databases, where the pearson correlation coefficient (PLCC) were only 0.5739 and 0.7623 in TID2013 and CSIQ database, respectively. Spatial domain features are the basis of NR-IQA-CDI architecture. Therefore, in this paper, the spatial domain features are complementary with curvelet domain features, in order to take advantage of the potent properties of the curvelet in extracting information from images such as multiscale and multidirectional. The experimental outcome rely on K-fold cross validation (K ranged 2-10) and statistical test showed that the performance of NR-IQA-CDI rely on curvelet domain features (NR-IQA-CDI-CvT) significantly surpasses those which are rely on five spatial domain features.

Multiply distorted image quality assessment based on feature level fusion and optimal feature selection

No reference image quality assessment (NR-IQA) has received considerable importance in the last decade due to a rise in the use of multimedia content in our daily lives. Due to limitations in technology, multiple distortions may be introduced in the images that need to be assessed. Recently feature selection has shown promising results for single distorted NR-IQA and their effectiveness on multiple distorted images still need to be addressed. In this paper, impact of feature level fusion and feature selection on multiple distorted image quality assessment is presented. To this end features are extracted from multiple distorted images using six NR-IQA techniques (BLIINDS-II, BRISQUE, CurveletQA, DIIVINE, GM-LOG, SSEQ) that extract features in different (discrete cosine transform, spatial, curvelet transform, wavelet transform, spatial and gradient, spatial and spectral) domains. The extracted features from different domains are fused to generate a single feature vector. All combinations of feature-level fusion from six different techniques have been evaluated. Three different feature selection algorithms (genetic search, linear forward search, particle swarm optimization) are then applied to select optimum features for NR-IQA. The selected features are then used by the support vector regression model to predict the quality score. The performance of the proposed methodology is evaluated for two multiple distorted IQA databases (LIVE multiple distorted image dataset (LIVEMD), multiply distorted image database (MDID2017)), two singly synthetically distorted IQA databases (Tampere image database (TID2013), Computational and subjective image quality database (CSIQ)), and one screen content IQA database (Screen content image quality database (SIQAD)). Experimental results show that the fusion of features from different domains gives better performance in comparison to existing multiple-distorted NR-IQA techniques with SROCC scores of 0.9555, 0.9587, 0.6892, 0.9452, and 0.7682 on the LIVEMD, MDID, TID2013, CSIQ, and SIQAD databases respectively. Moreover, the performance is further improved when the genetic search feature selection algorithm is applied to fused features to remove the redundant and irrelevant features. The SROCC scores are improved to 0.9691, 0.9723, and 0.6897 for LIVEMD, MDID, and TID2013 databases respectively.

A Novel Contrast Enhancement Technique using Gradient-Based Joint Histogram Equalization

Image enhancement by histogram equalization reduces the number of gray levels that lead to information loss and unnatural appearance. This paper aims to improve the contrast and preserve information and edge details by employing gradient-based joint histogram equalization. It is achieved by a multiscale-based dark pass filter, which gives the pixel’s edge information. A joint histogram is computed from the edge information and the gray-level distribution of the low contrast image to develop a discrete function. This discrete function is mapped to uniform distribution to get the final enhanced image. The proposed method is experimented on Kodak, USC-SIPI, and CSIQ databases and analyzed using various performance measures such as Contrast, standard deviation, contrast improvement index, structural similarity index, normalized entropy, and normalized mean brightness error. It is observed that the proposed method provides the highest Contrast values of 86.2, 85.79, and 86.02 in Kodak, USC-SIPI, and CSIQ databases, respectively. Normalized entropy value is found to be highest for the proposed method for all the databases. This is noticed to be 0.89, 0.84, and 0.85 for the databases Kodak, USC-SIPI, and CSIQ, respectively. The degree of the uniform distribution is measured by Kullback–Leibler distance. The proposed method produces more uniformity than other techniques available in the literature for all the three databases.

PIQI: perceptual image quality index based on ensemble of Gaussian process regression

Digital images contain a lot of redundancies, therefore, compression techniques are applied to reduce the image size without loss of reasonable image quality. Same become more prominent in the case of videos which contains image sequences and higher compression ratios are achieved in low throughput networks. Assessment of quality of images in such scenarios has become of particular interest. Subjective evaluation in most of the scenarios is infeasible so objective evaluation is preferred. Among the three objective quality measures, full-reference and reduced-reference methods require an original image in some form to calculate the image quality which is unfeasible in scenarios such as broadcasting, acquisition or enhancement. Therefore, a no-reference Perceptual Image Quality Index (PIQI) is proposed in this paper to assess the quality of digital images which calculates luminance and gradient statistics along with mean subtracted contrast normalized products in multiple scales and color spaces. These extracted features are provided to a stacked ensemble of Gaussian Process Regression (GPR) to perform the perceptual quality evaluation. The performance of the PIQI is checked on six benchmark databases and compared with twelve state-of-the-art methods and competitive results are achieved. The comparison is made based on RMSE, Pearson and Spearman’s correlation coefficients between ground truth and predicted quality scores. The scores of 0.0552, 0.9802 and 0.9776 are achieved respectively for these metrics on CSIQ database. Two cross-dataset evaluation experiments are performed to check the generalization of PIQI.

No-Reference Image Quality Assessment Based on Edge Pattern Feature in the Spatial Domain

This paper proposes a general-purpose no-reference image quality assessment (NR-IQA) method that investigates the image’s structure information from a new aspect, i.e., the characteristic of image edge profiles that depict the directional property of adjacent edge points in the spatial domain of the image. More specifically, we extracted the image’s edge map based on Laplacian of Gaussian (LoG) filtration and zero-crossing (ZC) detection and refined the edge map to be 1-pixel wide. We then explored the edge map by investigating edge profiles’ statistics in a local window with a $5\times 5$ -pixel size. Considering the consensus that natural images consist of directional structures, we found that the spatial distribution property of adjacent edge points can be represented through several edge profiles called edge patterns, which are selected from natural images with a proposed smooth criterion. With the proposed edge patterns and their statistical histogram for the image and the support vector regression technique, we proposed the NR-IQA model based on the edge patterns in the spatial domain, named EPISD. The proposed method has been extensively validated on the LIVE, CSIQ, TID2013, MDID2017, SIQAD, and SCID databases. The experimental results showed that EPISD has a competitive performance with state-of-the-art methods and works stably across different databases.

Taxonomy of Performance Measures for Contrast Enhancement

Contrast enhancement is a primitive step in image processing application for enhancing the visual perceptibility which can be treated as the increase in gray level differences between background and foreground with suppression of noises. In literature, various methods have been proposed for contrast enhancement such as bi- histogram and two-dimensional histogram-based methods, etc. Among various methods, the suitable contrast enhancement algorithm should be chosen based on different applications. To find suitability, qualitative and quantitative analysis is also needed. The quantitative analysis can be done with the help of the various performance measures. In this paper, a complete taxonomy of performance measure for contrast enhancement has been discussed and analyzed for the CSIQ database. Among the various performance measures, it is found that Absolute Mean Brightness Error (AMBE), entropy, contrast improvement index, and gradient magnitude similarity deviation (GMSD) are better in terms of the range of operation and the wide variation of low contrast images. It is interpreted that absolute mean brightness requires less time for computation and it is widely used by the researchers for quantitative analysis of contrast enhancement algorithm. This study provides a critical analysis of the several quantitative measures that validate the effectiveness of qualitative measures and motivates to propose an alternate measure that considers the strength of various measures discussed in the literature. These measures quantify the mean brightness shifting, contrast, and entropy preservation.

Perceptual Quality Assessment of Enhanced Images Using a Crowd-Sourcing Framework

In this work, we present a psychophysical study, in which, we analyzed the perceptual quality of images enhanced with several types of enhancement algorithms, including color, sharpness, histogram, and contrast enhancements. To estimate and compare the qualities of enhanced images, we performed a psychophysical experiment with 35 source images, obtained from publicly available databases. More specifically, we used images from the Challenge Database, the CSIQ database, and the TID2013 database. To generate the test sequences, we used 12 different image enhancement algorithms, generating a dataset with a total of 455 images. We used a Double Stimulus Continuous Quality Scale (DSCQS) experimental methodology, with a between-subjects approach where each subject scored a subset of the total database to avoid fatigue. Given the high number of test images, we designed a crowd-sourcing interface to perform an online psychophysical experiment. This type of interface has the advantage of making it possible to collect data from many participants. We also performed an experiment in a controlled laboratory environment and compared its results with the crowd-sourcing results. Since there are very few quality enhancement databases available in the literature, this works represents a contribution to the area of image quality.

Feature-segmentation strategy based convolutional neural network for no-reference image quality assessment

Convolutional neural networks (CNN) have been shown to deliver outstanding performance for image quality assessment (IQA). Most CNN models are trained using small image patches with a fixed resolution of 32 × 32. However, more information of image content and human visual system should be taken into account. This paper proposes a post segmentation based CNN model for no-reference quality assessment without any pre-processing. The network consists of five convolutional layers with max pooling, one special fully connected layer feature-segmentation and one output layer. This paper adopts the feature-segmentation strategy to assure enough training data. We modify the structure of fully connected layer and regard every feature vector of the last pooling maps as an independent sample to train our proposed model. In this way, the raw images can be fed into the input and do not need to be split into patches, to avoid any hand-crafted features. Moreover, the size of the input images is not fixed, and the size of the extracted feature vector is invariant. Experiments on LIVE, CSIQ and TID2008 databases demonstrate that our approach has high consistency with the subjective evaluation scores. The experimental results show that the proposed network outperforms state-of-the-art no-reference IQA algorithms and is comparable to some full-reference IQA algorithms.

Bitrate-Based No-Reference Video Quality Assessment Combining the Visual Perception of Video Contents

In video communication, the quality of video is mainly determined by bitrate in general. Moreover, the effect of video contents and their visual perception on video quality assessment (VQA) is often overlooked. However, in fact, for different videos, although the bitrates are the same, their VQA scores are still significantly different. Hence, it is assumed that the bitrate, video contents, and human visual characteristics mainly affect the VQA. Based on the above three aspects, in this paper, we designed a bitrate-based no-reference (NR) VQA metric combining the visual perception of video contents, namely, BRVPVC. In this metric, first an initial VQA model was proposed by only considering the bitrate alone. Then, the visual perception model for video contents was designed based on the texture complexity and local contrast of image, temporal information of video, and their visual perception features. Finally, two models were synthesized by adding certain weight coefficients into an overall VQA metric, namely, BRVPVC. Furthermore, ten reference videos and 150 distorted videos in the LIVE video database were used to test the metric. Moreover, based on the results of evaluating the videos in LIVE, VQEG, IRCCyN, EPFL-PoliMI, IVP, CSIQ, and Lisbon databases, the performance of BRVPVC is respectively compared with that of six full-reference (FR) metrics and ten NR VQA metrics. The results show that our VQA metric has a higher accuracy than six common FR VQA metrics and eight NR VQA metrics, and it is close to other two NR VQA metrics in accuracy. The corresponding values of Pearson linear correlation coefficient and Spearman rank order correlation coefficient reached 0.8547 and 0.8260, respectively. In addition, the computational complexity of proposed VQA metric is lower than video signal-to-noise ratio, video quality model, motion-based video integrity evaluation, spatiotemporal most apparent distortion, V-BLINDS, and V-CORNIA metrics. Moreover, the proposed metric has a better generalization property than these metrics.

A complete no-reference image quality assessment method based on local feature

ABSTRACTImage quality assessment (IQA) is widely used in digital image processing, and no-reference (NR) IQA has become research focus recently. This paper proposes a NR IQA method based on local features without access to prior knowledge of the images or their distortions. Four gradient masks are used to detect the maximum local gradient (MLG), and the analysis shows that the MLG of strong structure (such as region boundary) includes very tiny noise component, thus this paper assesses image visual quality by using MLGs of strong structures. The proposed method can assess noisy image and blurred image at the same time, and the quality score drops either when the test image becomes blurred or corrupted by random noise. The experiment results show that the proposed approach works well on LIVE, TID2013 and CSIQ databases, and it outperforms some state-of-the-art algorithms.