Blind Quality Evaluation Research Articles

Blind quality evaluator for multi-exposure fusion image via joint sparse features and complex-wavelet statistical characteristics

Blind quality evaluator for multi-exposure fusion image via joint sparse features and complex-wavelet statistical characteristics

Blind cartoon image quality assessment based on local structure and chromatic statistics

With the help of computer-assisted systems, cartoon synthesis has become convenient and efficient by reusing existing cartoon materials. However, quality evaluation of the obtained cartoon image still relies on labor-intensive subjective judgment. This accordingly raises an urgent demand for effective quality evaluation methods to automatically select a cartoon image of high quality from a set of candidates with different parameter settings. In this paper, a new blind image quality assessment metric is developed for evaluating the perceptual quality of cartoon images by considering structure and chromatic distortions. The extracted gradient-based local structure features and multiscale chromatic statistical features are integrated into one representation for an overall perceptual quality prediction. Experimental results on two benchmark cartoon image datasets, i.e., NBU-CIQAD and HFUT-CID, indicate that the proposed metric outperforms both the state-of-the-art blind quality evaluation methods designed for natural images or synthetic images.

Blind Quality Evaluator for Enhanced Colonoscopy Images by Integrating Local and Global Statistical Features

Blind Quality Evaluator for Enhanced Colonoscopy Images by Integrating Local and Global Statistical Features

Blind Quality Evaluator of Light Field Images by Group-Based Representations and Multiple Plane-Oriented Perceptual Characteristics

Due to the emergency of multi-view cameras and commercial Light Field (LF) cameras, the demand of high-performance LF quality evaluator is of great significance for guiding LF acquisition, processing and application and further promoting the visual perceived quality of LF visualizations. However, LF Images (LFIs), as high-dimensional data, suffer from various quality degradations not only in the spatial domain but also in the angular domain. Therefore, it is of great challenge to predict LF quality accurately. An effective LF evaluator should be able to represent these heterogeneous artifacts. In this paper, we provide a novel No-Reference LF Quality Assessment Evaluator (NR LF-QAE) to tackle this problem. Firstly, to measure angular consistency among viewports, we utilize group-based representations to character information similarity of aligned view stacks. Secondly, to better describe the texture information of LFIs, unifying spatial-angular texture statistic measurement is performed via Local Binary Patterns from Three Orthogonal Planes (LBP-TOP). Thirdly, we design 3D Log-Gabor filters to extract LF global structure information in Sub-Aperture Images (SAIs) as spatial feature characterizations and 2D Log-Gabor filters are adopted to characterize ray direction/depth information in Epipolar Plane Images (EPIs) as angular feature characterizations. By comprehensive LF information analyses in angular consistency and spatial-angular feature extraction with texture and structure descriptors, experimental results demonstrate the superiority of the proposed NR LF-QAE over the state-of-the-art comparative models in predicting the quality of LFIs on three available benchmark databases. The code will be released at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/zerosola/NR-LF-QAE</uri> .

3D Animation Production and Generation Based on Human Body Base 3D Models

Abstract With the gradual increase in the demand for parametric human 3D models, the requirements for the accuracy of modeling and the quality of generation are getting higher and higher. In this paper, the human body structure model is constructed by 3D human posture estimation and spatio-temporal graph convolutional network, and after normalizing the data of the human body base, the 3D human body model based on the human body base is generated by combining the free deformation method, and the local light model is used to make the human body model more three-dimensional and real. In addition, the quality of 3D animation production is investigated by combining blind quality evaluation and validation analysis of the human body-based 3D model. The 3D human body modeling simulation error assessment has an error assessment root-means-square error that falls within 1.5%. In the validation analysis of model generation quality, the quality evaluation model proposed in this paper has a quality correlation coefficient and prediction accuracy of 0.9456 and 0.9454 on the dataset, respectively, and the performance of the ablation experiments and generalization experiments is stable. The mean value of the error between the animation production character generation effect and the actual testers is not more than 10mm.Combined with the research in this paper, it effectively improves the accuracy of human body modeling and provides an evaluation method for the quality of human 3D animation production and generation, which is of practical significance for its development.

Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images for Electronic Display

The problem of reproducing high dynamic range (HDR) images on electronic display and photography with restricted dynamic range has gained a lot of interest in the consumer electronics community. There exist various approaches to this issue, e.g., tone mapping operators (TMOs) and multi-exposure fusion algorithms (MEFs). Many existing image quality assessment (IQA) methods have been proposed to compare images of quality degradation generated by TMOs/MEFs. Although promising performances have been achieved, they seldom consider local specific artifacts difference (i.e., abnormal exposure and color cast) related with the TMOs/MEFs. To address this limitation, this paper proposes a Blind Quality Evaluator of Tone-Mapped HDR and Multi-Exposure Fused Images (BQE-TM/MEFI). First, two purpose-designed segment models are utilized to distinguish well-exposedness dense patches (WEDPes) and non-WEDPes, color cast patches (CCPes) and non-CCPes respectively. Second, multiple quality-perception features are extracted to measure local artifacts: 1) structure and sharpness features from WEDPes, 2) saturation features from non-CCPes, and 3) edge structure features. Then, three new low-complexity regional features (over-exposure ratio, entropy and color confidence index) are calculated based on over-exposure segmentation model. Finally, all extracted features are aggregated into a machine-learning regression model to pool a quality score. The simplicity and good performance of the proposed method makes it suitable for electronic displays and other consumer electronics.

No-reference quality assessment of tone-mapped image using clustering perception and salient regions detection

In order to display a high dynamic range (HDR) image on a standard display device, we need to compress an HDR image into a tone-mapped image (TMI) using the tone-mapped operator (TMO). In order to evaluate the performance of different TMOs, a blind quality evaluation method of TMI is proposed. First, the TMI is divided into bright, dark and middle regions by clustering algorithm, and the Y channel is extracted in these three regions respectively to calculate the goodness of Gaussian fitness of the histogram. Second, the TMI is decomposed by tucker decomposition, and the texture features are extracted on the first sub-band. After that, the non-negative matrix is applied to calculate the salient regions of TMI, and features in the salient regions are extracted. Then, for the sake of comprehensive evaluation the quality of TMI, the naturalness feature is extracted in the gray scale domain and colorfulness features are extracted in HSI color space. Finally, the final score quality is predicted using support vector regression model training. Experimental results on TMID and ESPL-LIVE HDR database proved the proposed method is very effective and superior to the existing blind TMI quality assessment methods.

No Reference Quality Assessment for Screen Content Images Using Stacked Autoencoders in Pictorial and Textual Regions.

Recently, the visual quality evaluation of screen content images (SCIs) has become an important and timely emerging research theme. This article presents an effective and novel blind quality evaluation metric for SCIs by using stacked autoencoders (SAE) based on pictorial and textual regions. Since the SCI consists of not only the pictorial area but also the textual area, the human visual system (HVS) is not equally sensitive to their different distortion types. First, the textual and pictorial regions can be obtained by dividing an input SCI via an SCI segmentation metric. Next, we extract quality-aware features from the textual region and pictorial region, respectively. Then, two different SAEs are trained via an unsupervised approach for quality-aware features that are extracted from these two regions. After the training procedure of the SAEs, the quality-aware features can evolve into more discriminative and meaningful features. Subsequently, the evolved features and their corresponding subjective scores are input into two regressors for training. Each regressor can obtain one output predictive score. Finally, the final perceptual quality score of a test SCI is computed by these two predicted scores via a weighted model. Experimental results on two public SCI-oriented databases have revealed that the proposed scheme can compare favorably with the existing blind image quality assessment metrics.

Blind Quality Evaluation for Screen Content Images Based on Regionalized Structural Features

Currently, screen content images (SCIs) are widely used in our modern society. However, since SCIs have distinctly different properties compared to natural images, traditional quality assessment methods of natural images cannot precisely evaluate the quality of SCIs. Thus, we propose a blind quality evaluation method for SCIs based on regionalized structural features that are closely relevant to the intrinsic quality of SCIs. Firstly, the features of textual and pictorial regions of SCIs are extracted separately. For textual regions, since they contain noticeable structural information, we propose improved histograms of oriented gradients extracted from multi-order derivatives as structural features. For pictorial regions, since human vision is sensitive to texture information and luminance variation, we adopt texture as the structural feature; meanwhile, luminance is used as the auxiliary feature. The local derivative pattern and the shearlet local binary pattern are used to extract texture in the spatial and shearlet domains, respectively. Secondly, to derive the quality of textual and pictorial regions, two mapping functions are respectively trained from their features to subjective values. Finally, an activity weighting strategy is proposed to combine the quality of textual and pictorial regions. Experimental results show that the proposed method achieves better performance than the state-of-the-art methods.

Deep blind quality evaluator for multiply distorted images based on monogenic binary coding

Perceptual quality evaluation of multiply distorted images has become a very challenging research topic. In this paper, we present a novel and efficient deep blind quality evaluator for multiply distorted images based on monogenic binary coding (MBC). Local complementary structural information and a deep learning method are employed to blindly evaluate the quality of multiply distorted images. First, a monogenic signal representation is utilized to decompose a multiply distorted image into three complementary components: orientation, phase, and magnitude. The quality-predictive features are then determined from the complementary components. Finally, the features are mapped to the human quality score of the multiply distorted image based on the deep neural network. The results on two newly established multiply distorted image subjective databases confirm that our metric has a better prediction performance than existing state-of-the-art full-reference and classical blind metrics.

BLIQUE-TMI: Blind Quality Evaluator for Tone-Mapped Images Based on Local and Global Feature Analyses

High dynamic range (HDR) image, which has a powerful capacity to represent the wide dynamic range of real-world scenes, has been receiving attention from both academic and industrial communities. Although HDR imaging devices have become prevalent, the display devices for HDR images are still limited. To facilitate the visualization of HDR images in standard low dynamic range displays, many different tone mapping operators (TMOs) have been developed. To create a fair comparison of different TMOs, this paper proposes a BLInd QUality Evaluator to blindly predict the quality of Tone-Mapped Images (BLIQUE-TMI) without accessing the corresponding HDR versions. BLIQUE-TMI measures the quality of TMIs by considering the following aspects: 1) visual information; 2) local structure; and 3) naturalness. To be specific, quality-aware features related to the former two aspects are extracted in a local manner based on sparse representation, while quality-aware features related to the third aspect are derived based on global statistics modeling in both intensity and color domains. All the extracted local and global quality-aware features constitute a final feature vector. An emergent machine learning technique, i.e., extreme learning machine, is adopted to learn a quality predictor from feature space to quality space. The superiority of BLIQUE-TMI to several leading blind IQA metrics is well demonstrated on two benchmark databases.

Local and Global Feature Learning for Blind Quality Evaluation of Screen Content and Natural Scene Images.

The blind quality evaluation of screen content images (SCIs) and natural scene images (NSIs) has become an important, yet very challenging issue. In this paper, we present an effective blind quality evaluation technique for SCIs and NSIs based on a dictionary of learned local and global quality features. First, a local dictionary is constructed using local normalized image patches and conventional -means clustering. With this local dictionary, the learned local quality features can be obtained using a locality-constrained linear coding with max pooling. To extract the learned global quality features, the histogram representations of binary patterns are concatenated to form a global dictionary. The collaborative representation algorithm is used to efficiently code the learned global quality features of the distorted images using this dictionary. Finally, kernel-based support vector regression is used to integrate these features into an overall quality score. Extensive experiments involving the proposed evaluation technique demonstrate that in comparison with most relevant metrics, the proposed blind metric yields significantly higher consistency in line with subjective fidelity ratings.

Local and global sparse representation for no-reference quality assessment of stereoscopic images

No-reference/blind quality assessment of stereoscopic 3D images is much more challenging than 2D images due to the poor understanding of binocular vision. In this paper, we propose a BLind Quality Evaluator for stereoscopic 3D images by learning Local and Global Sparse Representations (BLQELGSR). Specifically, at the training stage, we first construct a large-scale training set by simulating some common distortions that are likely encountered by stereoscopic images, and propose a multi-modal sparse representation framework to characterize the relationship between the feature and quality spaces for all sources of information from left, right and cyclopean views in local and global manners. At the testing stage, based on the derived 3D quality prediction framework, the local and global quality scores from different sources are predicted and combined to drive a final 3D quality score. Experimental results on three 3D image quality databases show that in comparison with the existing methods, the devised BLQELGSR can achieve better prediction performance to be in line with subjective assessment.

Model-Based Referenceless Quality Metric of 3D Synthesized Images Using Local Image Description.

New challenges have been brought out along with the emerging of 3D-related technologies, such as virtual reality, augmented reality (AR), and mixed reality. Free viewpoint video (FVV), due to its applications in remote surveillance, remote education, and so on, based on the flexible selection of direction and viewpoint, has been perceived as the development direction of next-generation video technologies and has drawn a wide range of researchers' attention. Since FVV images are synthesized via a depth image-based rendering (DIBR) procedure in the "blind" environment (without reference images), a reliable real-time blind quality evaluation and monitoring system is urgently required. But existing assessment metrics do not render human judgments faithfully mainly because geometric distortions are generated by DIBR. To this end, this paper proposes a novel referenceless quality metric of DIBR-synthesized images using the autoregression (AR)-based local image description. It was found that, after the AR prediction, the reconstructed error between a DIBR-synthesized image and its AR-predicted image can accurately capture the geometry distortion. The visual saliency is then leveraged to modify the proposed blind quality metric to a sizable margin. Experiments validate the superiority of our no-reference quality method as compared with prevailing full-, reduced-, and no-reference models.

Towards efficient blind quality evaluation of screen content images based on edge‐preserving filter

The problem of quality assessment of screen content images from two respects is addressed, the edge-preserving filter based free energy and structural degradation model. For screen content images which always contain texts, edge information plays an important role in the process of evaluating the screen images quality. Inspired by this, the edge-preserving filter based free energy entropy and structural degradation model were combined to extract the quality features and accordingly the statistical model of screen content images was established. Experimental results prove that the proposed method can produce highly consistent with human perception, particularly superior to state-of-the-art full- and no-reference quality metrics on the SIQAD database dedicated to the quality assessment of screen images.

Learning a blind quality evaluation engine of screen content images

We in this paper investigate how to blindly predict the visual quality of a screen content image (SCI). With the popularity of multi-client and remote-controlling systems, SCIs and the relevant applications have been a hot research topic. In general, SCIs contain texts or graphics in cartoons, ebooks or captures of computer screens. As for blind quality assessment (QA) of natural scene images (NSIs), it has been well established since NSIs possess certain statistical properties. SCIs however do not have reliable statistic models so far and thus the associated blind QA task is hard to be addressed. Aiming at solving this problem, we first extract 13 perceptual-inspired features with the free energy based brain theory and structural degradation model. In order to avoid the overfitting and guarantee the independence of training and testing samples, we then collect 100,000 images and use their objective quality scores computed via a high-accuracy full-reference QA method for SCIs as labels, before learning a new blind quality measure from aforementioned 13 features to the objective quality score. Experimental results performed on a large-scale screen image quality assessment database (SIQAD) demonstrate that the proposed blind quality metric has a good correlation with human perception of quality, even superior to state-of-the-art full-, reduced- and no-reference QA algorithms.