Pattern Recognition in Semantic Feature Spaces for Image Colorization Quality Assessment

Measuring the quality of digital image is a complicated and importance task in image processing. This task is possible using Image Quality Assessment (IQA) metrics. Among them Pixel and edge-based IQA metrics are so crucial in dealing with a digital image. So, combination of edge and pixel features could handle not all but, almost all aspects of an image. Most recently using edge-based image quality metrics are popular, due to weakness of traditional image quality assessment metrics such as Peak Signal-to Noise Ratio. Also, majority of IQA metrics are belonged to color images, but recently new metrics for depth images are emerged. This paper proposes a new Full-Reference image quality assessment metric for color and depth images, which works based on edge and pixel features. Proposed method is a combination of improved Edge Based IQA and Peak Signal-to Noise Ratio methods. Proposed method is called Edge and Pixel-based Image Quality Assessment Metric (EPIQA). The system is validated using famous and benchmark performance metrics or quality measures such as Spearman Rank-Order Correlation Coefficient (SROCC), along with comparison with other similar methods on well-known related databases. Color databases have proper and diverse number of noises, but there is no proper depth noisy database, which it is decided to make one. Proposed method returned promising and satisfactory results in different tests.

Evaluation of light field image (LFI), especially micro-lens camera light field (LF), is a new and challenging work. The development of image quality assessment (IQA) metric of LFIs relies on the subjective quality assessment database. In this paper, we establish a perceptual quality assessment dataset consisting of 240 distorted images from 8 source images with five distortion types. Furthermore, a no-reference IQA metric is proposed by combining 2D and 3D characteristics of LFI with the Support Vector Regression (SVR) model. The performance of the proposed metric is demonstrated by comparing with some classical full reference IQA metrics both on the presented dataset and a third-party dataset. The experiment results show that our method has a better performance than others.

Most image processing and computer vision applications necessitate objective quality assessment of the input images during pre-processing or post-processing. This important inquiry promotes effective selection of quality images by imaging and vision machines for accurate image processing, analysis and interpretation. On this ground, peak signal-to-noise ratio (PSNR) has traditionally been used in a range of image enhancement and restoration problems, including denoising and super-resolution, to gauge quality of images. Despite its wide application, PSNR is an unbounded metric and cannot be intuitively interpreted with respect to ideal conditions of the input images. PSNR increases with an image quality, but fails to inform an upper bound of a quality value that an image can achieve. Additionally, PSNR explodes to an unreasonably large number under ideal situations (when mean squared error equals zero, or when original and restored images are equal). Furthermore, PSNR resonates poorly with the human visual system. These observations limit the scope of interpreting the metric, especially in situations where we intend to understand the performance degree of restoration methods. In this work, we have re-defined PSNR to ensure boundedness and, based on this new definition, three other alternative image quality assessment (IQA) metrics have been proposed: absolute percentage error, symmetric mean absolute percentage error and logarithmic relative error. In addition, we have introduced a normalized IQA metric, which ranges between 0% and 100%, based on the re-defined bounded PSNR metric (PSNRB). Empirical results show that our IQA metrics remain robust under varying degradation conditions. Furthermore, PSNRB correlates with PSNR within the established lower and upper bounds of PSNRB – an important observation highlighting the significance and validity of PSNRB. These metrics expand our understanding on quantifying and qualifying quality of features contained in the images. To allow for reproducibility of our results, we have shared implementation codes and datasets in the MATLAB File Exchange: https://www.mathworks.com/matlabcentral/fileexchange/136574-bounded-peak-signal-to-noise-ratio.

In developing novel image quality assessment (IQA) metrics, researchers should compare their proposed metrics with state-of-the-art metrics. A commonly adopted approach is by comparing two residuals between the nonlinearly mapped scores of two IQA metrics and the difference mean opinion score, which are assumed from Gaussian distributions with zero means. An F-test is then used to test the equality of variances of the two sets of residuals. If the variances are significantly different, then we conclude that the residuals are from different Gaussian distributions and that the two IQA metrics are significantly different. The F-test assumes that the two sets of residuals are independent. However, given that the IQA metrics are calculated on the same database, the two sets of residuals are paired and may be correlated. We note this improper usage of the F-test by practitioners, which can result in misleading comparison results of two IQA metrics. To solve this practical problem, we introduce the Pitman test to investigate the equality of variances for two sets of correlated residuals. Experiments on the Laboratory for Image and Video Engineering (LIVE) database show that the two tests can provide different conclusions.

Image and video compression aims at finding an optimal trade-off between rate and distortion. This is done through Rate-Distortion Optimization (RDO) in traditional en-coders with the use of Image Quality Assessment (IQA) metrics. While it is known that most IQA metrics are designed to be correlated with human perception, there is no evidence that this observation can be generalized in a Video Coding for Machines (VCM) context, where the receiver is not a human anymore but a machine. In this paper, we propose an evaluation protocol to measure the correlation level between conventional Full-Reference (FR) IQA metrics and machine perception through the semantic segmentation vision task. Experiments showed a relatively low correlation between them when measured on the block-level. This observation implies the need of RDO algorithms that are better suited for Machine-to-Machine (M2M) communications. In order to facilitate the emergence of IQA metrics that better reflect machine perception, the code and dataset used to perform this study is made freely available at https://github.com/albmarie/iqa_m2m_segmentation.

Image Quality Assessment (IQA) metrics play an important role in helping measure computed tomography (CT) image reconstruction algorithms and trade-off dose reduction and imaging quality. However, some studies have shown that the results of peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), which are commonly used IQA metrics are often inconsistent with the subjective opinions of radiologists when applied to medical images, and it is unknown whether other IQA algorithms which outperform PSNR and SSIM on natural images are equally good on CT images. In this paper, we explore the performance of nine IQA metrics on CT images by comparing their running time and their correlation with mean opinion score (MOS). Moreover, inspired by the two-step framework of traditional IQA metrics, we propose a combination IQA mdoel based on genetic algorithm to optimize the existing metrics from comparative features and pooling strategies. The obtained results show that most traditional FR-IQA metrics suffer from unsatisfactory accuracy or high computational cost, while the proposed method achieves superior performance in both aspects.

Face Sketch Recognition (FSR) presents a severe challenge to conventional recognition paradigms developed basically to match face photos. This challenge is mainly due to the large texture discrepancy between face sketches, characterized by shape exaggeration, and face photos. In this paper, we propose a training-free synthesized face sketch recognition method based on the rank-level fusion of multiple Image Quality Assessment (IQA) metrics. The advantages of IQA metrics as a recognition engine are combined with the rank-level fusion to boost the final recognition accuracy. By integrating multiple IQA metrics into the face sketch recognition framework, the proposed method simultaneously performs face-sketch matching application and evaluates the performance of face sketch synthesis methods. To test the performance of the recognition framework, five synthesized face sketch methods are used to generate sketches from face photos. We use the Borda count approach to fuse four IQA metrics, namely, structured similarity index metric, feature similarity index metric, visual information fidelity and gradient magnitude similarity deviation at the rank-level. Experimental results and comparison with the state-of-the-art methods illustrate the competitiveness of the proposed synthesized face sketch recognition framework.

Traditional image quality assessment (IQA) metrics are most based on the discrepancy between reference image and the distorted image does not correlate well with the perceptive mechanism of human visual system (HVS). In this paper, we found from the experimental investigations that perceptible quality distortions can lead to some measurable changes in the visual saliency (VS) map of the image, and then proposed a novel and effective full reference image quality assessment method by means of VS. In our method, VS serves two functions which one is as a feature for computing the distorted image's local quality map and the other is used for a weight to reflect the importance of a local region when pooling. However, VS map alone sometimes does not work quite well when image's contrast change or color distortion, so we also merged the gradient similarity map and chrominance similarity map into the IQA. We named our proposed IQA metric as visual saliency-based metric (VSM). The experiments carried out on a benchmark datasets demonstrate that our proposed VSM outperform most of the state-of-the-art IQA metrics in terms of the prediction accuracy.

Purpose: Motion artifacts in magnetic resonance (MR) images mostly undergo subjective evaluation, which is poorly reproducible, time consuming, and costly. Recently, full-reference image quality assessment (FR-IQA) metrics, such as structural similarity (SSIM), have been used, but they require a reference image and hence cannot be used to evaluate clinical images. We developed a convolutional neural network (CNN) model to quantify motion artifacts without using reference images. Approach: The brain MR images were obtained from an open dataset. The motion-corrupted images were generated retrospectively, and the peak signal-to-noise ratio, cross-correlation coefficient, and SSIM were calculated. The CNN was trained using these images and their FR-IQA metrics to predict the FR-IQA metrics without reference images. Receiver operating characteristic (ROC) curves were created for binary classification, with artifact scores indicating the need for rescanning. ROC curve analysis was performed on the binary classification of the real motion images. Results: The predicted FR-IQA metric having the highest correlation with the subjective evaluation was SSIM, which was able to classify images requiring rescanning with a sensitivity of 89.5%, specificity of 78.2%, and area under the ROC curve (AUC) of 0.930. The real motion artifacts were classified with the AUC of 0.928. Conclusions: Our CNN model predicts FR-IQA metrics with high accuracy, which enables quantitative assessment of motion artifacts in MR images without reference images. It enables classification of images requiring rescanning with a high AUC, which can improve the workflow of MR imaging examinations.

This paper explores the application of Blind Image Quality Assessment (BIQA) metrics to pulsed thermography. Two BIQA were used to subsample a sequence of images acquired using Pulse Thermography (PT). The experiments show that the sequences subsampled using BIQA significantly improve the results when applied to metallic samples but fail to capture informative features on composite materials. On metallic samples, when the PCT is applied, an average improvement of 139% of the Contrast to Noise Ratio (CNR) score is observed, on the sequences subsampled, compared with the entire sequence. Nonetheless, the CNR shows that processing the entire data sequence offers 240% improvement compared with subsampled sequences on composite materials.

This letter presents a wavelet based perceptive image quality assessment metric (WASH-Wavelet based Sharp features) based on HVS which accounts for the sensitivity of human vision to sharp features of the image, the sharpness and zero-crossings. The image is analyzed in the wavelet domain which is advantageous to the quality assessment based on human perception. The sharp regions are highly attentive to early vision and the edge points obtained by zero-crossing are important features that give good quality estimation based on the structural distortion of the image. The WASH algorithm was tested on the publicly available databases and gave best results with the state-of-the-art quality assessment metrics as well as the wavelet based perceptive quality assessment metrics.

We develop a no-reference image quality assessment metric to evaluate the quality of synthesized view rendered from the Multi-view Video plus Depth (MVD) format. Our metric is named Synthesized View Comparison (SVC), which is designed for real-time quality monitoring at the receiver side in a 3D-TV system. The metric utilizes the virtual views in the middle which are warped from left and right views by Depth-image-based rendering algorithm (DIBR), and compares the difference between the virtual views rendered from different cameras by Structural SIMilarity (SSIM), a popular 2D full-reference image quality assessment metric. The experimental results indicate that our no-reference quality assessment metric for the synthesized images has competitive prediction performance compared with some classic full-reference image quality assessment metrics.

Going beyond traditional 2D imaging is not only an emerging trend of imaging technology, but also the key to a more immersive user experience. Light Field Image (LFI) is a typical high-dimensional imaging format, and the quality evaluation of which is very challenging but necessary. In this paper, we propose a novel Pseudo Video-based Blind quality assessment metric for Light Field image (PVBLiF). In contrast to most previous Light Field Image Quality Assessment (LF-IQA) metrics, in which different types of 2D representations derived from LFI are used for quality assessment indirectly, our metric exploits a more intuitive 3D representation, named Pseudo Video Block Sequence (PVBS), to evaluate the perceptual quality of LFI. For this purpose, we first divide the LFI into a massive number of non-overlapping PVBSs, which simultaneously contain spatial and angular information of LFI. Then, we propose a novel network (named PVBSNet) based on Convolutional Neural Networks (CNNs) to extract the spatio-angular features of PVBS and further evaluate the PVBS quality. The proposed PVBSNet consists of four stages: multi-information division, intra-feature extraction, cross-feature fusion, and quality regression. Finally, a Saliency- and Variance-guided Pooling (SVPooling) method is presented to integrate all the PVBS quality into the overall quality of LFI. The proposed PVBLiF metric has been extensively evaluated on three widely-used LFI datasets: Win5-LID, NBU-LF1.0, and SHU. Experimental results demonstrate that our proposed PVBLiF metric outperforms state-of-the-art metrics and is capable of highly approximating the performance of human observers. The source code of PVBLiF is publicly available at <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/ZhengyuZhang96/PVBLiF</uri> .

View synthesis brings geometric distortions which are not handled efficiently by existing image quality assessment metrics. Despite the widespread of 3-D technology and notably 3D television (3DTV) and free-viewpoints television (FTV), the field of view synthesis quality assessment has not yet been widely investigated and new quality metrics are required. In this study, we propose a new full-reference objective quality assessment metric: the View Synthesis Quality Assessment (VSQA) metric. Our method is dedicated to artifacts detection in synthesized view-points and aims to handle areas where disparity estimation may fail: thin objects, object borders, transparency, variations of illumination or color differences between left and right views, periodic objects... The key feature of the proposed method is the use of three visibility maps which characterize complexity in terms of textures, diversity of gradient orientations and presence of high contrast. Moreover, the VSQA metric can be defined as an extension of any existing 2D image quality assessment metric. Experimental tests have shown the effectiveness of the proposed method.

영상 처리 및 컴퓨터 비전 분야에 있어서, 평균 제곱 오차(Mean Squared Error: MSE)는 좋은 수학적 특성(예를 들어, 척도성(metricability), 미분가능성(differentiability) 및 볼록 성질(convexity))을 가짐으로 인해 많은 영상 화질 최적화 문제의 객관적 척도로 사용되어 왔다. 그러나 MSE가 영상의 왜곡 신호에 대한 시각적 인지 화질과 상관도가 높지 않다는 것이 알려지면서, 이를 해결하기 위해 위에서 언급한 좋은 수학적 특성과 높은 영상 화질 예측 성능을 동시에 가지는 객관적 영상 화질 측정(Image Quality Assessment: IQA)척도가 활발히 연구되어 왔다. 비록 최근 제안된 좋은 수학적 성질을 만족시키는 IQA 척도들은 MSE와 비교하여 매우 향상된 주관적 화질 예측 성능을 보이지만, 상대적으로 높은 계산 복잡도를 가진다. 본 논문은 이를 해결하기 위해, 단순 라플라스 연산자를 이용한 좋은 수학적 특성을 가지는 새로운 IQA 척도를 제안한다. 제안 IQA 방법에 도입한 단순 라플라스 연산자는 인간 시각 체계의 망막에서의 광도 자극에 대한 시신경 반응을 효과적으로 모사할 뿐만 아니라 계산이 매우 단순하기 때문에, 제안 IQA 척도는 단순 라플라스 연산자를 사용하여 매우 빠른 계산 속도와 높은 주관적 화질 점수 예측력을 확보하였다. 제안 IQA 척도의 효과를 검증하기 위해, 최신 IQA 척도들과 광범위한 성능비교 실험을 수행하였다. 실험 결과, 제안하는 IQA 척도는 모든 테스트 IQA 척도들 중 MSE를 제외하고 가장 빠른 처리 속도를 보였을 뿐만 아니라, 가장 높은 주관적 화질예측 성능을 보였다. In image processing and computer vision fields, mean squared error (MSE) has popularly been used as an objective metric in image quality optimization problems due to its desirable mathematical properties such as metricability, differentiability and convexity. However, as known that MSE is not highly correlated with perceived visual quality, much effort has been made to develop new image quality assessment (IQA) metrics having both the desirable mathematical properties aforementioned and high prediction performances for subjective visual quality scores. Although recent IQA metrics having the desirable mathematical properties have shown to give some promising results in prediction performance for visual quality scores, they also have high computation complexities. In order to alleviate this problem, we propose a new fast IQA metric using a simple Laplace operator. Since the Laplace operator used in our IQA metric can not only effectively mimic operations of receptive fields in retina for luminance stimulus but also be simply computed, our IQA metric can yield both very fast processing speed and high prediction performance. In order to verify the effectiveness of the proposed IQA metric, our method is compared to some state-of-the-art IQA metrics. The experimental results showed that the proposed IQA metric has the fastest running speed compared the IQA methods except MSE under comparison. Moreover, our IQA metric achieves the best prediction performance for subjective image quality scores among the state-of-the-art IQA metrics under test.