A Benchmark of Variance of Opinion Scores in Image Quality Assessment

This paper deals with the subjective and objective image quality evaluation. The demand of an accurate image and video objective quality assessment tool is extremely important in modern multimedia systems. Possible enhancement of the performance in existent image quality assessment approaches using multiple quality measures with the support of the artificial neural network data processing is proposed. The analysis results of the known quality measures and their suitability for the particular image or video quality assessment problem are presented. The most suitable measures are used to implement the novel image quality assessment tool using artificial neural network data processing. Optimization of the proposed model has been done in order to achieve as highest generalization feature of the model as possible. Performance of the implemented model for the image quality assessment has been evaluated using the database of distorted images and subjective image quality assessment results with respect to the Mean Opinion Score (MOS) obtained by the group of observers. It is shown that the proposed image quality assessment model can achieve high correlation with the subjective image quality ratings.

We survey information theoretic approaches to solve a variety of visual quality assessment (QA) problems. These approaches are generally built on natural scene statistical models and lead to practical automatic QA algorithms delivering excellent performance in terms of correlation with human judgments of quality. We study all three categories of image QA models: full reference (FR), reduced reference (RR) and no reference (NR) image QA, as well as FR video QA and information weighting strategies for FR image and video QA. We demonstrate the application of information theory in each of these problems. Each of these problems presents its own challenges in the design of information theoretic QA indices leading to different algorithms under different scenarios. In the algorithms, we survey, FR image and video QA algorithms are based on mutual information or conditional Kolmogorov complexities; RR image QA algorithms either use relative entropy or entropic differences, while the NR QA algorithm applies Renyi entropy, and the weighting strategies rely on mutual information. We also discuss various open research questions, particularly in the realm of NR image QA and all classes of video QA.

— Many biometric applications are faced with enormous performance challenges due to submission of low quality facial images. In this study, adaptive regression splines (ARES) models were built for predicting algorithm matching scores (AMS) and overall quality scores (OQS). A face verification and image quality assessment (FVIQA) framework was adopted to extract five facial quality features from still images. The SCface database was adopted for the training and testing datasets with 2,093 and 897 images respectively. ARES models were built from the normalized individual quality scores and algorithm matching scores using ARESLab in the MATLAB environment. A black face surveillance camera (BFSC) database of 50 subjects was populated to mimic the SCface database and act as the target dataset for the model validation. Results from the study shows that FVIQA quality scores and other experimental results are comparable and consistent with previous research works. The model ANOVA decomposition showed that pose variation is the major determinant for model OQS and AMS with 0.046 and 0.261 respectively. From the performance evaluation, model OQS achieved 99.96% and 99.81% prediction accuracy on the test and target datasets while model AMS achieved 87.04% and 84.73% respectively. Subsequently, no failure-to-acquire (FTA) was recorded when superior face images were selected from the SCface database using the developed image verification and quality assessment (IVQA) number

영상 처리 및 컴퓨터 비전 분야에 있어서, 평균 제곱 오차(Mean Squared Error: MSE)는 좋은 수학적 특성(예를 들어, 척도성(metricability), 미분가능성(differentiability) 및 볼록 성질(convexity))을 가짐으로 인해 많은 영상 화질 최적화 문제의 객관적 척도로 사용되어 왔다. 그러나 MSE가 영상의 왜곡 신호에 대한 시각적 인지 화질과 상관도가 높지 않다는 것이 알려지면서, 이를 해결하기 위해 위에서 언급한 좋은 수학적 특성과 높은 영상 화질 예측 성능을 동시에 가지는 객관적 영상 화질 측정(Image Quality Assessment: IQA)척도가 활발히 연구되어 왔다. 비록 최근 제안된 좋은 수학적 성질을 만족시키는 IQA 척도들은 MSE와 비교하여 매우 향상된 주관적 화질 예측 성능을 보이지만, 상대적으로 높은 계산 복잡도를 가진다. 본 논문은 이를 해결하기 위해, 단순 라플라스 연산자를 이용한 좋은 수학적 특성을 가지는 새로운 IQA 척도를 제안한다. 제안 IQA 방법에 도입한 단순 라플라스 연산자는 인간 시각 체계의 망막에서의 광도 자극에 대한 시신경 반응을 효과적으로 모사할 뿐만 아니라 계산이 매우 단순하기 때문에, 제안 IQA 척도는 단순 라플라스 연산자를 사용하여 매우 빠른 계산 속도와 높은 주관적 화질 점수 예측력을 확보하였다. 제안 IQA 척도의 효과를 검증하기 위해, 최신 IQA 척도들과 광범위한 성능비교 실험을 수행하였다. 실험 결과, 제안하는 IQA 척도는 모든 테스트 IQA 척도들 중 MSE를 제외하고 가장 빠른 처리 속도를 보였을 뿐만 아니라, 가장 높은 주관적 화질예측 성능을 보였다. In image processing and computer vision fields, mean squared error (MSE) has popularly been used as an objective metric in image quality optimization problems due to its desirable mathematical properties such as metricability, differentiability and convexity. However, as known that MSE is not highly correlated with perceived visual quality, much effort has been made to develop new image quality assessment (IQA) metrics having both the desirable mathematical properties aforementioned and high prediction performances for subjective visual quality scores. Although recent IQA metrics having the desirable mathematical properties have shown to give some promising results in prediction performance for visual quality scores, they also have high computation complexities. In order to alleviate this problem, we propose a new fast IQA metric using a simple Laplace operator. Since the Laplace operator used in our IQA metric can not only effectively mimic operations of receptive fields in retina for luminance stimulus but also be simply computed, our IQA metric can yield both very fast processing speed and high prediction performance. In order to verify the effectiveness of the proposed IQA metric, our method is compared to some state-of-the-art IQA metrics. The experimental results showed that the proposed IQA metric has the fastest running speed compared the IQA methods except MSE under comparison. Moreover, our IQA metric achieves the best prediction performance for subjective image quality scores among the state-of-the-art IQA metrics under test.

Advances in imaging and computing hardware have led to an explosion in the use of color images in image processing, graphics and computer vision applications across various domains such as medical imaging, satellite imagery, document analysis and biometrics to name a few. However, these images are subjected to wide variety of distortions during its acquisition, subsequent compression, transmission, processing and then reproduction, which degrade their visual quality. Hence objective quality assessment of color images has emerged as one of the essential operation in image processing. During the last two decades, efforts have been put to design such an image quality metric which can be calculated simply but can accurately reflect subjective quality of human perception. In this paper, we evaluated the quality assessment of color images using CIE proposed Lab color space, which is considered to be perceptually uniform space. In addition we have used two different approaches of quality assessment namely, metric based and distance based.

We study the problem of automatic "reduced-reference" image quality assessment (QA) algorithms from the point of view of image information change. Such changes are measured between the reference- and natural-image approximations of the distorted image. Algorithms that measure differences between the entropies of wavelet coefficients of reference and distorted images, as perceived by humans, are designed. The algorithms differ in the data on which the entropy difference is calculated and on the amount of information from the reference that is required for quality computation, ranging from almost full information to almost no information from the reference. A special case of these is algorithms that require just a single number from the reference for QA. The algorithms are shown to correlate very well with subjective quality scores, as demonstrated on the Laboratory for Image and Video Engineering Image Quality Assessment Database and the Tampere Image Database. Performance degradation, as the amount of information is reduced, is also studied.

Nuclear medicine technologists (NMTs) are experts in the acquisition of myocardial perfusion (MP) images, in addition to the many other types of images acquired in nuclear medicine departments. NMTs are expected to ensure that images are of optimal quality in order to facilitate accurate interpretation by nuclear medicine physicians (NMPs). However, ensuring optimal image quality is a shared responsibility between NMTs and NMPs. The shared responsibilities have resulted in inconsistences in the assessment of MP image quality among NMTs in different departments. Little is known about the perceptions and experiences of NMTs on the assessment of MP image quality. Therefore, the focus of this research study was NMTs. The aim of this qualitative study was to explore and describe the perceptions and experiences of NMTs on the assessment of MP image quality. The research question was, "How do NMTs perform the responsibility of ensuring MP image quality?" Methods: The study followed a qualitative explorative approach using focus groups as a means of collecting data. Nineteen NMTs from 4 academic hospitals were purposefully selected to participate. A semistructured questionnaire was used to conduct the focus groups. The collected data were managed using a computer-aided qualitative data analysis software program to formulate codes, categories, and themes. Results: Two overarching themes emerged from the data: the management of MP images, and the resources required to support NMTs. NMTs differed in their management of MP images because of the prevailing circumstances in their respective departments. In addition, the results suggested that NMTs' level of involvement in the assessment of MP image quality was influenced by the availability of resources required for processing and assessing image quality. Conclusion: Despite the shared responsibility in the assessment of MP image quality with NMPs, NMTs considered themselves as playing a major role. However, resources to facilitate the assessment of image quality are considered necessary and should be available to support NMTs in submitting images of optimal quality for interpretation.

An optimized CNN-based quality assessment model for screen content image

Visual (image, video) quality assessments can be modelled by visual features in different domains, e.g., spatial, frequency, and temporal domains. Perceptual mechanism in the human visual system (HVS) play a crucial role in the generation of quality perception. This paper proposes a general framework for no-reference visual quality assessment using efficient windowed transformer architectures. A lightweight module for multi-stage channel attention is integrated into the Swin (shifted window) Transformer. Such module can represent appropriate perceptual mechanisms in image quality assessment (IQA) to build an accurate IQA model. Meanwhile, representative features for image quality perception in the spatial and frequency domains can also be derived from the IQA model, which are then fed into another windowed transformer architecture for video quality assessment (VQA). The VQA model efficiently reuses attention information across local windows to tackle the issue of expensive time and memory complexities of original transformer. Experimental results on both large-scale IQA and VQA databases demonstrate that the proposed quality assessment models outperform other state-of-the-art models by large margins.KeywordsImage quality assessmentNo-reference visual quality assessmentTransformerVideo quality assessmentVisual mechanisms

Objective image quality assessment (IQA) plays a very important role in transmission, encoding, and quality of service (QoS) of the image and video data. However, the existing IQA methods often do not consider image content features and their visual perception, so there is a certain gap between the objective IQA sores and the subjective perception. To solve this problem, in the study, we propose an objective IQA method based on the visual perception of image content, which combines the complexity characteristics of image content, and the properties of masking, contrast sensitivity and luminance perception nonlinearity of human visual system (HVS). In the proposed method, the image is first transformed using a nonlinear model of luminance perception to obtain the intensity perception image. Then, the intensity information is summed using the contrast sensitivity values of HVS and the average contrast values of the local image as a weighting factor of the intensity. The summed data information is taken as the content of human perceiving image, and an image perception model is constructed. Finally, the reference images and distorted images are perceived by simulating the HVS with this model. Moreover, the difference in intensity between two perceived images is calculated. Based on the intensity difference and peak signal-to -noise ratio model, an objective IQA model is constructed. Further, the simulation with 47 reference images and 1549 test images in the LIVE, TID2008, and CSIQ databases is conducted. Moreover, the experimental results are compared with those of four typical objective IQA models, namely SSIM, VSNR, FSIM, and PSNRHVS. In addition, we explore the factors that affect the IQA accuracy and a way to improve assessment accuracy by combining HVS characteristics, through analyzing the correlation between IQA results of the proposed model and the subjective mean opinion scores (MOSs) provided in the three image databases from the following two aspects. Namely, (1) all reference images in three image databases are distorted by multiple types, and the distorted images of each reference image are taken as a test sequence. Then, the proposed model is used to evaluate each test sequence to obtain the IQA scores. By analyzing the correlation between the IQA scores of each test sequence and the subjective MOSs and comparing them with the assessment results of SSIM, we explore the influence of the image content complexity on the objective IQA accuracy. (2) The test images which are distorted by each type and many distortion degrees are used as another sequence, and they are evaluated by the proposed IQA model. By analyzing the correlation between the subjective MOSs and the IQA results of each test sequence, and comparing them with assessment results of SSIM, we discuss the influence of image distortion mode on the IQA accuracy. The experimental results show that the coefficient values of Pearson linear correlation and Spearman rank order correlation between the objective IQA scores obtained by the proposed method and the subjective MOSs have been averagely improved by 9.5402% and 3.2852%, respectively, in comparison with IQA results from the SSIM method. Also, they are enhanced more significantly than those fom the PSNRHVS and VSNR methods. In summary, it is shown that the proposed IQA method is an effective and feasible method of objectively assessing the image quality; moreover, it is shown that in the objective assessment of image quality it is very helpful to improve the consistency of subjective and objective assessment of image quality by considering the content perception and complexity analysis of the images.

Authors' response.

Image and video quality assessment (QA) is a critical issue in image and video processing applications. General full-reference (FR) QA criteria such as peak signal-to-noise ratio (PSNR) and mean squared error (MSE) do not accord well with human subjective assessment. Some QA indices that consider human visual sensitivity, such as mean structural similarity (MSSIM) with structural sensitivity, visual information fidelity (VIF) with statistical sensitivity, etc., were proposed in view of the differences between reference and distortion frames on a pixel or local level. However, they ignore the role of human visual attention (HVA). Recently, some new strategies with HVA have been proposed, but the methods extracting the visual attention are too complex for real-time realization. We take advantage of the phase spectrum of quaternion Fourier transform (PQFT), a very fast algorithm we previously proposed, to extract saliency maps of color images or videos. Then we propose saliency-based methods for both image QA (IQA) and video QA (VQA) by adding weights related to saliency features to these original IQA or VQA criteria. Experimental results show that our saliency-based strategy can approach more closely to human subjective assessment compared with these original IQA or VQA methods and does not take more time because of the fast PQFT algorithm.

Conventionally, the reference image for image quality assessment (IQA) is completely available (full-reference IQA) or unavailable (no-reference IQA). Even for reduced-reference IQA, the features that are used to predict image quality are still extracted from the pristine reference image. However, the pristine reference image is always unavailable in many real scenarios. In contrast, it is convenient to obtain a number of similar reference images via retrieval from the Internet. These similar reference images may share similar contents and scenes with the image to be assessed. In this paper, we attempt to discuss the image quality assessment problem from the view of similar images, i.e. similar reference IQA. Although the similar reference images share similar contents with the degraded image, the difference between them still cannot be ignored. Therefore, we propose an IQA framework based on local feature matching, which can help to identify the similar regions and structures. Then the IQA features are computed only from these similar regions to predict the final image quality score. Besides, since there is no IQA databases for the similar reference IQA problem, we establish a novel IQA database that consists of 272 images from four scenes. The experiments demonstrate that the performance of our scheme goes beyond state-of-the-art no-reference IQA methods and some full-reference IQA algorithms.

We study non-reference image and video quality assessment methods, which are of great importance for computational video editing. The object of our work is image quality assessment (IQA) applicable for fast and robust frame-by-frame multipurpose video quality assessment (VQA) for short videos.

In computed tomography (CT) imaging, optimizing the balance between radiation dose and image quality is crucial due to the potentially harmful effects of radiation on patients. Although subjective assessments by radiologists are considered the gold standard in medical imaging, these evaluations can be time-consuming and costly. Thus, objective methods, such as the peak signal-to-noise ratio and structural similarity index measure, are often employed as alternatives. However, these metrics, initially developed for natural images, may not fully encapsulate the radiologists’ assessment process. Consequently, interest in developing deep learning-based image quality assessment (IQA) methods that more closely align with radiologists’ perceptions is growing. A significant barrier to this development has been the absence of open-source datasets and benchmark models specific to CT IQA. Addressing these challenges, we organized the Low-dose Computed Tomography Perceptual Image Quality Assessment Challenge in conjunction with the Medical Image Computing and Computer Assisted Intervention 2023. This event introduced the first open-source CT IQA dataset, consisting of 1,000 CT images of various quality, annotated with radiologists’ assessment scores. As a benchmark, this challenge offers a comprehensive analysis of six submitted methods, providing valuable insight into their performance. This paper presents a summary of these methods and insights. This challenge underscores the potential for developing no-reference IQA methods that could exceed the capabilities of full-reference IQA methods, making a significant contribution to the research community with this novel dataset. The dataset is accessible at https://zenodo.org/records/7833096.