Human Perceptual Evaluation Research Articles

Application of Machine Learning and Multi-Dimensional Perception in Urban Spatial Quality Evaluation: A Case Study of Shanghai Underground Pedestrian Street

The exploitation of urban subsurface space in urban inventory planning is closely connected to the quality of urban environments. Currently, the construction of underground pedestrian streets is characterised by inefficiency and traffic congestion, making them insufficient for fulfilling the demand for well-designed and human-centred spaces. In the study of spatial quality, traditional evaluation methods, such as satellite remote sensing and street maps, often suffer from low accuracy and slow updating rates, and they frequently overlook human perceptual evaluations. Consequently, there is a pressing need to develop a set of spatial quality evaluation methods incorporating pedestrian perspectives, thereby addressing the neglect of subjective human experiences in spatial quality research. This study first quantifies and clusters the characteristics of underground pedestrian spaces using spatial syntax. It then gathers multidimensional perception data from selected locations and ultimately analyses and predicts the results employing machine learning techniques, specifically Random Forest and XGBoost. The research results indicate variability in pedestrians’ evaluations of spatial quality across different functionally oriented spaces. Key factors influencing these evaluations include Gorgeous, Warm, Good Ventilation, and Flavour indicators. The study proposes a comprehensive and applicable spatial quality evaluation model integrating spatial quantification methods, machine learning algorithms, and multidimensional perception measurements. The development of this model offers valuable scientific guidance for the planning and construction of high-quality urban public spaces.

Interactive landscape–scale cloud animation using DCGAN

This article presents an interactive method for 3D cloud animation at the landscape scale by employing machine learning. To this end, we utilize deep convolutional generative adversarial network (DCGAN) on GPU for training on home-captured cloud videos and producing coherent animation frames. We limit the size of input images provided to DCGAN, thereby reducing the training time and yet producing detailed 3D animation frames. This is made possible through our preprocessing of the source videos, wherein several corrections are applied to the extracted frames to provide an adequate input training data set to DCGAN. A significant advantage of the presented cloud animation is that it does not require any underlying physics simulation. We present detailed results of our approach and verify its effectiveness using human perceptual evaluation. Our results indicate that the proposed method is capable of convincingly realistic 3D cloud animation, as perceived by the participants, without introducing too much computational overhead.

Perceptual Quality Assessment of Pan-Sharpened Images

Pan-sharpening (PS) is a method of fusing the spatial details of a high-resolution panchromatic (PAN) image with the spectral information of a low-resolution multi-spectral (MS) image. Visual inspection is a crucial step in the evaluation of fused products whose subjectivity renders the assessment of pansharpened data a challenging problem. Most previous research on the development of PS algorithms has only superficially addressed the issue of qualitative evaluation, generally by depicting visual representations of the fused images. Hence, it is highly desirable to be able to predict pan-sharpened image quality automatically and accurately, as it would be perceived and reported by human viewers. Such a method is indispensable for the correct evaluation of PS techniques that produce images for visual applications such as Google Earth and Microsoft Bing. Here, we propose a new image quality assessment (IQA) measure that supports the visual qualitative analysis of pansharpened outcomes by using the statistics of natural images, commonly referred to as natural scene statistics (NSS), to extract statistical regularities from PS images. Importantly, NSS are measurably modified by the presence of distortions. We analyze six PS methods in the presence of two common distortions, blur and white noise, on PAN images. Furthermore, we conducted a human study on the subjective quality of pristine and degraded PS images and created a completely blind (opinion-unaware) fused image quality analyzer. In addition, we propose an opinion-aware fused image quality analyzer, whose predictions with respect to human perceptual evaluations of pansharpened images are highly correlated.

Predicting the Quality of Fused Long Wave Infrared and Visible Light Images.

The capability to automatically evaluate the quality of long wave infrared (LWIR) and visible light images has the potential to play an important role in determining and controlling the quality of a resulting fused LWIR-visible light image. Extensive work has been conducted on studying the statistics of natural LWIR and visible images. Nonetheless, there has been little work done on analyzing the statistics of fused LWIR and visible images and associated distortions. In this paper, we analyze five multi-resolution-based image fusion methods in regards to several common distortions, including blur, white noise, JPEG compression, and non-uniformity. We study the natural scene statistics of fused images and how they are affected by these kinds of distortions. Furthermore, we conducted a human study on the subjective quality of pristine and degraded fused LWIR-visible images. We used this new database to create an automatic opinion-distortion-unaware fused image quality model and analyzer algorithm. In the human study, 27 subjects evaluated 750 images over five sessions each. We also propose an opinion-aware fused image quality analyzer, whose relative predictions with respect to other state-of-the-art models correlate better with human perceptual evaluations than competing methods. An implementation of the proposed fused image quality measures can be found at https://github.com/ujemd/NSS-of-LWIR-and-Vissible-Images. Also, the new database can be found at http://bit.ly/2noZlbQ.

Image fusion quality assessment based on discrete cosine transform and human visual system

With the rapid development of image fusion technology, image fusion quality evaluation plays a very important guiding role in selecting or designing image fusion algorithms. Objective image quality assessment is an interesting research subject in the field of image quality assessment. The ideal objective evaluation method is consistent with human perceptual evaluation. A new fusion image quality assessment method according with human vision system and discrete cosine transform (DCT) is introduced. Firstly, using the Sobel operator to calculate to gradient images for the source images and fused image, the gradient images are divided into 8×8 blocks and calculating the DCT coefficients for each block, and then based on the characteristics of human visual system, calculates the luminance masking, contrast masking to form the perceptual error matrix between input images and fused images. Finally, weighs the perceptual error matrix using the structural similarity. Experiments demonstrate that the new assessment maintains better consistency with human subjective perception.

Study and correlation analysis of linguistic, perceptual, and automatic machine translation evaluations

AbstractEvaluation of machine translation output is an important task. Various human evaluation techniques as well as automatic metrics have been proposed and investigated in the last decade. However, very few evaluation methods take the linguistic aspect into account. In this article, we use an objective evaluation method for machine translation output that classifies all translation errors into one of the five following linguistic levels: orthographic, morphological, lexical, semantic, and syntactic. Linguistic guidelines for the target language are required, and human evaluators use them in to classify the output errors. The experiments are performed on English‐to‐Catalan and Spanish‐to‐Catalan translation outputs generated by four different systems: 2 rule‐based and 2 statistical. All translations are evaluated using the 3 following methods: a standard human perceptual evaluation method, several widely used automatic metrics, and the human linguistic evaluation. Pearson and Spearman correlation coefficients between the linguistic, perceptual, and automatic results are then calculated, showing that the semantic level correlates significantly with both perceptual evaluation and automatic metrics.

Listeners’ weighting of acoustic cues to synthetic speech naturalness: A multidimensional scaling analysis

The quality of current commercial speech synthesis systems is now so high that system improvements are being made at subtle sub- and supra-segmental levels. Human perceptual evaluation of such subtle improvements requires a highly sophisticated level of perceptual attention to specific acoustic characteristics or cues. However, it is not well understood what acoustic cues listeners attend to by default when asked to evaluate synthetic speech. It may, therefore, be potentially quite difficult to design an evaluation method that allows listeners to concentrate on only one dimension of the signal, while ignoring others that are perceptually more important to them. The aim of the current study was to determine which acoustic characteristics of unit-selection synthetic speech are most salient to listeners when evaluating the naturalness of such speech. This study made use of multidimensional scaling techniques to analyse listeners’ pairwise comparisons of synthetic speech sentences. Results indicate that listeners place a great deal of perceptual importance on the presence of artifacts and discontinuities in the speech, somewhat less importance on aspects of segmental quality, and very little importance on stress/intonation appropriateness. These relative differences in importance will impact on listeners’ ability to attend to these different acoustic characteristics of synthetic speech, and should therefore be taken into account when designing appropriate methods of synthetic speech evaluation.

A new automated quality assessment algorithm for image fusion

Automated image quality assessment is highly desirable to evaluate the performance of various image fusion algorithms for night vision applications. In this paper we propose a perceptual quality evaluation method for image fusion which is based on human visual system (HVS) models. Our method assesses the image quality of a fused image using the following steps. First, the source and fused images are filtered by a contrast sensitivity function (CSF) after which a local contrast map is computed for each image. Second, a contrast preservation map is generated to describe the relationship between the fused image and each source image. Finally, the preservation maps are weighted by a saliency map to obtain an overall quality map. The mean of the quality map indicates the quality of the fused image. Experimental results compare the predictions made by our algorithm with human perceptual evaluations for several different parameter settings in our algorithm. The most popular existing algorithms are also evaluated. For some specific parameter settings, we find our algorithm provides better predictions, which are more closely matched to human perceptual evaluations, than the existing algorithms. The evaluations focus on the night vision application, but the algorithm we propose is applicable to other applications also.