Perceptual Visual Quality Research Articles

Urban planning analysis of the territory of the Republic of Abkhazia with a view to its 
 sustainable development

The article presents an urban planning analysis of the territory of the Republic of Abkhazia, including an analysis of climate, relief, historical and cultural heritage, existing functional zoning for the purpose of rational use of the territory;
 The analysis of the existing purpose for the future will improve the layout of buildings and structures, taking into account the ecological and landscape condition of the territory, the quality of transport infrastructure, compositional and visual perception of the territory

Layered input GradiNet for image denoising

In image denoising, the recovery of high-frequency regions, such as image edges, directly affects the quality of the denoised images. However, previous deep learning-based denoising methods fail to effectively allocate the transmission of different frequency information and have difficulty giving network attention to high-frequency regions. In this paper, we rethink the fusion of image gradients in a neural network and deeply mine the intrinsic structure of the input image to propose a novel layered input gradient network (LIGN) for image denoising. The core of our network focuses on the features of different frequencies through two networks, which contain several key elements: (a) The input noise image is layered to widen the shallow layer of the network and to promote the hierarchical learning of different types of frequencies. (b) A multiscale feature extraction (MFE) block and information shunting (IS) block are proposed to integrate and separate various frequency features. (c) A gradient network (GradiNet) is designed to extract high-frequency information by network training, and the information is adaptively added to the input of the parallel main network (MainNet) through normalization to obtain high-quality images. Furthermore, we propose a sharpening loss function to enhance the texture details of the denoised image and improve visual quality. Extensive experiments on synthetic and real-world datasets show that the proposed method greatly enhances perceptual visual quality and achieves state-of-the-art performance on both PSNR and SSIM. The source code and pretrained models are available at https://github.com/JerryYann/LIGN.

Infrared and Visible Image Fusion Method Using Salience Detection and Convolutional Neural Network.

This paper presents an algorithm for infrared and visible image fusion using significance detection and Convolutional Neural Networks with the aim of integrating discriminatory features and improving the overall quality of visual perception. Firstly, a global contrast-based significance detection algorithm is applied to the infrared image, so that salient features can be extracted, highlighting high brightness values and suppressing low brightness values and image noise. Secondly, a special loss function is designed for infrared images to guide the extraction and reconstruction of features in the network, based on the principle of salience detection, while the more mainstream gradient loss is used as the loss function for visible images in the network. Afterwards, a modified residual network is applied to complete the extraction of features and image reconstruction. Extensive qualitative and quantitative experiments have shown that fused images are sharper and contain more information about the scene, and the fused results look more like high-quality visible images. The generalization experiments also demonstrate that the proposed model has the ability to generalize well, independent of the limitations of the sensor. Overall, the algorithm proposed in this paper performs better compared to other state-of-the-art methods.

Adaptive Bilateral Texture Filter for Image Smoothing.

The biggest challenge of texture filtering is to smooth the strong gradient textures while maintaining the weak structures, which is difficult to achieve with current methods. Based on this, we propose a scale-adaptive texture filtering algorithm in this paper. First, the four-directional detection with gradient information is proposed for structure measurement. Second, the spatial kernel scale for each pixel is obtained based on the structure information; the larger spatial kernel is for pixels in textural regions to enhance the smoothness, while the smaller spatial kernel is for pixels on structures to maintain the edges. Finally, we adopt the Fourier approximation of range kernel, which reduces computational complexity without compromising the filtering visual quality. By subjective and objective analysis, our method outperforms the previous methods in eliminating the textures while preserving main structures and also has advantages in structure similarity and visual perception quality.

Fine Tuning of Deep Contexts Toward Improved Perceptual Quality of In-Paintings.

Over the recent years, a number of deep learning approaches are successfully introduced to tackle the problem of image in-painting for achieving better perceptual effects. However, there still exist obvious hole-edge artifacts in these deep learning-based approaches, which need to be rectified before they become useful for practical applications. In this article, we propose an iteration-driven in-painting approach, which combines the deep context model with the backpropagation mechanism to fine-tune the learning-based in-painting process and hence, achieves further improvement over the existing state of the arts. Our iterative approach fine tunes the image generated by a pretrained deep context model via backpropagation using a weighted context loss. Extensive experiments on public available test sets, including the CelebA, Paris Streets, and PASCAL VOC 2012 dataset, show that our proposed method achieves better visual perceptual quality in terms of hole-edge artifacts compared with the state-of-the-art in-painting methods using various context models.

Multiscale analysis of human social sensing of urban appearance and its effects on house price appreciation in Wuhan, China

• Explore the effect of human social sensing of urban appearance on the house price appreciation rate with consideration of temporal and scale heterogeneity. • Use machine learning with big data containing 34 variables. • Provide strategies to achieve sustainable smart urban layout plan. City managers seek to achieve people-oriented sustainable city development, which requires a clear understanding of socioeconomic effects of citizens’ perceptions of urban appearance. Traditional studies have investigated effects of urban appearance but have ignored the perception of place, which depends on people's unique social experiences. We distinguish “house” and “home” and propose an integrated model to test the prediction effect of combining conventional perception and social perception variables on house price appreciation. We establish a dataset that includes factors, e.g., housing structures, visual quality, and human physical perception. Then, we use machine learning models to extract features from multisource data, investigate the price appreciation of 1,032 houses in Wuhan from 2015 to 2020 and use multiscale geographically weighted regression (MGWR) to discuss spatial dependence. Finally, we perform a cluster analysis to understand the combined effects. The results show that human physical perception has the highest effect, the visual quality of urban streets has the highest impact in places where highly educated people gather, and the impact of service facilities is greatest in economically underdeveloped areas. Our findings provide novel insights into the interlinkages between human social sensing and appreciation rates, which can be efficiently applied to build sustainable smart cities.

Assessing the perception of urban visual quality: an approach integrating big data and geostatistical techniques

Human well-being is affected by the design quality of the city in which they live and walk. This depends primarily on specific physical characteristics and how they are aggregated together. Many studies have highlighted the great potential of photographic data shared on the Flickr platform for analyzing environmental perceptions in landscape and urban planning. Other researchers have used panoramic images from the Google Street View (GSV) web service to extract data on urban quality. However, at the urban level, there are no studies correlating quality perceptions detected by social media platforms with spatial geographic characteristics through geostatistical models. This work proposes the analysis of urban quality in different areas of the Livorno city through a methodological approach based on Geographical Random Forest regression. The result offers important insights into the physical characteristics of a street environment that contribute to the more abstract qualities of urban design.

Patients\u2019 dissatisfaction with multifocal intraocular lenses managed by exchange with other multifocal lenses of different optical profiles

BackgroundThe aim of the study was to evaluate the outcomes of dissatisfied patients reporting poor visual quality following implantation of multifocal intraocular lenses (MF-IOLs), managed by IOL exchange with another multifocal optical profile.MethodsThis is a retrospective series of cases. MF-IOL exchange was done in 15 dissatisfied patients (30 eyes) with the perception of poor visual quality for far distance affected by neuroadaptation failure. Patients underwent a bilateral exchange of a MF-IOL with another MF-IOL of a different optical profile. Visual outcomes and complications were analyzed. Questionnaires including Quality of Vision (QoV), Visual Function Index-14 (VF-14) and its Rasch-revised version (VF-8R) and a satisfaction questionnaire were also used for outcome evaluation.ResultsThe mean elapsed time from implantation to explantation-reimplantation was 11.8 months. The QoV scores improved significantly across all the three subscales. Visual function improved with a change in VF-14 score from 60.41 ± 24.81 to 90.16 ± 10.91 (P < 0.001). The VF-8R score improved as well. The uncorrected distance visual acuity improved from 0.24 to 0.12 logMAR after exchange (P < 0.001) and corrected distance visual acuity improved from 0.15 to 0.04 logMAR (P < 0.001). Safety and efficacy indexes reached 1.46 and 1.16, respectively. Concerning patients’ satisfaction following MF-IOL exchange, 80% of the patients reported they would have the MF-IOL reimplantation procedure again.ConclusionsPatient dissatisfaction with neuroadaptation failure following MF-IOL implantation can be managed in 80% of our cases by MF-IOL exchange with a different MF-IOL optical profile.

A two-stage modular blind denoising algorithm based on real scene

The use of deep learning methods has developed rapidly in image denoising. Now deep learning methods have a good improvement in denoising effects compared to traditional methods, but there are also some problems. On the one hand, these denoisers are more used to remove the noise of a specific distribution, while the image noise distribution in real life is not fixed, so it is more difficult and more practical to denoise the real image. On the other hand, more and more complex network structures and deeper and deeper network models seem to have become a necessary condition for better denoising effects, but its improvement is not linear, and deeper networks are likely to be removed. The improvement in noise effect is minimal. Because of this, we designed a relatively simple network structure to remove real image noise, which includes the noise level estimation stage using the channel attention mechanism, and the non-blind noise reduction stage using the micro-branch structure we designed. Experiments show that our method has good visual perception quality compared with other methods on commonly used image denoising data sets.

Whole-Body Magnetic Resonance Imaging in the Large Population-Based German National Cohort Study: Predictive Capability of Automated Image Quality Assessment for Protocol Repetitions.

Reproducible image quality is of high relevance for large cohort studies and can be challenging for magnetic resonance imaging (MRI). Automated image quality assessment may contribute to conducting radiologic studies effectively. The aims of this study were to assess protocol repetition frequency in population-based whole-body MRI along with its effect on examination time and to examine the applicability of automated image quality assessment for predicting decision-making regarding repeated acquisitions. All participants enrolled in the prospective, multicenter German National Cohort (NAKO) study who underwent whole-body MRI at 1 of 5 sites from 2014 to 2016 were included in this analysis (n = 11,347). A standardized examination program of 12 protocols was used. Acquisitions were carried out by certified radiologic technologists, who were authorized to repeat protocols based on their visual perception of image quality. Eleven image quality parameters were derived fully automatically from the acquired images, and their discrimination ability regarding baseline acquisitions and repetitions was tested. At least 1 protocol was repeated in 12% (n = 1359) of participants, and more than 1 protocol in 1.6% (n = 181). The repetition frequency differed across protocols (P < 0.001), imaging sites (P < 0.001), and over the study period (P < 0.001). The mean total scan time was 62.6 minutes in participants without and 67.4 minutes in participants with protocol repetitions (mean difference, 4.8 minutes; 95% confidence interval, 4.5-5.2 minutes). Ten of the automatically derived image quality parameters were individually retrospectively predictive for the repetition of particular protocols; for instance, "signal-to-noise ratio" alone provided an area under the curve of 0.65 (P < 0.001) for repetition of the Cardio Cine SSFP SAX protocol. Combinations generally improved prediction ability, as exemplified by "image sharpness" plus "foreground ratio" yielding an area under the curve of 0.89 (P < 0.001) for repetition of the Neuro T1w 3D MPRAGE protocol, versus 0.85 (P < 0.001) and 0.68 (P < 0.001) as individual parameters. Magnetic resonance imaging protocol repetitions were necessary in approximately 12% of scans even in the highly standardized setting of a large cohort study. Automated image quality assessment shows predictive value for the technologists' decision to perform protocol repetitions and has the potential to improve imaging efficiency.

Mimicking Individual Media Quality Perception with Neural Network based Artificial Observers

The media quality assessment research community has traditionally been focusing on developing objective algorithms to predict the result of a typical subjective experiment in terms of Mean Opinion Score (MOS) value. However, the MOS, being a single value, is insufficient to model the complexity and diversity of human opinions encountered in an actual subjective experiment. In this work we propose a complementary approach for objective media quality assessment that attempts to more closely model what happens in a subjective experiment in terms of single observers and, at the same time, we perform a qualitative analysis of the proposed approach while highlighting its suitability. More precisely, we propose to model, using neural networks (NNs) , the way single observers perceive media quality. Once trained, these NNs, one for each observer, are expected to mimic the corresponding observer in terms of quality perception. Then, similarly to a subjective experiment, such NNs can be used to simulate the users’ single opinions, which can be later aggregated by means of different statistical indicators such as average, standard deviation, quantiles, etc. Unlike previous approaches that consider subjective experiments as a black box providing reliable ground truth data for training, the proposed approach is able to consider human factors by analyzing and weighting individual observers. Such a model may therefore implicitly account for users’ expectations and tendencies, that have been shown in many studies to significantly correlate with visual quality perception. Furthermore, our proposal also introduces and investigates an index measuring how much inconsistency there would be if an observer was asked to rate many times the same stimulus. Simulation experiments conducted on several datasets demonstrate that the proposed approach can be effectively implemented in practice and thus yielding a more complete objective assessment of end users’ quality of experience.

A continuous bitstream-based blind video quality assessment using multi-layer perceptron

We propose a continuous blind/no-reference video quality assessment (NR-VQA) algorithm based on features extracted from the bitstream, i.e., without decoding the video. The resulting algorithm requires minimal training and adopts a simple multi-layer perceptron for score prediction. The algorithm is computationally appealing. To assess the performance of the algorithm, both the Pearson Correlation Coefficient (PCC) and the Spearman Rank Ordered Correlation Coefficient (SROCC) are computed between the predicted values and the quality scores of two databases. The proposed approach is shown to have a high correlation with human visual perception of quality.

Application of region of interest extraction method based on deep learning in UAV high performance image compression

UAV has been widely used in detecting targets, but the image transmission of UAV still faces the problems of distortion and frame loss. Most image compression methods based on deep learning are lossy compression, and lossy compression reduces image quality in exchange for higher compression ratio. In order to improve the quality of the region of interest (ROI) in the reconstructed image with a certain bit rate, an importance map extraction module is embedded in the encoder, and the importance map is generated by extracting the output features of the last layer of the encoder. Finally, the mask is generated to guide the efficient allocation of bit rate in the process of drop coding. At the same time, a decoder enhancement module is embedded in the decoder output to predict the high frequency components in the reconstructed image and improve the quality of the reconstructed image by enhancing the details in the reconstructed image. The experimental results show that the proposed method is superior to the comparison method when multi-scale structural similarity (MS-SSIM) is used as the evaluation index, and the proposed method achieves better visual perception quality.

Speckle Reduction in Medical Ultrasound Imaging based on Visual Perception Model

Ultrasound imaging technology is one of the most important clinical imaging modalities due to its safety, low cost, in addition to its versatile applications. The main technical problem in this technology is its noisy appearance due to the presence of speckle, which makes reading imaging more difficult. In this study, a new method of speckle reduction in medical ultrasound images is proposed based on adaptive shifting of the contrast sensitivity function of human vision using a bias field map estimated from the original image. The aim of this work is to have an effective image enhancement strategy that reduces speckle while preserving diagnostically useful image features and allowing practical implementation in real-time for medical ultrasound imaging applications. The new method is used to improve the visual perception of image quality of ultrasound images by adding a local brightness bias to the areas with speckle noise. This allows the variations in image pixels due to speckle noise to be better perceived by the human observer because of the visual perception model. The performance of the proposed method is objectively assessed using quantitative image quality metrics and compared to previous methods. Furthermore, given that image quality perception is subjective, the level of added bias is controlled by a single parameter that accommodates the different needs for different users and applications. This method has potential to offer better viewing conditions of ultrasound images, which translates to higher diagnostic accuracy.

Intermediate-Layer Transferable Adversarial Attack With DNN Attention

The widespread deployment of deep learning models in practice necessitates an assessment of their vulnerability, particularly in security-sensitive areas. As a result, transfer-based adversarial attacks have elicited increasing interest in assessing the security of deep learning models. However, adversarial samples usually exhibit poor transferability over different models because of overfitting of the particular architecture and feature representation of a source model. To address this problem, the Intermediate Layer Attack with Attention guidance (IAA) is proposed to alleviate overfitting and enhance the black-box transferability. The IAA works on an intermediate layer <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> of the source model. Guided by the model’s attention to the features of layer <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">l</i> , the attack algorithm seeks and undermines the key features that are likely to be adopted by diverse architectures. Significantly, IAA focuses on improving existing white-box attacks without introducing significant visual perceptual quality degradation. Namely, IAA maintains the white-box attack performance of the original algorithm while significantly enhancing its black-box transferability. Extensive experiments on ImageNet classifiers confirmed the effectiveness of our method. The proposed IAA outperformed all state-of-the-art benchmarks in various white-box and black-box settings.

Image enhancement with luminance preserving is a false proposition and others

The “image enhancement with luminance preservation” is a self-conflict proposition. Researchers, including propositional proposer Kim, gave the index AMBE (absolute mean luminance error) to judge if the “luminance preserving”, however, which just proved that the image enhancement with “luminance preserving” cannot be realized. We have proved that the visual quality of an image is a function of the image average luminance with convex feature. The image cannot be enhanced with luminance preservation, and the luminance cannot be preserved with image being enhanced. The visual perception quality of an image is not equal to the image contrast. We can experimentally prove what the higher the contrast of the image, the better the visual quality of the image is not true.

Visual Perceptual Quality Assessment Based on Blind Machine Learning Techniques.

This paper presents the construction of a new objective method for estimation of visual perceiving quality. The proposal provides an assessment of image quality without the need for a reference image or a specific distortion assumption. Two main processes have been used to build our models: The first one uses deep learning with a convolutional neural network process, without any preprocessing. The second objective visual quality is computed by pooling several image features extracted from different concepts: the natural scene statistic in the spatial domain, the gradient magnitude, the Laplacian of Gaussian, as well as the spectral and spatial entropies. The features extracted from the image file are used as the input of machine learning techniques to build the models that are used to estimate the visual quality level of any image. For the machine learning training phase, two main processes are proposed: The first proposed process consists of a direct learning using all the selected features in only one training phase, named direct learning blind visual quality assessment . The second process is an indirect learning and consists of two training phases, named indirect learning blind visual quality assessment . This second process includes an additional phase of construction of intermediary metrics used for the construction of the prediction model. The produced models are evaluated on many benchmarks image databases as , , and in the wild image quality challenge. The experimental results demonstrate that the proposed models produce the best visual perception quality prediction, compared to the state-of-the-art models. The proposed models have been implemented on an platform to demonstrate the feasibility of integrating the proposed solution on an image sensor.

Machine Learning Method for Functional Assessment of Retinal Models.

Challenges in the field of retinal prostheses motivate the development of retinal models to accurately simulate Retinal Ganglion Cells (RGCs) responses. The goal of retinal prostheses is to enable blind individuals to solve complex, reallife visual tasks. In this paper, we introduce the functional assessment (FA) of retinal models, which describes the concept of evaluating the performance of retinal models on visual understanding tasks. We present a machine learning method for FA: we feed traditional machine learning classifiers with RGC responses generated by retinal models, to solve object and digit recognition tasks (CIFAR-10, MNIST, Fashion MNIST, Imagenette). We examined critical FA aspects, including how the performance of FA depends on the task, how to optimally feed RGC responses to the classifiers and how the number of output neurons correlates with the model's accuracy. To increase the number of output neurons, we manipulated input images - by splitting and then feeding them to the retinal model and we found that image splitting does not significantly improve the model's accuracy. We also show that differences in the structure of datasets result in largely divergent performance of the retinal model (MNIST and Fashion MNIST exceeded 80% accuracy, while CIFAR-10 and Imagenette achieved ∼40%). Furthermore, retinal models which perform better in standard evaluation, i.e. more accurately predict RGC response, perform better in FA as well. However, unlike standard evaluation, FA results can be straightforwardly interpreted in the context of comparing the quality of visual perception.

High dynamic range imaging via gradient-aware context aggregation network

Obtaining a high dynamic range (HDR) image from multiple low dynamic range images with different exposures is an important step in various computer vision tasks. One of the ongoing challenges in the field is to generate HDR images without ghosting artifacts. Motivated by an observation that such artifacts are particularly noticeable in the gradient domain, in this paper, we propose an HDR imaging approach that aggregates the information from multiple LDR images with guidance from image gradient domain. The proposed method generates artifact-free images by integrating the image gradient information and the image context information in the pixel domain. The context information in a large area helps to reconstruct the contents contaminated by saturation and misalignments. Specifically, an additional gradient stream and the supervision in the gradient domain are applied to incorporate the gradient information in HDR imaging. To use the context information captured from a large area while preserving spatial resolution, we adopt dilated convolutions to extract multi-scale features with rich context information. Moreover, we build a new dataset containing 40 groups of real-world images from diverse scenes with ground truth to validate the proposed model. The samples in the proposed dataset include more challenging moving objects inducing misalignments. Extensive experimental results demonstrate that our proposed model outperforms previous methods on different datasets in terms of both quantitative measure and visual perception quality.

Visualizing evaluation model of human perceptions and characteristic indicators of landscape visual quality in urban green spaces by using nomograms

Understanding human perceptions and quantifying parameters that characterize landscape visual quality in an urban green space is important to describe these connections. Within this environment, the association between human perceptions of landscape visual quality (LVQ) have long been of interest to a wide variety of fields. Previous studies have relied on formulas which did not include a direct-viewing and visible information to predict human perceptions of the LVQ of urban green space. In this work, 300 photos of urban green space were taken and evaluated for LVQ by a questionnaire to obtain scores for six human perceptions (e.g., Beauty, Color, Depression, Complex, Safety, and Liveliness). Each photo presents an urban green space landscape and 28 characteristic indicators were identified to explain LVQ. We used logistic regression to identify indicators and Nomograms to display the relationship between characteristic indicators and human perceptions. Different from other formula-based models, the contribution of characteristic indicators on LVQ were clearly and visually expressed by Nomogram. In visualizing the study results with Nomogram, the 28 characteristic indicators were identified with their relationship to each of the six perceptions and the weight of each indicator could be displayed by the length of line. The study also showed that Tree health and Neatness contributed to all human perceptions. Specifically, Tree health affects Beauty, Complex and Liveliness most, while Color contrast has the greatest influence on Color. Recreation Road ratio is the most important factor for Depression and Safety. In practical application for urban designers and planners, the Nomogram results can quickly illustrate the influence of characteristic indicators on the changing of LVQ in urban green space.