Characteristics Of Human Visual Perception Research Articles

Feature-level contrastive learning for full-reference light field image quality assessment

Light field (LF), as a new visual information representation for immersive experiences in virtual reality and augmented reality, has attracted extensive attention. It is of significance to establish the light field image quality assessment (LF-IQA) that conforms to the characteristics of human visual perception. Due to the spatial-angular inter-twined imaging mechanism, the quality evaluation for LF images faces more challenges than ordinary 2D/3D IQA. This paper proposes a feature-level contrastive learning based full-reference LF-IQA with a divide-and-conquer learning strategy. It first employs the cross-attention mechanism to learn the local contrast features related to the reference features from three subsets of distorted sub-aperture images (SAIs). Then a Transformer-based feature fusion is used to predict the overall perceptual quality. The evaluation experiments are carried out on three commonly used LF-IQA datasets: Win5-LID, NBU-LF1.0, and SHU. The experimental results demonstrate that our model outperforms the existing state-of-the-art LF-IQA methods.

A modified atmospheric scattering model and degradation image clarification algorithm for haze environments

Addressing the limitation of existing image dehazing algorithms based on atmospheric scattering models, which often overlook the underlying physical causes of color distortion in hazy images, resulting in suboptimal algorithm robustness and image clarity when faced with varying levels of haze pollution, this study introduces a refined atmospheric scattering model. It also presents a haze image clarity algorithm based on this modified atmospheric scattering model by analyzing the fundamental factors contributing to image degradation and distortion caused by haze. Initially, the hazy image undergoes conversion to a new HSI color space. The hazy image is aligned more closely with human visual perception characteristics by implementing an adaptive saturation enhancement technique and adjusting contribution coefficients for the three RGB channels to the luminance component. Subsequently, the hazy image is partitioned into dense haze and non-dense haze regions based on haze distribution characteristics within the image. A global light intensity computation method is devised for the hazy image utilizing a quad-tree segmentation algorithm. Following this, calculations are performed to determine the atmospheric light value and transmittance for dense and non-dense haze regions separately. The global atmospheric light and transmittance values are then derived based on the probability density function of the dense haze region using Alpha Fusion. Finally, corrections are applied to the global atmospheric light values using an atmospheric light correction vector, while denoising the global transmittance is achieved through a fast non-local mean filtering algorithm. Experimental findings demonstrate that the proposed haze image clarity algorithm, grounded in the modified atmospheric scattering model, exhibits adaptability across various degrees and types of haze images. Comparative analysis against other prominent image dehazing algorithms reveals its efficacy in addressing severe color distortion in dense haze scenarios, achieving a more natural clarity effect, and showcasing superior performance across all numerical evaluation metrics.

A Remote Sensing Image Quality Interpretation Scale Characterization Method Based on the TTP Criterion

Accurate grading of remote sensing image interpretation is crucial for improving image classification and screening efficiency. Through extensive research, the General Image Quality Equation (GIQE) based on the National Imagery Interpretability Rating Scale (NIIRS) has been developed. However, poor reliability and low accuracy issues remain due to the failure to consider human visual characteristics. This paper introduces the Target Task Performance (TTP) criterion as a key parameter to reflect the cascading degradation factors of human visual perception characteristics and system imaging links, which improves the reliability of the model. A New Optimized Remote Sensing Image Quality Equation (NORSIQE), which effectively predicted the interpretability of image information, is constructed. Using 200 sets of test data, the quantitative relationship between key parameters (GSD, TTP, and SNR) of the NORSIQE and the subjective NIIRS level is obtained by a least squares regression fit, and the determination coefficient of the model is as high as 0.916. The model is evaluated for accuracy using 120 sets of validation data, showing an 87% improvement compared to the GIQE4. This method provides theoretical support for the development of new methods for remote sensing image quality evaluation and the design of payloads for remote sensing imaging systems.

Blind Video Quality Assessment for Ultra-High-Definition Video Based on Super-Resolution and Deep Reinforcement Learning.

Ultra-high-definition (UHD) video has brought new challenges to objective video quality assessment (VQA) due to its high resolution and high frame rate. Most existing VQA methods are designed for non-UHD videos-when they are employed to deal with UHD videos, the processing speed will be slow and the global spatial features cannot be fully extracted. In addition, these VQA methods usually segment the video into multiple segments, predict the quality score of each segment, and then average the quality score of each segment to obtain the quality score of the whole video. This breaks the temporal correlation of the video sequences and is inconsistent with the characteristics of human visual perception. In this paper, we present a no-reference VQA method, aiming to effectively and efficiently predict quality scores for UHD videos. First, we construct a spatial distortion feature network based on a super-resolution model (SR-SDFNet), which can quickly extract the global spatial distortion features of UHD videos. Then, to aggregate the spatial distortion features of each UHD frame, we propose a time fusion network based on a reinforcement learning model (RL-TFNet), in which the actor network continuously combines multiple frame features extracted by SR-SDFNet and outputs an action to adjust the current quality score to approximate the subjective score, and the critic network outputs action values to optimize the quality perception of the actor network. Finally, we conduct large-scale experiments on UHD VQA databases and the results reveal that, compared to other state-of-the-art VQA methods, our method achieves competitive quality prediction performance with a shorter runtime and fewer model parameters.

Rich Structural Index for Stereoscopic Image Quality Assessment.

The human visual system (HVS), affected by viewing distance when perceiving the stereo image information, is of great significance to study of stereoscopic image quality assessment. Many methods of stereoscopic image quality assessment do not have comprehensive consideration for human visual perception characteristics. In accordance with this, we propose a Rich Structural Index (RSI) for Stereoscopic Image objective Quality Assessment (SIQA) method based on multi-scale perception characteristics. To begin with, we put the stereo pair into the image pyramid based on Contrast Sensitivity Function (CSF) to obtain sensitive images of different resolution. Then, we obtain local Luminance and Structural Index (LSI) in a locally adaptive manner on gradient maps which consider the luminance masking and contrast masking. At the same time we use Singular Value Decomposition (SVD) to obtain the Sharpness and Intrinsic Structural Index (SISI) to effectively capture the changes introduced in the image (due to distortion). Meanwhile, considering the disparity edge structures, we use gradient cross-mapping algorithm to obtain Depth Texture Structural Index (DTSI). After that, we apply the standard deviation method for the above results to obtain contrast index of reference and distortion components. Finally, for the loss caused by the randomness of the parameters, we use Support Vector Machine Regression based on Genetic Algorithm (GA-SVR) training to obtain the final quality score. We conducted a comprehensive evaluation with state-of-the-art methods on four open databases. The experimental results show that the proposed method has stable performance and strong competitive advantage.

A measurable evaluation method of visual comfort in underground space by intelligent sorting and classification algorithms

Based on human perception and machine learning methods, this study proposes a measurable method for evaluating visual comfort in underground spaces. First, a comfort evaluation index based on the characteristics of human visual perception is proposed, and color features and segmentation extraction methods for intelligent methods are given. Then, using probability statistics and machine learning methods, a multi-class intelligent sorting and classification algorithm for ranking visual comfort levels is constructed and a comparison is made of the suitability of different intelligent methods for evaluating visual comfort. The random forest algorithm is then selected as the most effective measurable intelligent evaluation algorithm for underground spaces. Finally, the proposed method is compared to intelligent methods employed by previous research, and a case study, the Wujiaochang underground space in Shanghai, China, is applied as the background. Results show that the proposed method can effectively improve the quantification and refinement of human perception and evaluation of comfort in underground spaces; this method will also be useful in computer-aided generative design in the future.

Hierarchical Image Segmentation Based on Nonsymmetry and Anti-Packing Pattern Representation Model.

Image segmentation is the foundation of high-level image analysis and image understanding. How to effectively segment an image into regions that are "meaningful" to the human visual perception and ensure that the segmented regions are consistent at different resolutions is still a very challenging issue. Inspired by the idea of the Nonsymmetry and Anti-packing pattern representation Model in the Lab color space (NAMLab) and the "global-first" invariant perceptual theory, in this paper, we propose a novel framework for hierarchical image segmentation. Firstly, by defining the dissimilarity between two pixels in the Lab color space, we propose an NAMLab-based color image representation approach that is more in line with the human visual perception characteristics and can make the image pixels fast and effectively merge into the NAMLab blocks. Then, by defining the dissimilarity between two NAMLab-based regions and iteratively executing NAMLab-based merging algorithm of adjacent regions into larger ones to progressively generate a segmentation dendrogram, we propose a fast NAMLab-based algorithm for hierarchical image segmentation. Finally, the complexities of our proposed NAMLab-based algorithm for hierarchical image segmentation are analyzed in details. The experimental results presented in this paper show that our proposed algorithm when compared with the state-of-the-art algorithms not only can preserve more details of the object boundaries, but also it can better identify the foreground objects with similar color distributions. Also, our proposed algorithm can be executed much faster and takes up less memory and therefore it is a better algorithm for hierarchical image segmentation.

Objective assessment of image quality based on image content contrast perception

Image quality assessment (IQA) plays a very important role in acquiring, storing, transmitting and processing image and video. Using the characteristics of human visual perception and the features of the gray, gradient, local contrast, and blurring of image, an IQA method based on the image content contrast perception is proposed in the paper, which is called MPCC. In the proposed method, firstly, combining with the characteristics of human visual perception, based on the definition of the contrast in physics, a novel definition for image quality and its calculation method are proposed. Then, based on the gray gradient co-occurrence matrix, a novel concept, namely the gray-gradient entropy of image, and its calculation method, are proposed. And based on the gray-gradient entropy, local contrast and blurring of image, a method of describing the image content and their visual perception are proposed. Finally, based on the image content features and the image quality definition, an IQA method and its mathematical model are proposed by comprehensive analysis. Further, the proposed IQA model MPCC is tested by using 119 reference images and 6395 distorted images from the five open image databases (LIVE, CSIQ, TID2008, TID2013 and IVC). Moreover, the influences of the 52 distortion types on IQA are analyzed. In addition, in order to illustrate the advantages of the MPCC model, it is compared with the seven existing typical IQA models in terms of the accuracy, complexity and generalization performance of model. The experimental results show that the accuracy PLCC of the MPCC model can achieve 0.8616 at lowest and 0.9622 at most in the five databases; among the 52 distortion types, the two distortion types, namely the change of color saturation and the local block-wise distortions of different intensity, have the greatest influence on IQA, and the accuracy PLCC values of the seven existing IQA models are almost all below 0.6, but the PLCC of the MPCC model can reach more than 0.68; and the comprehensive benefit of the performance of the MPCC model is better than those of the seven existing IQA models. These results of test and comparison above show that the proposed IQA method is effective and feasible, and the corresponding model has an excellent performance.

Robust Image Watermarking via Perceptual Structural Regularity-based JND Model

A better tradeoff between robustness and invisibility will be realized by using the just noticeable (JND) model into the quantization-based watermarking scheme. The JND model is usually used to describe the perception characteristics of human visual systems (HVS). According to the research of cognitive science, HVS can adaptively extract the structure features of an image. However, the existing JND models in the watermarking scheme do not consider the structure features. Therefore, a novel JND model is proposed , which includes three aspects: contrast sensitivity function, luminance adaptation, and contrast masking (CM). In this model, the CM effect is modeled by analyzing the direction features and texture complexity, which meets the human visual perception characteristics and matches well with the spread transform dither modulation (STDM) watermarking framework by employing a new method to measure edge intensity. Compared with the other existing JND models, the proposed JND model based on structural regularity is more efficient and applicable in the STDM watermarking scheme. In terms of the experimental results, the proposed scheme performs better than the other watermarking scheme based on the existing JND models.

Multiview Stereoscopic Video Hole Filling Considering Spatiotemporal Consistency and Binocular Symmetry for Synthesized 3D Video

This paper proposes a new hole-filling method with spatiotemporal consistency and binocular symmetry for synthesized 3D videos in view extrapolation. Disocclusion regions in the synthesized views at virtual viewpoints result in regions with missing content. These regions will be referred to as hole regions. To provide the high-quality synthesized 3D videos via 3D display, the hole regions need to be filled considering the characteristics of human visual perception. From the perceptual point of view, binocular asymmetry between synthesized left- and right-eye videos (i.e., stereo pair video) is one of the most important factors that induce visual discomfort in stereoscopic viewing. In addition, binocular symmetry without temporal consistency between temporally neighboring frames could cause visual discomfort by annoying flickering artifacts. In this paper, to maintain the spatiotemporal consistency and binocular symmetry in synthesized 3D videos at multiple virtual viewpoints, we propose a global optimization-based hole-filling method using the information from the already filled adjacent view and previous frame. Furthermore, to reduce the computational cost of the global optimization, we propose a label propagation method, which propagates reliable labels used in the adjacent view and previous frame to the target image to be filled. The performance of the proposed method has been evaluated by objective assessments of 3D image quality, temporal consistency, and computational efficiency. In addition, subjective assessment is conducted for measuring the visual comfort and overall quality. The experimental results proved that the proposed method provides hole filling results with spatiotemporal consistency and binocular symmetry.

Blind Image Quality Assessment Using Statistical Structural and Luminance Features

Blind image quality assessment (BIQA) aims to develop quantitative measures to automatically and accurately estimate perceptual image quality without any prior information about the reference image. In this paper, we introduce a novel BIQA metric by structural and luminance information, based on the characteristics of human visual perception for distorted image. We extract the perceptual structural features of distorted image by the local binary pattern distribution. Besides, the distribution of normalized luminance magnitudes is extracted to represent the luminance changes in distorted image. After extracting the features for structures and luminance, support vector regression is adopted to model the complex nonlinear relationship from feature space to quality measure. The proposed BIQA model is called no-reference quality assessment using statistical structural and luminance features (NRSL). Extensive experiments conducted on four synthetically distorted image databases and three naturally distorted image databases have demonstrated that the proposed NRSL metric compares favorably with the relevant state-of-the-art BIQA models in terms of high correlation with human subjective ratings. The MATLAB source code and validation results of NRSL are publicly online at http://www.ntu.edu.sg/home/wslin/Publications.htm .

Technique for multi-focus image fusion based on fuzzy-adaptive pulse-coupled neural network

Multi-focus image fusion technique can solve the problem that not all the targets in an image are clear in case of imaging in the same scene. In this paper, a novel multi-focus image fusion technique is presented, which is developed by using the nonsubsampled contourlet transform (NSCT) and a proposed fuzzy logic based adaptive pulse-coupled neural network (PCNN) model. In our method, sum-modified Laplacian (SML) is calculated as the motivation for PCNN neurons in NSCT domain. Since the linking strength plays an important role in PCNN, we propose an adaptively fuzzy way to determine it by computing each coefficient’s importance relative to the surrounding coefficients. Combined with human visual perception characteristics, the fuzzy membership value is employed to automatically achieve the degree of importance of each coefficient, which is utilized as the linking strength in PCNN model. Experimental results on simulated and real multi-focus images show that the proposed technique has a superior performance to series of exist fusion methods.

Region-based saliency estimation for 3D shape analysis and understanding

The detection of salient regions is an important pre-processing step for many 3D shape analysis and understanding tasks. This paper proposes a novel method for saliency detection in 3D free form shapes. Firstly, we smooth the surface normals by a bilateral filter. Such a method is capable of smoothing the surfaces and retaining the local details. Secondly, a novel method is proposed for the estimation of the saliency value of each vertex. To this end, two new features are defined: Retinex-based Importance Feature (RIF) and Relative Normal Distance (RND). They are based on the human visual perception characteristics and surface geometry respectively. Since the vertex based method cannot guarantee that the detected salient regions are semantically continuous and complete, we propose to refine such values based on surface patches. The detected saliency is finally used to guide the existing techniques for mesh simplification, interest point detection, and overlapping point cloud registration. The comparative studies based on real data from three publicly accessible databases show that the proposed method usually outperforms five selected state of the art ones both qualitatively and quantitatively for saliency detection and 3D shape analysis and understanding.

Depth video spatial and temporal correlation enhancement algorithm based on just noticeable rendering distortion model

Spatial and temporal inconsistency of depth video deteriorates encoding efficiency in three dimensional video systems. A depth video processing algorithm based on human perception is presented. Firstly, a just noticeable rendering distortion (JNRD) model is formulated by combining the analyses of the influence of depth distortion on virtual view rendering with human visual perception characteristics. Then, depth video is processed based on the JNRD model from two aspects, spatial and temporal correlation enhancement. During the process of spatial correlation enhancement, depth video is segmented into edge, foreground, and background regions, and smoothened by Gaussian and mean filters. The operations of the temporal correlation enhancement include temporal–spatial transpose (TST), temporal smoothing filter and inverse TST. Finally, encoding and virtual view rendering experiments are conducted to evaluate the proposed algorithm. Experimental results show that the proposed algorithm can greatly reduce the bit rate while it maintains the quality of virtual view.

Perception-Inspired Background Subtraction

Developing universal and context-invariant methods is one of the hardest challenges in computer vision. Background subtraction (BS), an essential precursor in most machine vision applications used for foreground detection, is no exception. Due to overreliance on statistical observations, most BS techniques show unpredictable behavior in dynamic unconstrained scenarios in which the characteristics of the operating environment are either unknown or change drastically. To achieve superior foreground detection quality across unconstrained scenarios, we propose a new technique, called perception-inspired background subtraction (PBS), which avoids overreliance on statistical observations by making key modeling decisions based on the characteristics of human visual perception. PBS exploits the human perception-inspired confidence interval to associate an observed intensity value with another intensity value during both model learning and background-foreground classification. The concept of perception-inspired confidence interval is also used for identifying redundant samples, thus ensuring the optimal number of samples in the background model. Furthermore, PBS dynamically varies the model adaptation speed (learning rate) at pixel level based on observed scene dynamics to ensure faster adaptation of changed background regions, as well as longer retention of stationary foregrounds. Extensive experimental evaluations on a wide range of benchmark datasets validate the efficacy of PBS compared to the state of the art for unconstraint video analytics.

모바일 환경을 위해 에지맵 보간과 개선된 고속 Back Projection 기법을 이용한 Super Resolution 알고리즘

Recently, as the prevalence of high-performance mobile devices and the application of the multimedia content are expanded, Super Resolution (SR) technique which reconstructs low resolution images to high resolution images is becoming important. And in the mobile devices, the development of the SR algorithm considering the operation quantity or memory is required because of using the restricted resources. In this paper, we propose a new single frame fast SR technique suitable for mobile devices. In order to prevent color distortion, we change RGB color domain to HSV color domain and process the brightness information V (Value) considering the characteristics of human visual perception. First, the low resolution image is enlarged by the improved fast back projection considering the noise elimination. And at the same time, the reliable edge map is extracted by using the LoG (Laplacian of Gaussian) filtering. Finally, the high definition picture is reconstructed by using the edge information and the improved back projection result. The proposed technique removes effectually the unnatural artefact which is generated during the super resolution restoration, and the edge information which can be lost is amended and emphasized. The experimental results indicate that the proposed algorithm provides better performance than conventional back projection and interpolation methods.

Depth perception enhancement based on chromostereopsis in a 3D display

This study was conducted to enhance the cubic effect by representing an image with a sense of three-dimensional (3D) depth, using chromostereopsis, among the characteristics of human visual perception. An algorithm that enhances the cubic effect, based on the theory that the cubic effect of the chromostereoptic effect and the chromostereoptic reversal effect depends on the lightness of the background, classifies the layers of the 3D image input into the foreground, middle, and background layers according to the depth of the image input. It suits the characteristics of human visual perception because it controls the color factor that was adaptively detected through experiments on each layer; and it can achieve an enhanced cubic effect that is suitable for the characteristics of the image input.

Research on Vision-Based Web Interface Design

Through the in-depth analysis of the psychological and physiological characteristics of human visual perception, combined with the technical characteristics of web page itself, the characteristics of visual process, and the aesthetic theories, this paper analyzes the main components of web interface such as page layout, text, color, and animation in detail, and proposes the corresponding design principles, with which to improve the clarity, accuracy, and effectiveness of the visual information transmission in the web page.

51.3: An Ultra‐Low‐Cost 2‐D/3‐D Video‐Conversion System

AbstractIn this paper, we propose an ultra low cost 2D‐to‐3D conversion system which generates depth maps using the human visual perception characteristics of luminance and color. The proposed method has two major parts, edge feature‐based global scene depth gradient and texture‐based local depth refinement. The near‐to‐far global scene depth is generated by analyzing the edge complexity on each row. Then the local pixel value, Y, Cb, and Cr component of the video content is used to refine the detail depth value by enhance the texture lighting gradient on object surface. Combining the global depth gradient and local depth refinement, the depth map has a comfortable and vivid quality with very low computational complexity. The algorithm can be easily implemented by only line‐based architecture and integrated into the display chip.

Invisible Calibration Pattern based on Human Visual Perception

In this paper, we propose an arrangement and detection method of an invisible calibration pattern based on characteristics of human visual perception. A calibration pattern is arranged around contents where invisible data is embedded, as some feature points between an original image and the scanned image for normalization of the scanned image. However, it is clear that conventional methods interfere with page layout and artwork of contents. Moreover, conventional visible patterns show a third person the position of embedded data. Therefore, visible calibration patterns are not suitable for security service. The most important part of human visual perception in the proposed method is the spectral luminous efficiency characteristic and the chromatic spatial frequency characteristic. In addition, a back ground color in surrounding of contents is not restricted to uniform color by using the proposed calibration pattern. It is suggest that the proposed method protect page layout and artwork.