Sensitivity Of Human Visual System Research Articles

VPRF: Visual Perceptual Radiance Fields for Foveated Image Synthesis.

Neural radiance fields (NeRF) has achieved revolutionary breakthrough in the novel view synthesis task for complex 3D scenes. However, this new paradigm struggles to meet the requirements for real-time rendering and high perceptual quality in virtual reality. In this paper, we propose VPRF, a novel visual perceptual based radiance fields representation method, which for the first time integrates the visual acuity and contrast sensitivity models of human visual system (HVS) into the radiance field rendering framework. Initially, we encode both the appearance and visual sensitivity information of the scene into our radiance field representation. Then, we propose a visual perceptual sampling strategy, allocating computational resources according to the HVS sensitivity of different regions. Finally, we propose a sampling weight-constrained training scheme to ensure the effectiveness of our sampling strategy and improve the representation of the radiance field based on the scene content. Experimental results demonstrate that our method renders more efficiently, with higher PSNR and SSIM in the foveal and salient regions compared to the state-of-the-art FoV-NeRF. The results of the user study confirm that our rendering results exhibit high-fidelity visual perception.

Imperceptible adversarial attack via spectral sensitivity of human visual system

Imperceptible adversarial attack via spectral sensitivity of human visual system

A Visual Sensitivity Aware ABR Algorithm for DASH via Deep Reinforcement Learning

In order to cope with the fluctuation of network bandwidth and provide smooth video services, adaptive video streaming technology is proposed. In particular, the adaptive bitrate (ABR) algorithm is widely used in dynamic adaptive streaming over HTTP (DASH) to improve quality of experience (QoE). However, existing ABR algorithms still ignore the inherent visual sensitivity of human visual system (HVS). As the final receiver of video, HVS has different sensitivity to the quality distortion of different video content, and video content with high visual sensitivity needs to allocate more bitrate resources. Therefore, existing ABR algorithms still have limitations in reasonably allocating bitrate and maximizing QoE. To solve this problem, this paper designs an adaptive bitrate strategy from the perspective of user vision, studies the modeling of visual sensitivity, and proposes a visual sensitivity aware ABR algorithm. We extract a set of content features and attribute features from the video, and consider the simulation of HVS to establish a total masking effect model that reflects the visual sensitivity more accurately. Further, the network status, buffer occupancy, and visual sensitivity are comprehensively considered under a deep reinforcement learning framework to select the appropriate bitrate for maximizing QoE. We implement the proposed algorithm over a realistic trace-driven evaluation and compare its performance with several latest algorithms. Experimental results show that our algorithm can align ABR strategy with visual sensitivity to achieve better QoE in high visual sensitivity content, and improves the average perceptual video quality and overall user QoE by 18.3% and 22.8%, respectively. Additionally, we prove the feasibility of our algorithm through subjective evaluation in the real environment.

Just Noticeable Difference Model for Images with Color Sensitivity.

The just noticeable difference (JND) model reflects the visibility limitations of the human visual system (HVS), which plays an important role in perceptual image/video processing and is commonly applied to perceptual redundancy removal. However, existing JND models are usually constructed by treating the color components of three channels equally, and their estimation of the masking effect is inadequate. In this paper, we introduce visual saliency and color sensitivity modulation to improve the JND model. Firstly, we comprehensively combined contrast masking, pattern masking, and edge protection to estimate the masking effect. Then, the visual saliency of HVS was taken into account to adaptively modulate the masking effect. Finally, we built color sensitivity modulation according to the perceptual sensitivities of HVS, to adjust the sub-JND thresholds of Y, Cb, and Cr components. Thus, the color-sensitivity-based JND model (CSJND) was constructed. Extensive experiments and subjective tests were conducted to verify the effectiveness of the CSJND model. We found that consistency between the CSJND model and HVS was better than existing state-of-the-art JND models.

CPRM: Color perception and representation model for fabric image based on color sensitivity of human visual system

As an important attribute of fabric appearance and the first element that affects vision, color plays an important part in fabric and garment design, development, production and sales. Fabrics are becoming increasingly colorful which raises the difficulty of automatic recognition. With the development of computer vision technology, it has become a mainstream research hotspot to process and recognize fabric colors through novel computer vision techniques. In this paper, a color perception and representation model (CPRM) for fabric image based on color sensitivity of the human visual system is proposed to deal with the color changes caused by the human eye’s perception of fabric appearance. Inspired by the human visual biological mechanisms, we designed a CPRM by using the color sensitive function. In order to verify the effectiveness of the proposed CPRM, two types of computer simulation experiments are simulated, which are color shifts and color matching. The former verifies that our proposed model can effectively simulate the human visual system in color perception actions, which is consistent with the results verified by biological experimental data; the latter shows that the proposed model’s synergy of assimilation and contrast is closer to the biological model during the color perception stage. Finally, the CPRM is applied on three different fabric image data sets, and extensive experiments are simulated by the proposed CPRM. The CPRM could not only effectively characterize the color contrasts, but also accurately express the color changes caused by the fine adjustment of dyed fibers. The experimental results show the validity of the proposed CPRM, which can effectively perceive and represent fabric color. The proposed CPRM has important application prospects in improving color matching efficiency and intelligent production level in the textile industry.

Blind watermarking algorithm combiningNSCT,DWT,SVD, andHVS

AbstractIn order to solve the contradiction between watermark transparency and robustness, this paper combines Non‐subsampled Contourlet Transformation (NSCT), Discrete Wavelet Transformation (DWT), Singular Value Decomposition (SVD), and Human Visual System (HVS) to improve a watermarking algorithm. Firstly, this algorithm performs a first‐level NSCT decomposition on the carrier image to obtain its low‐frequency part, and then performs a second‐level DWT on the low‐frequency part, extracts its second‐level low‐frequency sub bands and block these sub bands to sub‐blocks according to their size. Secondly, SVD is performed for each sub‐block and the best embedding position of the watermark according to the sensitivity of HVS for the average brightness and the modified cosine similarity measurement method is calculated. Finally, the watermark scrambled and encrypted by Arnold is embedded into the selected singular value. This algorithm can realize the blind extraction of watermark. Experimental results show that the algorithm can not only satisfy the transparency but the robustness of the watermark at the same time as well.

Means of harmonization of composition and optimization of the design of photographic images

The subject of this research is such a means of harmonization of composition of the image in a plane as the contrast. The article assesses the possibility and factors of using contrast for increasing the attractiveness of the designed photographic image. The author reviews the examples of photo images with appropriate use of contrast; provides the research data on the perception[WU1]  of contrast by human visual system.; explores correlation between contrast and the conditions for a harmonious composition; as well as reviews the connection between the properties of compositional elements of photography and the components of visual perception, discovered by R. Barthes in his work “Camera Lucida”. The author carries out the analysis of photographic images; compares the empirical data on the study of perception of contrast by a human; applies the methods of induction and deduction in analyzing the acquitted results and comparing them with the characteristics of a harmonious composition. The author’s special contribution consists in comparison of the results of empirical research of perception of contrast by a human in the context of dependence of the sensitivity of human visual system on the size of observable alternating contrast lines and the size of the dominant in relation to the image field object of composition. It is proposed to use high contrasts of the elements of detail drawing for reaching harmonization of composition of the designed image elements.  [WU1]

Experimental Study for Revising Visual Noise Measurement of ISO 15739

Visual noise is a metric for measuring the amount of noise perception in images taking into account the properties of the human visual system (HVS). A visual noise measurement method is specified in Annex B of ISO 15739, which has been used as a useful measurement metric over the years since its introduction. As several issues have been questioned recently, it is now being investigated for revision involving changes to the CSF, color space used for rms noise value measurements, and visual noise formula combining rms values in three color channels. To derive visual noise formula involving color noise weighting coefficients representing HVS sensitivity to noise, subjective experiments using simulated luminance and color noise images were done. The improvement of calculation stability and HVS correspondence was verified using simulation and real camera images under various conditions. Finally, subjective experiments using real camera images were performed to validate the revised method and update the color noise coefficients considering practical measurement cases.

Local Dominant Directional Symmetrical Coding Patterns for Facial Expression Recognition.

To overcome the shortcomings of inaccurate textural direction representation and high-computational complexity of Local Binary Patterns (LBPs), we propose a novel feature descriptor named as Local Dominant Directional Symmetrical Coding Patterns (LDDSCPs). Inspired by the directional sensitivity of human visual system, we partition eight convolution masks into two symmetrical groups according to their directions and adopt these two groups to compute the convolution values of each pixel. Then, we encode the dominant direction information of facial expression texture by comparing each pixel's convolution values with the average strength of its belonging group and obtain LDDSCP-1 and LDDSCP-2 codes, respectively. At last, in view of the symmetry of two groups of direction masks, we stack these corresponding histograms of LDDSCP-1 and LDDSCP-2 codes into the ultimate LDDSCP feature vector which has effects on the more precise facial feature description and the lower computational complexity. Experimental results on the JAFFE and Cohn-Kanade databases demonstrate that the proposed LDDSCP feature descriptor compared with LBP, Gabor, and other traditional operators achieves superior performance in recognition rate and computational complexity. Furthermore, it is also no less inferior to some state-of-the-art local descriptors like as LDP, LDNP, es-LBP, and GDP.

ChaboNet : Design of a deep CNN for prediction of visual saliency in natural video

Prediction of visual saliency in images and video is needed for video understanding, search and retrieval, coding, watermarking and other applications. The majority of prediction models are founded only on “bottom-up” features. Nevertheless, the “top-down” component of human visual attention becomes prevalent as human observers explore the visual scene. Visual saliency which is always a mix of bottom-up and top-down cues can be predicted on the basis of seen data. In this paper, a model of prediction of visual saliency in video on the basis of Deep convolutional neural networks (CNNs) is proposed. A Deep CNN architecture is designed. Various input channels for a CNN architecture are studied: using the known sensitivity of human visual system to residual motion, pixel colour values are completed with residual motion map. The latter is a normalized energy of residual motion in video frames with regard to the estimated global affine motion model. The experiments show that the choice of the input features for the Deep CNN depends on visual task: for highly dynamic content, the proposed model with residual motion is more efficient and gives decent results with relatively shallow Deep architecture.

A New Robust Reversible Blind Watermarking in Wavelet-Domain for Color Images

Reversible watermarking methods are used for copyright protection and are able to recover the host image without distortion. Robust reversible watermarking technique should resist against intentional and unintentional image processing attacks. Robust reversible watermarking techniques should have three features namely imperceptibility, reversibility and robustness. In this paper, it is proposed to develop a new robust reversible blind watermarking for color images based on histogram construction of the wavelet coefficients constructed from the cover image. In the proposed approach, the red component of a host color image is decomposed into wavelet coefficients. Motivated by the excellent spatio-frequency localization properties of wavelets, this technique is proposed in the wavelet domain. The pixels are adjusted before watermark embedding such that both overflow and underflow of pixels during embedding is avoided and image is recovered without distortion. Based on histogram construction and the local sensitivity of Human Visual System (HVS) in wavelet domain, the watermark is embedded. For watermark extraction without host image, k-means clustering algorithm is proposed. The experimental results show that the proposed technique has good performance in terms of reversibility and robustness with the high quality of the watermarked image. The PSNR value of the recovered image is around 48[Formula: see text]dB which proves that the quality of the recovered image is not degraded.

Mechanisms of orientation sensitivity of human visual system: Part II. Neural patterns of early processing of information about line orientation

The high density EEG was registered in 41 healthy subjects (20 males, 21 females) in the cardinal (horizontal and vertical) and oblique (45 and 135 deg) line orientation identification task. The analysis of the adaptive amplitude maximum (4 ms averaging) of P1 and N1 evoked potentials in the symmetrical occipital, parietal and inferior temporal areas and dipole source modelling showed the anisotropy of cortical responses in the 80-150 ms interval. The amplitude is higher on the oblique orientations as comparison with cardinal ones. The temporal and regional features of cortical answers were discovered. The earlier selective response (~90 ms latency) is registered.in the parietal areas, while the later (~145 ms latency) is found in the occipital ones. We discovered a number of sex-related differences in the early stages of line orientation detection. In males, the amplitude of components is higher; they have broader area of localisation of their dipole sources: in addition to the occipital and parietal regions, cortex of the temporal regions is involved. Theobtained data are discussed in the context of the idea of effective neural coding (Barlow, 1959) and the features of spatial information processing in the visual system of males and females.

Contrast Sensitivity 함수를 이용한 영상화질 개선 방법

We develop the signal processing method for adaptive implementing direction of signal and the frequency sensitivity of human visual system(HVS). Existing multibnad energy scaling method makes ringing artifact because it does not consider signal direction. To solve this problem, we use block gradient for signal direction in addition to existing method. And we use the fact that frequency component of signal is more sensitive than value of signal over human eyes. we enhance the signal according to contrast sensitivity function(CSF) which is the model of frequency sensitivity of human eye. Compared that the existing analysis models only improve the efficiencies in the existing systems, the developed method can process the image signals to be more desirable and suitable to HVS.

An efficient human visual system based quality metric for 3D video

Stereoscopic video technologies have been introduced to the consumer market in the past few years. A key factor in designing a 3D system is to understand how different visual cues and distortions affect the perceptual quality of stereoscopic video. The ultimate way to assess 3D video quality is through subjective tests. However, subjective evaluation is time consuming, expensive, and in some cases not possible. The other solution is developing objective quality metrics, which attempt to model the Human Visual System (HVS) in order to assess perceptual quality. Although several 2D quality metrics have been proposed for still images and videos, in the case of 3D efforts are only at the initial stages. In this paper, we propose a new full-reference quality metric for 3D content. Our method mimics HVS by fusing information of both the left and right views to construct the cyclopean view, as well as taking to account the sensitivity of HVS to contrast and the disparity of the views. In addition, a temporal pooling strategy is utilized to address the effect of temporal variations of the quality in the video. Performance evaluations showed that our 3D quality metric quantifies quality degradation caused by several representative types of distortions very accurately, with Pearson correlation coefficient of 90.8 %, a competitive performance compared to the state-of-the-art 3D quality metrics.

DCT Based Fuzzy Image Watermarking

Image watermarking implants an ownership parameter in a digital image or video. This paper works on authentication of the image file. Robust image watermarking technique is used to model the Human Visual System (HVS) using Fuzzy Logic. The proposed Fuzzy Logic System is trained using inference rules considering three sensitivities of HVS as Brightness, Contrast and Edge Sensitivity. The Fuzzy network uses the image captured in realtime and computes block wise for producing a single output weighting factor used to embed unique identification numbers generated from the confidential data as watermark, which is for authorization and verification of the original image. The robustness of the watermark embedded is checked by Stirmark image processing attacks. Recovered watermark's computed value using SIM (X, r) parameter for the image verified it as good watermark recovery process.

New just-Noticeable-Distortion Model Based on the Depth Information and its Application in Multiview Video Coding

To reach higher compression ratios without perceptually degrading the reconstructed signal, the properties of the human visual system (HVS) need to be better exploited. Research simulated the sensitivity of HVS with the just-noticeable-distortion (JND) model, which describes the perception redundancy quantitatively. This paper proposed a new JND model in multiview video coding (MVC) which explored the property of HVS to stereoscopic masking effect. The proposed model contained the spatial JND, temporal JND and the JND in depth. A novel asymmetric stereoscopic video coding method was proposed based on the new JND model. For each macroblock (MB), the quantization parameters (QPs) were optimized adaptively based on the new JND. Compared with the original JMVC scheme without JND, our method gets remain perceptual quality and save bits averagely 24.18%~36.24%.

A 3-Level Secure Histogram Based Image Steganography Technique

Steganography is an art that involves communication of secret data in an appropriate carrier, eg. images, audio, video, etc. with a goal to hide the very existence of embedded data so as not to arouse an eavesdropper's suspicion. In this paper, a steganographic technique with high level of security and having a data hiding capacity close to 20% of cover image data has been developed. An adaptive and matched bit replacement method is used based on the sensitivity of Human Visual System (HVS) at different intensities. The proposed algorithm ensures that the generated stego image has a PSNR greater than 38.5 and is also resistant to visual attack. A three level security is infused into the algorithm which makes data retrieval from the stego image possible only in case of having all the right keys.

New visual perceptual pooling strategy for image quality assessment

Most of Image Quality Assessment (IQA) metrics consist of two processes. In the first process, quality map of image is measured locally. In the second process, the last quality score is converted from the quality map by using the pooling strategy. The first process had been made effective and significant progresses, while the second process was always done in simple ways. In the second process of the pooling strategy, the optimal perceptual pooling weights should be determined and computed according to Human Visual System (HVS). Thus, a reliable spatial pooling mathematical model based on HVS is an important issue worthy of study. In this paper, a new Visual Perceptual Pooling Strategy (VPPS) for IQA is presented based on contrast sensitivity and luminance sensitivity of HVS. Experimental results with the LIVE database show that the visual perceptual weights, obtained by the proposed pooling strategy, can effectively and significantly improve the performances of the IQA metrics with Mean Structural SIMilarity (MSSIM) or Phase Quantization Code (PQC). It is confirmed that the proposed VPPS demonstrates promising results for improving the performances of existing IQA metrics.

Fourier Transform-Based Scalable Image Quality Measure

We present a new image quality assessment (IQA) algorithm based on the phase and magnitude of the 2D (twodimensional) Discrete Fourier Transform (DFT). The basic idea is to compare the phase and magnitude of the reference and distorted images to compute the quality score. However, it is well known that the Human Visual Systems (HVSs) sensitivity to different frequency components is not the same. We accommodate this fact via a simple yet effective strategy of nonuniform binning of the frequency components. This process also leads to reduced space representation of the image thereby enabling the reduced-reference (RR) prospects of the proposed scheme. We employ linear regression to integrate the effects of the changes in phase and magnitude. In this way, the required weights are determined via proper training and hence more convincing and effective. Lastly, using the fact that phase usually conveys more information than magnitude, we use only the phase for RR quality assessment. This provides the crucial advantage of further reduction in the required amount of reference image information. The proposed method is therefore further scalable for RR scenarios. We report extensive experimental results using a total of 9 publicly available databases: 7 image (with a total of 3832 distorted images with diverse distortions) and 2 video databases (totally 228 distorted videos). These show that the proposed method is overall better than several of the existing fullreference (FR) algorithms and two RR algorithms. Additionally, there is a graceful degradation in prediction performance as the amount of reference image information is reduced thereby confirming its scalability prospects. To enable comparisons and future study, a Matlab implementation of the proposed algorithm is available at http://www.ntu.edu.sg/home/wslin/reduced_phase.rar.

On spatiochromatic visual sensitivity and peripheral color LOD management

Empirical findings from a gaze-contingent color degradation study report the effects of artificial reduction of the human visual system's sensitivity to peripheral chromaticity on visual search performance. To our knowledge, this is the first such investigation of peripheral color reduction. For unimpeded performance, results suggest that, unlike spatiotemporal content, peripheral chromaticity cannot be reduced within the central 20° visual angle. Somewhat analogous to dark adaptation, reduction of peripheral color tends to simulate scotopic viewing conditions. This holds significant implications for chromatic Level Of Detail management. Specifically, while peripheral spatiotemporal detail can be attenuated without affecting visual search, often dramatically (e.g., spatial detail can be so reduced up to 50% at about 5°), peripheral chromatic reduction is likely to be noticed much sooner. Therefore, color LOD reduction (e.g., via compression), should be maintained isotropically across the central 20° visual field.