Human Visual Model Research Articles

Robust, sensor-independent target detection and recognition based on computational models of human vision

Most current artificial vision systems lack robustness and are applicable only to a narrow range of tasks. Crevier (1997) has suggested that this is due to their reliance on a small number of vision mechanisms and to lack of knowledge about how vision algorithms should be integrated. We suggest a systems approach to artificial vision based on computational vision research. The capabilities of biological vision systems are contrasted with those of current piecemeal approaches to artificial vision. A mature, comprehensive vision system, called the Georgia Tech Vision (GTV) simulation is described. GTV incorporates quasilinear filter mechanisms to simulate the processing performed by simple and complex cortical cells. The outputs of these mechanisms are adaptively combined to discriminate targets from clutter and/or one another. GTV outputs predictions of human search and detection performance and/or targeting metrics for automatic target recognition (ATR) applications. Studies validating GTV as a model of human search and detection performance and demonstrating its performance as an ATR are presented.

Watermarking algorithm based on a human visual model

This paper presents an additive watermarking technique for grey-scale pictures. It consists in secretly embedding copyright information (a binary code) into the picture without degrading its quality. Those bits are encoded through the phase of maximal length sequences (MLS). MLS are binary sequences with good correlation properties. The result of the autocorrelation is much greater than crosscorrelations, i.e. correlations made with shifted versions of this sequence. The embedded bits are retrieved from the result of the correlations. The core of the embedding process is underlaid by a masking criterion that guarantees the invisibility of the watermark. It is combined with an edge and texture discrimination to determine the embedding level of the MLS, whose bits are actually spread over 32×8 pixel blocks. Eventually, some results are presented, which analyze the efficiency of the retrieval as well as the resistance of the watermark to compression and its robustness against malevolent manipulation.

Theoretical analysis of perceptual responses to flashed sinusoidal waves using a multichannel spatiotemporal human vision model

A human vision model with a multiple-channel structure that can quantitatively simulate spatiotemporal perceptual responses to flickering and drifting images as well as still images has already been proposed. This model was mathematically formulated as an adaptive focus adjustable feedback system based on two essential properties of the human vision system: the interdependent relation between a retinal image's blur and a visual field size, and the spatiotemporal frequency characteristics for human vision's X- and Y-channels, originating from the X- and Y-retinal ganglion cells. However, since the channels' temporal frequency characteristics embedded in the model were only gain characteristics without phase characteristics, which are difficult to measure experimentally, the use of the model was restricted to applications to temporally and periodically changing images, such as flickering or drifting images. This paper discusses ways to improve the model to a more advanced vision model applicable to images that have all types of temporal changes: two kinds of temporal impulse responses of human vision's X- and Y-channels are theoretically derived from the embedded gain characteristics and the phase characteristics estimated by using the Hilbert transform, and are incorporated into the previous model. As an appropriate application, this improved model is used to reproduce human vision's perceptual response characteristics (the relation between exposure duration and threshold contrast, and the spatial frequency characteristics of contrast sensitivity) to flashed sinusoidal waves, a typical kind of image without temporal periods. The results show that the theoretical perceptual response characteristics derived from the improved model agree well with the experimentally measured characteristics. This suggests that this improved model is also applicable to images with various kinds of temporal changes. © 1998 Scripta Technica. Electron Comm Jpn Pt 3, 81(4): 51–62, 1998

Color Theory and Color Imaging Systems: Past, Present and Future

James Clerk Maxwell demonstrated the first color photograph in a lecture to the Royal Society of Great Britain in 1861. He used the demonstration to illustrate Thomas Young's idea that human vision uses three kinds of light sensors. This demonstration led to a great variety of color photographic systems using both additive and subtractive color. Today, we have image-capture devices that are photographic, video, still, and scanning. We have hardcopy printers that are electrophotographic, ink jet, thermal and holographic, as well as displays that use cathode ray tubes, liquid-crystal and other light emission color devices. The major effort today is to get control of all these technologies so that the user can, without effort, move a color digital image from one technology to another without changing the appearance of the image. The strategy of choice is to use colorimetry to calibrate each device. If all prints and displays sent the same colorimetric values from every pixel, then the images, regardless of the display, would appear identical. The problem with matching prints and displays is that they have very different color gamuts. A more satisfactory solution is needed. In my view, the future emphasis of color research will be in models of human vision. The purpose of these models will shift from calculating color matches to calculating color sensations. All the technologies listed above work one pixel at a time. The response at every pixel is dependent on the input at that pixel, regardless of whether the imaging system is chemical, photonic, or electrical. Humans are different. The color they see at a pixel is controlled by that pixel and all the other pixels in the field of view. Human color vision uses a spatial calculation involving the whole image. Except for human vision, all other color systems have the same output from a single input. In other words, if an input pixel has a value of 128, and the image processing changes that value to 155, then all pixels with 128 in will have 155 out. Human vision is unique among color imaging systems because a single input value (128) will generate a range of output values (0, or 55, or 128, or 255), depending on the values of other pixels in the image. Despite the remarkable progress in our ability to control the placement of dyes and pigments on paper, we must now return to the study of Maxwell's interest—color theory—for the next advancements in color systems. In the future, we will see more models that compute the color appearance from spatial information and write color sensations on media, rather than attempting to write the quanta catch of visual receptors.

Chromatic effect on apparent brightness in interior spaces III: Chromatic brightness model

In its simplest form the opponent colour model of human colour vision includes an achromatic channel having V(λ) sensitivity, and two opponent colour channels. Perceived brightness is considered to be a sum of the total activity in these three channels. From fitting experimental data it is suggested that at suprathreshold levels the achromatic channel is suppressed, and that brightness can be modelled adequately by the two chromatic channels alone. This model is termed Chromatic Brightness.

人間の視覚・聴覚情報 視覚系の数理モデルを用いた画像表示システム性能評価尺度とその基本的性質

Three new objective performance evaluation criteria, C/Vn (Carrier to Visual noise ratio), N/Vn (Noise to Visual noise ratio), and C/N (Carrier to Noise ratio), for imaging systems are proposed. These criteria are quantified based on visual noise characteristics derived from a human vision model formulated by the author in an output image from imaging systems, to which a sine-wave is inputted, C/Vn and N/Vn are defined, respectively, as the quantity of the perceptible sine-wave component with the frequency equal to the input sine-wave and as the quantity of all the perceptible noise components, and C/N is a simplified expression for integrating C/Vn and N/Vn. These criteria are applied to the performance evaluation of imaging systems based on the density-gradation-method such as CRTs, LCDs, printers and so on. They are calculated as parameters of pixel aperture ratio, quantizing level number, and pixel size. In actual computation, pulse-amplitude-modulated sinusoidal waves are used as output images from such imaging systems. The calculated fundamental characteristics suggest that the objective performance evaluation criteria are effective in quantitatively expressing human subjective evaluation judgments.

Lightness determination at curved surfaces with applications to dynamic range compression and model-based coding of facial images

Lightness algorithms, which have been proposed as a model for human vision, are aimed at recovering surface reflectance in a close approximation. They attempt to separate reflectance data from illumination data by thresholding a spatial derivative of the image intensity. This, however, works only reliable in a world of plane Mondrians. An extension of the classical lightness approach of Land and McCann (1971) to curved surfaces is presented. Assuming smooth surfaces with Lambertian reflection properties and leaving aside occlusions and cast shadows, the separation of those components of the intensity gradient due to reflectance from those due to irradiance is posed as a constraint minimization problem. To do so, two classification operators were introduced which identify potential reflectance and irradiance data using a scale-space filtering approach. Two exemplary applications of the proposed extended lightness algorithm in the field of visual telecommunications are presented: (i) the simulation of more uniformly illuminated videophone portrait scenes to give dynamic range compressed images with a most realistic appearance and (ii) the synthesis of videophone portrait images from model-based coded data with a correct illumination effect. In both applications, the extended lightness algorithm is employed for estimating the reflectance functions at facial surfaces. Results obtained by applying the extended lightness algorithm are compared with results obtained by conventional methods known from literature.

Visualizing voltage profiles for large scale power systems

To be effective for large-scale power system simulation, the bus voltage representation has several issues to be resolved. The selected method must be able to handle a large amount of information effectively. For that purpose, each element must be simple and small, easy to identify and to discern in any environment. This is the principal objective of the authors' research, and of this article. They detail how their representations use a scale of carefully chosen colors based on characteristics of human visual perception and mental models of color usage in the power industry.

Properties and performance of a center/surround retinex

The last version of Land's (1986) retinex model for human vision's lightness and color constancy has been implemented and tested in image processing experiments. Previous research has established the mathematical foundations of Land's retinex but has not subjected his lightness theory to extensive image processing experiments. We have sought to define a practical implementation of the retinex without particular concern for its validity as a model for human lightness and color perception. We describe the trade-off between rendition and dynamic range compression that is governed by the surround space constant. Further, unlike previous results, we find that the placement of the logarithmic function is important and produces best results when placed after the surround formation. Also unlike previous results, we find the best rendition for a "canonical" gain/offset applied after the retinex operation. Various functional forms for the retinex surround are evaluated, and a Gaussian form is found to perform better than the inverse square suggested by Land. Images that violate the gray world assumptions (implicit to this retinex) are investigated to provide insight into cases where this retinex fails to produce a good rendition.

Theoretical reproduction of perceptual responses to drifting image using a spatio‐temporal human vision model with a multiple‐channel structure

A human vision model with a multiple-channel structure that can quantitatively simulate spatio-temporal perceptual responses to flickering sinusoidal waves as well as still images has already been proposed. This model was mathematically formulated as an adaptive focus adjustable feedback system based on two essential properties of the human vision system: the interdependent relationship between a retinal image's blur and a visual field size; and the spatio-temporal response characteristics of retinal ganglion cells. However, as the latter characteristics were actually formulated by utilizing spatio-temporal frequency characteristics experimentally measured using flickering sinusoidal waves, it is very important to confirm the model's applicability to images whose temporal change characteristics differ from flickering images. This paper discusses the model's applicability to drifting sinusoidal waves, which have a typical kind of temporal change characteristic except for flickering images. Consequently, it was clarified that the model's spatial frequency characteristics of contrast sensitivity and those of optimum drifting speed are in good agreement with experimental data. These results suggest that this model is also applicable to images with various kinds of temporal change characteristics other than flickering patterns. This provides the model's validity or generality. Moreover, the model was applied to the theoretical prediction of perceptual response characteristics of drifting square waves. How drifting square waves were perceived is also discussed. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(2): 35–45, 1997

Theoretical reproduction of perceptual responses to drifting image using a spatio-temporal human vision model with a multiple-channel structure

A human vision model with a multiple-channel structure that can quantitatively simulate spatio-temporal perceptual responses to flickering sinusoidal waves as well as still images has already been proposed. This model was mathematically formulated as an adaptive focus adjustable feedback system based on two essential properties of the human vision system: the interdependent relationship between a retinal image's blur and a visual field size; and the spatio-temporal response characteristics of retinal ganglion cells. However, as the latter characteristics were actually formulated by utilizing spatio-temporal frequency characteristics experimentally measured using flickering sinusoidal waves, it is very important to confirm the model's applicability to images whose temporal change characteristics differ from flickering images. This paper discusses the model's applicability to drifting sinusoidal waves, which have a typical kind of temporal change characteristic except for flickering images. Consequently, it was clarified that the model's spatial frequency characteristics of contrast sensitivity and those of optimum drifting speed are in good agreement with experimental data. These results suggest that this model is also applicable to images with various kinds of temporal change characteristics other than flickering patterns. This provides the model's validity or generality. Moreover, the model was applied to the theoretical prediction of perceptual response characteristics of drifting square waves. How drifting square waves were perceived is also discussed. © 1997 Scripta Technica, Inc. Electron Comm Jpn Pt 3, 80(2): 35–45, 1997

High-level vision: object recognition and visual cognition

Object recognition: shape-based recognition what is recognition? why object recognition is difficult. Approaches to object recognition: invariant properties and feature spaces parts and structural descriptions the alignment approach which is the correct approach?. The alignment of pictorial descriptions: using corresponding features the use of multiple models for 3-D objects aligning pictorial descriptions transforming the image or the models? before and after alignment. The alignment of smooth bounding contours: the curvate method accuracy of the curvature method empirical testing. Recognition by the combination of views: modelling objects by view combinations objects with sharp edges using two views only using a single view the use of depth values summary of the basic scheme objects with smooth boundaries recognition by image combinations extensions to the view-combination scheme psychophysical and physiological evidence interim conclusions: recognition by multiple views. Classifications: classification and identification the role of object classification class-based processing using class prototypes pictorial classification evidence from psychology and biology are classes in the world or in our head? the organization of recognition memory. Image and model correspondence: feature correspondence contour matching correspondence-less methods correspondence processes in human vision model construction compensating for illumination changes. Segmentation and saliency: is segmentation feasible? bottom-up and top-down segmentation extracting globally salient structures saliency, selection, and completion what can bottom-up segmentation achieve? Visual cognition and visual routines: perceiving inside and outside spatial analysis by visual routines conclusions and open problems the elemental operations the assembly and storage of routines routines and recognition. Sequence seeking and counter streams - a model for visual cortex: the sequence-seeking scheme biological embodiment summary. Appendices: alignment by feature the curvature method errors of the curvature method locally affine matching definitions.

Edge-based binocular stereopsis algorithm—a matching mechanism with probabilistic feedback

A novel algorithm for edge-based stereopsis which has a probabilistic feedback mechanism is introduced. Although the “second algorithm” of Marr and Poggio is one of the most effective edge-based stereopsis algorithms, it contains an inter-resolutional correspondence problem. To solve this problem, we developed an algorithm based on the probabilistic feedback from the “reconstruction process” to the “matching process”. Its architecture is reasonable as a model of human stereo vision, based upon anatomical and physiological fact. We also explain the computational theory of binocular stereopsis and its implementation based on the “second algorithm” in single resolution, show the advantage of the probabilistic feedback and the double eye-dominant mechanisms, and also discuss its adequacy as a model of human stereo vision.

視覚モデルを用いた画質評価プロセスの定量的再現 : 眼を近付けた場合の画質など(エレクトログラフィ特集〜高画質化技術の動向〜)

視覚モデルを用いた画質評価プロセスの定量的再現 : 眼を近付けた場合の画質など(エレクトログラフィ特集〜高画質化技術の動向〜)

Human visual model and wavelets for multi-resolution image coding

In this paper we present the results of a research activity which addresses the problem of high-quality image compression. The approach consists of embedding a model of the human visual system into the framework of the biorthogonal wavelet expansion. After a brief overview on the application of wavelets on image compression and a comparison with other transform-based approaches, we describe the mathematical aspect of the algorithm and the coding method used in the compression scheme. Some examples of the experimental results we have obtained end the paper.

Method for Shape Recovery from Monocular Image Considering Shading.

A human vision model is formulated in this paper, considering that line drawings are not limited to projection only from a polyhedron. To select a line drawing which can serve as an elementary picture for human stereo vision, we performed a psychological experiment. The model was proposed based on the experimental data to describe a property of human vision. Several examples were solved to validate the proposed model. The image-irradiate equation given by the Poisson equation is solved to reconstruct a face. Numerical simulation demonstrates that faces can be recovered precisely by considering the shading on the picture. This successful procedure involves solving the Dirichlet problem of the Poisson equation, specifying the boundary condition which is produced by the reconstructed edges. Numerical examples involve a drawing comprised only of lines, and a line drawing with shading.

An integrated computational model of three-dimensional vision.

This article presents the details of and background for a computational model of three-dimensional vision. The basic idea embodied in this model is that a veridical approximation to a three-dimensional scene can best be produced by combining several operators that act on acquired two-dimensional images to reconstruct surface shape and distance. Stereo, shape from shading (SFS), and shape from structured light (SFSL) operators are combined to produce a reconstruction that is superior to any that might be produced by one alone. The advantages and disadvantages of each independent operator and the generic difficulties faced by members of this class of operators are discussed. Collectively, this package of combined algorithms represents a functional model of human spatial vision.

Multistage integration model for human egomotion perception

Human computational vision models that attempt to account for the dynamic perception of egomotion and relative depth typically assume a common three-stage process: first, compute the optical flow field based on the dynamically changing image; second, estimate the egomotion states based on the flow; and third, estimate the relative depth/shape based on the egomotion states and possibly on a model of the viewed surface. We propose a model more in line with recent work in human vision, employing multistage integration. Here the dynamic image is first processed to generate spatial and temporal image gradients that drive a mutually interconnected state estimator and depth/shape estimator. The state estimator uses the image gradient information in combination with a depth/shape estimate of the viewed surface and an assumed model of the viewer's dynamics to generate current state estimates; in tandem, the depth/shape estimator uses the image gradient information in combination with the viewer's state estimate and assumed shape model to generate current depth/shape estimates. In this paper, we describe the model and compare model predictions with empirical data.

Modelling the increase of contrast sensitivity with grating area and exposure time

We extended the contrast detection model of human vision to temporal integration by taking into account the effect of exposure duration on contrast sensitivity for stationary gratings. The extended model thus comprised: (i) low-pass filtering due to the optical modulation transfer function of the eye; (ii) high-pass filtering (lateral inhibition) due to the neural modulation transfer function of the visual pathways; (iii) addition of internal neural noise; and (iv) detection by a local matched filter whose efficiency for gratings decreased with increasing area and exposure duration. To test the model we measured binocular contrast sensitivity in foveal photopic vision as a function of exposure duration and area for sinusoidal gratings with equiluminous surround at spatial frequencies of 0.25–16 c/deg. In agreement with the model, contrast sensitivity at all grating areas first increased in proportion to √ t when exposure duration ( t) was shorter than critical duration. Thereafter the increase saturated and contrast sensitivity became independent of exposure time. Critical exposure duration was found to be independent of grating area but increased with spatial frequency. Similarly, at all exposure durations contrast sensitivity first increased in proportion to √ A when grating area ( A) was smaller than critical area. Thereafter the increase saturated and contrast sensitivity became independent of area. Critical area was found to be independent of exposure duration but decreased with increasing spatial frequency. The extended model explained 95–97% of the total variance of our contrast sensitivity data at the spatial frequencies studied. Our results also mean that spatial and temporal integration processes are mutually independent and thus area and time are separable variables in the detection of stationary gratings.

Neural modulation transfer function of the human visual system at various eccentricities

We measured r.m.s. contrast sensitivity as a function of retinal illuminance at various spatial frequencies within 3–37 deg of eccentricity in the nasal visual field. In dim light contrast sensitivity increased in proportion to the square root of retinal illuminance obeying the De Vries-Rose law but in bright light contrast sensitivity was independent of luminance following Weber's law. Critical retinal illuminance ( I c ) marking the transition between the laws was found to be independent of grating area but proportional to the spatial frequency squared at all eccentricities, in agreement with the Van Nes-Bouman law of foveal vision. In addition, the proportionality constant was found to be independent of eccentricity and similar to that of the fovea. According to our contrast detection model of human vision the modulation transfer function ( P MTF ) of the neural visual pathways squared is directly proportional to the critical retinal illuminance. On this basis our result means that P MTF is similar, i.e. equal to spatial frequency across the visual field, thus attenuating low spatial frequencies relatively more than high spatial frequencies. Hence, up to the spatial cut-off frequency determined by the lowest neural sampling density of each retinal location the neural modulation transfer function is independent of visual location.