Gestalt Laws Research Articles

Object recognition using a generalized robust invariant feature and Gestalt's law of proximity and similarity

In this paper, we propose a new context-based method for object recognition. We first introduce a neuro-physiologically motivated visual part detector. We found that the optimal form of the visual part detector is a combination of a radial symmetry detector and a corner-like structure detector. A general context descriptor, named G-RIF (generalized-robust invariant feature), is then proposed, which encodes edge orientation, edge density and hue information in a unified form. Finally, a context-based voting scheme is proposed. This proposed method is inspired by the function of the human visual system, called figure-ground discrimination. We use the proximity and similarity between features to support each other. The contextual feature descriptor and contextual voting method, which use contextual information, enhance the recognition performance enormously in severely cluttered environments.

Automatic Object Extraction Over Multiscale Edge Field for Multimedia Retrieval

In this work, we focus on automatic extraction of object boundaries from Canny edge field for the purpose of content-based indexing and retrieval over image and video databases. A multiscale approach is adopted where each successive scale provides further simplification of the image by removing more details, such as texture and noise, while keeping major edges. At each stage of the simplification, edges are extracted from the image and gathered in a scale-map, over which a perceptual subsegment analysis is performed in order to extract true object boundaries. The analysis is mainly motivated by Gestalt laws and our experimental results suggest a promising performance for main objects extraction, even for images with crowded textural edges and objects with color, texture, and illumination variations. Finally, integrating the whole process as feature extraction module into MUVIS framework allows us to test the mutual performance of the proposed object extraction method and subsequent shape description in the context of multimedia indexing and retrieval. A promising retrieval performance is achieved, and especially in some particular examples, the experimental results show that the proposed method presents such a retrieval performance that cannot be achieved by using other features such as color or texture.

Gestalt Perception and Local-Global Processing in High-Functioning Autism

This study examined gestalt perception in high-functioning autism (HFA) and its relation to tasks indicative of local visual processing. Data on of gestalt perception, visual illusions (VI), hierarchical letters (HL), Block Design (BD) and the Embedded Figures Test (EFT) were collected in adult males with HFA, schizophrenia, depression and normative controls. Individuals with HFA processed gestalt stimuli less in accord with gestalt laws, particularly regarding the principle of similarity. Gestalt processing correlated positively with global processing of the HL. EFT and BD performance correlated negatively with VI susceptibility in HFA. All clinical groups succumbed less to VI than the normative sample. Results suggest decreased gestalt perception in HFA, being associated with a more general local visual processing bias.

Enhancement of Perceptually Salient Contours using a Parallel Artificial Cortical Network

In this paper we present a parallel artificial cortical network inspired by the Human visual system, which enhances the salient contours of an image. The network consists of independent processing elements, which are organized into hypercolumns. They process concurrently the distinct orientations of all the edges of the image. These processing elements are a new set of orientation kernels appropriate for the discrete lattice of the hypercolumns. The Gestalt laws of proximity and continuity that describe the process of saliency extraction in the human brain are encoded by means of weights. These weights interconnect the kernels according to a novel connection pattern based on co-exponentiality. The output of every kernel is modulated by the outputs of its neighboring kernels, according to a new affinity function. This function takes into account the degree of difference between the facilitation of the two lobes of the kernel. Saliency enhancement results as a consequence of the local interactions between the kernels. The network was tested on real and synthetic images and displays promising results for both. Comparisons with other methods with the same scope, demonstrate that the proposed method performs adequately. Furthermore it exhibits O(N) complexity with execution times that have never been reported by any other method so far, even though it is executed on a conventional PC.

Lighting, backlighting and watercolor illusions and the laws of figurality

We report some novel 'lighting' and 'backlighting' effects in plane figures similar to those which induce the 'watercolor illusion', that is, figures made with outlines composed of juxtaposed parallel lines varying in brightness and chromatic color. These new effects show 'illumination' as an emergent percept, and show how arrangements of 'dark and light' along the boundaries of various plane figures model the volume and strengthen the illusion of depth. To account for these various effects we propose several phenomenological 'laws of figurality' to add to the Gestalt laws of organization and figure-ground segregation. We offer a set of meta-laws which are speculative but which serve to integrate and organize the phenomenological laws. These laws indicate how luminance gradient profiles across boundary contours define both the 3D appearance of figures and the properties of the light reflected from their volumetric shapes.

The influence of perceptual grouping on motion detection

In this paper, we present a perceptual organization-based method for detecting moving objects from image sequences. To achieve the characteristics of real-time, efficiency, and robustness, a perceptual computation model of edge partitioning and grouping was proposed for the extraction of edge traces on the fly. Each edge trace is made up of generic edge tokens (GETs) which are perceptual features, and defined qualitatively based on the principles of Gestalt laws. Motion detection uses two basic computations: (1) segment motion GETs (MGETs) by computing the gradient differences between GET streams in consecutive frames; and (2) detect motion objects by perceptually grouping MGETs into object clusters. The MGETs in each cluster are constrained by the proximity of the features, and the motion continuation of the cluster measured by motion persistence, etc. Experimental results are provided.

Attentional modulation of perceptual grouping in human visual cortex: Functional MRI studies

When presented with a complex visual scene, our visual system has to organize the discrete entities present into useful perceptual units. The current work investigated the neural substrates of perceptual grouping defined by Gestalt laws of proximity and similarity of shape, and whether the neural substrates underlying perceptual grouping are modulated by task relevance and spatial attention. In visual discrimination tasks, subjects identified the orientations of perceptual groups formed by proximity or similarity of local elements or alternatively identified colors of either dots around the grouped stimuli or the fixation cross. Using functional magnetic resonance imaging (fMRI), we identified that the calcarine cortex was involved in proximity grouping but not in the grouping process defined by similarity of shape. Moreover, we showed evidence that the neural correlates of proximity grouping in the calcarine cortex were weakened when the elements were of low task relevance and fell outside an attended area of field. The findings reveal the neural basis for basic grouping operations, as well as illustrating how attention and proximity grouping interact in human visual cortex.

Salient closed boundary extraction with ratio contour

We present ratio contour, a novel graph-based method for extracting salient closed boundaries from noisy images. This method operates on a set of boundary fragments that are produced by edge detection. Boundary extraction identifies a subset of these fragments and connects them sequentially to form a closed boundary with the largest saliency. We encode the Gestalt laws of proximity and continuity in a novel boundary-saliency measure based on the relative gap length and average curvature when connecting fragments to form a closed boundary. This new measure attempts to remove a possible bias toward short boundaries. We present a polynomial-time algorithm for finding the most-salient closed boundary. We also present supplementary preprocessing steps that facilitate the application of ratio contour to real images. We compare ratio contour to two closely related methods for extracting closed boundaries: Elder and Zucker's method based on the shortest-path algorithm and Williams and Thornber's method based on spectral analysis and a strongly-connected-components algorithm. This comparison involves both theoretic analysis and experimental evaluation on both synthesized data and real images.

Neural mechanisms of human texture processing: Texture boundary detection and visual search

Texture of various appearances, geometric distortions, spatial frequency content and densities is utilized by the human visual system to segregate items from background and to enable recognition of complex geometric forms. For automatic, or pre-attentive, segmentation of a visual scene, sophisticated analysis and comparison of surface properties over wide areas of the visual field are required. We investigated the neural substrate underlying human texture processing, particularly the computational mechanisms of texture boundary detection. We present a neural network model which uses as building blocks model cortical areas that are bi-directionally linked to implement cycles of feedforward and feedback interaction for signal detection, hypothesis generation and testing within the infero-temporal pathway of form processing. In the spirit of Jake Beck's early investigations our model particularly builds upon two key hypotheses, namely that (i) texture segregation is based on boundary detection, rather than clustering homogeneous items, and (ii) texture boundaries are detected mainly on the basis of larger scenic contexts mediated by higher cortical areas, such as area V4. The latter constraint provides a basis for element grouping in accordance to the Gestalt laws of similarity and good continuation. It is shown through simulations that the model integrates a variety of psychophysical findings on texture processing and provides a link to the underlying physiology. The functional role of feedback processing is demonstrated by context dependent modulation of V1 cell activation, leading to sharply localized detection of texture boundaries. It furthermore explains why pre-attentive processing in visual search tasks can be directly linked to texture boundary processing as revealed by recent EEG studies on visual search.

A computational feature binding model of human texture perception

We present a computational model for human texture perception which assigns functional principles to the Gestalt laws of similarity and proximity. Motivated by early vision mechanisms, in the first stage, local texture features are extracted by utilizing multi-scale filtering and nonlinear spatial pooling. In the second stage, features are grouped according to the spatial feature binding model of the competitive layer model (CLM; Wersing et al. 2001). The CLM uses cooperative and competitive interactions in a recurrent network, where binding is expressed by the layer-wise coactivation of feature-representing neurons. The Gestalt law of similarity is expressed by a non-Euclidean distance measure in the abstract feature space with proximity being taken into account by a spatial component. To choose the stimulus dimensions which allow the most salient similarity-based texture segmentation, the feature similarity metrics is reduced to the directions of maximum variance. We show that our combined texture feature extraction and binding model performs segmentation in strong conformity with human perception. The examples range from classical microtextures and Brodatz textures to other classical Gestalt stimuli, which offer a new perspective on the role of texture for more abstract similarity grouping.

Towards Criteria for Visual Layout of Instructional Multimedia Interfaces

This article investigates the use of 11 refined Gestalt laws as the layout rules for instructional multimedia screen design by way of the redesign of the interface of a multimedia computer application and the subsequent evaluation of this redesigned interface. These layout rules are suitable for designing interfaces for complex learning materials in order to achieve more effective learning environments. To test the usefulness of these layout methodologies in visual screen design, they were applied to the redesign of an instructional multimedia application. Three different user groups evaluated the new screen design by comparing the designs. The evaluation results were overwhelmingly positive. Overall, 85% of the evaluators rated positively both the new design and the value of applying the 11 layout rules to improve the learning. This set of layout rules may also be successfully applied to design many different instructional multimedia applications.

Multi-modal estimation of collinearity and parallelism in natural image sequences*

In this paper we address the statistics of second-order relations of feature vectors derived from image sequences. We compute the individual vector components corresponding to the visual modalities orientation, contrast transition, optic flow, and colour by conventional low-level early vision algorithms. As a main result, we observe that collinear (or parallel) line pairs are, with very great likelihood, also associated with other identical features, for example sharing the same flow pattern, or colour or even sharing multiple feature combinations. It is known that low level processes, such as edge detection, optic flow estimation and stereo are ambiguous. Our results provide support for the assumption that the ambiguity of low level processes can be substantially reduced by integrating information across visual modalities. Furthermore, the attempt to model the application of Gestalt laws in computer vision systems based on statistical measurements, as suggested recently by some researchers (Krüger N 1998 Neural Process. Lett. 8 117-29; Elder H and Goldberg R M 1998 Perception Suppl. 27 11; Sigman M, Cecchi G A, Gilbert C D and Magnasco M O 200.1 Proc. Natl Acad. Sci. USA 98 1935-49; Geisler W S, Perry J S, Super B J and Gallogly D P 2002 Vis. Res. 41 711-24), gets further support and the results in this paper suggest formulation of Gestalt principles in artificial vision systems in a multi-modal way.

Visual binding of luminance, motion and disparity edges

Visual binding of edge segments embedded in noise and created by luminance, motion and disparity contrasts were studied in three experiments. The results showed that path formation was limited by the same rules across all attributes tested. The first experiment showed that binding could be accomplished with either attribute used in isolation. The second experiment showed that closed paths were easier to detect than open paths irrespectively of the attributes used to create the path elements. No additive effects were found in either Experiment 1 or 2 when the path elements were created with several attributes superimposed on the same positions, compared to when only one attribute was used along the path. In Experiment 3 it was found that when another attribute was added between the positions of the first attribute along the path, so that two attributes alternated along the path, the performance of path detection was better than expected by probability summation estimated from the single attribute conditions. These results provide evidence for attribute-invariant Gestalt laws and provide clues about the underlying neural mechanisms.

Ecological statistics of Gestalt laws for the perceptual organization of contours.

Although numerous studies have measured the strength of visual grouping cues for controlled psychophysical stimuli, little is known about the statistical utility of these various cues for natural images. In this study, we conducted experiments in which human participants trace perceived contours in natural images. These contours are automatically mapped to sequences of discrete tangent elements detected in the image. By examining relational properties between pairs of successive tangents on these traced curves, and between randomly selected pairs of tangents, we are able to estimate the likelihood distributions required to construct an optimal Bayesian model for contour grouping. We employed this novel methodology to investigate the inferential power of three classical Gestalt cues for contour grouping: proximity, good continuation, and luminance similarity. The study yielded a number of important results: (1) these cues, when appropriately defined, are approximately uncorrelated, suggesting a simple factorial model for statistical inference; (2) moderate image-to-image variation of the statistics indicates the utility of general probabilistic models for perceptual organization; (3) these cues differ greatly in their inferential power, proximity being by far the most powerful; and (4) statistical modeling of the proximity cue indicates a scale-invariant power law in close agreement with prior psychophysics.

Multi-modal estimation of collinearity and parallelism in natural image sequences

In this paper we address the statistics of second-order relations of feature vectors derived from image sequences. We compute the individual vector components corresponding to the visual modalities orientation, contrast transition, optic flow, and colour by conventional low-level early vision algorithms. As a main result, we observe that collinear (or parallel) line pairs are, with very great likelihood, also associated with other identical features, for example sharing the same flow pattern, or colour or even sharing multiple feature combinations.It is known that low level processes, such as edge detection, optic flow estimation and stereo are ambiguous. Our results provide support for the assumption that the ambiguity of low level processes can be substantially reduced by integrating information across visual modalities. Furthermore, the attempt to model the application of Gestalt laws in computer vision systems based on statistical measurements, as suggested recently by some researchers (Krüger N 1998 Neural Process. Lett. 8 117–29; Elder H and Goldberg R M 1998 Perception Suppl. 27 11; Sigman M, Cecchi G A, Gilbert C D and Magnasco M O 2001 Proc. Natl Acad. Sci. USA 98 1935–49; Geisler W S, Perry J S, Super B J and Gallogly D P 2002 Vis. Res. 41 711–24), gets further support and the results in this paper suggest formulation of Gestalt principles in artificial vision systems in a multi-modal wayThis Work has, to a large part, been performed during Norbert Krüger's stay in the Conginitive System Group at the University of Kiel, Germany.

Perceptual organization for inferring object boundaries in an image

We are concerned with object recognition in the framework of perceptual organization. The approach presented incorporates a number of concepts from human visual analysis especially the Gestalt laws of organization. Fuzzy techniques are used for the definition and evaluation of the grouping/non-grouping properties as well as for the construction of structures from grouped input tokens. This method takes as input the initially fitted line segments (tokens) and then recursively groups these tokens into higher level structures (tokens) such as lines, u-structures, quadrilaterals, etc. The output high-level structures can then be used to compare with object models and thus lead to object recognition. In this paper inference (grouping) of line segments, line symmetry, junctions, closed regions and strands is presented. The approach is supported by experimental results on 2D images of an office scene environment.

Interactions between perceptual organization based on Gestalt laws and those based on hierarchical processing.

Our recent research with compound stimuli (Han, Humphreys, & Chen, 1999a) suggests that grouping between local elements facilitates the perception of global structure, whereas encoding closure in local elements enhances their segmentation. The present study presents further evidence supporting this assertion. Experiment 1 first developed a new paradigm in which grouping between local elements was manipulated. Subjects responded to the orientations of perceptual groups consisting of local arrows or triangles embedded in background crosses. Responses to the orientations of the groups were slowed as a function of the increased contrast of the crosses, indicating that the strength of grouping between local arrows or triangles was gradually weakened by increasing the contrast of the crosses. Using a similar paradigm, Experiments 2 and 3 investigated the role of Gestalt factors in hierarchical analysis. Global arrows or triangles made up of local arrows, or triangles were embedded in background crosses. Subjects responded to global or local stimuli in terms of orientation or closure. Increasing the contrast of the background crosses produced stronger effects on global responses than on local responses and resulted in elimination of the global precedence effect and emerging of a local precedence effect, which was stronger for closure discrimination than for orientation discrimination. These results provide new evidence supporting our previous claim about the role of Gestalt factors in hierarchical analysis.

Curve detection in a noisy image.

In this paper we propose a new theory of the Gestalt law of good continuation. In this theory perceptual processes are modeled by an exponential pyramid algorithm. To test the new theory we performed three experiments. The subject's task was to detect a target (a set of dots arranged along a straight or curved line) among background dots. Detectability was high when: (a) the target was long; (b) the density of target dots relative to the density of background dots was large; (c) the local change of angle was small along the entire line; (d) local properties of the target were known to the subject. These results are consistent with our new model and they contradict prior models.

Pattern-Driven Visual Search for Multiple Defects

A simulation task for circuit-board inspection was designed with different image patterns to analyze inspectors' search strategies including the scan-path and the stopping policy. This study focused on how people fixate and locate defects, and the subsequent decision making processes. According to Gestalt laws of organization, a complex searching field would be decomposed in groups for separate scans. The analysis of scan-path shows that subjects had their own respective search pattern for all trials. Although the scan-paths were found different, search behavior within image patterns were similar across subjects. It implies that the visual search is driven by image patterns.

Neurophysiology of shape processing

Recent physiological findings are reviewed and synthesized into a model of shape processing and object recognition. Gestalt laws (e.g. good continuation, closure) and ‘non-accidental’ image properties (e.g. colinear terminating lines) are resolved in prestriate visual cortex, (areas V2 and V3) to support the extraction of 2D shape boundaries. Processing of shape continues along a ventral route through inferior temporal (IT) cortex where a vast catalogue of 2D shape primitives is established. Each catalogue entry is size-specific (±0.5 log scale unit) and orientation-specific (±45°), but can generalize over position (±150 degree 2). Several shape components are used to activate representations of the approximate appearance of one object type at one view, orientation and size. Subsequent generalization, first over orientation and size, then over view, and finally over object sub-component, is achieved in the anterior temporal cortex by combining descriptions of the same object from different orientations and views, through associative learning. This scheme provides a route to 3D object recognition through 2D shape description and reduces the problem of perceptual invariance to a series of independent analyses with an associative link established between the outputs. The system relies on parallel processing with computations performed in a series of hierarchical steps with relatively simple operations at each stage.