Perceptual Grouping Principles Research Articles

Mixed Evidence for Gestalt Grouping in Deep Neural Networks

Gestalt psychologists have identified a range of conditions in which humans organize elements of a scene into a group or whole, and perceptual grouping principles play an essential role in scene perception and object identification. Recently, Deep Neural Networks (DNNs) trained on natural images (ImageNet) have been proposed as compelling models of human vision based on reports that they perform well on various brain and behavioural benchmarks. Here we test a total of 16 networks covering a variety of architectures and learning paradigms (convolutional, attention-based, supervised and self-supervised, feed-forward and recurrent) on dots (Experiment 1) and more complex shapes (Experiment 2) stimuli that produce strong Gestalts effects in humans. In Experiment 1 we found that convolutional networks were indeed sensitive in a human-like fashion to the principles of proximity, linearity, and orientation, but only at the output layer. In Experiment 2, we found that most networks exhibited Gestalt effects only for a few sets, and again only at the latest stage of processing. Overall, self-supervised and Vision Transformer appeared to perform worse than convolutional networks in terms of human similarity. Remarkably, no model presented a grouping effect at the early or intermediate stages of processing. This is at odds with the widespread assumption that Gestalts occur prior to object recognition, and indeed, serve to organize the visual scene for the sake of object recognition. Our overall conclusion is that, albeit noteworthy that networks trained on simple 2D images support a form of Gestalt grouping for some stimuli at the output layer, this ability does not seem to transfer to more complex features. Additionally, the fact that this grouping only occurs at the last layer suggests that networks learn fundamentally different perceptual properties than humans.

Towards intentional aesthetics within topology optimization by applying the principle of unity-in-variety

Topology optimization is increasingly applied to design consumer products, for which aesthetics plays an important role to consumer acceptance. In industrial design, it is known that preferences or taste judgement obey certain rules or principles. These principles are not directly quantifiable, but can qualitatively predict and explain aesthetic responses. In this paper, we empirically evaluate whether or not these design principles are effective for increasing the appealingness of topology optimized shapes. Our starting point is an overarching principle known as Unity-in-Variety. Variety stimulates our interests, whilst unity helps us make sense of a design in its entirety. According to this principle, aesthetic appreciation is maximized when a balance in unity and variety is attained. Since designs from topology optimization often exhibit remarkable complexity and variety, we hypothesize that increasing unity is the key to reach a balance and thus to elevate aesthetic appreciation in topology optimization. In our experimental setup, designs from topology optimization were manually post-processed, with the intention to increase unity, by following the “principles of perceptual grouping”, known as Gestalt principles. Our user study shows that in 11 out of the 12 pairs of topology optimized designs and their modified counterparts, the modified designs are perceived by the majority as visually more appealing, confirming our hypothesis. These findings provide a good basis for improving the aesthetic pleasure of topology optimized designs, either manually or ultimately by integrating them in the topology optimization formulation. It is expected that this eventually will contribute to a wider acceptance of topology optimization for consumer product design.

A General Spline-Based Method for Centerline Extraction from Different Segmented Road Maps in Remote Sensing Imagery

Road centerline extraction is the foundation for integrating the segmented road map from a remote sensing image into a geographic information system (GIS) database. Considering that existing approaches tend to have a decline in performance for centerline and junction extraction when segmented road structures are irregular, this paper proposes a novel method which models the road network as a sequence of connected spline curves. Based on this motivation, the ratio of cross operators is firstly proposed to detect direction and width features of roads. Then, road pixels are divided into different clusters by local features using three perceptual grouping principles (i.e., direction grouping, proximity grouping, and continuity grouping). After applying a polynomial curve fitting on each cluster using pixel coordinates as observation data, the internal control points are determined according to the adjacency relation between clusters. Finally, road centerlines are generated based on spline fitting with constraints. We test our approach on segmented road maps which were obtained previously by machine recognition, or manual extraction from real optical (WorldView-2) and synthetic aperture radar (TerraSAR-X, Radarsat-2) images. Depending on the accuracy of the input segmented road maps, experimental results from our test data show that both the completeness and correctness of extracted centerlines are over 84% and 68% for optical and radar images, respectively. Furthermore, experiments also demonstrate the advantages of our proposed method, in contrast to existing methods for gaining smooth centerlines and precise junctions.

Cortical circuits for top-down control of perceptual grouping

A fundamental characteristic of human visual perception is the ability to group together disparate elements in a scene and treat them as a single unit. The mechanisms by which humans create such groupings remain unknown, but grouping seems to play an important role in a wide variety of visual phenomena, and a good understanding of these mechanisms might provide guidance for how to improve machine vision algorithms. Here, we build on a proposal that some groupings are the result of connections in cortical area V2 that join disparate elements, thereby allowing them to be selected and segmented together. In previous instantiations of this proposal, connection formation was based on the anatomy (e.g., extent) of receptive fields, which made connection formation obligatory when the stimulus conditions stimulate the corresponding receptive fields. We now propose dynamic circuits that provide greater flexibility in the formation of connections and that allow for top-down control of perceptual grouping. With computer simulations we explain how the circuits work and show how they can account for a wide variety of Gestalt principles of perceptual grouping, texture segmentation tasks, amodal illusory contours, and ratings of perceived groupings. We propose that human observers use such top-down control to implement task-dependent connection strategies that encourage particular groupings of stimulus elements in order to promote performance on various visual tasks.

Ecological origins of perceptual grouping principles in the auditory system.

Events and objects in the world must be inferred from sensory signals to support behavior. Because sensory measurements are temporally and spatially local, the estimation of an object or event can be viewed as the grouping of these measurements into representations of their common causes. Perceptual grouping is believed to reflect internalized regularities of the natural environment, yet grouping cues have traditionally been identified using informal observation and investigated using artificial stimuli. The relationship of grouping to natural signal statistics has thus remained unclear, and additional or alternative cues remain possible. Here, we develop a general methodology for relating grouping to natural sensory signals and apply it to derive auditory grouping cues from natural sounds. We first learned local spectrotemporal features from natural sounds and measured their co-occurrence statistics. We then learned a small set of stimulus properties that could predict the measured feature co-occurrences. The resulting cues included established grouping cues, such as harmonic frequency relationships and temporal coincidence, but also revealed previously unappreciated grouping principles. Human perceptual grouping was predicted by natural feature co-occurrence, with humans relying on the derived grouping cues in proportion to their informativity about co-occurrence in natural sounds. The results suggest that auditory grouping is adapted to natural stimulus statistics, show how these statistics can reveal previously unappreciated grouping phenomena, and provide a framework for studying grouping in natural signals.

On Visually-Grounded Reference Production: Testing the Effects of Perceptual Grouping and 2D/3D Presentation Mode.

When referring to a target object in a visual scene, speakers are assumed to consider certain distractor objects to be more relevant than others. The current research predicts that the way in which speakers come to a set of relevant distractors depends on how they perceive the distance between the objects in the scene. It reports on the results of two language production experiments, in which participants referred to target objects in photo-realistic visual scenes. Experiment 1 manipulated three factors that were expected to affect perceived distractor distance: two manipulations of perceptual grouping (region of space and type similarity), and one of presentation mode (2D vs. 3D). In line with most previous research on visually-grounded reference production, an offline measure of visual attention was taken here: the occurrence of overspecification with color. The results showed effects of region of space and type similarity on overspecification, suggesting that distractors that are perceived as being in the same group as the target are more often considered relevant distractors than distractors in a different group. Experiment 2 verified this suggestion with a direct measure of visual attention, eye tracking, and added a third manipulation of grouping: color similarity. For region of space in particular, the eye movements data indeed showed patterns in the expected direction: distractors within the same region as the target were fixated more often, and longer, than distractors in a different region. Color similarity was found to affect overspecification with color, but not gaze duration or the number of distractor fixations. Also the expected effects of presentation mode (2D vs. 3D) were not convincingly borne out by the data. Taken together, these results provide direct evidence for the close link between scene perception and language production, and indicate that perceptual grouping principles can guide speakers in determining the distractor set during reference production.

Perceptually motivated algorithm for grouping objects in complex scenes

Region proposal algorithms are beneficial for enhancing the performance of object detection and recognition methods. We present a method for grouping region proposals based on perceptual grouping principles. The grouping principles are simulated to extract image features, and the region proposals are segmented by solving a sequence of parametric maxflow problems. In order to extract complex objects from natural images, the element connectedness cue is introduced in the parametric energy functions. This newly introduced cue is propitious to group objects with diversified patterns. To effectively fuse the grouping principles, a multiclassify-based learning algorithm is proposed to optimize an ensemble of binary segmentation models. The training samples are first divided into groups to pretrain each individual model, and the algorithm adaptively adjusts the sample groups in the iteration procedure to learn an optimal set of models. We conduct the experiments on the PASCAL Visual Object Classes Challenge 2012 segmentation dataset but also in the context of region proposals in optical remote sensing images, and the results show that the proposed method can achieve a favorable performance compared to the existing algorithms.

Perception and conception in understanding evolutionary trees

Relationships depicted in evolutionary trees depend solely on levels of most recent common ancestry. Integrating discipline-based education research in biology with perceptual/cognitive psychology, the authors predicted, however, that the Gestalt principles of perceptual grouping would affect how students interpret these relationships. Experiment 1 (N = 93) found that students segment 6–9 branch trees in accordance with the Gestalt principle of connectedness. Experiment 2 (N = 310) found that students in introductory through advanced biology classes predominantly believed, incorrectly, that the evolutionary relationships among a set of target taxa differed in two trees because the grouping of those taxa differed. Experiment 3 (N = 99) found that students from these same classes were more likely to make inferences consistent with the depicted evolutionary relationships when Gestalt grouping supported those inferences. The authors discuss implications for improving students’ understanding of cladograms.

Hearing triangles: Perceptual clarity, opacity, and symmetry of spectrotemporal sound shapes.

In electroacoustic music, the spectromorphological approach commonly employs analogies to non-sonic phenomena like shapes, gestures, or textures. In acoustical terms, sound shapes can concern simple geometries on the spectrotemporal plane, for instance, a triangle that widens in frequency over time. To test the auditory relevance of such triangular sound shapes, two psychoacoustic experiments assessed if and how these shapes are perceived. Triangular sound-shape stimuli, created through granular synthesis, varied across the factors grain density, frequency and amplitude scales, and widening vs narrowing orientations. The perceptual investigation focused on three auditory qualities, derived in analogy to the visual description of a triangle: the clarity of the triangular outline, the opacity of the area enclosed by the outline, and the symmetry along the vertical dimension. These morphological qualities seemed to capture distinct perceptual aspects, each linked to different acoustical factors. Clarity of shape was conveyed even for sparse grain densities, while also exhibiting a perceptual bias for widening orientations. Opacity varied as a function of grain texture, whereas symmetry strongly depended on frequency and amplitude scales. The perception of sound shapes could relate to common perceptual cross-modal correspondences and share the same principles of perceptual grouping with vision.

Temporal Reference, Attentional Modulation, and Crossmodal Assimilation.

Crossmodal assimilation effect refers to the prominent phenomenon by which ensemble mean extracted from a sequence of task-irrelevant distractor events, such as auditory intervals, assimilates/biases the perception (such as visual interval) of the subsequent task-relevant target events in another sensory modality. In current experiments, using visual Ternus display, we examined the roles of temporal reference, materialized as the time information accumulated before the onset of target event, as well as the attentional modulation in crossmodal temporal interaction. Specifically, we examined how the global time interval, the mean auditory inter-intervals and the last interval in the auditory sequence assimilate and bias the subsequent percept of visual Ternus motion (element motion vs. group motion). We demonstrated that both the ensemble (geometric) mean and the last interval in the auditory sequence contribute to bias the percept of visual motion. Longer mean (or last) interval elicited more reports of group motion, whereas the shorter mean (or last) auditory intervals gave rise to more dominant percept of element motion. Importantly, observers have shown dynamic adaptation to the temporal reference of crossmodal assimilation: when the target visual Ternus stimuli were separated by a long gap interval after the preceding sound sequence, the assimilation effect by ensemble mean was reduced. Our findings suggested that crossmodal assimilation relies on a suitable temporal reference on adaptation level, and revealed a general temporal perceptual grouping principle underlying complex audio-visual interactions in everyday dynamic situations.

DRIVER PREFERENCE CONCERNING IN-CAR ROUTE GUIDANCE AND NAVIGATION SYSTEM MAPS FOR DRIVERS WITH COLOR VISION DEFICIENCY

Abstract: In-car Route Guidance and Navigation Systems (RGNS) are used to help drivers navigate. These maps have mainly been designed to accommodate drivers with normal color vision. However, the color perception of people with normal color vision differs from that of people with color vision deficiency. When navigating, understanding certain kinds of information presented by RGNS maps can be a more complex task for colorblind drivers and traffic safety may be impacted negatively. An important aspect related to the graphic design of RGNS maps is the use of a good combination of colors to improve map legibility. Cartographic representations with good legibility aid drivers in comprehending information and making appropriate decisions during driving tasks. This paper evaluates driver preference for RGNS maps designed for drivers with color vision deficiency. A total of 14 subjects participated in an experiment performed in a parked car. Maps were designed to accommodate red-blinds and green-blinds by using a color simulator and principles of perceptual grouping and figure-ground segregation. Based on the results, we conclude that the map grouping symbols representing car and direction arrows in blue segregated from the route in black was more acceptable to drivers compared to other combinations. It is recommended that RGNS should offer a specific graphic design to support drivers with color vision deficiency in navigating.

Learning to group discrete graphical patterns

We introduce a deep learning approach for grouping discrete patterns common in graphical designs. Our approach is based on a convolutional neural network architecture that learns a grouping measure defined over a pair of pattern elements. Motivated by perceptual grouping principles, the key feature of our network is the encoding of element shape, context, symmetries, and structural arrangements. These element properties are all jointly considered and appropriately weighted in our grouping measure. To better align our measure with human perceptions for grouping, we train our network on a large, human-annotated dataset of pattern groupings consisting of patterns at varying granularity levels, with rich element relations and varieties, and tempered with noise and other data imperfections. Experimental results demonstrate that our deep-learned measure leads to robust grouping results.

GRASS

We introduce a novel neural network architecture for encoding and synthesis of 3D shapes, particularly their structures. Our key insight is that 3D shapes are effectively characterized by their hierarchical organization of parts, which reflects fundamental intra-shape relationships such as adjacency and symmetry. We develop a recursive neural net (RvNN) based autoencoder to map a flat, unlabeled, arbitrary part layout to a compact code. The code effectively captures hierarchical structures of man-made 3D objects of varying structural complexities despite being fixed-dimensional: an associated decoder maps a code back to a full hierarchy. The learned bidirectional mapping is further tuned using an adversarial setup to yield a generative model of plausible structures, from which novel structures can be sampled. Finally, our structure synthesis framework is augmented by a second trained module that produces fine-grained part geometry, conditioned on global and local structural context, leading to a full generative pipeline for 3D shapes. We demonstrate that without supervision, our network learns meaningful structural hierarchies adhering to perceptual grouping principles, produces compact codes which enable applications such as shape classification and partial matching, and supports shape synthesis and interpolation with significant variations in topology and geometry.

Principles of perceptual grouping: implications for image-guided surgery.

Gestalt theory has provided perceptual science with a conceptual framework which has inspired researchers ever since, taking the field of perceptual organization into the 21st century. This opinion article discusses the importance of rules of perceptual organization for the testing and design of visual interface technology. It is argued that major Gestalt principles, such as the law of good continuation or the principle of Praegnanz (suggested translation: salience), taken as examples here, are important to our understanding of visual image processing by a human observer. Perceptual integration of contrast information across collinear space, and the organization of objects in the 2D image plane into figure and ground are of a particular importance here. Visual interfaces for image-guided surgery illustrate the criticality of these two types of perceptual processes for reliable decision making and action. It is concluded that Gestalt theory continues to generate powerful concepts and insights for perceptual science placed within the context of major technological challenges of today.

A temporo-spatial analysis of the neural correlates of extrinsic perceptual grouping in vision

Principles of perceptual grouping can be divided into intrinsic grouping cues, which are based on built-in properties of the grouped elements (e.g., their shape, position, colour, etc.) like most of the classical Gestalt laws, and extrinsic grouping principles, based on relations between the discrete elements and other external stimuli that induce them to group (e.g., common region, connectedness). Several studies have explored the neural correlates of intrinsic grouping factors but, to our knowledge, no previous study has studied the neural correlates of extrinsic principles. The present study aimed to shed light on this issue by exploiting the high temporal resolution of event-related potentials (ERPs) and recent advances in source localization. Specifically, grouping by common region was compared with two comparison conditions, an intrinsic grouping (luminance similarity) and a uniform stimulus condition, in a perceptual discrimination task. We reported three main neural effects associated with grouping by common region. First, a posterior N210 component with a neural origin in the left extrastriate cortex was related to perceptual analysis of extrinsic elements inducing grouping and the formation of a visual group. Second, an enhanced posterior P280, which presumably reflects higher confidence decisions during response selection. Finally, a P550 originated in the right superior parietal cortex that seems to be associated with top-down suppression activity connected with the termination of the processing of the current trial. Overall, our results suggest that common region cues belong to the category of long latency grouping principles that mainly involve activity in extrastriate cortices.

Object Detection for Robotic Applications Using Perceptual Organization in 3D

Object segmentation of unknown objects with arbitrary shape in cluttered scenes is still a challenging task in computer vision. A framework is introduced to segment RGB-D images where data is processed in a hierarchical fashion. After pre-segmentation and parametrization of surface patches, support vector machines are used to learn the importance of relations between these patches. The relations are derived from perceptual grouping principles. The proposed framework is able to segment objects, even if they are stacked or jumbled in cluttered scenes. Furthermore, the problem of segmenting partially occluded objects is tackled.

Learning of perceptual grouping for object segmentation on RGB-D data

Object segmentation of unknown objects with arbitrary shape in cluttered scenes is an ambitious goal in computer vision and became a great impulse with the introduction of cheap and powerful RGB-D sensors. We introduce a framework for segmenting RGB-D images where data is processed in a hierarchical fashion. After pre-clustering on pixel level parametric surface patches are estimated. Different relations between patch-pairs are calculated, which we derive from perceptual grouping principles, and support vector machine classification is employed to learn Perceptual Grouping. Finally, we show that object hypotheses generation with Graph-Cut finds a globally optimal solution and prevents wrong grouping. Our framework is able to segment objects, even if they are stacked or jumbled in cluttered scenes. We also tackle the problem of segmenting objects when they are partially occluded. The work is evaluated on publicly available object segmentation databases and also compared with state-of-the-art work of object segmentation.

Phenomenal Causality II: Integration and Implication

The empirical literature on phenomenal causality (the notion that causality can be perceived) is reviewed. Different potential types of phenomenal causality and variables that influence phenomenal causality were considered in Part I (Hubbard 2012b) of this two-part series. In Part II, broader questions regarding properties of phenomenal causality and connections of phenomenal causality to other perceptual or cognitive phenomena (different types of phenomenal causality, effects of spatial and temporal variance, phenomenal causality in infancy, effects of object properties, naive physics, spatial localization, other illusions, amodal completion, Gestalt principles of perceptual grouping, effects of context, differences between physical and social causality, effects of learning and experience, individual differences, effects of predictability, asymmetry in phenomenal causality, differences between perceived causality and perceived force, phenomenal causality in nonhuman animals) are considered. Potential mechanisms of phenomenal causality (inference from contiguity, a priori understanding, ampliation, perceptual learning, stimulus activity, beliefs regarding kinematics, haptic experience, beliefs regarding impetus, postdiction, innateness, modularity, specific neural structures) are also considered.

To what extent do Gestalt grouping principles influence tactile perception?

Since their formulation by the Gestalt movement more than a century ago, the principles of perceptual grouping have primarily been investigated in the visual modality and, to a lesser extent, in the auditory modality. The present review addresses the question of whether the same grouping principles also affect the perception of tactile stimuli. Although, to date, only a few studies have explicitly investigated the existence of Gestalt grouping principles in the tactile modality, we argue that many more studies have indirectly provided evidence relevant to this topic. Reviewing this body of research, we argue that similar principles to those reported previously in visual and auditory studies also govern the perceptual grouping of tactile stimuli. In particular, we highlight evidence showing that the principles of proximity, similarity, common fate, good continuation, and closure affect tactile perception in both unimodal and crossmodal settings. We also highlight that the grouping of tactile stimuli is often affected by visual and auditory information that happen to be presented simultaneously. Finally, we discuss the theoretical and applied benefits that might pertain to the further study of Gestalt principles operating in both unisensory and multisensory tactile perception.

To be or not to be…included in an event file: Integration and retrieval of distractors in stimulus–response episodes is influenced by perceptual grouping.

The binding of stimulus and response features into stimulus-response (S-R) episodes or "event files" is a basic process for the efficient control of behavior. However, relevant information is usually accompanied by information that is irrelevant for the selection of action. Recent studies showed that even irrelevant information is bound into event files. In this study, we investigated the boundary conditions of distractor-response binding and subsequent distractor-based response retrieval processes. In particular, we tested whether the inclusion of distractor information into S-R episodes is modulated by whether the distractor and target stimulus are perceived as belonging to the same object or as belonging to different objects. We argue that distracting information is only bound into S-R episodes if it is perceived as belonging to the same object as the relevant information, whereas no binding occurs when the distracting information is perceived as belonging to a separate object. In 6 experiments, we found evidence for the modulation of distractor-response bindings according to perceptual grouping principles.