Representation Of Object Shape Research Articles

Gradual change of cortical representations with growing visual expertise for synthetic shapes

Abstract Objective: Visual expertise for particular categories of objects (e.g., mushrooms, birds, flowers, minerals, and so on) is known to enhance cortical responses in parts of the ventral occipitotemporal cortex. How is such additional expertise integrated into the prior cortical representation of life-long visual experience? To address this question, we presented synthetic visual objects rotating in three dimensions and recorded multivariate BOLD responses as initially unfamiliar objects gradually became familiar. Main results: An analysis of pairwise distances between multivariate BOLD responses (“representational similarity analysis,” RSA) revealed that visual objects were linearly discriminable in large parts of the ventral occipital cortex, including the primary visual cortex, as well as in certain parts of the parietal and frontal cortex. These cortical representations were present from the start, when objects were still unfamiliar, and even though objects were shown from different sides. As shapes became familiar with repeated viewing, the distribution of responses expanded to fill more of the available space. In contrast, the distribution of responses to novel shapes (which appeared only once) contracted and shifted to the margins of the available space. Conclusion: Our results revealed cortical representations of object shape and gradual changes in these representations with learning and consolidation. The cortical representations of once-viewed shapes that remained novel diverged dramatically from repeatedly viewed shapes that became familiar. This disparity was evident in both the similarity and the diversity of multivariate BOLD responses.

The multisensory and multidimensional nature of object representation.

Recent functional magnetic resonance imaging (fMRI) experiments revealed similar neural representations across different types of two-dimensional (2-D) visual stimuli; however, real three-dimensional (3-D) objects affording action differentially affect neural activation and behavioral results relative to 2-D objects. Recruitment of multiple sensory regions during unisensory (visual, haptic, and auditory) object shape tasks suggests that shape representation may be modality invariant. This mini-review explores the overlapping neural regions involved in object shape representation, across 2-D, 3-D, visual, and haptic experiments.

Computation on Demand: Action-Specific Representations of Visual Task Features Arise during Distinct Movement Phases.

The intention to act influences the computations of various task-relevant features. However, little is known about the time course of these computations. Furthermore, it is commonly held that these computations are governed by conjunctive neural representations of the features. But, support for this view comes from paradigms arbitrarily combining task features and affordances, thus requiring representations in working memory. Therefore, the present study used electroencephalography and a well-rehearsed task with features that afford minimal working memory representations to investigate the temporal evolution of feature representations and their potential integration in the brain. Female and male human participants grasped objects or touched them with a knuckle. Objects had different shapes and were made of heavy or light materials with shape and weight being relevant for grasping, not for "knuckling." Using multivariate analysis showed that representations of object shape were similar for grasping and knuckling. However, only for grasping did early shape representations reactivate at later phases of grasp planning, suggesting that sensorimotor control signals feed back to the early visual cortex. Grasp-specific representations of material/weight only arose during grasp execution after object contact during the load phase. A trend for integrated representations of shape and material also became grasp-specific but only briefly during the movement onset. These results suggest that the brain generates action-specific representations of relevant features as required for the different subcomponents of its action computations. Our results argue against the view that goal-directed actions inevitably join all features of a task into a sustained and unified neural representation.

Vision matters for shape representation: Evidence from sculpturing and drawing in the blind

Shape is a property that could be perceived by vision and touch, and is classically considered to be supramodal. While there is mounting evidence for the shared cognitive and neural representation space between visual and tactile shape, previous research tended to rely on dissimilarity structures between objects and had not examined the detailed properties of shape representation in the absence of vision. To address this gap, we conducted three explicit object shape knowledge production experiments with congenitally blind and sighted participants, who were asked to produce verbal features, 3D clay models, and 2D drawings of familiar objects with varying levels of tactile exposure, including tools, large nonmanipulable objects, and animals. We found that the absence of visual experience (i.e., in the blind group) led to stronger differences in animals than in tools and large objects, suggesting that direct tactile experience of objects is essential for shape representation when vision is unavailable. For tools with rich tactile/manipulation experiences, the blind produced overall good shapes comparable to the sighted, yet also showed intriguing differences. The blind group had more variations and a systematic bias in the geometric property of tools (making them stubbier than the sighted), indicating that visual experience contributes to aligning internal representations and calibrating overall object configurations, at least for tools. Taken together, the object shape representation reflects the intricate orchestration of vision, touch and language.

Exploiting sublimated deep features for image retrieval

Deep learning techniques can be used to describe image content, which is a good way to reduce the semantic gap between low-level and high-level features. However, convolutional neural networks (CNNs) exhibit texture bias and largely ignore global object shape. This does not conform to human perception and can fail to gain the advantages of both low-level features and deep features. To address this problem, in this work, the main focus was shifted from using simple deep features to the use of sublimated deep features, which incorporate global object shape and color features. Along these lines, in this work, a novel image retrieval method named the sublimated deep feature histogram (SDFH) was proposed. The main highlights are: 1) An effect orientation feature was introduced, namely, orientation-selective feature, to mimic the orientation-selection mechanism. This provides a good representation of global object shape and reduces the side-effects of texture bias. 2) A new concept was developed, namely color perceptual feature, to address the shortcoming of deep features—that they discard color features. This includes color cues in the deep features and provides a more discriminating representation. 3) The orientation selective and color perception mechanisms were effectively mimicked to provide a compact yet efficient representation, and propose an effective transfer learning method called gain whitening learning. By carrying out comparative experiments, it was demonstrated that sublimated deep features can provide highly competitive retrieval performance (in terms of mean average precision) using a pre-trained CNN model applied to well-known benchmark datasets. Our results provide new insights into image retrieval based on the mechanisms of the primary visual cortex (V1). Furthermore, the method is more in line with human perception than other methods.

Similar object shape representation encoded in the inferolateral occipitotemporal cortex of sighted and early blind people.

We can sense an object's shape by vision or touch. Previous studies suggested that the inferolateral occipitotemporal cortex (ILOTC) implements supramodal shape representations as it responds more to seeing or touching objects than shapeless textures. However, such activation in the anterior portion of the ventral visual pathway could be due to the conceptual representation of an object or visual imagery triggered by touching an object. We addressed these possibilities by directly comparing shape and conceptual representations of objects in early blind (who lack visual experience/imagery) and sighted participants. We found that bilateral ILOTC in both groups showed stronger activation during a shape verification task than during a conceptual verification task made on the names of the same manmade objects. Moreover, the distributed activity in the ILOTC encoded shape similarity but not conceptual association among objects. Besides the ILOTC, we also found shape representation in both groups' bilateral ventral premotor cortices and intraparietal sulcus (IPS), a frontoparietal circuit relating to object grasping and haptic processing. In contrast, the conceptual verification task activated both groups' left perisylvian brain network relating to language processing and, interestingly, the cuneus in early blind participants only. The ILOTC had stronger functional connectivity to the frontoparietal circuit than to the left perisylvian network, forming a modular structure specialized in shape representation. Our results conclusively support that the ILOTC selectively implements shape representation independently of visual experience, and this unique functionality likely comes from its privileged connection to the frontoparietal haptic circuit.

A survey on geometric shape representation of objects based on medial axis transform

<div><p>Geometric shape representation algorithms are key technologies in the fields of computer graphics and geometric modeling. The Medial Axis Transform (MAT) is an important geometric model description tool that provides a simplified representation of complex geometric shapes while ensuring accurate descriptions of geometric shape and topology. Therefore, it can meet the requirements of many modern research fields, including geometric modeling, pattern recognition, model segmentation, model deformation, physical simulation, path planning, and more. This paper first introduces the basic concept of the medial axis transform, including the definition of the medial axis transform and the concept of medial axis primitives. It then describes the extraction algorithms for the medial axis transform, specific research on the medial axis transform in computer vision and computer graphics, potential applications of the medial axis transform, and medial axis transform datasets. Finally, the disadvantages and advantages of the medial axis transform are discussed, and some suggestions on possible future research directions are presented.</p></div>

Role of Tactile Noise in the Control of Digit Normal Force.

Whenever we grasp and lift an object, our tactile system provides important information on the contact location and the force exerted on our skin. The human brain integrates signals from multiple sites for a coherent representation of object shape, inertia, weight, and other material properties. It is still an open question whether the control of grasp force occurs at the level of individual fingers or whether it is also influenced by the control and the signals from the other fingers of the same hand. In this work, we approached this question by asking participants to lift, transport, and replace a sensorized object, using three- and four-digit grasp. Tactile input was altered by covering participant's fingertips with a rubber thimble, which reduced the reliability of the tactile sensory input. In different experimental conditions, we covered between one and three fingers opposing the thumb. Normal forces at each finger and the thumb were recorded while grasping and holding the object, with and without the thimble. Consistently with previous studies, reducing tactile sensitivity increased the overall grasping force. The gasping force increased in the covered finger, whereas it did not change from baseline in the remaining bare fingers (except the thumb for equilibrium constraints). Digit placement and object tilt were not systematically affected by rubber thimble conditions. Our results suggest that, in each finger opposing thumb, digit normal force is controlled locally in response to the applied tactile perturbation.

Satellite Image Target Super-Resolution With Adversarial Shape Discriminator

Substantial progress in generative modeling has facilitated the development of perceptual-driven methods for image super-resolution (SR), which shows superior visual quality as rated by human observers. However, we discover that most existing methods are biased to synthesize fake details or enhanced textures and fail to learn representations of object shape. They are often misguided to recover the shape of small objects and focus on generating less meaningful high-frequency components. This is problematic when super-resolving satellite imagery since it contains many relatively small and clustered objects in a broad area. In this letter, we propose a new perceptual-driven SR method that has a stronger preference toward shape information. We integrate the scale-space filtering with the discriminator to attenuate high-frequency components in its decision space. It effectively encourages our discriminator to concentrate on structural shape information in differentiating real and fake images. We also devise a cross-scale aggregation network for the generator architecture. Experiments on WorldView data set demonstrate that our method achieves state-of-the-art performance compared to recent perceptual-driven methods.

High-acuity vision from retinal image motion.

A mathematical model and a possible neural mechanism are proposed to account for how fixational drift motion in the retina confers a benefit for the discrimination of high-acuity targets. We show that by simultaneously estimating object shape and eye motion, neurons in visual cortex can compute a higher quality representation of an object by averaging out non-uniformities in the retinal sampling lattice. The model proposes that this is accomplished by two separate populations of cortical neurons — one providing a representation of object shape and another representing eye position or motion — which are coupled through specific multiplicative connections. Combined with recent experimental findings, our model suggests that the visual system may utilize principles not unlike those used in computational imaging for achieving “super-resolution” via camera motion.

Orthogonal Representations of Object Shape and Category in Deep Convolutional Neural Networks and Human Visual Cortex

Deep Convolutional Neural Networks (CNNs) are gaining traction as the benchmark model of visual object recognition, with performance now surpassing humans. While CNNs can accurately assign one image to potentially thousands of categories, network performance could be the result of layers that are tuned to represent the visual shape of objects, rather than object category, since both are often confounded in natural images. Using two stimulus sets that explicitly dissociate shape from category, we correlate these two types of information with each layer of multiple CNNs. We also compare CNN output with fMRI activation along the human visual ventral stream by correlating artificial with neural representations. We find that CNNs encode category information independently from shape, peaking at the final fully connected layer in all tested CNN architectures. Comparing CNNs with fMRI brain data, early visual cortex (V1) and early layers of CNNs encode shape information. Anterior ventral temporal cortex encodes category information, which correlates best with the final layer of CNNs. The interaction between shape and category that is found along the human visual ventral pathway is echoed in multiple deep networks. Our results suggest CNNs represent category information independently from shape, much like the human visual system.

Shape description and recognition by implicit Chebyshev moments

Chebyshev Moments(CMs) have been applied to representation and recognition of 2D object shapes in image processing and computer vision. However they still suffer from poor representation power and difficulty in computing invariants for shapes. In this work, we present Implicit Chebyshev Moments (ICMs) to overcome these issues. Firstly, we use Euclid distance transformation to generate a series of level sets based on a given shape. Secondly, we fit an implicit Chebyshev polynomial to the data set consisting of the obtained level sets together with all the boundary points on the original shape and call the obtained coefficients of the fitted implicit Chebyshev polynomial ICMs. Finally, we propose a new approach to derive geometric invariants based on ICMs. In addition, we also develop an algorithm for the determination of a suitable degree for implicit Chebyshev polynomials before representing a given shape. Experimental results show the ICMs are more efficient for representing complex shapes than CMs.

TOWARDS BETTER CLASSIFICATION OF LAND COVER AND LAND USE BASED ON CONVOLUTIONAL NEURAL NETWORKS

Abstract. Land use and land cover are two important variables in remote sensing. Commonly, the information of land use is stored in geospatial databases. In order to update such databases, we present a new approach to determine the land cover and to classify land use objects using convolutional neural networks (CNN). High-resolution aerial images and derived data such as digital surface models serve as input. An encoder-decoder based CNN is used for land cover classification. We found a composite including the infrared band and height data to outperform RGB images in land cover classification. We also propose a CNN-based methodology for the prediction of land use label from the geospatial databases, where we use masks representing object shape, the RGB images and the pixel-wise class scores of land cover as input. For this task, we developed a two-branch network where the first branch considers the whole area of an image, while the second branch focuses on a smaller relevant area. We evaluated our methods using two sites and achieved an overall accuracy of up to 89.6% and 81.7% for land cover and land use, respectively. We also tested our methods for land cover classification using the Vaihingen dataset of the ISPRS 2D semantic labelling challenge and achieved an overall accuracy of 90.7%.

Surface diagnosticity predicts the high-level representation of regular and irregular object shape in human vision

The human visual system has an extraordinary capacity to compute three-dimensional (3D) shape structure for both geometrically regular and irregular objects. The goal of this study was to shed new light on the underlying representational structures that support this ability. Observers (N = 85) completed two complementary perceptual tasks. Experiment 1 involved whole–part matching of image parts to whole geometrically regular and irregular novel object shapes. Image parts comprised either regions of edge contour, volumetric parts, or surfaces. Performance was better for irregular than for regular objects and interacted with part type: volumes yielded better matching performance than surfaces for regular but not for irregular objects. The basis for this effect was further explored in Experiment 2, which used implicit part–whole repetition priming. Here, we orthogonally manipulated shape regularity and a new factor of surface diagnosticity (how predictive a single surface is of object identity). The results showed that surface diagnosticity, not object shape regularity, determined the differential processing of volumes and surfaces. Regardless of shape regularity, objects with low surface diagnosticity were better primed by volumes than by surfaces. In contrast, objects with high surface diagnosticity showed the opposite pattern. These findings are the first to show that surface diagnosticity plays a fundamental role in object recognition. We propose that surface-based shape primitives—rather than volumetric parts—underlie the derivation of 3D object shape in human vision.

Real-world size coding of solid objects, but not 2-D or 3-D images, in visual agnosia patients with bilateral ventral lesions

Patients with visual agnosia show severe deficits in recognizing two-dimensional (2-D) images of objects, despite the fact that early visual processes such as figure-ground segmentation, and stereopsis, are largely intact. Strikingly, however, these patients can nevertheless show a preservation in their ability to recognize real-world objects –a phenomenon known as the ‘real-object advantage’ (ROA) in agnosia. To uncover the mechanisms that support the ROA, patients were asked to identify objects whose size was congruent or incongruent with typical real-world size, presented in different display formats (real objects, 2-D and 3-D images). While recognition of images was extremely poor, real object recognition was surprisingly preserved, but only when physical size matched real-world size. Analogous display format and size manipulations did not influence the recognition of common geometric shapes that lacked real-world size associations. These neuropsychological data provide evidence for a surprising preservation of size-coding of real-world-sized tangible objects in patients for whom ventral contributions to image processing are severely disrupted. We propose that object size information is largely mediated by dorsal visual cortex and that this information, together with detailed representation of object shape which is also subserved by dorsal cortex, serve as the basis of the ROA.

How are object shape axes defined? Evidence from mirror-image confusions.

A central goal in research on visual perception is to understand how the visual system represents the shapes of objects. According to many theorists, axes defined on the basis of object geometry provide a coordinate system for representing the locations and orientations of object parts. An important question that has received little attention concerns how object axes are defined-that is, what aspects of object geometry determine how axes are assigned to shapes? We evaluated 2 hypotheses. According to the elongated-part hypothesis, axes are defined on the basis an object's most elongated part, such that, for example, the principal axis for a hatchet would coincide with the long axis of the hatchet's handle. In contrast, the global-shape hypothesis holds that axes are defined on the basis of an object's overall shape (e.g., for the hatchet, as the longest axis that spans the entire hatchet). Using a novel paradigm involving analysis of mirror-image confusions, we obtained evidence strongly supporting the elongated-part hypothesis. Our results also point to a role for secondary as well as principal axes in object shape representation. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

Words affect visual perception by activating object shape representations

Linguistic labels are known to facilitate object recognition, yet the mechanism of this facilitation is not well understood. Previous psychophysical studies have suggested that words guide visual perception by activating information about visual object shape. Here we aimed to test this hypothesis at the neural level, and to tease apart the visual and semantic contribution of words to visual object recognition. We created a set of object pictures from two semantic categories with varying shapes, and obtained subjective ratings of their shape and category similarity. We then conducted a word-picture matching experiment, while recording participants’ EEG, and tested if the shape or the category similarity between the word’s referent and target picture explained the spatiotemporal pattern of the picture-evoked responses. The results show that hearing a word activates representations of its referent’s shape, which interacts with the visual processing of a subsequent picture within 100 ms from its onset. Furthermore, non-visual categorical information, carried by the word, affects the visual processing at later stages. These findings advance our understanding of the interaction between language and visual perception and provide insights into how the meanings of words are represented in the brain.

Automatic Extraction of the Centerline of Corpus Callosum from Segmented Mid-Sagittal MR Images.

The centerline, as a simple and compact representation of object shape, has been used to analyze variations of the human callosal shape. However, automatic extraction of the callosal centerline remains a sophisticated problem. In this paper, we propose a method of automatic extraction of the callosal centerline from segmented mid-sagittal magnetic resonance (MR) images. A model-based point matching method is introduced to localize the anterior and posterior endpoints of the centerline. The model of the endpoint is constructed with a statistical descriptor of the shape context. Active contour modeling is adopted to drive the curve with the fixed endpoints to approximate the centerline using the gradient of the distance map of the segmented corpus callosum. Experiments with 80 segmented mid-sagittal MR images were performed. The proposed method is compared with a skeletonization method and an interactive method in terms of recovery error and reproducibility. Results indicate that the proposed method outperforms skeletonization and is comparable with and sometimes better than the interactive method.

Disentangling representations of shape and action components in the tool network

Shape and how they should be used are two key components of our knowledge about tools. Viewing tools preferentially activated a frontoparietal and occipitotemporal network, with dorsal regions implicated in computation of tool-related actions and ventral areas in shape representation. As shape and manners of manipulation are highly correlated for daily tools, whether they are independently represented in different regions remains inconclusive. In the current study, we collected fMRI data when participants viewed blocks of pictures of four daily tools (i.e., paintbrush, corkscrew, screwdriver, razor) where shape and action (manner of manipulation for functional use) were orthogonally manipulated, to tease apart these two dimensions. Behavioral similarity judgments tapping on object shape and finer aspects of actions (i.e., manners of motion, magnitude of arm movement, configuration of hand) were also collected to further disentangle the representation of object shape and different action components. Information analysis and representational similarity analysis were conducted on regional neural activation patterns of the tool-preferring network. In both analyses, the bilateral lateral occipitotemporal cortex showed robust shape representations but could not effectively distinguish between tool-use actions. The frontal and precentral regions represented kinematic action components, whereas the left parietal region (in information analyses) exhibited coding of both shape and tool-use action. By teasing apart shape and action components, we found both dissociation and association of them within the tool network. Taken together, our study disentangles representations for object shape from finer tool-use action components in the tool network, revealing the potential dissociable roles different tool-preferring regions play in tool processing.

Tactile object categories can be decoded from the parietal and lateral-occipital cortices

The visual system classifies objects into categories, and distinct populations of neurons within the temporal lobe respond preferentially to objects of a given perceptual category. We can also classify the objects we recognize with the sense of touch, but less is known about the neuronal correlates underlying this cognitive function. To address this question, we performed a multivariate pattern analysis (MVPA) of functional magnetic resonance imagining (fMRI) activity to identify the cortical areas that can be used to decode the category of objects explored with the hand. We observed that tactile object category can be decoded from the activity patterns of somatosensory and parietal areas. Importantly, we found that categories can also be decoded from the lateral occipital complex (LOC), which is a multimodal region known to be related to the representation of object shape. Furthermore, a hyperalignment analysis showed that activity patterns are similar across subjects. Our results thus indicate that tactile object recognition generates category-specific patterns of activity in a multisensory area known to encode objects, and that these patterns have a similar functional organization across individuals.