Texture Cues Research Articles

Scale-Aware Saliency for Application to Frame Rate Upconversion

Our understanding of human visual perception has been paramount in the development of tools for digital video processing. For this reason, saliency detection, i.e., the determination of visual importance in a scene, has come to the forefront in recent literature. In the proposed work, a new method for scale-aware saliency detection is introduced. Scale determination is afforded through a scale-space model utilizing color and texture cues. Scale information is fed back to a discriminant saliency engine by automatically tuning center-surround parameters through a soft weighting. Excellent results are demonstrated for the proposed method through its performance against a database of measured human fixations. Further evidence of the proposed algorithm's performance is demonstrated through an application to frame rate upconversion. The ability of the algorithm to detect salient objects at multiple scales allows for class-leading performance both objectively, in terms of peak signal-to-noise ratio/structural similarity index, and subjectively. Finally, the need for operator tuning of saliency parameters is dramatically reduced by the inclusion of scale information. The proposed method is well suited for any application requiring automatic saliency determination for images or video.

ASSESSING THE SHAPES AND SPEECH SOUNDS THAT CONSUMERS ASSOCIATE WITH DIFFERENT KINDS OF CHOCOLATE

ABSTRACTReliable crossmodal correspondences have been demonstrated between dark and mint chocolates with angular shapes and sharper‐sounding speech sounds on one hand, and between milk chocolate with organic shapes and rounder‐sounding speech sounds on the other. In the present study, a panel of consumers was presented with four different chocolates: two mints (solid and fondant), one dark and one milk. They either tasted (Experiment 1) or simply imagined tasting (Experiment 2) the chocolates and indicated whether the perceived flavor matched one or other of the items (nonsense words or simple outline shapes) anchoring various line scales by marking a point along each scale. Dark and solid mint chocolates were more angular‐shaped and associated with sharp meaningless speech sounds (e.g., “tuki” and “takete”). Mint fondant, by contrast, was considered less angular and more pleasant than dark or solid mints, while milk chocolate was more pleasant and strongly associated with organic shapes and rounded speech sounds (e.g., “lula” and “maluma”). These results corroborate and build upon recent findings concerning sound symbolism in the taste and flavor domain by highlighting the fact that oral‐somatosensory textural cues play an important role in determining the crossmodal correspondences that regular consumers have for foodstuffs such as chocolate.PRACTICAL APPLICATIONSWhether taste or texture, or some combination of the two sensory attributes, is responsible for the systematic associations between various chocolates and certain angular/sharp or organic shapes and speech sounds remains unclear. The results of the present study demonstrate that both attributes play an important role in determining consumers' crossmodal correspondences. Thus, when conducting research on various food qualities, these two factors should be carefully considered and/or systematically controlled. Additionally, the fact that these results held across different testing protocols (laboratory‐based vs. online) confirms the potential utility of internet‐based food testing for quantitatively and, importantly, qualitatively valuable/sound data collection (at least for foods that consumers are familiar with). The results of experiments such as those reported here can, in the future, be used to provide insights regarding the speech sounds and abstract imagery that should be associated with specific oral‐somatosensory and taste/flavor attributes in commercial food products. Such results may be used to develop abstract imagery for product packaging and/or brand names/logos that more effectively capture the shape/sound symbolic properties of the food concerned.

Integration of disparity and texture cues to slant in adolescents with an autism spectrum disorder

Integration of disparity and texture cues to slant in adolescents with an autism spectrum disorder

3D Human model adaptation by frame selection and shape–texture optimization

We present a novel approach for 3D human body shape model adaptation to a sequence of multi-view images, given an initial shape model and initial pose sequence. In a first step, the most informative frames are determined by optimization of an objective function that maximizes a shape–texture likelihood function and a pose diversity criterion (i.e. the model surface area that lies close to the occluding contours), in the selected frames. Thereafter, a batch-mode optimization is performed of the underlying shape- and pose-parameters, by means of an objective function that includes both contour and texture cues over the selected multi-view frames. Using above approach, we implement automatic pose and shape estimation using a three-step procedure: first, we recover initial poses over a sequence using an initial (generic) body model. Both model and poses then serve as input to the above mentioned adaptation process. Finally, a more accurate pose recovery is obtained by means of the adapted model. We demonstrate the effectiveness of our frame selection, model adaptation and integrated pose and shape recovery procedure in experiments using both challenging outdoor data and the HumanEva data set.

Visual Effects of Haptic Feedback Are Large but Local

Vision generally provides reliable predictions for touch and motor-control, but some classes of stimuli evoke visual illusions. Using haptic feedback on virtual 3-D surfaces, we tested the function of touch in such cases. Our experiments show that in the perception of 3-D shapes from texture cues, haptic information can dominate vision in some cases, changing percepts qualitatively from convex to concave and concave to slant. The effects take time to develop, do not outlive the cessation of the feedback, are attenuated by distance, and drastically reduced by gaps in the surface. These dynamic shifts in qualitative perceived shapes could be invaluable in neural investigations that test whether haptic feedback modifies selective activation of neurons or changes the shape-tuning of neurons responsible for percepts of 3-D shapes.

Adaptation and the perception of facial age

We examined how the perceived age of adult faces is affected by adaptation to younger or older adult faces. Observers viewed images of a synthetic male face simulating ageing over a modelled range from 15 to 65 years. Age was varied by changing shape cues or textural cues. Age level was varied in a staircase to find the observer's subjective category boundary between “old” and “young”. These boundaries were strongly biased by adaptation to the young or old face, with significant aftereffects induced by either shape or textural cues. A further experiment demonstrated comparable aftereffects for photorealistic images of average older or younger adult faces, and found that aftereffects showed some selectivity for a change in gender but also strongly transferred across gender. This transfer shows that adaptation can adjust to the attribute of age somewhat independently of other facial attributes. These findings suggest that perceived age, like many other natural facial dimensions, is highly susceptible to adaptation, and that this adaptation can be carried by both the structural and textural changes that normally accompany facial ageing.

Facial masculinity is a cue to women’s dominance

Although there is compelling evidence for associations between facial masculinity and indices of dominance in men, comparatively few studies have tested for corresponding associations in women. Here we found that (1) ratings of women’s facial masculinity were correlated with their scores on a dominance questionnaire, and (2) prototypes with the average facial characteristics of women with high scores on the dominance questionnaire were judged to be more masculine than prototypes with the average facial characteristics of women with low scores, even when color and texture cues were kept constant to control for effects of makeup use. These findings suggest an association between facial masculinity and dominance in women, complementing prior work reporting that masculine women are perceived to be more dominant than their relatively feminine peers.

Fusion of disparity and texture cues to slant is not mandatory in children

Fusion of disparity and texture cues to slant is not mandatory in children

Haptic learning disambiguates but does not override texture cues to 3-D shape

Haptic learning disambiguates but does not override texture cues to 3-D shape

Labeling and family resemblance in the discrimination of polymorphous categories by pigeons

Two experiments examined whether pigeons discriminate polymorphous categories on the basis of a single highly predictive feature or overall similarity. In the first experiment, pigeons were trained to discriminate between categories of photographs of complex real objects. Within these pictures, single features had been manipulated to produce a highly salient texture cue. Either the picture or the texture provided a reliable cue for discrimination during training, but in probe tests, the picture and texture cues were put into conflict. Some pigeons showed a significant tendency to discriminate on the basis of the picture cue (overall similarity or family resemblance), whereas others appeared to rely on the manipulated texture cue. The second experiment used artificial polymorphous categories in which one dimension of the stimulus provided a completely reliable cue to category membership, whereas three other dimensions provided cues that were individually unreliable but collectively provided a completely reliable basis for discrimination. Most pigeons came under the control of the reliable cue rather than the unreliable cues. A minority, however, came under the control of single dimensions from the unreliable set. We conclude that cue salience can be more important than cue reliability in determining what features will control behavior when multiple cues are available.

Adaptive hybrid likelihood model for visual tracking based on Gaussian particle filter

We present a new scheme based on multiple-cue integration for visual tracking within a Gaussian particle filter framework. The proposed method integrates the color, shape, and texture cues of an object to construct a hybrid likelihood model. During the measurement step, the likelihood model can be switched adaptively according to environmental changes, which improves the object representation to deal with the complex disturbances, such as appearance changes, partial occlusions, and significant clutter. Moreover, the confidence weights of the cues are adjusted online through the estimation using a particle filter, which ensures the tracking accuracy and reliability. Experiments are conducted on several real video sequences, and the results demonstrate that the proposed method can effectively track objects in complex scenarios. Compared with previous similar approaches through some quantitative and qualitative evaluations, the proposed method performs better in terms of tracking robustness and precision.

Texture and haptic cues in slant discrimination: Measuring the effect of texture type on cue combination

Texture and haptic cues in slant discrimination: Measuring the effect of texture type on cue combination

Nonlinear integration of texture and shading cues on a slant discrimination task

Most studies of visual cue integration have found that observers' data can be modeled by a linear cue combination rule. We conducted an experiment in which, intuitively, a linear cue combination rule should not provide a satisfying model of observers' data. The experiment used a surface defined by texture and shading cues. At a coarse-level of detail, the surface was planar, but it also contained deformations resembling random bumps or waves. Subjects performed a slant discrimination task—they judged whether comparison surfaces were more or less slanted than a standard surface—in three conditions, namely when surfaces were defined by the texture cue alone, by the shading cue alone, or by both cues. The results indicate that when both cues were available subjects' discrimination thresholds were significantly lower than predicted by a linear model. We propose that the texture and shading cues to slant are each highly ambiguous when there is great uncertainty about the shape of the surface. When only a single cue is available, observers find it difficult to estimate surface shape (due to its irregularities) and, thus, find it difficult to interpret that cue for the purpose of judging slant. When both cues are available, however, estimates of surface shape are more accurate, thereby making it easier to interpret the cues as indicators of slant. The results are consistent with the hypothesis that observers estimate surface shape, and use this estimate when judging surface slant.

Perception of 3-D carved surfaces from monocular texture cues

Perception of 3-D carved surfaces from monocular texture cues

The efficiency of the use of feedback in perceptual learning

In vision science the currently most popular models for depth perception are weak fusion models in which the final depth estimate results from a weighted average of the independent depth estimates obtained from each cue [2]. In these models a more reliable cue has a larger weight in the combined estimate. Furthermore, recent studies report that human observes combine depth cues as to obtain the minimal variance unbiased estimator of depth [1]. Different texture types can elicit different performance in a slant discrimination task [3]. In the present study we ask whether the reliability-sensitive weighting is observed in slant discrimination based on texture and haptic cues, when interchanging the texture type on the stimuli (see figure below). In the first experiment, with texture and haptic cues depicting slant consistently, we tested a minimal variance unbiased estimator of slant. That is, whether performance for the haptic and texture cues combined followed:

Concrete Column Recognition in Images and Videos

The automated detection of structural elements (e.g., columns and beams) from visual data can be used to facilitate many construction and maintenance applications. The research in this area is under initial investigation. The existing methods solely rely on color and texture information, which makes them unable to identify each structural element if these elements connect each other and are made of the same material. The paper presents a novel method of automated concrete column detection from visual data. The method overcomes the limitation by combining columns’ boundary information with their color and texture cues. It starts from recognizing long vertical lines in an image/video frame through edge detection and Hough transform. The bounding rectangle for each pair of lines is then constructed. When the rectangle resembles the shape of a column and the color and texture contained in the pair of lines are matched with one of the concrete samples in knowledge base, a concrete column surface is assumed to be located. This way, one concrete column in images/videos is detected. The method was tested using real images/videos. The results are compared with the manual detection ones to indicate the method’s validity.

Active Contour Model Based on Salient Boundary Point Image for Object Contour Detection in Natural Image

This paper combines the LBP operator and the active contour model. It introduces a salient gradient vector flow snake (SGVF snake), based on a novel edge map generated from the salient boundary point image (SBP image). The MDGVM criterion process helps to reduce feature detail and background noise as well as retaining the salient boundary points. The resultant SBP image as an edge map gives powerful support to the SGVF snake because of the inherent combination of the intensity, gradient and texture cues. Experiments prove that the MDGVM process has high efficiency in reducing outliers and the SGVF snake is a large improvement over the GVF snake for contour detection, especially in natural images with low contrast and small texture background.

Cue dynamics underlying rapid detection of animals in natural scenes

Humans are known to be good at rapidly detecting animals in natural scenes. Evoked potential studies indicate that the corresponding neural signals can emerge in the brain within 150 msec of stimulus onset (S. Thorpe, D. Fize, & C. Marlot, 1996) and eye movements toward animal targets can be initiated in roughly the same timeframe (H. Kirchner & S. J. Thorpe, 2006). Given the speed of this discrimination, it has been suggested that the underlying visual mechanisms must be relatively simple and feedforward, but in fact little is known about these mechanisms. A key step is to understand the visual cues upon which these mechanisms rely. Here we investigate the role and dynamics of four potential cues: two-dimensional boundary shape, texture, luminance, and color. Results suggest that the fastest mechanisms underlying animal detection in natural scenes use shape as a principal discriminative cue, while somewhat slower mechanisms integrate these rapidly computed shape cues with image texture cues. Consistent with prior studies, we find little role for luminance and color cues throughout the time course of visual processing, even though information relevant to the task is available in these signals.

Kernel-based Tracking Based on Adaptive Fusion of Multiple Cues

A scheme is proposed to integrate multiple cues with kernel tracking by adaptive fusion to improve the robustness of object tracking in the time-variant scenario.The tracked object is represented by a set of submodels of each cue,and then the multiple cues are combined by linear weighting to realize kernel-based tracking.According to the discriminability of each cue between target and background,measured by Fisher rule,an adaptive mechanism is presented to update the cue weight.Furthermore,a selective submodel update strategy is utilized to alleviate the model drift.In experiments,we employ color cue and local binary pattern(LBP)texture cue to implement the scheme,and the results demonstrate the effectiveness of the proposed method in several real sequences testing.

Consumer freshness perceptions of breads, biscuits and cakes

Freshness is a holistic attribute of a food product, with meaning that most often includes how recently produced or harvested a food currently is, to what extent it has been preserved, and it is the opposite of stale or spoiled. The sensory properties of foods contribute very significantly to consumer freshness perceptions, but these properties are not easily identified and described as they will vary considerably between different product types. The aim of this investigation was to improve the understanding of the sensory properties that contribute to the perceived freshness of different baked product types, including breads, biscuits and cakes. Two separate consumer studies were conducted. In the first study, consumers evaluated product freshness based on appearance, flavour and texture and in the second, consumers only evaluated product odour. Qualitative information detailing the product characteristics that consumers most associated with freshness was collected following assessment at each study. In order to better understand the sensory properties associated with product freshness, a trained sensory panel carried out descriptive analysis with the baked products, using a defined vocabulary. Individual differences in perceived product freshness were represented on two-dimensional Freshness Maps generated using Principal Component Analysis. Results showed the relative similarities and differences between the product types on the basis of individual freshness perceptions. Consumers used similar terms to describe product freshness, but different terms were associated with specific product types. Relationships between consumer freshness ratings and descriptive analysis showed attributes that were positively and negatively associated with freshness for each product type when evaluated following either consumption, or by odour only. It was found that sensory characteristics perceived to be fresh in one product type was not necessarily fresh in another. Comparisons between consumer and sensory vocabularies showed specific appearance, odour, taste and textural cues that consumers associated with product freshness were used to differentiate the same products cognitively.