Look first, feel faster: Prior visual information accelerates haptic material exploration

Perception during active touch essentially depends on the executed exploratory movements. Humans use different movement schemes to perceive different haptic properties, the so-called exploratory procedures (EPs). The stereotypically used EPs are normally superior to other EPs in perceiving the associated property and it has been speculated that the EPs are a means of maximising pickup of the relevant sensory information. However, EPs are not always executed identically as they vary in a number of ways. For instance, the peak force and the number of fingers used during exploration are not fixed. This chapter reviews existing findings on the exploratory movement strategies that humans use in softness perception and gives an overview on how different manners of exploration affect the performance in softness tasks. It is shown that observers adapt their movement strategies depending on variations of the stimulus value and the exact conditions of the exploratory task, and that different movement parameters, e.g. the peak exploratory forces, considerably affect performance. Overall, results suggest that humans adjust their exploratory strategies to achieve the highest levels of performance in softness discrimination.KeywordsPeak ForceDiscrimination PerformanceFinger ForceWeber FractionExploratory MovementThese keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

The neural mechanisms behind active and passive touch are not yet fully understood. Using fMRI we investigated the brain correlates of these exploratory procedures using a roughness categorization task. Participants either actively explored a surface (active touch) or the surface was moved under the participant's stationary finger (passive touch). The stimuli consisted of three different grades of sandpaper which participants were required to categorize as either coarse, medium, or fine. Exploratory procedure did not affect performance although the coarse and fine surfaces were more easily categorized than the medium surface. An initial whole brain analysis revealed activation of sensory and cognitive areas, including post-central gyrus and prefrontal cortical areas, in line with areas reported in previous studies. Our main analysis revealed greater activation during active than passive touch in the contralateral primary somatosensory region but no effect of stimulus roughness. In contrast, activation in the parietal operculum (OP) was significantly affected by stimulus roughness but not by exploration procedure. Active touch also elicited greater and more distributed brain activity compared with passive touch in areas outside the somatosensory region, possibly due to the motor component of the task. Our results reveal that different cortical areas may be involved in the processing of surface exploration and surface texture, with exploration procedures affecting activations in the primary somatosensory cortex and stimulus properties affecting relatively higher cortical areas within the somatosensory system.

In order to acquire information concerning the geometry and material of handheld objects, people tend to execute stereotypical hand movement patterns called haptic Exploratory Procedures (EPs). Manual annotation of haptic exploration trials with these EPs is a laborious task that is affected by subjectivity, attentional lapses, and viewing angle limitations. In this paper we propose an automatic EP annotation method based on position and orientation data from motion tracking sensors placed on both hands and inside a stimulus. A set of kinematic variables is computed from these data and compared to sets of predefined criteria for each of four EPs. Whenever all criteria for a specific EP are met, it is assumed that that particular hand movement pattern was performed. This method is applied to data from an experiment where blindfolded participants haptically discriminated between objects differing in hardness, roughness, volume, and weight. In order to validate the method, its output is compared to manual annotation based on video recordings of the same trials. Although mean pairwise agreement is less between human-automatic pairs than between human-human pairs (55.7% vs 74.5%), the proposed method performs much better than random annotation (2.4%). Furthermore, each EP is linked to a specific object property for which it is optimal (e.g., Lateral Motion for roughness). We found that the percentage of trials where the expected EP was found does not differ between manual and automatic annotation. For now, this method cannot yet completely replace a manual annotation procedure. However, it could be used as a starting point that can be supplemented by manual annotation.

ABSTRACTIntroduction: When children with physical impairments cannot perform hand movements for haptic exploration, they may miss opportunities to learn the properties of objects. Assistive robots may enable them to make manipulation actions. Objective: To examine the differences between using a robotic teleoperation system with haptic feedback and manual exploration when making perceptual comparisons about object properties. Accuracy and exploratory procedures (EP) using the system were compared to those in manual exploration. Method: Twenty adults without physical disabilities and ten typically developing children manipulated four pairs of objects and chose one based on size, roughness, hardness and shape. All participants completed the task with the robotic system (Tech) and manual exploration (No Tech), with the order counterbalanced. Results and conclusion: Participants performed a previously unidentified EP, “tapping”, in the Tech condition. Enclosure was not possible with the robot end effector, but tapping afforded the required perceptual information. Adults’ perceptual comparisons were always accurate and they predominantly performed the optimum EP in both conditions. Even when children performed the optimum EP with the system, their answers were less accurate than with manual exploration. Most gave the correct answer, except for hardness, which was likely due to mechanical flexibility in the robotic system.

Several studies have demonstrated that observation of a dummy or mirror-reflected hand being stroked or moving at the same time as the hidden hand evokes a feeling that the dummy hand is one’s own, such as the rubber hand illusion (RHI) and mirror visual feedback (MVF). Under these conditions, participants also report sensing the tactile stimulation applied to the fake hands, suggesting that tactile perception is modulated by visual information during the RHI and MVF. Previous studies have utilized passive stimulation conditions; however, active touch is more common in real-world settings. Therefore, we investigated whether active touch is also modulated by visual information during an MVF scenario. Twenty-three participants (13 men and 10 women; mean age ± SD: 21.6 ± 2.0 years) were required to touch a polyurethane pad with both hands synchronously, and estimate the hardness of the pad while observing the mirror reflection. When participants observed the mirror reflection of the other hand pushing a softer or harder pad, perceived hardness estimates were significantly biased toward softer or harder, respectively, even though the physical hardness of the pad remained constant. Furthermore, perceived hardness exhibited a strong correlation with finger displacement of the mirrored, but not hidden, hand. The modulatory effects on perceived hardness diminished when participants touched the pad with both hands asynchronously or with their eyes closed. Moreover, participants experienced ownership of the mirrored hand when they touched the pad with both hands synchronously but not asynchronously. These results indicate that hardness estimates were modulated by observation of the mirrored hand during synchronous touch conditions. The present study demonstrates that, similar to passive touch, active touch is also modulated by visual input.

When humans walk, it is important for them to have some measure of the distance they have traveled. Typically, many cues from different modalities are available, as humans perceive both the environment around them (for example, through vision and haptics) and their own walking. Here, we investigate the contribution of visual cues and nonvisual self-motion cues to distance reproduction when walking on a treadmill through a virtual environment by separately manipulating the speed of a treadmill belt and of the virtual environment. Using mobile eye tracking, we also investigate how our participants sampled the visual information through gaze. We show that, as predicted, both modalities affected how participants (N = 28) reproduced a distance. Participants weighed nonvisual self-motion cues more strongly than visual cues, corresponding also to their respective reliabilities, but with some interindividual variability. Those who looked more toward those parts of the visual scene that contained cues to speed and distance tended also to weigh visual information more strongly, although this correlation was nonsignificant, and participants generally directed their gaze toward visually informative areas of the scene less than expected. As measured by motion capture, participants adjusted their gait patterns to the treadmill speed but not to walked distance. In sum, we show in a naturalistic virtual environment how humans use different sensory modalities when reproducing distances and how the use of these cues differs between participants and depends on information sampling.NEW & NOTEWORTHY Combining virtual reality with treadmill walking, we measured the relative importance of visual cues and nonvisual self-motion cues for distance reproduction. Participants used both cues but put more weight on self-motion; weight on visual cues had a trend to correlate with looking at visually informative areas. Participants overshot distances, especially when self-motion was slow; they adjusted steps to self-motion cues but not to visual cues. Our work thus quantifies the multimodal contributions to distance reproduction.

Individuals with more elastic, more hydrated or smaller fingers usually show better performance in several passive touch tasks. In active touch, people use different exploratory procedures when evaluating object properties, and tune their exploratory parameters. For example, they indent stimuli to assess softness and optimize their peak forces to get relevant information. In this study, we aim to understand whether finger pad size, elasticity and hydration affect individuals' force-tuning and discrimination performance in active softness perception. Participants performed two softness tasks in two different sessions. In one session, hyaluronic acid was applied to their finger pads to soften it, in the other they received no treatment. We assessed individual elasticity and hydration values with cutometer and corneometer in each session, and measured finger pad size in three dimension by caliper. In each task, two pairs of stimuli were presented to the participants (Young's Modulus: 41.5 vs. 45.0; 28.7 vs. 31.3 kPa) who chose the softer stimulus. In the restricted task, they could apply force only up to 2 Newton, whereas there was no force limit in the unconstrained task. We found that participants with smaller finger pad size exerted less force in the restricted task and participants with more hydrated and elastic fingers exerted less force in the unconstrained task. The force-tuning disappeared in the unconstrained task when treatment was applied. These results indicate that people employ strategies according to their finger parameters and to the availability of cues whereas adaptation to treatment is likely to need longer practice.

Haptic exploration of tools: Insight into the processes that drive haptic exploration in preschool-aged children

Prior probability information and visual kinematic information are essential for action anticipation in athletes. The aims of this study were to examine how conflicting prior information influences anticipatory judgment in athletes vs. non-athletes and to explore the underlying cognitive mechanisms. The aim of Experiment 1 was to determine the moment when prior information influenced action anticipation in athletes vs. non-athletes. To that end, 17 semi-elite soccer goalkeepers and 18 non-athletes received prior information about the probability of the direction that a player on a video would kick a ball into the goal. Participants then anticipated the trajectory of the ball when the action of the player's kick on the video was truncated at the moment the foot contacted the ball (time T) or one frame (T-1; 50 ms) or two frames (T-2; 100 ms) before the foot-ball contact. The aim of Experiment 2 was to elucidate the adaptive cognitive-motor behavior exhibited by highly trained soccer players at the moment when their anticipatory performance was most influenced by prior information. Experiment 2 included 27 different semi-elite soccer players with many years of experience as a goalkeeper and 27 different non-athletes. Participants anticipated the direction of the kick when the kinematic action of the kicker at the moment the anticipatory performance of the participants was most influenced by prior information (as determined in Experiment 1) was congruent, incongruent, or neutral. Action anticipation accuracy and response time were evaluated for both experiments, whereas event-related potential components N1, N2, and P3 were assessed only in Experiment 2. The results of Experiment 1 showed that anticipatory accuracy was significantly higher among athletes than non-athletes and that anticipatory accuracy with directional information given was significantly higher than that when no prior information was given or when prior information without directional information was given (p < 0.001) for both T-1 (p's ≤ 0.034) and T-2 (p's < 0.001) occlusion points. In Experiment 2 using those two video occlusion times, the amplitude of the N1 component, which reflects selective attention to stimulus properties, was significantly higher in athletes than in non-athletes (p < 0.001). The amplitude of the N2 component, which has been associated with conflict monitoring, for the incongruent condition was significantly higher than that for both neutral (p < 0.001) and congruent (p < 0.001) conditions in athletes. Non-athletes exhibited no significant N2 amplitude differences for any prior information condition. Integrating prior information enhanced action anticipation in semi-elite soccer players, particularly 50 and 100 ms before the foot-ball contact. Semi-elite soccer players prioritized early selective attention and conflict monitoring of kinematic information, facilitating action anticipation using the prior information.

Many statistical problems contain an infinite parameter space or are analyzed as if they contained one. In a Bayesian analysis of such a problem, there is often, for one or more of several reasons, an attraction in employing an infinite measure on parameter space in the role of `prior distribution' of the parameter. The employment of such a quasi prior distribution consists in its formal substitution as the prior density in Bayes's theorem to produce a quasi posterior distribution. (We will qualify the "posterior distribution," obtained in this way, as quasi, even if it is a probability distribution with integral one, as will be assumed for the rest of this paper). The attractions referred to are that the quasi prior distribution (i) may be thought to represent "ignorance" about the parameter; (ii) may give (quasi) posterior distributions satisfying some "natural" invariance requirement; (iii) may itself satisfy some "natural" invariance requirement (the Jeffreys invariants); (iv) may give (quasi) posterior distributions on the basis of which statistical statements may be constructed which closely resemble those of classical statistics. [A separate argument for the quasi prior distribution is that, for an infinite parameter space, the class of Bayes decision functions may be complete only if the class includes those derived from quasi prior distributions (e.g. Sacks (1963)); but in this paper we will go no further than consideration of posterior distributions.] In the foundations of Bayesian statistics, associated with the names of Ramsey, de Finetti and Savage (but not Jeffreys), quasi prior distributions do not appear. When finally arrived at, subjective prior distributions are finite measures. Moreover they are, for any given person, uniquely determined, so that there should be no question of choice. Hence, matters such as the representation of ignorance, invariance and the degree of resemblance to classical statistics are not relevant. But it is possible to accept this standpoint and then to argue that the consequences of using quasi prior distributions are worth investigating if only as convenient approximations in some sense. As Welch [(1958), p. 778] reveals, such an attitude must have been implicitly adopted by those nineteenth century followers of Bayes and Laplace who ascribed a probability content to the interval between probable error limits of some astronomical or geodetic observation. (With a normal distribution of known variance taken for the observations, the implicit quasi prior distribution was, of course, uniform on the real line.) One sense of the "approximation" is straightforward. Imagine fixed data. A quasi prior distribution may be a satisfactory approximation to the actual prior distribution for this set of data if it locally resembles or simulates the actual distribution over some important compact set of parameter points determined by the data; the fact that the quasi prior distribution is not integrable on the complement of this compact set may not be important. For example, if, in sampling from a normal distribution, the quasi prior resembles the actual prior in the region where the likelihood function is not close to zero, then the approximation may be satisfactory. Thus, for given data, actual prior distribution and criterion of satisfactory approximation, the question of deciding whether a certain quasi prior distribution is employable is theoretically straightforward, the answer being based on a direct comparison of the actual posterior and quasi posterior distributions. Furthermore, even without knowledge of the data, a calculation of the prior probability (evaluated by the actual prior distribution) of obtaining data for which the quasi prior distribution is employable will provide the necessary prospective analysis. However, when no actual prior distribution is given, two courses of justification of a given quasi prior distribution are available. One, suggested by a referee, would consist of the demonstration that, for each member of a wide class of proper (and possibly actual) prior distributions, the corresponding posterior distributions are (with high probability) satisfactorily close to the quasi posterior distribution. The other, which formally avoids any decision as to when two distributions are satisfactorily close to each other, is asymptotic and would ask the question "Does there exist a sequence of proper prior distributions such that, as we proceed down the sequence, the posterior distributions converge in some sense to the quasi posterior distribution?" Jeffreys [(1957), p. 68] and Wallace [(1959), p. 873] have adopted the latter course. Wallace shows without difficulty that, roughly speaking, given a quasi prior distribution, there exists a sequence of proper prior densities whose corresponding posterior densities tend to the quasi posterior density for each fixed set of data. Reintroducing the concept of satisfactory approximation, the existence of this type of convergence, which may be called Jeffreys-Wallace convergence, assures us that, for all reasonable criteria of approximation, given the data there will be a member of the constructed sequence of prior distributions whose corresponding posterior distribution will be satisfactorily approximated by the quasi posterior distribution. Since this particular prior distribution could be the actual prior distribution of some experimenter, a certain justification of the quasi posterior distribution is thereby provided. This justification can be made separately for each set of data thereby yielding an apparently prospective justification of the quasi prior distribution itself (that is, a justification of its use for all sets of data). However, it is clear that the justification is essentially retrospective, since the prior selected may depend on the data or, in other words, the convergence (of posterior distributions to quasi posterior distribution) may not be uniform with respect to different data. In this paper, we will, like Jeffreys and Wallace, adopt the asymptotic justification course (fearing that the more statistically relevant alternative is exceedingly complex) but our justification will be genuinely prospective. To obtain this prospective asymptotic justification of a given quasi prior distribution, we may, perhaps, impose the condition that the convergence be uniform. However, even where it is possible, such a requirement is stronger than is statistically necessary. All we need is convergence in probability defined as follows. (The precise definition, applied to a special context, is given later.) There is convergence in probability to the quasi posterior distributions corresponding to a given quasi prior distribution if there exists a sequence of proper prior distributions such that, at each parameter point, the corresponding sequence of posterior densities converges in probability to the quasi posterior density. The latter use of "converges in probability" is the customary one with the Bayesian slant that the sequence of probability distributions with respect to which it is defined are the marginal distributions of data corresponding to the sequence of prior distributions. From this definition it is clear that, for all reasonable criteria of approximation, for each $\\epsilon &gt; 0$ there will be a member of the postulated sequence of prior distributions such that the prior probability of obtaining data for which the quasi posterior distribution is a satisfactory approximation to the posterior distribution corresponding to the member will exceed $1 - \\epsilon$. Since this member could be the actual prior distribution of some experimenter, a genuinely prospective justification of the given quasi prior distribution is thereby provided. Wallace's theorem shows that there is Jeffreys-Wallace convergence to all quasi prior distributions, so that all quasi prior distributions are asymptotically justified in the Jeffreys-Wallace sense. However, no equivalent theorem is available for convergence in probability, which requires demonstration for each quasi prior distribution considered. The above distinction has been drawn previously by Stone (1963), (1964) for data from the normal distribution-univariate and multivariate. The present work provides a generalisation of the case. Two analytical restrictions are made. The first, essential for the results obtained, is that the experiment generating the data should have the property of invariance under a group of transformations. (Following Fraser (1961), the supposed invariance will generally be conditional on an ancillary statistic). Many statistical problems have this group invariance structure. The second restriction (which may well be inessential) is that, given any quasi prior distribution, only sequences of prior distributions obtained by truncations of the quasi distribution to compact parameter sets will be considered. In Section 2, the group invariant structure of the experiment is outlined. In Section 3, we follow Hartigan (1964) by introducing relatively invariant prior distributions. In Section 4, convergence in probability in this context is defined and in Theorems 4.1 and 4.2 it is shown that right Haar measure (as quasi prior distribution) is, under certain conditions, sufficient and necessary (among relatively invariant prior distributions) for convergence in probability. In Section 5, general statistical applications are considered.

Oculomotor behaviors integrate sensory and prior information to overcome sensory-motor delays and noise. After much debate about this process, reliability-based integration has recently been proposed and several models of smooth pursuit now include recurrent Bayesian integration or Kalman filtering. However, there is a lack of behavioral evidence in humans supporting these theoretical predictions. Here, we independently manipulated the reliability of visual and prior information in a smooth pursuit task. Our results show that both smooth pursuit eye velocity and catch-up saccade amplitude were modulated by visual and prior information reliability. We interpret these findings as the continuous reliability-based integration of a short-term memory of target motion with visual information, which support modeling work. Furthermore, we suggest that saccadic and pursuit systems share this short-term memory. We propose that this short-term memory of target motion is quickly built and continuously updated, and constitutes a general building block present in all sensorimotor systems.

In this paper, we present the results of a pilot study that examines whether restricting how people can explore objects haptically effects the object attributes they notice and the efficiency with which they can perform a simple sorting task. 25 observers were each randomly assigned to one of five exploration conditions: two hands (the control), one hand, thumb/forefinger, one finger, or probe. All observers performed a series of two-bin sorts. Stimuli were eight multi-propertied cubes which could be divided into two equal bins according to three properties: size, texture, and compliance. Preliminary results indicate that the restrictions on manual exploration we imposed affected both the exploratory procedures observers chose to use and the efficiency with which they could perform the task. Haptic interface designs inevitably restrict the exploratory procedures available to the user. This study attempts to determine the cost of these restrictions on the efficiency with which a user can explore multi-propertied objects in a virtual or telepresence environment.

When haptically exploring softness, humans use higher peak forces when indenting harder versus softer objects. Here, we investigated the influence of different channels and types of prior knowledge on initial peak forces. Participants explored two stimuli (hard vs. soft) and judged which was softer. In Experiment 1 participants received either semantic (the words "hard" and "soft"), visual (video of indentation), or prior information from recurring presentation (blocks of harder or softer pairs only). In a control condition no prior information was given (randomized presentation). In the recurring condition participants used higher initial forces when exploring harder stimuli. No effects were found in control and semantic conditions. With visual prior information, participants used less force for harder objects. We speculate that these findings reflect differences between implicit knowledge induced by recurring presentation and explicit knowledge induced by visual and semantic information. To test this hypothesis, we investigated whether explicit prior information interferes with implicit information in Experiment 2. Two groups of participants discriminated softness of harder or softer stimuli in two conditions (blocked and randomized). The interference group received additional explicit information during the blocked condition; the implicit-only group did not. Implicit prior information was only used for force adaptation when no additional explicit information was given, whereas explicit interfered with movement adaptation. The integration of prior knowledge only seems possible when implicit prior knowledge is induced-not with explicit knowledge.

Spatial orientation strongly relies on visual and whole-body information available while moving through space. As virtual environments allow to isolate the contribution of visual information from the contribution of whole-body information, they are an attractive methodological means to investigate the role of visual information for spatial orientation. Using an elementary spatial orientation task (triangle completion) in a simple virtual environment we studied the effect of amount of simultaneously available visual information (geometric field of view) and triangle layout on the integration and uptake of directional (turn) and distance information under visual simulation conditions. While the amount of simultaneously available visual information had no effect on homing errors, triangle layout substantially affected homing errors. Further analysis of the observed homing errors by means of an Encoding Error Model revealed that subjects navigating under visual simulation conditions had problems in accurately taking up and representing directional (turn) information, an effect which was not observed in experiments reported in the literature from similar whole-body conditions. Implications and prospects for investigating spatial orientation by means of virtual environments are discussed considering the present experiments as well as other work on spatial cognition using virtual environments.

Both visual and haptic softness perception have recently been shown to have multiple dimensions, such as deformability, granularity, fluidity, surface softness, and roughness. During haptic exploration, people adjust their hand motions (exploratory procedures, EPs) based on the material qualities of the object and the particular information they intend to acquire. Some of these EPs are also shown to be associated with perceived softness dimensions, for example, stroking a silk blouse or applying pressure to a pillow. Here, we aimed to investigate whether we can manipulate observers' judgments about softness attributes through exposure to videos of others performing various EPs on everyday soft materials. In two experiments, participants watched two videos of the same material: one with a corresponding EP and the other without correspondence; then, they judged these materials based on 12 softness-related adjectives (semantic differentiation method). The results of the second experiment suggested that when the EP is congruent with the dimension from which the material is chosen, the ratings for the adjectives from the same dimension are higher than the incongruent EP. This study provides evidence that participants can assess material properties from optic and mechanical cues without needing haptic signals. Additionally, our findings indicate that manipulating the hand motion can selectively facilitate material-related judgments.