Abstract
When we hold an object while looking at it, estimates from visual and haptic cues to size are combined in a statistically optimal fashion, whereby the “weight” given to each signal reflects their relative reliabilities. This allows object properties to be estimated more precisely than would otherwise be possible. Tools such as pliers and tongs systematically perturb the mapping between object size and the hand opening. This could complicate visual-haptic integration because it may alter the reliability of the haptic signal, thereby disrupting the determination of appropriate signal weights. To investigate this we first measured the reliability of haptic size estimates made with virtual pliers-like tools (created using a stereoscopic display and force-feedback robots) with different “gains” between hand opening and object size. Haptic reliability in tool use was straightforwardly determined by a combination of sensitivity to changes in hand opening and the effects of tool geometry. The precise pattern of sensitivity to hand opening, which violated Weber's law, meant that haptic reliability changed with tool gain. We then examined whether the visuo-motor system accounts for these reliability changes. We measured the weight given to visual and haptic stimuli when both were available, again with different tool gains, by measuring the perceived size of stimuli in which visual and haptic sizes were varied independently. The weight given to each sensory cue changed with tool gain in a manner that closely resembled the predictions of optimal sensory integration. The results are consistent with the idea that different tool geometries are modeled by the brain, allowing it to calculate not only the distal properties of objects felt with tools, but also the certainty with which those properties are known. These findings highlight the flexibility of human sensory integration and tool-use, and potentially provide an approach for optimizing the design of visual-haptic devices.
Highlights
When humans manipulate objects with their hands while looking at them, visual and haptic size information is integrated in a manner that is highly consistent with statistically optimal models of sensory integration (Ernst and Banks, 2002; Gepshtein and Banks, 2003; Helbig and Ernst, 2007a)
Relative to the 1:1 tool, perceived size was on average 2–3 mm larger with the 0.7:1 tool, which magnified the hand opening relative to object size, and a similar amount smaller with the 1.4:1 tool, which reduced the hand opening. These are relatively small effects, given the variation in actual hand opening across conditions. This result suggests that haptic size estimates when using a tool are rescaled to account for the tool gain, but that this “compensation” is incomplete
In Experiment 2, we found that the brain took account of these changes in haptic reliability introduced by different tool geometries, and adjusted the weighting of haptic and visual signals in a manner broadly consistent with statistically optimal sensory integration
Summary
When humans manipulate objects with their hands while looking at them, visual and haptic size information is integrated in a manner that is highly consistent with statistically optimal models of sensory integration (Ernst and Banks, 2002; Gepshtein and Banks, 2003; Helbig and Ernst, 2007a). Such models describe how, under the assumptions that estimates from each sense are on average unbiased, and their noises are independent and Gaussian distributed, the minimum-variance unbiased estimate (SVH; Equation 1) is a weighted sum of visual and haptic estimates (SV , SH) where the weight given to each signal (wV , wH) is proportional to the inverse of its variance (Equation 2; for a review see Oruç et al, 2003). The reliability of visual estimates varies substantially with variations in any number of “geometrical” properties of the stimulus including the type of surface texture, the object’s orientation with respect to the viewer, and viewing distance (Knill, 1998a,b; Gepshtein and Banks, 2003; Knill and Saunders, 2003; Hillis et al, 2004; Keefe et al, 2011)
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
