Abstract
To move real objects, our hand needs to get in direct physical contact with the object. However, this is not necessarily the case when interacting with virtual objects, for example when displacing objects on tablets by swipe movements. Here, we performed two experiments to study the behavioral strategies of these movements, examining how visual information about the virtual object is mapped into a swipe that moves the object into a goal location. In the first experiment, we investigated how swiping behavior depends on whether objects were located within or outside the swiping workspace. Results show that participants do not start the swipe movement by placing their finger on the virtual object, as they do when reaching to real objects, but rather keep a systematic distance between the object location and the initial swipe location. This mismatch, which was experimentally imposed by placing the object outside the workspace, also occurred when the object was within the workspace. In the second experiment, we investigated which factors determine this mismatch by systematically manipulating the initial hand location, the location of the object and the location of the goal. Dimensionality reduction of the data showed that three factors are taken into account when participants choose the initial swipe location: the expected total movement distance, the distance between their finger on the screen and the object, and a preference not to cover the object. The weight given to each factor differed among individuals. These results delineate, for the first time, the flexibility of visuomotor associations in the virtual world.
Highlights
Interacting with objects is a fundamental characteristic of human behavior
We investigated the role of the locations of start, goal and object on the swiping strategy that we observed in the Free Task in Experiment 1
We found three factors that participants may have taken into account in deciding where to initiate their swipes: a preference to have the finger near the object, an aversion to obscure the object with the finger, and a preference to minimize the total expected movement distance
Summary
Interacting with objects is a fundamental characteristic of human behavior. We unscrew lids from jars, catch balls or play the piano. We swipe across the screen to bring objects or icons in vision that are initially not seen, or swipe to rotate virtual 3D-objects, like globes or cars, to view them from other perspectives While these types of interactions feel straightforward and intuitive, they evoke new computational challenges compared to the manipulation of real objects. In touchscreen control the visuomotor mapping is not rule-based, but freely chosen by the participant This flexible mapping bears resemblance with how humans use tools, but without the effects of object dynamics or interaction forces. We tested how the visuomotor mapping of swiping movements varied for flexible vs instructed conditions (Experiment 1) and which spatial variables (i.e., location of the object on the screen, initial hand location and goal location) are used as a reference for the swiping behavior (Experiment 2). We discuss a model, which allows us to interpret which criteria participants used to choose the initial swipe location
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
