Continuous Visual Information Research Articles

Utilization of Image/Force/Tactile Sensor Data for Object-Shape-Oriented Manipulation: Wiping Objects With Turning Back Motions and Occlusion

There has been an increasing demand for housework robots to handle various objects. It is, however, difficult to achieve object-shape-oriented tasks in conventional research owing to the requirement for dealing with multiple surfaces, invisible area, and occlusion; moreover, robots must perceive shapes and adjust movements even if they cannot be seen directly. Humans usually tackle questions by integrating several sensory information; inspired by this perception mechanism of humans, in this study, we considered the effective utilization of image/force/tactile data in constructing a multimodal deep neural networks (DNN) model for the shape of an object perception and motion generation. As an example, we constructed a robot to wipe around the outside of objects that are imitating light shades. The wiping motions include the moment when the hands of the robot must be away from the surface as well as the turning directions required to wipe the next surface, even though some parts of the surfaces, such as the backside or parts occluded by the arm of the robot, may not be seen directly. If DNN model uses continuous visual information, it is badly influenced by the occluded images. Hence, the best-performing DNN model is the one that uses an image of the initial time-step to approximately perceive the shape and size and then generate motions by integrating the perception and sense of tactile and force. We conclude that the effective approach to object-shape-oriented manipulation is to initially utilize image to outline the target shape and, thereafter, to use force and tactile to understand concrete features while performing tasks.

A brief glimpse at a haptic target is sufficient for multisensory integration in reaching movements

Goal-directed aiming movements toward visuo-haptic targets (i.e., seen and handheld targets) are generally more precise than those toward visual only or haptic only targets. This multisensory advantage stems from a continuous inflow of haptic and visual target information during the movement planning and execution phases. However, in everyday life, multisensory movements often occur without the support of continuous visual information. Here we investigated whether and to what extent limiting visual information to the initial stage of the action still leads to a multisensory advantage. Participants were asked to reach a handheld target while vision was briefly provided during the movement planning phase (50 ms, 100 ms, 200 ms of vision before movement onset), or during the planning and early execution phases (400 ms of vision), or during the entire movement. Additional conditions were performed in which only haptic target information was provided, or, only vision was provided either briefly (50 ms, 100 ms, 200 ms, 400 ms) or throughout the entire movement. Results showed that 50 ms of vision before movement onset were sufficient to trigger a direction-specific visuo-haptic integration process that increased endpoint precision. We conclude that, when a continuous support of vision is not available, endpoint precision is determined by the less recent, but most reliable multisensory information rather than by the latest unisensory (haptic) inputs.

The Visual Mechanisms of Seeing in Experiencing the Interior

This paper discusses the visual mechanisms of seeing and their significance in experiencing an interior space. The discussion investigates what the observers can obtain from seeing activities. The aim is to emphasise on the role of seeing as a way of constructing the relation between human and the interior environment. The paper explores the mechanisms of seeing by focusing on two different ways, which are seeing in a static position from a point of observation, and seeing while moving through a path of observation. The exploration in a hospital setting finds out that seeing from a point of observation gave a visual range determined by the body's shaft motion, head motion, and eye movement. This way of seeing produces visual information on interior space, which consists of vertical and horizontal fields. Seeing while moving will create a path of observation that gave an optical flow containing dynamic and continuous visual information. The understanding of seeing mechanisms in interior environment can generate a design with better human-interior relation.

Visual control of steering in curve driving.

This pair of studies investigated steering in the absence of continuous visual information. In a driving simulator, participants steered a curving path that was displayed either continuously or intermittently. Optic flow conditions were manipulated to alter the nature of the heading information with respect to the path being steered. Removing or biasing heading information had little effect on steering even during long and frequent path occlusions as long as turn rate was available. This demonstrates that participants can use intermittent views of the path to plan their steering actions and optic flow to accurately update vehicle turns with respect to that path.

An interference model of visual working memory.

The article introduces an interference model of working memory for information in a continuous similarity space, such as the features of visual objects. The model incorporates the following assumptions: (a) Probability of retrieval is determined by the relative activation of each retrieval candidate at the time of retrieval; (b) activation comes from 3 sources in memory: cue-based retrieval using context cues, context-independent memory for relevant contents, and noise; (c) 1 memory object and its context can be held in the focus of attention, where it is represented with higher precision, and partly shielded against interference. The model was fit to data from 4 continuous-reproduction experiments testing working memory for colors or orientations. The experiments involved variations of set size, kind of context cues, precueing, and retro-cueing of the to-be-tested item. The interference model fit the data better than 2 competing models, the Slot-Averaging model and the Variable-Precision resource model. The interference model also fared well in comparison to several new models incorporating alternative theoretical assumptions. The experiments confirm 3 novel predictions of the interference model: (a) Nontargets intrude in recall to the extent that they are close to the target in context space; (b) similarity between target and nontarget features improves recall, and (c) precueing-but not retro-cueing-the target substantially reduces the set-size effect. The success of the interference model shows that working memory for continuous visual information works according to the same principles as working memory for more discrete (e.g., verbal) contents. Data and model codes are available at https://osf.io/wgqd5/. (PsycINFO Database Record

Hand interception of occluded motion in humans: a test of model-based vs. on-line control.

Two control schemes have been hypothesized for the manual interception of fast visual targets. In the model-free on-line control, extrapolation of target motion is based on continuous visual information, without resorting to physical models. In the model-based control, instead, a prior model of target motion predicts the future spatiotemporal trajectory. To distinguish between the two hypotheses in the case of projectile motion, we asked participants to hit a ball that rolled down an incline at 0.2 g and then fell in air at 1 g along a parabola. By varying starting position, ball velocity and trajectory differed between trials. Motion on the incline was always visible, whereas parabolic motion was either visible or occluded. We found that participants were equally successful at hitting the falling ball in both visible and occluded conditions. Moreover, in different trials the intersection points were distributed along the parabolic trajectories of the ball, indicating that subjects were able to extrapolate an extended segment of the target trajectory. Remarkably, this trend was observed even at the very first repetition of movements. These results are consistent with the hypothesis of model-based control, but not with on-line control. Indeed, ball path and speed during the occlusion could not be extrapolated solely from the kinematic information obtained during the preceding visible phase. The only way to extrapolate ball motion correctly during the occlusion was to assume that the ball would fall under gravity and air drag when hidden from view. Such an assumption had to be derived from prior experience.

Knowledge of Performance is Insufficient for Implicit Visuomotor Rotation Adaptation

ABSTRACT The ability to adapt is a fundamental and vital characteristic of the motor system. The authors altered the visual environment and focused on the ability of humans to adapt to a rotated environment in a reaching task, in the absence of continuous visual information about their hand location. Subjects could not see their arm but were provided with post trial knowledge of performance depicting hand path from movement onset to final position. Subjects failed to adapt under these conditions. The authors sought to find out whether the lack of adaptation is related to the number of target directions presented in the task, and planned 2 protocols in which subjects were gradually exposed to 22.5° visuomotor rotation. These protocols differed only in the number of target directions: 8 and 4 targets. The authors found that subjects had difficulty adapting without the existence of continuous visual feedback of their performance regardless of the number of targets presented in task. In the 4-target protocol, some of the subjects noticed the rotation and explicitly aimed to the correct direction. The results suggest that real-time feedback is required for motor adaptation to visual rotation during reaching movements.

Prospective versus predictive control in timing of hitting a falling ball

Debate exists as to whether humans use prospective or predictive control to intercept an object falling under gravity (Baurès et al. in Vis Res 47:2982-2991, 2007; Zago et al. in Vis Res 48:1532-1538, 2008). Prospective control involves using continuous information to regulate action. τ, the ratio of the size of the gap to the rate of gap closure, has been proposed as the information used in guiding interceptive actions prospectively (Lee in Ecol Psychol 10:221-250, 1998). This form of control is expected to generate movement modulation, where variability decreases over the course of an action based upon more accurate timing information. In contrast, predictive control assumes that a pre-programmed movement is triggered at an appropriate criterion timing variable. For a falling object it is commonly argued that an internal model of gravitational acceleration is used to predict the motion of the object and determine movement initiation. This form of control predicts fixed duration movements initiated at consistent time-to-contact (TTC), either across conditions (constant criterion operational timing) or within conditions (variable criterion operational timing). The current study sought to test predictive and prospective control hypotheses by disrupting continuous visual information of a falling ball and examining consistency in movement initiation and duration, and evidence for movement modulation. Participants (n=12) batted a ball dropped from three different heights (1, 1.3 and 1.5m), under both full-vision and partial occlusion conditions. In the occlusion condition, only the initial ball drop and the final 200ms of ball flight to the interception point could be observed. The initiation of the swing did not occur at a consistent TTC, τ, or any other timing variable across drop heights, in contrast with previous research. However, movement onset was not impacted by occluding the ball flight for 280-380ms. This finding indicates that humans did not need to be continuously coupled to vision of the ball to initiate the swing accurately, but instead could use predictive control based on acceleration timing information (TTC2). However, other results provide evidence for movement modulation, a characteristic of prospective control. Strong correlations between movement initiation and duration and reduced timing variability from swing onset to arrival at the interception point, both support compensatory variability. An analysis of modulation within the swing revealed that early in the swing, the movement acceleration was strongly correlated to the required mean velocity at swing onset and that later in the swing, the movement acceleration was again strongly correlated with the current required mean velocity. Rather than a consistent movement initiated at the same time, these findings show that the swing was variable but modulated for meeting the demands of each trial. A prospective model of coupling τ (bat-ball) with τ (ball-target) was found to provide a very strong linear fit for an average of 69% of the movement duration. These findings provide evidence for predictive control based on TTC2 information in initiating the swing and prospective control based on τ in guiding the bat to intercept the ball.

Corticospinal excitability during preparation for an anticipatory action is modulated by the availability of visual information

To intercept rapidly moving objects, people must predict the right time to initiate their actions. The timing of movement initiation in interceptions is thought to be determined when a perceptual variable specifying time to contact reaches a criterion value. If a response needs to be aborted, the performer must make a decision before this moment. It has been recently shown that the minimal time to suppress an anticipatory action takes longer during motion extrapolation than during continuous visual information. In experiment 1, we sought to determine whether or not the availability of visual information would 1) affect the latency to inhibit an anticipatory action, and 2) modulate the level of excitability in the motor cortex (M1). The behavioral results showed that the absence of visual information prolonged the latency to stop the movement as previously reported. The neurophysiological data indicated that corticospinal excitability levels were affected by the availability of visual information. In experiment 2, we sought to verify whether corticospinal excitability levels would also differ between the two visual conditions when the task did not involve response suppression. The results of experiment 2 indicated that excitability levels did not differ between visual conditions. Overall, our findings indicated that the buildup of motor activation can also play a role in determining different latencies to inhibit an anticipatory action. They also suggest that the buildup of motor activation in the corticospinal pathways can be strategically modulated to the requirements of the task during continuous visual information.

Delayed inhibition of an anticipatory action during motion extrapolation

BackgroundContinuous visual information is important for movement initiation in a variety of motor tasks. However, even in the absence of visual information people are able to initiate their responses by using motion extrapolation processes. Initiation of actions based on these cognitive processes, however, can demand more attentional resources than that required in situations in which visual information is uninterrupted. In the experiment reported we sought to determine whether the absence of visual information would affect the latency to inhibit an anticipatory action.MethodsThe participants performed an anticipatory timing task where they were instructed to move in synchrony with the arrival of a moving object at a determined contact point. On 50% of the trials, a stop sign appeared on the screen and it served as a signal for the participants to halt their movements. They performed the anticipatory task under two different viewing conditions: Full-View (uninterrupted) and Occluded-View (occlusion of the last 500 ms prior to the arrival at the contact point).ResultsThe results indicated that the absence of visual information prolonged the latency to suppress the anticipatory movement.ConclusionWe suggest that the absence of visual information requires additional cortical processing that creates competing demand for neural resources. Reduced neural resources potentially causes increased reaction time to the inhibitory input or increased time estimation variability, which in combination would account for prolonged latency.

Human vision focuses on information relevant to a task, to the detriment of information that is not relevant

Glover offers an account for why some pictorial illusions influence early but not late phases of an action. His proposed corrective control process, however, functions normally in the absence of continuous visual information, suggesting that the stimulus is registered veridically prior to action onset. Here I consider an alternative account, based on differing informational constraints of behaviors (and phases of behaviors).

Asymmetries in Coupling Dynamics of Perception and Action.

The influence of information-based dynamics on coordination dynamics of rhythmic movement was examined with special reference to the expression of asymmetries. In Experiment 1, right-handed subjects performed unimanual, rhythmical movements in coordination with either a discrete or continuous visual display. The right hand-visual display system defined a more stable perception-action collective than the left, particularly when continuous visual information was available. In Experiment 2, the same subjects performed rhythmic bimanual movements in coordination with a continuous visual display. The action collective was inherently more stable than the perception-action collective, although similar patterns were observed at both levels. Importantly, the dynamics of the perception-action collective impinged upon the dynamics of the action collective in terms of stability. Asymmetries remained evident between limbs in the bimanual preparations, with the left hand exhibiting greater limit-cycle variability and also a tendency to more often effect transitions at the action couple. Features of dynamical models that capture characteristics of manual asymmetries are discussed.

Continuous visual information may be important after all: A failure to replicate Thomson (1983).

Continuous visual information may be important after all: A failure to replicate Thomson (1983).

Continuous visual information may be important after all: a failure to replicate Thomson (1983).

The purpose of the two experiments reported here was to replicate previous research (Thomson, 1983) which suggests that visual information useful in the control of movement persists for up to 8 s after visual occlusion. Contrary to other findings (Thomson, 1980, 1983), little evidence was found for an 8-s visual representation of the environmental layout, indicating there is no substitute for continuous visual information in the control of movement. Methodological and statistical problems with Thomson's work are discussed.

Descriptive dynamic models for goal-directed arm movements

Human goal directed arm movements are studied by making use of mathematical models. Such models that describe movements as outputs on input step tracking tasks, consist of a linear differential equation. The approach is based on control engineering concepts and finds its origin in the field of man-machine systems. Starting from two existing models for continuous tracking tasks, five related models for step tracking are investigated. Two types of experiments have been executed with a manipulator having the characteristics of a lightly damped mass and a controlled system characterised by a simple gain factor. It was found that a simple feedback model with an additional delay time outside the control loop was very satisfactory for the four subjects engaged in the experiments, for forward and backward movements and for movements with continuous visual information as well as for movements with interrupted visual information. An extension of the model with a feedforward element was found to be rather irrelevant for the experimental conditions applied.