Cursor Trajectories Research Articles

Complexity of sensorimotor transformations alters hand perception

When using tools effects in body space and distant space often do not correspond or are even in conflict. The ideomotor principle holds that actors select, initiate and execute movements by activating the anticipatory codes of the movements’ sensory effects (Greenwald, 1970; James, 1890). These may be representations of body-related effects and/or representations of more distal effects. Previous studies have demonstrated that distant action effects dominate action control, while body-related effects are attenuated (e.g., Müsseler and Sutter, 2009). In two experiments, participants performed closed-loop controlled movements on a covered digitizer tablet to control a cursor on a monitor. Different gains perturbed the relation between hand and cursor amplitude, so that the hand amplitude varied and the cursor amplitude remained constant, and vice versa. Within a block the location of amplitude perturbation randomly varied (low predictability) or not (high predictability). In Experiment 1 both trajectories of hand and cursor followed the same linear path, in Experiment 2 a linear hand trajectory produced a curved cursor trajectory on the monitor. When participants were asked to evaluate their hand movement, they were extremely uncertain about their trajectories. Both, predictability of amplitude perturbation and shape of cursor trajectory modulated the awareness of one’s own hand movements. We will discuss whether the low awareness of proximal action effects originates from an insufficient quality of the humans’ tactile and proprioceptive system or from an insufficient spatial reconstruction of this information in memory.

The Representation of Visual and Motor Aspects of Reaching Movements in the Human Motor Cortex

The human primary motor cortex (M1) is robustly activated during visually guided hand movements. M1 multivoxel patterns of functional MRI activation are more correlated during repeated hand movements to the same targets than to greatly differing ones, and therefore potentially contain information about movement direction. It is unclear, however, whether direction specificity is due to the motor command, as implicitly assumed, or to the visual aspects of the task, such as the target location and the direction of the cursor's trajectory. To disambiguate the visual and motor components, different visual-to-motor transformations were applied during an fMRI scan, in which participants made visually guided hand movements in various directions. The first run was the "baseline" (i.e., visual and motor mappings were matched); in the second run ("rotation"), the cursor movement was rotated by 45° with respect to the joystick movement. As expected, positive correlations were seen between the M1 multivoxel patterns evoked by the baseline run and by the rotation run, when the two movements were matched in their movement direction but the visual aspects differed. Importantly, similar correlations were observed when the visual elements were matched but the direction of hand movement differed. This indicates that M1 is sensitive to both motor and visual components of the task. However, repeated observation of the cursor movement without concurrent joystick control did not elicit significant activation in M1 or any correlated patterns of activation. Thus, visual aspects of movement are encoded in M1 only when they are coupled with motor consequences.

Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array

The ongoing pilot clinical trial of the BrainGate neural interface system aims in part to assess the feasibility of using neural activity obtained from a small-scale, chronically implanted, intracortical microelectrode array to provide control signals for a neural prosthesis system. Critical questions include how long implanted microelectrodes will record useful neural signals, how reliably those signals can be acquired and decoded, and how effectively they can be used to control various assistive technologies such as computers and robotic assistive devices, or to enable functional electrical stimulation of paralyzed muscles. Here we examined these questions by assessing neural cursor control and BrainGate system characteristics on five consecutive days 1000 days after implant of a 4 × 4 mm array of 100 microelectrodes in the motor cortex of a human with longstanding tetraplegia subsequent to a brainstem stroke. On each of five prospectively-selected days we performed time-amplitude sorting of neuronal spiking activity, trained a population-based Kalman velocity decoding filter combined with a linear discriminant click state classifier, and then assessed closed-loop point-and-click cursor control. The participant performed both an eight-target center-out task and a random target Fitts metric task which was adapted from a human-computer interaction ISO standard used to quantify performance of computer input devices. The neural interface system was further characterized by daily measurement of electrode impedances, unit waveforms and local field potentials. Across the five days, spiking signals were obtained from 41 of 96 electrodes and were successfully decoded to provide neural cursor point-and-click control with a mean task performance of 91.3% ± 0.1% (mean ± s.d.) correct target acquisition. Results across five consecutive days demonstrate that a neural interface system based on an intracortical microelectrode array can provide repeatable, accurate point-and-click control of a computer interface to an individual with tetraplegia 1000 days after implantation of this sensor.

Type of visual feedback during practice influences the precision of the acquired internal model of a complex visuo-motor transformation

This study investigated the influence of the type of visual feedback during practice with a complex visuo-motor transformation of a sliding two-sided lever on the acquisition of an internal model of the transformation. Three groups of participants, who practised with different types of visual feedback, were compared with regard to movement accuracy, curvature and movement time. One group had continuous visual feedback during practice and two groups were presented terminal visual feedback, either only the end position of the movement or the end position together with the trajectory of the cursor. Results showed that continuous visual feedback led to more precise movement end positions during practice than terminal visual feedback, but to less precise movements during open-loop tests. This finding indicates that terminal visual feedback supports the development of a precise internal model of a new visuo-motor transformation. However, even terminal feedback of the cursor trajectory during practice did not result in an internal model, which includes appropriate curvatures of hand movements. Statement of Relevance:This paper presents results on the influence of type of visual feedback on learning the complex motor skill of controlling a sliding lever. These findings contribute to the conceptual basis of optimised training procedures for the acquisition of sensori-motor skills required for the mastery of instruments utilised in minimally invasive surgery.

Reorganization of Finger Coordination Patterns During Adaptation to Rotation and Scaling of a Newly Learned Sensorimotor Transformation

We examined how people organize redundant kinematic control variables (finger joint configurations) while learning to make goal-directed movements of a virtual object (a cursor) within a low-dimensional task space (a computer screen). Subjects participated in three experiments performed on separate days. Learning progressed rapidly on day 1, resulting in reduced target capture error and increased cursor trajectory linearity. On days 2 and 3, one group of subjects adapted to a rotation of the nominal map, imposed either stepwise or randomly over trials. Another group experienced a scaling distortion. We report two findings. First, adaptation rates and memory-dependent motor command updating depended on distortion type. Stepwise application and removal of the rotation induced a marked increase in finger motion variability but scaling did not, suggesting that the rotation initiated a more exhaustive search through the space of viable finger motions to resolve the target capture task than did scaling. Indeed, subjects formed new coordination patterns in compensating the rotation but relied on patterns established during baseline practice to compensate the scaling. These findings support the idea that the brain compensates direction and extent errors separately and in computationally distinct ways, but are inconsistent with the idea that once a task is learned, command updating is limited to those degrees of freedom contributing to performance (thereby minimizing energetic or similar costs of control). Second, we report that subjects who learned a scaling while moving to just one target generalized more narrowly across directions than those who learned a rotation. This contrasts with results from whole-arm reaching studies, where a learned scaling generalizes more broadly across direction than rotation. Based on inverse- and forward-dynamics analyses of reaching with the arm, we propose the difference in results derives from extensive exposure in reaching with familiar arm dynamics versus the novelty of the manual task.

How Age Affects Pointing With Mouse and Touchpad: A Comparison of Young, Adult, and Elderly Users

Effects of age on pointing performance have become increasingly important as computers have become extensively used by still larger parts of the population. This study empirically investigates young (12–14 years), adult (25–33 years), and elderly (61–69 years) participants' performance when pointing with mouse and touchpad. The goal is to provide an integrated analysis of (a) how these three age groups differ in pointing performance, (b) how these differences are affected by the two pointing devices, and (c) how the submovement structure of cursor trajectories may explain performance differences. Results show that adult participants perform better than both young and elderly participants in that adult participants make fewer errors than young participants and complete trials quicker than elderly participants. Moreover, young participants are quicker than elderly participants, who make neither more nor less errors than young and adult participants. All three age groups were slower and made more errors with the touchpad than the mouse, but the touchpad slowed down elderly participants more than young participants, who in turn were slowed down more than adult participants. Adult participants made more efficient submovements than elderly participants; young participants had an intermediate position in that they were similar to adult participants for some submovement measures and similar to elderly participants for others.

Do Electrode Properties Create a Problem in Interpreting Local Field Potential Recordings?

Local field potential (LFP) recordings within the brain have become an important tool used by neuroscientists to make inferences about the activity of a population of cells near an electrode. Each passing year analysis of LFPs in neuroscience seems to bring important new insights on the possible workings of networks in the brain to produce behavior (Buschman and Miller 2007; Canolty et al. 2006; Gregoriou et al. 2009; Liu and Newsome 2006; Lubenov and Siapas 2009; Pesaran et al. 2008; Womelsdorf et al. 2006). Indeed LFPs have become a near-ubiquitous tool in neurophysiology seemingly in use anywhere extracellular spikes are also recorded.

Comparison of brain–computer interface decoding algorithms in open-loop and closed-loop control

Neuroprosthetic devices such as a computer cursor can be controlled by the activity of cortical neurons when an appropriate algorithm is used to decode motor intention. Algorithms which have been proposed for this purpose range from the simple population vector algorithm (PVA) and optimal linear estimator (OLE) to various versions of Bayesian decoders. Although Bayesian decoders typically provide the most accurate off-line reconstructions, it is not known which model assumptions in these algorithms are critical for improving decoding performance. Furthermore, it is not necessarily true that improvements (or deficits) in off-line reconstruction will translate into improvements (or deficits) in on-line control, as the subject might compensate for the specifics of the decoder in use at the time. Here we show that by comparing the performance of nine decoders, assumptions about uniformly distributed preferred directions and the way the cursor trajectories are smoothed have the most impact on decoder performance in off-line reconstruction, while assumptions about tuning curve linearity and spike count variance play relatively minor roles. In on-line control, subjects compensate for directional biases caused by non-uniformly distributed preferred directions, leaving cursor smoothing differences as the largest single algorithmic difference driving decoder performance.

Mapping broadband electrocorticographic recordings to two-dimensional hand trajectories in humans

Brain-machine interfaces (BMIs) aim to translate the motor intent of locked-in patients into neuroprosthetic control commands. Electrocorticographical (ECoG) signals provide promising neural inputs to BMIs as shown in recent studies. In this paper, we utilize a broadband spectrum above the fast gamma ranges and systematically study the role of spectral resolution, in which the broadband is partitioned, on the reconstruction of the patients' hand trajectories. Traditionally, the power of ECoG rhythms (<200-300 Hz) has been computed in short duration bins and instantaneously and linearly mapped to cursor trajectories. Neither time embedding, nor nonlinear mappings have been previously implemented in ECoG neuroprosthesis. Herein, mapping of neural modulations to goal-oriented motor behavior is achieved via linear adaptive filters with embedded memory depths and as a novelty through echo state networks (ESNs), which provide nonlinear mappings without compromising training complexity or increasing the number of model parameters, with up to 85% correlation. Reconstructed hand trajectories are analyzed through spatial, spectral and temporal sensitivities. The superiority of nonlinear mappings in the cases of low spectral resolution and abundance of interictal activity is discussed.

Pointing Movements on Multi-Layered Displays

A Multi Layered Display TM (MLD) uses depth to alter the salience of information. This study considered how the use of depth might influence cursor positioning. As pointing movements to targets may be influenced by flanking non- targets, a MLD was used in an attempt to reduce any distracting effects of non-targets. Participants were 14 skilled computer users who moved a computer cursor using a mouse to virtual targets at two distances in the presence of flanking non-targets on the same or different layer, before or after the target. Cursors avoided the non-targets. Kinematic analysis of cursor trajectories implied less interference in initial cursor trajectories when non-targets were on a different layer. Enhancing target and flanker separation using the MLD reduced the distracting effects of flankers upon the early planned part of cursor trajectories. For the latter part of cursor trajectories a MLD may reduce a perceived potential for impact with non-targets.

Analysis Method of Skill Acquisition Process using Electromyogram for Autonomous Assistive Devices

For development of an adaptable assist device, it requires to quantify users' skillfulness. In this paper, we propose and verify a method to index skillfulness on arm reaching tasks for motor learning. Process analysis of motor learning gives us substantial information on functional adaptability of human motion activities. Hence, we define skillfulness as reproducibility and motion accuracy of motion patterns. In our experiment, a computer monitor displays a moving cursor (e.g., cyclical and random movements), and seven subjects track the cursor with a joystick. We measure electromyogram (EMG) and cursor trajectories on skill acquisition process of the subjects. Then, we evaluate their skillfulness with the internal conditions: the reproducibility is calculated from the variance value of EMG patterns and the motion accuracy is calculated from the trajectory error. Eventually, the results indicated variety of skill acquisition processes, and proved that the proposed method quantitatively-analyzed their skillfulness.

Does a Multi-Layer Display Reduce the Effect of Non-Target Flankers?

Pointing movements to targets may be influenced by non-target flankers. A Multi Layered DisplayTM (MLD) was used in an attempt to reduce any distracting effects of flanking targets. A ‘response vector model’ predicts movements away from highly salient flankers, while a ‘response activation model’ can predict movements towards. Participants were 14 skilled computer users who moved a computer cursor using a mouse to virtual targets. Kinematic analysis revealed significant effects consistent with the response vector model. Separating target and flanker separation using the MLD reduced the distracting effects of flankers upon the early “planned” part of cursor trajectories but not the latter part of cursor trajectories.

Kinematic Analysis of Cursor Trajectories Controlled With a Touchpad

To investigate the characteristics of a computer touchpad as a pointing device, 14 participants used their right hand to manipulate the touchpad on a laptop computer. They were required to move a cursor over different distances (7.5 cm, 15 cm) from a home location to targets of different diameters (8 mm, 16 mm), situated to the upper left, middle, or right of a computer screen. A kinematic analysis of movement onsets and cursor trajectories indicated the nature of inefficiencies of the touchpad compared with other devices, primarily excessive submovements. Upper leftward movements were poorer, which can be explained by asymmetries in the finger-wrist system. This result implies that screen accessibility can vary as a function of users' interaction with cursor controllers and that the default placements of key icons might need to vary as a consequence.

Forward/up directional incompatibilities during cursor placement within graphical user interfaces

Within graphical user interfaces, an indirect relationship between display and control may lead to directional incompatibilities when a forward mouse movement codes upward cursor motions. However, this should not occur for left/right movements or direct cursor controllers (e.g. touch sensitive screens). In a four-choice reaction time task, 12 participants performed movements from a central start location to a target situated at one of four cardinal points (top, bottom, left, right). A 2 × 2 × 2 design varied directness of controller (moving cursor on computer screen or pen on graphics tablet), compatibility of orientation of cursor controller with screen (horizontal or vertical) and axis of desired cursor motion (left/right or up/down). Incompatibility between orientation of controller and motion of cursor did not affect response latencies, possibly because both forward and upward movements are away from the midline and go up the visual field. However, directional incompatibilities between display and controller led to slower movement with prolonged accelerative phases. Indirect relationships between display and control led to less efficient movements with prolonged decelerative phases and a tendency to undershoot movements along the bottom/top axis. More direct cursor control devices, such as touch sensitive screens, should enhance the efficiency of aspects of cursor trajectories.

A submovement analysis of cursor trajectories

Understanding human movement is key to improving input devices and interaction techniques. This paper presents a study of mouse movements of motion-impaired users, with an aim to gaining a better understanding of impaired movement. The cursor trajectories of six motion-impaired users and three able-bodied users are studied according to their submovement structure. Performance measures based on submovement structure are described, including the frequency and duration of pauses between submovements, verification times, the number of submovements, the peak speed of submovements and the accuracy of submovements in two dimensions. The measures are shown to be sensitive to differences between users with dissimilar physical capabilities. Results include findings that some motion-impaired users pause more often and for longer than able-bodied users, require up to five times more submovements to complete the same task, and exhibit a greater decline in accuracy with increasing speed than able-bodied users.

Mouse movements of motion-impaired users: a submovement analysis

Understanding human movement is key to improving input devices and interaction techniques. This paper presents a study of mouse movements of motion-impaired users, with an aim to gaining a better understanding of impaired movement. The cursor trajectories of six motion-impaired users and three able-bodied users are studied according to their submovement structure. Several aspects of the movement are studied, including the frequency and duration of pauses between submovements, verification times, the number of submovements, the peak speed of submovements and the accuracy of submovements in two-dimensions. Results include findings that some motion-impaired users pause more often and for longer than able-bodied users, require up to five times more submovements to complete the same task, and exhibit a correlation between error and peak submovement speed that does not exist for able-bodied users.

Effects of Cursor Orientation and Required Precision on Positioning Movements on Computer Screens

Arrowhead cursors used within graphic-user interfaces include implicit directional cues that may not be compatible with desired axis of motion. In addition, arrowhead cursors may not afford the best cues for location, and such effects may be exacerbated when there is a greater need for precise cursor placement. To address the impact of the cursor's orientation on its positioning, 12 participants were required to move cursors (pointing arrows) leftward or rightward to targets (small, medium, or large) on a computer screen. Response latencies were influenced by compatibilities between movement direction and cursor shape but only when moderate levels of precision were required. Rightward movements were slower and less accurate, and movements were slower and less efficient when arrows pointed in compatible directions. Implicit directional cues only elicited compatibility effects with moderate precision requirements. Arrowhead cursors compatible with direction of motion led to slower cursor movements and less efficient cursor trajectories. Where response initiation is important, compatible arrowheads are beneficial, but if speed of cursor placement is an issue, an orientation neutral cursor with area effect might be preferable, or a more direct interface (e.g., touch-sensitive screen) might be more appropriate.

Characteristics of cursor trajectories controlled by the computer mouse

Characteristics of cursor trajectories controlled by the computer mouse

Characteristics of cursor trajectories controlled by the computer mouse

An analysis of computer screen cursor trajectories can provide insights into the factors limiting efficient cursor positioning and can assist in the design of humancomputer interfaces. Cursor locations as controlled by a Microsoft computer mouse with standard settings were therefore sampled at 5ms intervals and kinematic analyses addressed the proportions of time spent in the initiation, accelerative and terminal guidance phases of cursor positioning. Twelve participants used a computer mouse to move a cursor over different distances (7.5cm, 15cm) from a home location in the lower centre of the screen to targets of different diameters (8mm, 16mm), situated to the left, middle or right of the computer screen. Cursor trajectories were irregular, and participants regularly overshot their targets, spending 70% of movement duration in terminal guidance. Participants appeared to use the initial part of their movement to establish mappings between controller and display. Interventions should seek to reduce the terminal guidance phase of cursor positioning.

Cursor Control Device Characteristics

An analysis of cursor positioning may provide guidelines for improvements in cursor control. Four experiments addressed efficiency of cursor control devices (Mouse, Digitising Pen, Accupoint, Trackball). Participants moved a cursor leftwards, upwards or rightwards, positioning it in large or small targets situated in near or far space on the computer screen. Cursor coordinates were sampled every 5 ms. The number of submovements and the proportion of time spent in deceleration were analysed. Participants could not plan movements controlled by an Accupoint. Cursor trajectories were more variable in near space for detachable manipulanda due to potential cartesian coordinate system incompatibilities