Cursor Trajectories Research Articles

Quantifying Similarities Between MediaPipe and a Known Standard to Address Issues in Tracking 2D Upper Limb Trajectories: Proof of Concept Study.

Markerless motion tracking methods have promise for use in a range of domains, including clinical settings where traditional marker-based systems for human pose estimation are not feasible. Artificial intelligence (AI)-based systems can offer a markerless, lightweight approach to motion capture. However, the accuracy of such systems, such as MediaPipe, for tracking fine upper limb movements involving the hand has not been explored. The aim of this study is to evaluate the 2D accuracy of MediaPipe against a known standard. Participants (N=10) performed a touchscreen-based shape-tracing task requiring them to trace the trajectory of a moving cursor using their index finger. Cursor trajectories created a reoccurring or random shape at 5 different speeds (500-2500 ms, in increments of 500 ms). Movement trajectories on each trial were simultaneously captured by the touchscreen and a separate video camera. Movement coordinates for each trial were extracted from the touchscreen and compared to those predicted by MediaPipe. Specifically, following resampling, normalization, and Procrustes transformations, root-mean-squared error (RMSE; primary outcome measure) was calculated between predicted coordinates and those generated by the touchscreen computer. Although there was some size distortion in the frame-by-frame estimates predicted by MediaPipe, shapes were similar between the 2 methods and transformations improved the general overlap and similarity of the shapes. The resultant mean RMSE between predicted coordinates and those generated by the touchscreen was 0.28 (SD 0.06) normalized px. Equivalence testing revealed that accuracy differed between MediaPipe and the touchscreen, but that the true difference was between 0 and 0.30 normalized px (t114=-3.02; P=.002). Additional analyses revealed no differences in resultant RMSE between methods when comparing across lower frame rates (30 and 60 frames per second [FPS]), although there was greater RMSE for 120 FPS than for 60 FPS (t35.43=-2.51; P=.03). Overall, we quantified similarities between one AI-based approach to motion capture and a known standard for tracking fine upper limb movements, informing applications of such systems in domains such as clinical and research settings. Future work should address accuracy in 3 dimensions to further validate the use of AI-based systems, including MediaPipe, in such domains.

Input device matters for measures of behaviour in online experiments

Studies of perception, cognition, and action increasingly rely on measures derived from the movements of a cursor to investigate how psychological processes unfold over time. This method is one of the most sensitive measures available for remote experiments conducted online, but experimenters have little control over the input device used by participants, typically a mouse or trackpad. These two devices require biomechanically distinct movements to operate, so measures extracted from cursor tracking data may differ between input devices. We investigated this in two online experiments requiring participants to execute goal-directed movements. We identify several measures that are critically influenced by the choice of input device using a kinematic decomposition of the recorded cursor trajectories. Those using a trackpad were slower to acquire targets, mainly attributable to greater times required to initiate movements and click on targets, despite showing greater peak speeds and lower variability in their movements. We believe there is a substantial risk that behavioural disparities caused by the input device used could be misidentified as differences in psychological processes. We urge researchers to collect data on input devices in online experiments and carefully consider and account for the effect they may have on their experimental data.

De novo sensorimotor learning through reuse of movement components.

From tying one's shoelaces to driving a car, complex skills involving the coordination of multiple muscles are common in everyday life; yet relatively little is known about how these skills are learned. Recent studies have shown that new sensorimotor skills involving re-mapping familiar body movements to unfamiliar outputs cannot be learned by adjusting pre-existing controllers, and that new task-specific controllers must instead be learned "de novo". To date, however, few studies have investigated de novo learning in scenarios requiring continuous and coordinated control of relatively unpractised body movements. In this study, we used a myoelectric interface to investigate how a novel controller is learned when the task involves an unpractised combination of relatively untrained continuous muscle contractions. Over five sessions on five consecutive days, participants learned to trace a series of trajectories using a computer cursor controlled by the activation of two muscles. The timing of the generated cursor trajectory and its shape relative to the target improved for conditions trained with post-trial visual feedback. Improvements in timing transferred to all untrained conditions, but improvements in shape transferred less robustly to untrained conditions requiring the trained order of muscle activation. All muscle outputs in the final session could already be generated during the first session, suggesting that participants learned the new task by improving the selection of existing motor commands. These results suggest that the novel controllers acquired during de novo learning can, in some circumstances, be constructed from components of existing controllers.

Mouse cursor trajectories capture the flexible adaptivity of predictive sentence processing.

Recent psycholinguistic findings raise fundamental questions about comprehenders' ability to rationally adapt their predictions during sentence processing. Two mouse cursor tracking experiments (each N = 85) assessed this adaptivity by manipulating the reliability of verb-based semantic cues. In Experiment 1, predictive mouse cursor movements to targets (e.g., bike) versus distractors (e.g., kite) were measured while participants heard equal proportions of nonpredictive (e.g., "spot … the bike"), predictive (e.g., "ride … the bike"), and antipredictive (e.g., "fly … the bike") sentences. In Experiment 2, participants heard equal proportions of nonpredictive and antipredictive sentences. Participants were observed to flexibly adapt their predictions, such that they disengaged prediction in Experiment 1 when verb-based cues were unreliable and as likely to be disconfirmed as confirmed, while they generated adapted predictions in Experiment 2 when verb-based cues were reliably disconfirmed. However, links to individual differences in cognitive control were not observed. These results are interpreted as supporting rational theoretical approaches. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Anticipatory postural control in adaptation of goal-directed lower extremity movements.

Skilled football players can adapt their kicking movements depending on external environments. Predictive postural control movements, known as anticipatory postural adjustments (APAs), are needed preceding kicking movements to precisely control them while maintaining a standing posture only with the support leg. We aimed to clarify APAs of the support leg in the process of adaptation of goal-directed movements with the lower limb. Participants replicated ball-kicking movements such that they reached a cursor, representing a kicking-foot position towards a forward target while standing with the support leg. APAs were observed as the centre of pressure of the support leg shifted approximately 300ms in advance of the onset of movement of the kicking foot. When the cursor trajectory of the kicking foot was visually rotated during the task, the kicking-foot movement was gradually modified to reach the target, indicating adaptation to the novel visuomotor environment. Interestingly, APAs in the mediolateral direction were also altered following the change in kicking-foot movements. Additionally, the APAs modified more slowly than the kicking-foot movements. These results suggest that flexible changes in predictive postural control might support the adaptation of goal-directed movements of the lower limb.

Color me honest! Time pressure and (dis)honest behavior

We introduce three modifications to the die-in-a-cup paradigm to gain novel insights into dishonest behavior under time pressure. The regular die is substituted with a custom one that has a distinct color on each side as a way of manipulating familiarity with the decision situation. The cup is substituted with a ‘dice tower' to control the randomization process. Alongside outcome data, we capture mouse cursor trajectories. Results from our preregistered laboratory experiment involving 229 subjects suggest that time pressure increases dishonesty only when the regular die is used. Mouse tracking analysis suggests that it takes more effort to be honest than to lie outright and that partial lying is most difficult.

How Consumers Resolve Conflict over Branded Products: Evidence from Mouse Cursor Trajectories

Consumers differ in the extent to which brands drive their choices. The current research investigates the psychology underlying such decisions by using a cursor-tracking paradigm that captures consumers’ decision-making processes in real time. Results indicate that while consumers typically process brand attributes relatively later than product attributes, the timing of this processing varies across individuals and affects choice. Specifically, when consumers trade off brand and product desirability (i.e., when deciding between a more [less] preferred product from a less [more] preferred brand), the earlier that brand attributes are considered, the more likely consumers are to choose the option from the preferred brand. Increasing the prominence of brand (vs. product) attributes leads to earlier brand attribute processing and a higher likelihood of choosing the preferred brand. These findings hold across a limited number of choice trials and for decisions involving three attributes (brand, product, and price). This research highlights the applicability of cursor tracking in revealing the psychological drivers of consumer choices in real time.

Deconstructing the functional significance of the error-related negativity (ERN) and midline frontal theta oscillations using stepwise time-locking and single-trial response dynamics

Error-related electroencephalographic potentials have been used for decades to develop theoretical models of response monitoring processes, study altered cognitive functioning in clinical populations, and more recently, to improve the performance of brain-computer interfaces. However, the vast majority of this research relies on discrete behavioral responses that confound error detection, response cancelation, error correction, and post-error cognitive and affective processes. By contrast, the present study demonstrates a novel, complementary method for isolating the functional correlates of error-related electroencephalographic responses using single-trial kinematic analyses of cursor trajectories and a stepwise time-locking analysis. The results reveal that the latency of the ERN, Pe, and medial-frontal theta oscillations are all strongly positively correlated with the latency at which an initiated error response is canceled, as indicated by the peak deceleration of the initiated movement prior to a corrective response. Results are discussed with respect to current theoretical models of error-related brain potentials and potential relevance to clinical applications.

Incompatibility Influences Cursor Placement When Pointing to Images of Cups.

As images are used within graphical user interfaces to signify menu selection, it is important to understand how image properties can influence cursor placement online. Objects have multiple dimensions that create potential ambiguity and Stroop-like confusions for the operator if a previously habitual response conflicts with the required response. To examine the impact of compatibility and other contextual factors, 41 participants used a computer Mouse or touch screen to place the cursor upon images of full or empty cups that varied in size, and direction of handle. Cursor placements took longer using the Mouse than touch screen. Although participants were placing the cursor on images, the size of the cup, whether it was empty or full, and the handle orientation interacted in their effects upon response duration and cursor placement. The effects of cup size reversed for empty cups or those with incompatible handles. Context can influence cursor placement. Perceived spillability influenced precision requirements, and the cup handle can serve as target or a flanking distractor. Image content can influence screen hotspots. As performance can change with cup spillability, this bodes well for attempts to detect intent from cursor trajectories.

Online mouse cursor trajectories distinguish phonological activation by linguistic and nonlinguistic sounds

Four online mouse cursor tracking experiments (total N = 208) examined the activation of phonological representations by linguistic and nonlinguistic auditory stimuli. Participants hearing spoken words (e.g., “bell”) produced less direct mouse cursor trajectories toward corresponding pictures or text when visual arrays also included phonologically related competitors (e.g., belt) as compared with unrelated distractors (e.g., hose), but no such phonological competition was observed during environmental sounds (e.g., the ring of a bell). While important similarities have been observed between spoken words and environmental sounds, these experiments provide novel mouse cursor evidence that environmental sounds directly activate conceptual knowledge without needing to engage linguistic knowledge, contrasting with spoken words. Implications for theories of conceptual knowledge are discussed.

Upper Limb Sensory-Motor Control During Exposure to Different Mechanical Environments in Multiple Sclerosis Subjects With No Clinical Disability.

Multiple sclerosis (MS) is an autoimmune and neurodegenerative disease resulting in motor impairments associated with muscle weakness and lack of movement coordination. The goal of this work was to quantify upper limb motor deficits in asymptomatic MS subjects with a robot-based assessment including performance and muscle synergies analysis. A total of 7 subjects (MS: 3 M−4 F; 42 ± 10 years) with clinically definite MS according to McDonald criteria, but with no clinical disability, and 7 age- and sex-matched subjects without a history of neurological disorders participated in the study. All subjects controlled a cursor on the computer screen by moving their hand or applying forces in 8 coplanar directions at their self-selected speed. They grasped the handle of a robotic planar manipulandum that generated four different environments: null, assistive or resistive forces, and rigid constraint. Simultaneously, the activity of 15 upper body muscles was recorded. Asymptomatic MS subjects generated less smooth and less accurate cursor trajectories than control subjects in controlling a force profile, while the end-point error was significantly different also in the other environments. The EMG analysis revealed different muscle activation patterns in MS subjects when exerting isometric forces or when moving in presence of external forces generated by a robot. While the two populations had the same number and similar structure of muscle synergies, they had different activation profiles. These results suggested that a task requiring to control forces against a rigid environment allows better than movement tasks to detect early sensory-motor signs related to the onset of symptoms of multiple sclerosis and to differentiate between stages of the disease.

Cognitive miserliness in argument literacy? Effects of intuitive and analytic thinking on recognizing fallacies

Abstract Fallacies are a particular type of informal argument that are psychologically compelling and often used for rhetorical purposes. Fallacies are unreasonable because the reasons they provide for their claims are irrelevant or insufficient. Ability to recognize the weakness of fallacies is part of what we call argument literacy and imporatant in rational thinking. Here we examine classic fallacies of types found in textbooks. In an experiment, participants evaluated the quality of fallacies and reasonable arguments. We instructed participants to think either intuitively, using their first impressions, or analytically, using rational deliberation. We analyzed responses, response times, and cursor trajectories (captured using mouse tracking). The results indicate that instructions to think analytically made people spend more time on the task but did not make them change their minds more often. When participants made errors, they were drawn towards the correct response, while responding correctly was more straightforward. The results are compatible with “smart intuition” accounts of dual-process theories of reasoning, rather than with corrective default-interventionist accounts. The findings are discussed in relation to whether theories developed to account for formal reasoning can help to explain the processing of everyday arguments.

Punishment Feedback Impairs Memory and Changes Cortical Feedback-Related Potentials During Motor Learning.

Reward and punishment have demonstrated dissociable effects on motor learning and memory, which suggests that these reinforcers are differently processed by the brain. To test this possibility, we use electroencephalography to record cortical neural activity after the presentation of reward and punishment feedback during a visuomotor rotation task. Participants were randomly placed into Reward, Punishment, or Control groups and performed the task under different conditions to assess the adaptation (learning) and retention (memory) of the motor task. These conditions featured an incongruent position between the cursor and the target, with the cursor trajectory, rotated 30° counterclockwise, requiring the participant to adapt their movement to hit the target. Feedback based on error magnitude was provided during the Adaptation condition in the form of a positive number (Reward) or negative number (Punishment), each representing a monetary gain or loss, respectively. No reinforcement or visual feedback was provided during the No Vision condition (retention). Performance error and event-related potentials (ERPs) time-locked to feedback presentation were calculated for each participant during both conditions. Punishment feedback reduced performance error and promoted faster learning during the Adaptation condition. In contrast, punishment feedback increased performance error during the No Vision condition compared to Control and Reward groups, which suggests a diminished motor memory. Moreover, the Punishment group showed a significant decrease in the amplitude of ERPs during the No Vision condition compared to the Adaptation condition. The amplitude of ERPs did not change in the other two groups. These results suggest that punishment feedback impairs motor retention by altering the neural processing involved in memory encoding. This study provides a neurophysiological underpinning for the dissociative effects of punishment feedback on motor learning.

Racial bias in face perception is sensitive to instructions but not introspection.

Faces with typically African features are perceived as darker than they really are. We investigated how early in processing the bias emerges, whether participants are aware of it, and whether it can be altered by explicit instructions. We presented pairs of faces sequentially, manipulated the luminance and morphological features of each, and asked participants which was lighter, and how confident they were in their responses. In Experiment 1, pre-response mouse cursor trajectories showed that morphology affected motor output just as early as luminance did. Furthermore, participants were not slower to respond or less confident when morphological cues drove them to give a response that conflicted with the actual luminance of the faces. However, Experiment 2 showed that participants could be instructed to reduce their reliance on morphology, even at early stages of processing. All stimuli used, code to run the experiments reported, raw data, and analyses scripts and their outputs can be found at https://osf.io/brssn.

A Review of Mouse-Tracking Applications in Economic Studies

Since mouse-tracking paradigm came under the spotlight two decades ago, by providing mouse cursor trajectories, it has been applied by behavioral scientists to a variety of topics to help understand real-time psychological state when people are faced with multiple choices. In this article, we provide a comprehensive, documentation of experimental economics studies with mouse-tracking paradigm. Among these studies, some focus on measuring choice uncertainty including subject uncertainty, temporal uncertainty, and probabilistic uncertainty; the rest are concerned with economic games including bargaining games and social dilemma games. Why and how these works employ mouse-tracking technique in their experiments is elaborated in detail. Finally, limitations of mouse-tracking paradigm are discussed, and research opportunities are proposed. Basic know-hows are appended as a general guide for interested readers.

Increased error-correction leads to both higher levels of variability and adaptation

In order to intercept moving objects, we need to predict the spatiotemporal features of the motion of both the object and our hand. Our errors can result in updates of these predictions to benefit interceptions in the future (adaptation). Recent studies claim that task-relevant variability in baseline performance can help adapt to perturbations, because initial variability helps explore the spatial demands of the task. In this study, we examined whether this relationship is also found in interception (temporal domain) by looking at the link between the variability of hand-movement speed during baseline trials, and the adaptation to a temporal perturbation. 17 subjects performed an interception task on a graphic tablet with a stylus. A target moved from left to right or vice versa, with varying speed across trials. Participants were instructed to intercept this target with a straight forward movement of their hand. Their movements were represented by a cursor that was displayed on a screen above the tablet. To prevent online corrections we blocked the hand from view, and a part of the cursor's trajectory was occluded. After a baseline phase of 80 trials, a temporal delay of 100 ms was introduced to the cursor representing the hand (adaptation phase: 80 trials). This delay initially caused participants to miss the target, but they quickly accounted for these errors by adapting to most of the delay of the cursor. We found that variability in baseline movement velocity is a good predictor of temporal adaptation (defined as a combination of the rate of change and the asymptotic level of change after a perturbation), with higher variability during baseline being associated with better adaptation. However, cross-correlation results suggest that the increased variability is the result of increased error correction, rather than exploration.

ImCAPTCHA: Imperceptible CAPTCHA Based on Cursor Trajectories

In the process of network interaction, CAPTCHA (completely automated public Turing test to tell computers and humans apart) is usually required. At present, the main method for differentiating between humans and machines is to use CAPTCHA, which includes images and text. Users need to interface with CAPTCHA for verification, which prolongs the interaction step. The interactive experience is poor, and the process of verification is not concealed. To simplify the verification process, optimize the interaction behavior and increase the security of verification, this paper proposes a CAPTCHA method based on a user's mouse operation behavior, and carries out a study on cursor trajectory recognition. This method can reduce the manual procedure and improve the user interactive experience. In this article, we propose an ensemble learning algorithm model based on sliding sampling to classify and recognize cursor trajectory data. Experiments show that the classification performance of the model is reliable for recognition based on human–machine cursor trajectories, which is a new concept for CAPTCHA.

Social Judgments of Digitally Manipulated Stuttered Speech: An Evaluation of Self-Disclosure on Cognition.

Purpose Persons who stutter (PWS) may be susceptible to discrimination because of negative judgments made by listeners. The current study sought to determine how the cognitive system's explicit (i.e., conscious) and implicit (i.e., nonconscious) biases about PWS are impacted by self-disclosure. Method A computer mouse-tracking paradigm was used to evaluate categorical social judgments about PWS. Computer mouse trajectories, which have been shown to reveal underlying cognitive pull or competition between opposing concepts, were used to measure implicit bias (i.e., nonconscious stereotypes). Participants were asked to explicitly categorize the speaker as either intelligent or unintelligent before and after listening to a speaker self-disclose. Mouse cursor trajectories during the explicit response categorization were used to evaluate implicit bias associated with the decision-making process. Results Results indicated that participants chose "intelligent" for a higher proportion of the trials in the disclosure condition compared to baseline, showing that listeners' explicit biases changed after listening to a self-disclosure that the speaker stutters. Results also indicated listeners exhibited a more negative implicit bias, based on computer mouse trajectories, when rating the PWS relative to the "persons who do not stutter" talker, but this negative implicit bias did seem to reduce over time after the disclosure was made. Conclusions These findings indicate that, even though explicit and implicit biases were evident when listeners heard stuttering, both explicit and implicit biases seemed to extinguish over time after a self-disclosure. Although the bias was not completely extinguished, these results provide promising evidence toward developing methods to reduce negative beliefs and reactions toward PWS. Supplemental Material http://osf.io/mwrp7/.

Upper Limb Motor Skills Performance Evaluation Based on Point-and-Click Cursor Trajectory Analysis: Application in Early Multiple Sclerosis Detection

We present an enhanced version of the input device evaluation application (IDEA) system as an objective method for evaluating upper limb motor skills performance. By introducing three new metrics for mouse cursor trajectory analysis, along with the application of the two-dimensional (2D) experiment in the case of multiple sclerosis (MS), we examine the sensitivity of the IDEA system for differentiating patients with early-stage MS and healthy participants. The IDEA system calculates multiple kinematic metrics for point-and-click tasks: movement time, index of difficulty, effective target width, effective index of difficulty, throughput, missed clicks, target re-entry, task axis crossing, movement direction change, orthogonal direction change, movement variability, movement error, movement offset, mean velocity, velocity peaks, and maximum/mean velocity ratio. The results reveal that the IDEA system sensitivity has been improved in comparison with previous studies, which is high enough to detect the presence of early-stage MS with a 70.9% success rate in the 2D experiment.

East is not right: Spatial compatibility differs between egocentric and cardinal retrieval.

Four experiments examined perceptuo-motor associations involved in spatial knowledge encoding and retrieval. Participants learned spatial information by studying a map or by navigating through a real environment and then verified spatial descriptions based on either egocentric or cardinal directional terms. Participants moved the computer mouse to a YES or NO button to verify each statement. We tracked mouse cursor trajectories to examine perceptuo-motor associations in spatial knowledge. An encoding hypothesis predicts that perceptuo-motor associations depend on the involvement of perceptions and actions during encoding, regardless of how spatial knowledge would be used. The retrieval hypothesis predicts that perceptuo-motor associations change as a function of retrieval demands, regardless of how they are learned. The results supported the retrieval hypothesis. Participants showed action compatibility effects with egocentric retrieval, regardless of how spatial information was learned. With well-developed spatial knowledge, a reliable compatibility effect emerged during egocentric retrieval, but no or limited compatibility effects emerged with cardinal retrieval. With less-developed knowledge, the compatibility effects evident during cardinal retrieval suggest a process of egocentric recoding. Other factors of environment learning, such as location proximity and orientation changes, also impacted the compatibility effect, as revealed in the temporal dynamics of mouse movements. Taken together, the results demonstrate that retrieval demands differentially rely upon perceptuo-motor associations in long-term spatial knowledge. This effect is also modulated by environment experience, proximity of learned locations, and experienced orientations.