Manual Control Tasks Research Articles

Human Factors Design and Evaluation of China’s Space Manipulator Teleoperation System

Space manipulators are important for the assembly, construction, operation, and maintenance of space stations. To ensure that the manipulator system on the China Space Station (CSS) is user-friendly, the human factors engineering team has carried out extensive studies on the human factors design and evaluation of the space manipulator teleoperation system from three aspects: the technical process, research on ergonomic requirements, and ergonomic evaluation. This article systematically combs the technical process and index system of the human factors design and requirements research on CSS manipulator teleoperation system, which has guided the optimization of space manipulator engineering. This article expounds the research work on ergonomic requirements and suggestions related to manipulator teleoperation through experimental research on cognitive characteristics and human factors in display and control interfaces. We focus on human factors experimental research for camera type and handle polarity, and a recommended scheme for camera display and handle polarity has been screened out accordingly. Finally, the human-in-the-loop ergonomic evaluation method is demonstrated with a cooperative manual control task. The research results have facilitated the human factors design and optimization improvement of the space manipulator manipulator system, and have contributed significantly to the safe and smooth completion of astronaut extravehicular activities on the CSS.

Dual-task interference while receiving information on a head mounted display and manual tracking with and without auditory warnings

BackgroundDual-task interference is a concern when users attempt to use head mounted displays (HMD) while performing seated manual control tasks. Auditory warnings of information displayed on the HMD may or may not assist users while multi-tasking. ObjectiveIn this study, we investigated the dual-task interference of words, to be remembered later, presented on a HMD and motor control in a manual tracking task. The word presentations were warned or not warned with auditory signals and the impact of audio warnings were examined. MethodsParticipants were required to perform five tasks: (1) a compensatory tracking task performed alone, (2) a word memory task performed alone, (3) a tracking and word memory task, (4) a word memory task with words preceded by audial signals, and (5) a tracking and word memory task with words preceded by audial signals. In addition, in the dual-tasks, half of the word presentations were paired with directions changes in the tracking task to test for immediate perceptual interference. ResultsThere were significant dual task costs for both mean tracking error and later word recall. Additionally, participants took significantly longer to respond to motion changes paired with word presentations than motion changes not paired with word presentations. However, the impact of auditory warnings on tracking performance was nuanced. ConclusionEven with an in field of view transparent HMD momentary and sustained cognitive dual-task interference remains. Reaction times are affected most in the worst case scenario, when task critical events occur at the same time as a text message. ApplicationThe use of HMDs in time critical manual control tasks (such as operating machinery) should be limited. The use of audial warnings to alert operators to information displayed on an HMD requires further research; it may disrupt appropriate or natural task ordering.

Application of Nonlinear Correction Method for Attitude Control and Landing Oscillations Prevention

The purpose of this work is to demonstrate the application of the nonlinear correction method to prevent attitude oscillations of flying vehicles. This paper investigates the complications of manual control tasks in aeronautics caused by employing the integral component in the control loop. If the actuators of aircraft controlling surfaces reach their rate limits, the aircraft oscillations can appear for both manually and automatically piloted cases. In this paper, the oscillations alleviation problem is solved using the nonlinear correction method. A general methodology for pilot-induced oscillations prevention is described briefly by the example of an aircraft pitch control system. The obtained pilot model parameters are used for altitude control system examination. The nonlinear systems analysis is carried out by calculating the generalized sensitivity function for the control and disturbance input signal. In the second part, the application of the nonlinear correction method to prevent aircraft and satellite oscillations caused by integrator windup. A comparative analysis of the nonlinear correction and the customary anti-windup feedback is carried out. The results obtained demonstrate that a sequential nonlinear corrective device can be efficiently used to suppress pilot-induced and integrator windup oscillations.

The Effect of Noisy Galvanic Vestibular Stimulation on Learning of Functional Mobility and Manual Control Nulling Sensorimotor Tasks.

Galvanic vestibular stimulation (GVS) is a non-invasive method of electrically stimulating the vestibular system. We investigated whether the application of GVS can alter the learning of new functional mobility and manual control tasks and whether learning can be retained following GVS application. In a between-subjects experiment design, 36 healthy subjects performed repeated trials, capturing the learning of either (a) a functional mobility task, navigating an obstacle course on a compliant surface with degraded visual cues or (b) a manual control task, using a joystick to null self-roll tilt against a pseudo-random disturbance while seated in the dark. In the “learning” phase of trials, bilateral, bipolar GVS was applied continuously. The GVS waveform also differed between subjects in each task group: (1) white noisy galvanic vestibular stimulation (nGVS) at 0.3 mA (2) high-level random GVS at 0.7 mA (selected from pilot testing as destabilizing, but not painful), or (3) with the absence of stimulation (i.e., sham). Following the “learning” trials, all subjects were blindly transitioned to sham GVS, upon which they immediately completed another series of trials to assess any aftereffects. In the functional mobility task, we found nGVS significantly improved task learning (p = 0.03, mean learning metric 171% more than the sham group). Further, improvements in learning the functional mobility task with nGVS were retained, even once the GVS application was stopped. The benefits in learning with nGVS were not observed in the manual control task. High level GVS tended to inhibit learning in both tasks, but not significantly so. Even once the high-level stimulation was stopped, the impaired performance remained. Improvements in learning with nGVS may be due to increased information throughput resulting from stochastic resonance. The benefit of nGVS for functional mobility, but not manual control nulling, may be due to the multisensory (e.g., visual and proprioceptive), strategic, motor coordination, or spatial awareness aspects of the former task. Learning improvements with nGVS have the potential to benefit individuals who perform functional mobility tasks, such as astronauts, firefighters, high performance athletes, and soldiers.

Feedforward modulation of gamma motor neuron activity can improve motor command accuracy

Objective. Coactivation of gamma and alpha motor neuron activity ensures that muscle spindle responsiveness is maintained during muscle contractions. However, some evidence suggests that the activity of gamma motor neurons is phase-advanced with respect to that of alpha motor neurons during manual control tasks. We hypothesized that this might be a deliberate control strategy to maximize movement accuracy. Approach. Using a computational model of the neural activation of a muscle and its type Ia sensory feedback to the motor neurons, we systematically investigated the impact of the phase difference between oscillatory descending input to alpha and dynamic gamma motor neurons. Specifically, the amplification of the alpha motor neuron drive to the muscle was investigated as a function of the frequency of the synaptic input (1–9 Hz individually or superimposed) and the alpha–gamma phase difference (0–2π). Main results. Simulation results showed that when the phase advance of the dynamic gamma drive resulted in delays between muscle velocity and type Ia afferent feedback similar to those previously observed experimentally, low-frequency components (1 and 2 Hz) of the motor neuron synaptic input were amplified (gain up to 1.7). On the other hand, synaptic input at higher frequencies was unaffected. Significance. This finding suggests that by imposing a phase advance of the input to dynamic gamma motor neurons, components of the neural drive usually associated with voluntary control are amplified. In this way, our study suggests that this neural strategy increases the control-to-neural-noise ratio of the motor output during movement.

Assessing Cognitive Capacity by P3 During a Complex Manual Control Task

Abstract. Our aim was to adapt a classical P3 method to assess the free cognitive capacity during spacecraft docking training in space. Electroencephalogram (EEG) measurement in space is limited by several conditions. Based on experience with our own EEG experiments on MIR and ISS, we decided to use dry electrodes and restricted the electrode placement to the forehead. We examined whether P3 can be reliably obtained under these conditions. Subjects had to perform a manually controlled docking task simultaneously with an acoustic monitory task. The P3 component was evoked by the acoustic stimuli of the secondary task. Twenty-six subjects participated in this study, situated in a space simulation on earth. After a familiarization session, they performed the docking tasks at three difficulty levels: low, medium, and difficult. In the secondary task, subjects had to discriminate between a low (750 Hz) and a high (1,000 Hz) tone, which differed in probability of 90% and 10%, respectively. The subjects had to count the high tone and after 10 relevant tones and had to give a voice command to a power supply configuration. P3 amplitude was largest and the latency shortest during the medium difficult task. A decision matrix based on differences between the relevant and irrelevant P3 was calculated for each subject and each task. The results suggest that P3 can be recorded during a complex manual control task and can be used to assess individual free cognitive capacity.

Effects of Target Trajectory Bandwidth on Manual Control Behavior in Pursuit and Preview Tracking

The 1960s crossover model is widely applied to quantitatively predict a human controller's (HC's) manual control behavior. Unfortunately, the theory captures only compensatory tracking behavior and, as such, a limited range of real-world manual control tasks. This article finalizes recent advances in manual control theory toward more general pursuit and preview tracking tasks. It is quantified how HCs adapt their control behavior to a final crucial task variable: the target trajectory bandwidth. Beneficial adaptation strategies are first explored offline with computer simulations, using an extended crossover model theory for pursuit and preview tracking. The predictions are then verified with data from a human-in-the-loop experiment, in which participants tracked a target trajectory with bandwidths of 1.5, 2.5, and 4 rad/s, using compensatory, as well as pursuit and preview displays. In stark contrast to the crossover regression found in compensatory tasks, humans attenuate only their feedforward response when tracking higher-bandwidth trajectories in pursuit tasks, while their behavior is generally invariant in preview tasks. A full quantitative theory is now available to predict HC manual control behavior in tracking tasks, which includes HC adaptation to all key task variables.

Haptic Assistance for Helicopter Control Based on Pilot Intent Estimation

Haptic support systems have been widely used for supporting human operators when performing a manual control task. These systems are commonly designed to track known target trajectories. However, the trajectory to track is not known in many realistic cases. For instance, the pilot-intended trajectory is not known beforehand when considering a helicopter pilot flying in free-flight. This paper proposes a possible approach to design a haptic shared control system when the target trajectory is not known a priori. Especially, the aim of the proposed design is to help minimally trained pilots during a 2-degree-of-freedom lateral/longitudinal helicopter free-flight task. To accomplish this goal, first, a Pilot Intent Estimator (PIE) is developed to infer pilot intent. Then, the corresponding intended trajectory is generated. Finally, a haptic feedback is designed to track the estimated intended trajectory. The designed PIE was evaluated in a preliminary test with an experienced helicopter pilot. Then, a human-in-the-loop experiment with minimally trained participants was conducted to assess the proposed shared control system. Results showed the effectiveness of the PIE to estimate the correct direction of motion chosen by the pilot. Furthermore, the haptic feedback helped participants to accomplish the control task with better task performance (i.e., lower tracking errors and lower amount of control activity) compared with manual control.

Memory Pattern Identification for Feedback Tracking Control in Human-Machine Systems.

The aim of this paper was to identify the characteristics of memory patterns with respect to a visual input, perceived by the human operator during a manual control task, which consisted in following a moving target on a display with a cursor. Manual control tasks involve nondeclarative memory. The memory encodings of different motor skills have been referred to as procedural memories. The procedural memories have a pattern, which this paper sought to identify for the particular case of a one-dimensional tracking task. Specifically, data recorded from human subjects controlling dynamic systems with different fractional order were investigated. A finite impulse response (FIR) controller was fitted to the data, and pattern analysis of the fitted parameters was performed. Then, the FIR model was further reduced to a lower order controller; from the simplified model, the stability analysis of the human-machine system in closed-loop was conducted. It is shown that the FIR model can be used to identify and represent patterns in human procedural memories during manual control tasks. The obtained procedural memory pattern presents a time scale of about 650 ms before decay. Furthermore, the fitted controller is stable for systems with fractional order less than or equal to 1. For systems of different fractional order, the proposed control scheme-based on an FIR model-can effectively characterize the linear properties of manual control in humans. This research supports a biofidelic approach to human manual control modeling over feedback visual perceptions. Relevant applications of this research are the following: the development of shared-control systems, where a virtual human model assists the human during a control task, and human operator state monitoring.

Goal-related feedback guides motor exploration and redundancy resolution in human motor skill acquisition.

The plasticity of the human nervous system allows us to acquire an open-ended repository of sensorimotor skills in adulthood, such as the mastery of tools, musical instruments or sports. How novel sensorimotor skills are learned from scratch is yet largely unknown. In particular, the so-called inverse mapping from goal states to motor states is underdetermined because a goal can often be achieved by many different movements (motor redundancy). How humans learn to resolve motor redundancy and by which principles they explore high-dimensional motor spaces has hardly been investigated. To study this question, we trained human participants in an unfamiliar and redundant visually-guided manual control task. We qualitatively compare the experimental results with simulation results from a population of artificial agents that learned the same task by Goal Babbling, which is an inverse-model learning approach for robotics. In Goal Babbling, goal-related feedback guides motor exploration and thereby enables robots to learn an inverse model directly from scratch, without having to learn a forward model first. In the human experiment, we tested whether different initial conditions (starting positions of the hand) influence the acquisition of motor synergies, which we identified by Principal Component Analysis in the motor space. The results show that the human participants’ solutions are spatially biased towards the different starting positions in motor space and are marked by a gradual co-learning of synergies and task success, similar to the dynamics of motor learning by Goal Babbling. However, there are also differences between human learning and the Goal Babbling simulations, as humans tend to predominantly use Degrees of Freedom that do not have a large effect on the hand position, whereas in Goal Babbling, Degrees of Freedom with a large effect on hand position are used predominantly. We conclude that humans use goal-related feedback to constrain motor exploration and resolve motor redundancy when learning a new sensorimotor mapping, but in a manner that differs from the current implementation of Goal Babbling due to different constraints on motor exploration.

Traceability Management of Systemsof Systems: A Systematic Reviewin the Assisted Reproduction Domain

Over last decade, Assisted Reproductive Treatment (ART) has become a very used health service by more and more people around the world because of problems such as the delay in the maternity age, single-parent couples, etc. In this context, health agencies have performed innovations to improve healthcare processes of ARTs, to optimize the performance of health professionals who work in fertilization laboratories and to improve Biological Sample Management (BSM) and sample traceability in ART. However, there are important handicaps in ART processes from the point of view of quality, safety and management. On the one hand, these processes are mainly based on manual execution tasks and manual control tasks. This excess of manual tasks could lead to fatal traceability and safety errors during BSM. On the other hand, ART processes require real, interoperable and traceable communications between different software systems that have to collaborate together (health information systems, biological sample management systems, patient management systems, etc.), but, at present, it is possible to identify some limitations in this domain, that is, the domain of systems of systems (SoS). This paper aims to conduct an exhaustive study was carried out both in the research community and in the commercial field to identify and analyze SoS solutions and theoretical proposals for BSM in ART processes. We have applied the Systematic Literature Review (SLR) methodology to carry out our study and we conclude it is a very young research line that shows a growing trend and that in the actuality there are very few technologies that deal with the problem of the BSM in ART.After analyzing the results, this paper presents as future work an initial Model-Driven conceptual solution to improve BSM in ART.

Framework for a Two-step Evaluation of Haptic Displays

Haptic displays can greatly facilitate manual control tasks. Their capacity of allowing the operator to perceive the desired dynamics is an important design parameter. However, attempts to evaluate haptic displays on the basis of what dynamics humans actually perceive are scarce. This paper proposes a two-step framework which incorporates the characteristics of human haptic perception into the evaluation of haptic displays. The first step is to evaluate the haptic display based on a recently developed model of the threshold for changes in the perception of system dynamics. It allows us to know the frequency spectrum in which a haptic device alters the operator’s perception of the system dynamics. The second step is then to understand how the perceived dynamic distortions affect the operator’s characterization of the system dynamics. Findings from recent psychophysical studies allow us to relate the changes in perception caused by the haptic device to changes in the perceived mechanical properties of the system. A numerical example illustrates how haptic displays can be evaluated using the proposed framework.

Individual Learning Curves in Manual Control of Six Degrees of Freedom

A newly developed tool to facilitate learning processes in complex perceptual-motor tasks was tested for its efficiency characteristics. The tool is based on a demanding manual control task, in which a target object has to be manipulated with six degrees of freedom. During the self-sufficient learning process the task difficulty is increased from very easy to the complex level of a manual spacecraft docking on a space station. The type and the difficulty of the next task were defined by a regulation system based on individual performance. The tool was successfully applied to 22 participants in a laboratory setting. No performance differences were found between genders. Age was correlated with training duration but slower learners also succeeded eventually. The tool provides improvements in the learning process for operational tasks like controlling a robot arm or to perform a docking maneuver. Beside the foreseen practical application in space this is a time effective and economic way to educate participants in ground based research requiring such complex skills.

Development of Pilot Mathematical Model in the Preview Manual Control Task

The model for perception of the preview target trajectory is being proposed. The model is a weighted sum of the slopes of the preview target trajectory elements. A procedure for the selection of the weighting coefficients as well as other pilot adaptation parameters is proposed. The results of the mathematical modelling of the pilot-aircraft system corresponding to the preview tracking and ground-based simulation are compared.

Human manual control precision depends on vestibular sensory precision and gravitational magnitude.

Precise motion control is critical to human survival on Earth and in space. Motion sensation is inherently imprecise, and the functional implications of this imprecision are not well understood. We studied a "vestibular" manual control task in which subjects attempted to keep themselves upright with a rotational hand controller (i.e., joystick) to null out pseudorandom, roll-tilt motion disturbances of their chair in the dark. Our first objective was to study the relationship between intersubject differences in manual control performance and sensory precision, determined by measuring vestibular perceptual thresholds. Our second objective was to examine the influence of altered gravity on manual control performance. Subjects performed the manual control task while supine during short-radius centrifugation, with roll tilts occurring relative to centripetal accelerations of 0.5, 1.0, and 1.33 GC (1 GC = 9.81 m/s2). Roll-tilt vestibular precision was quantified with roll-tilt vestibular direction-recognition perceptual thresholds, the minimum movement that one can reliably distinguish as leftward vs. rightward. A significant intersubject correlation was found between manual control performance (defined as the standard deviation of chair tilt) and thresholds, consistent with sensory imprecision negatively affecting functional precision. Furthermore, compared with 1.0 GC manual control was more precise in 1.33 GC (-18.3%, P = 0.005) and less precise in 0.5 GC (+39.6%, P < 0.001). The decrement in manual control performance observed in 0.5 GC and in subjects with high thresholds suggests potential risk factors for piloting and locomotion, both on Earth and during human exploration missions to the moon (0.16 G) and Mars (0.38 G). NEW & NOTEWORTHY The functional implications of imprecise motion sensation are not well understood. We found a significant correlation between subjects' vestibular perceptual thresholds and performance in a manual control task (using a joystick to keep their chair upright), consistent with sensory imprecision negatively affecting functional precision. Furthermore, using an altered-gravity centrifuge configuration, we found that manual control precision was improved in "hypergravity" and degraded in "hypogravity." These results have potential relevance for postural control, aviation, and spaceflight.

Effects of Preview Time in Manual Tracking Tasks

In manual control tasks, preview of the target trajectory ahead is often limited by poor lighting, objects, or display edges. This paper investigates the effects of limited preview, or preview time, in manual tracking tasks with single- and double-integrator controlled element dynamics. A quasi-linear human controller model is used to predict the human behavior adaptations offline, by finding the model parameters that yield optimal performance at each preview time. These predictions are then verified by fitting the same model to measurements from a human-in-the-loop experiment, where subjects performed a tracking task with eight different preview time settings between 0 and 2 s. Results show that the tracking performance improves and the model's “look-ahead” time parameters increase with increasing preview time. Beyond a certain preview time, approximately 0.6 s and 1.15 s in single- and double-integrator tasks, respectively, additional preview evokes no further adaptations. The offline model predictions closely match the experimental results, which thereby promises to facilitate similar quantitative insights in other tasks with restricted preview.

Effects of Linear Perspective on Human Use of Preview in Manual Control

Due to linear perspective, the visual stimulus provided by a previewed reference trajectory reduces with increasing distance ahead. This paper investigates the effects of linear perspective on human use of preview in manual control tasks. Results of a human-in-the-loop tracking experiment are presented, where the linear perspective's horizontal and vertical deformations along the previewed trajectory were applied separately and combined, or were absent (plan-view task). Measurements are analyzed with both nonparametric and parametric system identification techniques, in combination with a quasi-linear human controller model for plan-view preview tracking tasks. Results show that reduced visual stimuli in perspective tasks evoke less aggressive control behavior, but that the human's underlying control mechanisms are still accurately captured by the model. We conclude that human controllers use preview information similar in plan-view and perspective tasks.

Tool use changes the spatial extension of the magnetic touch illusion.

Characterizing the brain mechanisms that allow humans to use tools to interact with the environment is a major goal in neuroscience. It has been proposed that handheld tools are incorporated into the multisensory representation of the body and its surrounding (peripersonal) space, underlying our remarkable tool use ability. One single-cell recording study in tool-using monkeys provided qualitative support for this hypothesis, and the results from a vast number of human studies employing different experimental paradigms have been ambiguous. Here, we made use of the recently reported magnetic touch illusion-a perceptual correlate of peripersonal space-to examine the effect of tool use on the representation of visuotactile peripersonal space. The results showed that active tool use leads to an extension of the "illusion volume" around the entire length of a tool, which was significantly greater compared with a manual control task. These findings support the notion that the multisensory representation of peripersonal space is extended to incorporate handheld tools and provide a three-dimensional estimation of this remapping process. (PsycINFO Database Record

Physiological Measures for Human Performance Analysis in Human-Robot Teamwork: Case of Tele-Exploration

Continuous monitoring of mental workload and situation awareness in operational environments is useful for understanding and prediction of human performance. Such information can be used to develop real-time adaptive systems to enhance human performance. In this paper, we investigate the use of workload- and attention-related physiological measures to predict operator performance and situation awareness in the context of tele-exploration with a small team of robots. A user study is conducted based on a simulated scenario involving visual scanning and manual control tasks with varying levels of task-load. Brain activity and eye movements of the participants are monitored across the experimental tasks using electroencephalogram and eye tracker sensors. The performances of the subjects are evaluated in terms of target detection and situation awareness (primary metrics) as well as reaction time and false detection (secondary metrics). Moreover, individual differences in two specific visual skills, visual search (VS) and multi-object tracking (MOT) are considered as between-subject factors in the experimental design. The main effects of task type and individual differences reveal that VS and MOT skill have significant effects on target detection and situation awareness, respectively. The correlations of physiological measures with the task performance and situation awareness are analyzed. The results suggest that brain-based features (mental workload and distraction) which represent the covert aspect of attention are better suited to predict the secondary performance metrics. On the other hand, glance-based features which represent the overt aspect of attention are shown to be the best predictors of the primary performance metrics.

Bilateral human-in-the-loop tele-haptic interface for controlling a robotic manipulator

Effective human-robot interaction is a factor deemed necessary for control of industrial robots and manipulators. Fine manual control and calibration tasks for an industrial manipulator can be daunting for human operators. In this paper, the authors proposed a bilateral haptic interface for controlling a robotic manipulator using a haptic device. This allows the human operator to intuitively jog and fell the manipulator from a remote station. The proposed system consists of slave robot manipulators controlled by the master haptic devices that can give haptic feedback to the user when the slave robot touches rigid and deformable surfaces. Such a human-in-the-loop approach to control a remote robotic system offers a pragmatic solution when the robotic system is operating under dynamically changing environments and dexterous tasks. The inclusion of haptic feedback offers the potential to augment the operators' control task and overcome the limitations of visual perception within the robots operating environment.