Human Steering Research Articles

Hierarchical MPC-based authority allocation strategy for human–machine shared vehicles considering human–machine conflict

The uncertainty of driver behavior is an important factor affecting the safety of human–machine co-driving vehicles. Traditional rule-based models often fail to capture the nonlinear and complex characteristics of human steering behavior. To overcome this, we propose a data network-driven approach utilizing gated recurrent unit (GRU) neural networks to accurately predict driver steering behavior. The GRU-based driver model is integrated with vehicle dynamics to construct a control-oriented driver–vehicle model. Considering that human–machine conflict may cause vehicle instability, an exponential function combining proportional–integral–derivative is proposed to quantify the human–machine conflict level based on steering difference. To reasonably allocate human–computer permissions based on human–machine interaction, a hierarchical authority allocation framework is proposed. The upper layer provides a reference authority allocation via an exponential function, while the lower layer employs a real-time model predictive control (MPC) optimizer to track this reference, ensuring optimal vehicle path tracking and stability. The proposed system’s effectiveness is validated through driver-in-the-loop testing, demonstrating significant improvements in safety and performance. The results show that in the human–machine conflict scenario, the proposed authority allocation strategy can still ensure the path tracking and safety of the vehicle.

Generalized two-point visual control model of human steering for accurate state estimation

Abstract We derive and validate a generalization of the two-point visual control model, an accepted cognitive science model for human steering behavior. The generalized model is needed as current steering models are either insufficiently accurate or too complex for online state estimation. We demonstrate that the generalized model replicates specific human steering behavior with high precision (85% reduction in modeling error) and integrate this model into a human-as-advisor framework where human steering inputs are used for state estimation. As a benchmark study, we use this framework to decipher ambiguous lane markings represented by biased lateral position measurements. We demonstrate that, with the generalized model, the state estimator can accurately estimate the true vehicle state, providing lateral state estimates with under 0.15 m error across participants. However, without the generalized model, the estimator cannot accurately estimate the vehicle's lateral state.

Context-dependence of deterministic and nondeterministic contributions to closed-loop steering control.

In natural circumstances, sensory systems operate in a closed loop with motor output, whereby actions shape subsequent sensory experiences. A prime example of this is the sensorimotor processing required to align one's direction of travel, or heading, with one's goal, a behavior we refer to as steering. In steering, motor outputs work to eliminate errors between the direction of heading and the goal, modifying subsequent errors in the process. The closed-loop nature of the behavior makes it challenging to determine how deterministic and nondeterministic processes contribute to behavior. We overcome this by applying a nonparametric, linear kernel-based analysis to behavioral data of monkeys steering through a virtual environment in two experimental contexts. In a given context, the results were consistent with previous work that described the transformation as a second-order linear system. Classically, the parameters of such second-order models are associated with physical properties of the limb such as viscosity and stiffness that are commonly assumed to be approximately constant. By contrast, we found that the fit kernels differed strongly across tasks in these and other parameters, suggesting context-dependent changes in neural and biomechanical processes. We additionally fit residuals to a simple noise model and found that the form of the noise was highly conserved across both contexts and animals. Strikingly, the fitted noise also closely matched that found previously in a human steering task. Altogether, this work presents a kernel-based analysis that characterizes the context-dependence of deterministic and non-deterministic components of a closed-loop sensorimotor task.

Processing of complex traffic scenes for effective steering and collision avoidance: a perspective, from research into human control, on the challenges for sensor-based autonomous vehicles on urban roads.

An overview is provided of behavioral research into human steering and collision avoidance including the processing of optic flow, optical looming and the role of the human mobile gaze system. A consideration is then made of the issues that may occur for autonomous vehicles (AV) when they move from grid-type road networks into complex inner-city streets and interact with human drivers, pedestrians and cyclists. Comparisons between human processing and AV processing of these interactions are made. This raises issues as to whether AV control systems need to mimic human visual processing more closely and highlights the need for AV systems to develop a "theory of road users" that allows attribution of intent to other drivers, cyclists or pedestrians. Guidelines for the development of a "theory of road users" for AVs are suggested.

Using naturalistic and driving simulator data to model driver responses to unintentional lane departures

Unintentional lane departures on straight roads cause many road fatalities each year. The objective of this study was to explore and model drivers’ recovery steering maneuvers in unintentional drift situations, to enable the prospective safety benefit assessment of lane departure warning and avoidance systems through counterfactual simulations. The timing and amplitude of the steering adjustments drivers make to avoid lane departure were studied over three data sets with different origins, consisting of both naturalistic data and experimental data from a driving simulator study. With respect to timing, the main finding was that visually distracted drivers often initiate the corrective steering response prior to looking back towards the road, demonstrating that lane-keeping information in the visual periphery is sufficient to trigger the response. As for steering amplitude, the observed amplitudes were correlated against different lane departure risk metrics from the literature, resulting in a model capable of accounting for human behavior across all three data sets with good performance. Surprisingly, a very simple model (which describes the steering amplitude as increasing quadratically with the vehicle’s orientation to the road) predicted the amplitude of the primary corrective steering adjustment better than models based on more complex lane departure risk metrics. This result indicates that drivers scale the amplitude of their steering adjustment to the steering input needed to get the vehicle back in the lane already at first response. However, it was possible to obtain a similar model fit using a more complex threshold model, with different dynamics depending on the vehicle’s current orientation to the road. We discuss how these findings can be applied to models of human steering for safety benefit assessment.

Validation of Stochastic Optimal Control Models for Goal-Directed Human Movements on the Example of Human Driving Behavior

Stochastic Optimal Control models represent the state-of-the-art in modeling goal-directed human movements. The linear-quadratic sensorimotor (LQS) model based on signal-dependent noise processes in state and output equation is the current main representative. With our newly introduced Inverse Stochastic Optimal Control algorithm building upon two bi-level optimizations, we can identify its unknown model parameters, namely cost function matrices and scaling parameters of the noise processes, for the first time. In this paper, we use this algorithm to identify the parameters of a deterministic linear-quadratic, a linear-quadratic Gaussian and a LQS model from human measurement data to compare the models’ capability in describing goal-directed human movements. Human steering behavior in a simplified driving task shown to posses similar features as point-ot-point human hand reaching movements serves as our example movement. The results show that the identified LQS model outperforms the others with statistical significance. Particularly, the average human steering behavior is modeled significantly better by the LQS model. This validates the positive impact of signal-dependent noise processes on modeling human average behavior.

Modeling Human Steering Behavior in Haptic Shared Control of Autonomy-Enabled Unmanned Ground Vehicles.

A human steering model for teleoperated driving is extended to capture the human steering behavior in haptic shared control of autonomy-enabled Unmanned Ground Vehicles (UGVs). Prior studies presented human steering models for teleoperation of a passenger-sized Unmanned Ground Vehicle, where a human is fully in charge of driving. However, these models are not applicable when a human needs to interact with autonomy in haptic shared control of autonomy-enabled UGVs. How a human operator reacts to the presence of autonomy needs to be studied and mathematically encapsulated in a module to capture the collaboration between human and autonomy. Human subject tests are conducted to collect data in haptic shared control for model development and validation. The ACT-R architecture and two-point steering model used in the previous literature are adopted to predict the operator's desired steering angle. A torque conversion module is developed to convert the steering command from the ACT-R model to human torque input, thus enabling haptic shared control with autonomy. A parameterization strategy is described to find the set of model parameters that optimize the haptic shared control performance in terms of minimum average lane keeping error (ALKE). The model predicts the minimum ALKE human subjects achieve in shared control. The extended model can successfully predict the best haptic shared control performance as measured by ALKE. This model can be used in place of human operators, enabling fully simulation-based engineering, in the development and evaluation of haptic shared control technologies for autonomy-enabled UGVs, including control negotiation strategies and autonomy capabilities.

Human steering angle estimation in video based on key point detection and Kalman filter

Human steering angle estimation in video based on key point detection and Kalman filter

Autonomous Driving using CNN

Human beings are currently addicted to automation and robotics technologies. The state-of-the-art in deep learning technologies and AI is the subject of this autonomous driving. Driving with automated driving systems promises to be safe, enjoyable, and efficient.. It is preferable to train in a virtual environment first and then move to a real-world one. Its goal is to enable a vehicle to recognise its surroundings and navigate without the need for human intervention. The raw pixels from a single front-facing camera were directly transferred to driving commands using a convolution neural network (CNN). This end-to-end strategy proved to be remarkably effective, The system automatically learns internal representations of the essential processing stages such as detecting useful road components using only the human steering angle as the training signal. We never expressly taught it to recognise the contour of roadways, for example. In comparison to explicit issue decomposition, such as lane marking detection, Our end-to-end solution optimises all processing processes at the same time, including path planning and control. We believe that this will lead to improved performance and smaller systems in the long run. Internal components will self-optimize to maximise overall system performance, resulting in improved performance.

Modeling Human Steering Behavior in Teleoperation of Unmanned Ground Vehicles With Varying Speed

This paper extends a prior human operator model to capture human steering performance in the teleoperation of unmanned ground vehicles (UGVs) in path-following scenarios with varying speed. A prior study presented a human operator model to predict human steering performance in the teleoperation of a passenger-sized UGV at constant speeds. To enable applications to varying speed scenarios, the model needs to be extended to incorporate speed control and be able to predict human performance under the effect of accelerations/decelerations and various time delays induced by the teleoperation setting. A strategy is also needed to parameterize the model without human subject data for a truly predictive capability. This paper adopts the ACT-R cognitive architecture and two-point steering model used in the previous work, and extends the model by incorporating a far-point speed control model to allow for varying speed. A parameterization strategy is proposed to find a robust set of parameters for each time delay to maximize steering performance. Human subject experiments are conducted to validate the model. Results show that the parameterized model can predict both the trend of average lane keeping error and its lowest value for human subjects under different time delays. The proposed model successfully extends the prior computational model to predict human steering behavior in a teleoperated UGV with varying speed. This computational model can be used to substitute for human operators in the process of development and testing of teleoperated UGV technologies and allows fully simulation-based development and studies.

A Universal Control Scheme of Human-Like Steering in Multiple Driving Scenarios

To incorporate the inherent superiorities of an experienced, proficient driver, a human-like control scheme for automated steering systems is proposed in this paper based on the concept of “staying within the safety zone”. Designed in the precognitive architecture, the novel control scheme takes into account key characteristics of human steering behaviors in high-speed conditions by determining the steering activation and the steering intensity based on the inversed-time-to-lane-crossing (iTLC), generating smooth, moderate steering actions to maintain the vehicle within a predefined safety zone. The proposed control scheme provides a universal solution for steering control in multiple driving scenarios by accordingly allocating the safety zone to ensure the driving safety therein. A simulator experiment was conducted to test both the fully autonomous and the driver-in-the-loop performance of the proposed control scheme in lane keeping and lane changing. A traditional error-minimizing control algorithm was implemented as a comparison. Results indicate that the novel scheme outperforms the conventional control schemes by yielding higher steering performance, lower steering effort, and higher robustness, proving the benefits of imitating the patterns of human control behaviors in the control algorithm of an automated steering system.

Human Path-Planning for Autonomous Spacecraft Guidance at Binary Asteroids

Rather than using baseline-oriented approaches, autonomous spacecraft trajectory planning within asteroid systems may be based on human behavior that is learned by demonstration. In this paper, we conducted a numerical experiment to observe a human agent steering in real-time spacecraft motion within simulated binary asteroid environments. The resulting collection of human-planned trajectories demonstrates effective guidance mechanisms and reveals underlying path-planning processes that may serve for more agile and autonomous asteroid exploration.

On optical data-guided optimal control simulations of human motion

This work addresses the synergistic fusion of optimal control simulations and marker-based optical measurements of human motion. The latter is a widespread capturing technology in biomechanics and movement science. In the context of optimal control simulations, the idea is to improve the computational performance by using a realistic initial guess and to increase the realism of the simulated motion through data-guiding. In the context of motion capturing, the idea is to use biomechanical simulations in order to maintain accurate capturings also with reduced measurement frequencies and points. This would greatly improve the usability of such systems in terms of setup time and wearing comfort. In this work, we investigate different methods for combining physical laws, 3D marker positions obtained from the optical system, and physiologically motivated objectives in an optimal control framework. Moreover, we explore the potential of obtaining reasonable results—in terms of motion trajectories and torques that are close to reference obtained from using all available information—with a reduced measurement frequency and a reduced number of markers. The tests are performed on a human steering and throwing motion, where a human arm was captured with seven retroreflective markers at $120\text{ Hz}$. Our results show, that a significant reduction of exploited measurements still provides feasible simulation results in our proposed method, given that the physiologically motivated objective reflects the actual movement. Furthermore, it turns out that neglecting markers close to the shoulder has less influence on the simulation results than neglecting markers close to the hand.

Stability and Control of a Rider–Bicycle System: Analysis and Experiments

We present stability and control analysis of a rider–bicycle system under human steering and body movements. The dynamic model of rider–bicycle interactions is first constructed to integrate the rider’s body movement with the moving bicycle platform. We then present human balance control strategies based on human riding experiments. The closed-loop system stability is analyzed and discussed. Quantitative influences of the bicycle physical parameters, the human control gains, and the time delays are also analyzed and discussed. Extensive experiments are conducted to validate the human control models and demonstrate human balance performance using the bikebot, an instrumented bicycle platform. The presented modeling and analysis results can be potentially used for further development of bicycle-assisted rehabilitation for postural balance patients. Note to Practitioners —Understanding human balance and control of bicycles is crucial for not only designing bicycle-based rehabilitation devices but also studying physical human–machine interactions for healthcare automation. This paper takes the rider and bicycle as an example of physical human–machine interactions to understand how human use their limbs and body movement to stabilize an unstable platform (i.e., bicycles). We develop an instrumented bicycle system, called bikebot, to conduct human riding experiments. Using experimental data, we build the dynamic models for human steering and leaning control actions. Using these actuation models, stability analyses are conducted for rider–bicycle interactions and then validated by experiments. We also obtain the stability results by perturbing human visual and sensorimotor feedback. These results reveal that the visual feedback and the time delay in sensorimotor feedback mechanism play critical roles in stabilizing the unstable bicycle platform.

Using Driver Control Models to Understand and Evaluate Behavioral Validity of Driving Simulators

For a driving simulator to be a valid tool for research, vehicle development, or driver training, it is crucial that it elicits similar driver behavior as the corresponding real vehicle. To assess such behavioral validity, the use of quantitative driver models has been suggested but not previously reported. Here, a task-general conceptual driver model is proposed, along with a taxonomy defining levels of behavioral validity. Based on these theoretical concepts, it is argued that driver models without explicit representations of sensory or neuromuscular dynamics should be sufficient for a model-based assessment of driving simulators in most contexts. As a task-specific example, two parsimonious driver steering models of this nature are developed and tested on a dataset of real and simulated driving in near-limit, low-friction circumstances, indicating a clear preference of one model over the other. By means of closed-loop simulations, it is demonstrated that the parameters of this preferred model can generally be accurately estimated from unperturbed driver steering data, using a simple, open-loop fitting method, as long as the vehicle positioning data are reliable. Some recurring patterns between the two studied tasks are noted in how the model's parameters, fitted to human steering, are affected by the presence or absence of steering torques and motion cues in the simulator.

Modeling Human Steering Behavior During Path Following in Teleoperation of Unmanned Ground Vehicles.

This paper presents a behavioral model representing the human steering performance in teleoperated unmanned ground vehicles (UGVs). Human steering performance in teleoperation is considerably different from the performance in regular onboard driving situations due to significant communication delays in teleoperation systems and limited information human teleoperators receive from the vehicle sensory system. Mathematical models capturing the teleoperation performance are a key to making the development and evaluation of teleoperated UGV technologies fully simulation based and thus more rapid and cost-effective. However, driver models developed for the typical onboard driving case do not readily address this need. To fill the gap, this paper adopts a cognitive model that was originally developed for a typical highway driving scenario and develops a tuning strategy that adjusts the model parameters in the absence of human data to reflect the effect of various latencies and UGV speeds on driver performance in a teleoperated path-following task. Based on data collected from a human subject test study, it is shown that the tuned model can predict both the trend of changes in driver performance for different driving conditions and the best steering performance of human subjects in all driving conditions considered. The proposed model with the tuning strategy has a satisfactory performance in predicting human steering behavior in the task of teleoperated path following of UGVs. The established model is a suited candidate to be used in place of human drivers for simulation-based studies of UGV mobility in teleoperation systems.

Online Sensing of Human Steering Intervention Torque for Autonomous Driving Actuation Systems

Identification of human driver intervention is important for the safety and performance of autonomous vehicles. Effective sensing of driver steering torque will enable human drivers to take over controls when needed. Due to the special operation mode of autonomous vehicles, both the human driver and the actuator motor apply active torque to the steering wheel simultaneously during the take-over process. This makes it impossible to detect the human interventions by simply using a torque sensor due to the coupled torques. To tackle this new problem, a model-based strategy to estimate human steering intervention torque is presented for autonomous vehicles in this paper. The model of autonomous steering wheel actuation system is established first. Online human steering torque estimation algorithm is then proposed and the stability condition is given to guarantee both the estimation convergence and accuracy. Numerical simulation is performed to validate the feasibility of the proposed estimator based upon three typical steering intervention cases. The influence factors of the estimation performance are discussed, and a trade-off policy is utilized to determine the optimal estimation gain. Furthermore, experimental studies are carried out on a passenger car equipped with an autonomous driving actuation system. Both the simulation and experimental results demonstrate the effectiveness of the proposed approach.

Synchronous Tracking Control for Agricultural Wide-Span Implement Carrier (WSIC)

Abstract. The mobile operating mode of agricultural Wide-Span Implement Carriers (WSICs) requires accurate synchronization of the forward motions of the two supporting tractors. The guidance task for WSIC operations is currently performed by human operators, which is labor-intensive and can be inaccurate. To replace the traditional human steering and control process, an automatic guidance system adapted to the mobile mode of the WSIC was designed and tested. A master-slave cooperative control method was applied, and the steering angle control law for both the master and slave vehicles and the velocity control law for the slave vehicle were derived. A scaled-down robotic platform was developed by integrating two heavy-duty mobile robots, a dual-rover real-time kinematic GPS (RTK-GPS), two inertial measurement units (IMUs), two XBee-PRO wireless communication modules, and two groups of control processors. Validation experiments were performed using the robotic platform on a flat and firm surface. Results of the single-robot path-following experiment verified the effectiveness of the steering angle control law. Results of the two synchronous tracking experiments, one with a constant reference velocity of 0.3 m s-1 and one with a varying reference velocity of 0.2 to 0.6 m s-1, showed that the two robots could perform their individual path-following tasks accurately while keeping their relative offset to less than 0.1 m. In the experiment with a constant reference velocity, the average, root mean square (RMS), maximum lead, and maximum lag of the offset errors were 0.01, 0.03, 0.05, and 0.06 m, respectively. Slightly larger average, RMS, and maximum lead offset errors were observed in the experiment with a varying reference velocity, which were 0.02, 0.04, and 0.09 m, respectively. These experimental results confirmed that the designed control laws for the synchronous tracking of the WSIC vehicles were effective and the developed automatic guidance system was reliable and applicable. Keywords: Automatic guidance, Master-slave control, Synchronous tracking, Wide-Span Implement Carrier (WSIC).

Herding the Crowd: Using Automated Planning for Better Crowdsourced Planning

One subclass of human computation applications are those directed at tasks that involve planning (e.g. tour planning) and scheduling (e.g. conference scheduling). Interestingly, work on these systems shows that even primitive forms of automated oversight on the human contributors helps in significantly improving the effectiveness of the humans/crowd. In this paper, we argue that the automated oversight used in these systems can be viewed as a primitive automated planner, and that there are several opportunities for more sophisticated automated planning in effectively steering crowdsourced planning. Straightforward adaptation of current planning technology is however hampered by the mismatch between the capabilities of human workers and automated planners. We identify and partially address two important challenges that need to be overcome before such adaptation of planning technology can occur: (i) interpreting inputs of the human workers (and the requester) and (ii) steering or critiquing plans produced by the human workers, armed only with incomplete domain and preference models. To these ends, we describe the implementation of AI-MIX, a tour plan generation system that uses automated checks and alerts to improve the quality of plans created by human workers.

Measurement and mathematical model of a driver's intermittent compensatory steering control

The compensatory (feedback) component of a human driver's steering control is examined. In particular, the effect of the cognitive process is studied. Model predictive control theory is used to implement models of intermittency in cognitive processing. Experiments using a fixed-base driving simulator with periodic occlusion of the visual display are used to reveal the nature of the driver's steering behaviour. An intermittent serial-ballistic control strategy is found to match the measured behaviour better than intermittent zero-order hold or continuous control. The findings may enable some insight to driver-vehicle interaction and vehicle handling qualities.