Maneuver Decision Research Articles

Synchronous Maneuver Searching and Trajectory Planning for Autonomous Vehicles in Dynamic Traffic Environments

In the real-time decision-making and local planning process of autonomous vehicles in dynamic environments, the autonomous driving system may fail to find a reasonable policy or even gets trapped in some situation due to the complexity of global tasks and the incompatibility between upper-level maneuver decisions with the low-level lower level trajectory planning. To solve this problem, this paper presents a synchronous maneuver searching and trajectory planning (SMSTP) algorithm based on the topological concept of homotopy. Firstly, a set of alternative maneuvers with boundary limits are enumerated on a multi-lane road. Instead of sampling numerous paths in the whole spatio-temporal space, we, for the first time, propose using Trajectory Profiles (TPs) to quickly construct the topological maneuvers represented by different routes, and put forward a corridor generation algorithm based on graph-search. The bounded corridor further constrains the maneuver's space in the spatial space. A step-wise heuristic optimization algorithm is then proposed to synchronously generate a feasible trajectory for each maneuver. To achieve real-time performance, we initialize the states to be optimized with the boundary constraints of maneuvers, and we set some heuristic states as terminal targets in the quadratic cost function. The solution of a feasible trajectory is always guaranteed only if a specific maneuver is given. The simulation and realistic driving-test experiments verified that the proposed SMSTP algorithm has a short computation time which is less than 37ms, and the experimental results showed the validity and effectiveness of the SMSTP algorithm.

Collision Avoidance of Hexacopter UAV Based on LiDAR Data in Dynamic Environment

A reactive three-dimensional maneuver strategy for a multirotor Unmanned Aerial Vehicle (UAV) is proposed based on the collision cone approach to avoid potential collision with a single moving obstacle detected by an onboard sensor. A Light Detection And Ranging (LiDAR) system is assumed to be mounted on a hexacopter to obtain the obstacle information from the collected point clouds. The collision cone approach is enhanced to appropriately deal with the moving obstacle with the help of a Kalman filter. The filter estimates the position, velocity, and acceleration of the obstacle by using the LiDAR data as the associated measurement. The obstacle state estimate is utilized to predict the future trajectories of the moving obstacle. The collision detection and obstacle avoidance maneuver decisions are made considering the predicted trajectory of the obstacle. Numerical simulations, including a Monte Carlo campaign, are conducted to verify the performance of the proposed collision avoidance algorithm.

Maneuver Decision of UAV in Short-Range Air Combat Based on Deep Reinforcement Learning

With the development of artificial intelligence and integrated sensor technologies, unmanned aerial vehicles (UAVs) are more and more applied in the air combats. A bottleneck that constrains the capability of UAVs against manned vehicles is the autonomous maneuver decision, which is a very challenging problem in the short-range air combat undergoing highly dynamic and uncertain maneuvers of enemies. In this paper, an autonomous maneuver decision model is proposed for the UAV short-range air combat based on reinforcement learning, which mainly includes the aircraft motion model, one-to-one short-range air combat evaluation model and the maneuver decision model based on deep Q network (DQN). However, such model includes a high dimensional state and action space which requires huge computation load for DQN training using traditional methods. Then, a phased training method, called “basic-confrontation”, which is based on the idea that human beings gradually learn from simple to complex is proposed to help reduce the training time while getting suboptimal but efficient results. Finally, one-to-one short-range air combats are simulated under different target maneuver policies. Simulation results show that the proposed maneuver decision model and training method can help the UAV achieve autonomous decision in the air combats and obtain an effective decision policy to defeat the opponent.

Evasive Maneuver Strategy for UCAV in Beyond-Visual-Range Air Combat Based on Hierarchical Multi-Objective Evolutionary Algorithm

This study deals with the autonomous evasive maneuver strategy of unmanned combat air vehicle (UCAV), which is threatened by a high-performance beyond-visual-range (BVR) air-to-air missile (AAM). Considering tactical demands of achieving self-conflicting evasive objectives in actual air combat, including higher miss distance, less energy consumption and longer guidance support time, the evasive maneuver problem in BVR air combat is defined and reformulated into a multi-objective optimization problem. Effective maneuvers of UCAV used in different evasion phases are modeled in three-dimensional space. Then the three-level decision space structure is established according to qualitative evasive tactical planning. A hierarchical multi-objective evolutionary algorithm (HMOEA) is designed to find the approximate Pareto-optimal solutions of the problem. The approach combines qualitative tactical experience and quantitative maneuver decision optimization method effectively. Simulations are used to demonstrate the feasibility and effectiveness of the approach. The results show that the obtained set of decision variables constitutes nondominated solutions, which can meet different evasive tactical requirements of UCAV while ensuring successful evasion.

Cooperative Trajectory Planning for Haptic Shared Control Between Driver and Automation in Highway Driving

This paper addresses the driver–automation shared driving control for lane keeping and obstacle avoidance of automated vehicles in highway traffic. The proposed shared control framework is established from a novel cooperative trajectory planning algorithm and a fuzzy steering controller. Based on polynomial functions, the cooperative trajectory planning is formulated by judiciously exploiting the information on the maneuver decision, the conflict management, and the driver monitoring. As a result, the planned trajectory of the vehicle is continuously adapted according to the driver's actions and intentions. By means of Lyapunov stability arguments, sufficient conditions in terms of linear matrix inequalities are given to design a Takagi–Sugeno fuzzy model-based controller. This robust steering controller provides a necessary assistive torque to track the planned vehicle trajectory. The new shared driving control framework allows reducing effectively the driver–automation conflict issue while offering the driver more freedom to swerve within a predefined lane. The advantages of the proposed approach are evaluated using both objective and subjective results, experimentally obtained from several human drivers and an advanced interactive dynamic driving simulator.

Nautical Collision Avoidance : The Cognitive Challenges of Balancing Safety, Efficiency, and Procedures.

Experimentally investigate maneuver decision preferences in navigating ships to avoid a collision. How is safety (collision avoidance) balanced against efficiency (deviation from path and delay) and rules of the road under conditions of both trajectory certainty and uncertainty. Human decision error is a prominent factor in nautical collisions, but the multiple factors of geometry of collisions and role of uncertainty have been little studied in empirical human factors literature. Eighty-seven Mechanical Turk participants performed in a lower fidelity ship control simulation, depicting ownship and a cargo ship hazard on collision or near-collision trajectories of various conflict geometries, while controlling heading and speed with the sluggish relative dynamics. Experiment 1 involved the hazard on a straight trajectory. In Experiment 2, the hazard could turn on unpredictable trials. Participants were rewarded for efficiency and penalized for collisions or close passes. Participants made few collisions, but did so more often when on a collision path. They sometimes violated the instructed rules of the road by maneuvering in front of the hazard ship's path. They preferred speed control to heading control. Performance degraded in conditions of uncertainty. Data reveal an understanding of maneuver decisions and conditions that affect the balance between safety and efficiency. The simulation and data highlight the degrading role of uncertainty and provide a foundation upon which more complex questions can be asked, asked of more trained navigators, and decision support tools examined.

Multi-attribute decision-making method for prioritizing maritime traffic safety influencing factors of autonomous ships’ maneuvering decisions using grey and fuzzy theories

Ship maneuvering decisions are influenced by several factors, and it is essential to prioritize the main influencing factors for efficient selection of the corresponding maneuvering decisions. Meanwhile, the autonomous ship maneuvering decision-making influencing factors constitute a typical grey system, which is suitable for research by grey relational analysis. Furthermore, in the fuzzy approach, linguistic assessment of factors is evaluated to obtain priorities numbers. Therefore, this study mainly focuses on the concept of human-like maneuvering for autonomous ships. Based on experimental data of experienced seafarers and using a simulation platform under the scenario of the Shanghai Waigaoqiao wharf, an inference model utilizing grey and fuzzy theories is proposed. The proposed model combined with expert linguistic terms in order to select the ship maneuvering decision-making main influencing factors from multi-source influencing factors (in overall and separated categories of natural environment, ship motion, force parameters, draft, and position), and to study the decision-making prioritization for maritime traffic safety for specific ship maneuvering scenarios. This method can prioritize the main factors which affect maneuvering decisions as well as guide an autonomous ship-assisted or automatic maneuvering evaluation system for the research of human-like maneuvering behavior. This study provides a new perspective on the identification of main ship maneuvering decision-making influencing factors in theory and in practice. It can be utilized for better decision-making concerning maritime traffic safety of autonomous ship maneuvering, which in turn makes shipping safer and promote the application and spreading of autonomous ships.

Influence of environmental factors on human-like decision-making for intelligent ship

To date, the increasing density of water traffic has caused the ship's navigation environment to deteriorate, resulting in frequent water traffic accidents. In addition, a majority of maritime accidents are caused by human factors, and one of the important ways to solve the ship accidents caused by human factors is to utilize intelligent maneuvering of ships. Based on the actual crews' operational data from full-task handling simulation platform, this study combines a 30,000-ton bulk carrier inbound navigation scenario and uses the decision tree method to propose a knowledge learning model under multiple environmental constraints to give intelligent ships the ability to make decisions like a human: An intelligent ship Human-like Decision-making Maneuvering Decision Recognition (HDMDR) model. The decision-making mechanism for the maneuvering behavior of Officer On Watch (OOW) under the influence of the specific water traffic environment in the inbound scenario is analyzed, and the OOW's decision-making knowledge is automatically acquired and represented. The validation tests and the comparative analysis with the classic classification algorithms of k-Nearest Neighbours (k-NN) and Support Vector Machine (SVM) are performed to demonstrate the accuracy of the proposed HDMDR model. This paper provides a feasible basis for the human-like decision-making analysis of intelligent ships.

Study on UCAV Robust Maneuvering Decision in Automatic Air Combat Based on Target Accessible Domain

The paper presents a deep study on the robust maneuvering decision of unmanned combat aerial vehicle (UCAV) in air combat. First, a method to descript the target accessible domain by angles is designed. Then the state prediction model of the target with variable weight is built based on the change of target tactical intention. Second, robust membership functions that reflect the air combat situation are denoted to building up the decision method, robust multi-objective optimization (RMO). Finally, grey wolf optimizer (GWO) is adopted to complete the optimization. The simulation result shows that, the model of target state prediction based on accessible domain can reflect the tactical intention and the state prediction can obtain better situation advantage than that without prediction. Moreover, the RMO can get the promising result and the GWO can satisfy the need of maneuvering decision.

Sensitivity of Safe Trajectory in a Game Environment to Determine Inaccuracy of Radar Data in Autonomous Navigation.

This article provides an analysis of the autonomous navigation of marine objects, such as ships, offshore vessels and unmanned vehicles, and an analysis of the accuracy of safe control in game conditions for the cooperation of objects during maneuvering decisions. A method for determining safe object strategies based on a cooperative multi-person positional modeling game is presented. The method was used to formulate a measure of the sensitivity of safe control in the form of a relative change in the payment of the final game; to determine the final deviation of the safe trajectory from the set trajectory of the autonomous vehicle movement; and to calculate the accuracy of information in terms of evaluating the state of the control process. The sensitivity of safe control was considered in terms of both the degree of the inaccuracy of radar information and changes in the kinematics and dynamics of the object itself. As a result of the simulation studies of the positional game algorithm, which used an example of a real situation at sea of passing one's own object with nine other encountered objects, the sensitivity characteristics of safe trajectories under conditions of both good and restricted visibility at sea are presented.

Predictive Maneuver Planning for an Autonomous Vehicle in Public Highway Traffic

This paper outlines a predictive maneuver-planning method for autonomous vehicle navigating public highway traffic. The method integrates discrete maneuvering decisions, i.e., lane and reference speed selection automata, with a model predictive control-based motion trajectory-planning scheme. A key notion is to apply a predictive reference speed pre-planning for each lane at each time step of a selected prediction horizon. This is done based on the predicted likely motion of the autonomous vehicle and other object vehicles subject to sensor noise and environmental disturbances. Then, an optimization problem is configured that computes safe, sub-optimal plans for the trajectories of both the motion states (and inputs) and maneuver references for the prediction horizon to accomplish maneuvers like lane keeping, lane change, or obstacle avoidance. While a first formulation of the problem results in a mixed-integer nonlinear programming problem, it is shown that a relaxation can be adopted that reduces the computational complexity to a low-order polynomial time nonlinear program that can be solved more efficiently. Through simulation of a series of multi-lane highway scenarios and comparison with one-maneuver planning approach and an adaptive cruise control approach, the proposed predictive maneuver planning is illustrated to better accommodate the traffic environment with feasible execution time. Also, the reference speed pre-planning improves the optimality and the robustness of the maneuver decision in trajectory planning without adding computational complexity to the optimization problem.

Model Predictive Trajectory Planning for Automated Driving

In order to enable automated driving systems on the road, several key challenges need to be solved. One of these issues is real-time maneuver decision and trajectory planning. This paper introduces a general framework for maneuver and trajectory planning with model predictive methods. It discusses several representations of this framework in distinct complexity levels. In general, sophisticated models require to nonquadratic objective functions with nonlinear constraints, leading to increased computational complexities during calculation. Yet, oversimplified models can neither cope with vehicle dynamics in critical maneuvers nor do they represent complex traffic scenes with several maneuver options appropriately. A scheme to partition the trajectory space into homotopy regions is proposed. In each homotopy class, linearization about a trajectory from this class is applied. Demonstrations by simulation and with experimental vehicles show the capability of the proposed method in selecting optimal maneuvers and trajectories. This is even valid during extreme maneuvers, such as in last moment collision avoidance.

A differential game for cooperative target defense

Multi-player pursuit–evasion games are crucial for addressing the maneuver decision problem arising in the cooperative control of multi-agent systems. This work addresses a particular pursuit–evasion game with three players, Target, Attacker, and Defender. The Attacker aims to capture the Target, while avoiding being captured by the Defender and the Defender tries to defend the Target from being captured by the Attacker, while trying to capture the Attacker at an opportune moment. A two-pronged pursuit–evasion problem in this game is considered and we focus on two aspects: the cooperation between the Target and Defender and balancing the roles of the Attacker between pursuer and evader. A barrier based on the explicit policy method and geometric analysis method is constructed to separate the whole state space into two disjoint parts that correspond to two winning regions for the Attacker and Target–Defender team. The main contributions of this work are obtaining the players’ winning regions and providing a complete game solution by analyzing the optimal strategies and trajectories of the players based on the barrier.

Research on Air Confrontation Maneuver Decision-Making Method Based on Reinforcement Learning

With the development of information technology, the degree of intelligence in air confrontation is increasing, and the demand for automated intelligent decision-making systems is becoming more intense. Based on the characteristics of over-the-horizon air confrontation, this paper constructs a super-horizon air confrontation training environment, which includes aircraft model modeling, air confrontation scene design, enemy aircraft strategy design, and reward and punishment signal design. In order to improve the efficiency of the reinforcement learning algorithm for the exploration of strategy space, this paper proposes a heuristic Q-Network method that integrates expert experience, and uses expert experience as a heuristic signal to guide the search process. At the same time, heuristic exploration and random exploration are combined. Aiming at the over-the-horizon air confrontation maneuver decision problem, the heuristic Q-Network method is adopted to train the neural network model in the over-the-horizon air confrontation training environment. Through continuous interaction with the environment, self-learning of the air confrontation maneuver strategy is realized. The efficiency of the heuristic Q-Network method and effectiveness of the air confrontation maneuver strategy are verified by simulation experiments.

Dynamic Games Methods in Synthesis of Safe Ship Control Algorithms

The paper presents application of dynamic games methods, multistage positional and multistep matrix games, to automate the process control of moving objects, on the example of safe control of own ship in collision situations when passing many ships encountered. Taking into consideration two types of ships cooperation, for each of the two types of games, positional and matrix, four control algorithms for determining a safe ship trajectory supporting the navigator’s maneuvering decision in a collision situation are presented. The considerations are illustrated by examples of computer simulation in Matlab/Simulink software of safe trajectories of a ship in a real situation at sea. Taking into account the smallest final deviation game trajectory from the reference trajectory of movement, in good visibility at sea, the best is trajectory for cooperative matrix game, but in restricted visibility at sea, the best is trajectory for cooperative positional game.

Stochastic Dynamics of and Collision Prediction for Low Altitude Earth Satellites

Air drag B∗ factors from Earth satellite element sets often show the characteristic near Gaussian distribution and autocorrelation exponential decay typical of a Gauss-Markov process. Assuming the “most current” set of orbital elements are correct, earlier elements can be used to construct covariance matrices as a function of prediction time into the future. If resolved in cylindrical orbit frame coordinates, these are remarkably structured, essentially showing only in-track error growth. Often the in-track position covariance element growth follows a fourth power in time rule, and is definitely forced by the uncertainty in the air drag factor. This observation is confirmed both by perturbation theory and by modeling stochastic state covariance propagation. Realizing that almost all error growth under the SGP4 model is in track, the Cosmos 2251/Iridium 33 event is reexamined. While a collision prediction from the last elements shows a minimum miss distance of about 700 m, those same elements show a closest approach distance of the orbits of only 32 m. Given large in-track uncertainty, minimum orbit separation may be a much more reliable metric for maneuver decisions.

Visualizing complexities: the human limits of air traffic control

Air traffic management is organized into filters in order to prevent tactical controllers from dealing with complex conflicting situations. In this article, we describe an experiment showing that a dynamic conflict display could improve human performance on complex conflict situations. Specifically, we designed a display tool that represents the conflicting portions of aircraft trajectories and the evolution of the conflict zone when the user adds a maneuver to an aircraft. The tool allows the user to dynamically check the potential conflicting zones with the computer mouse before making a maneuver decision. We tested its utility on a population of forty students: twenty air traffic controller (ATC) students at the end of their initial training and twenty engineering students with the same background but no ATC training. They had to solve conflicts involving 2–5 aircraft with a basic display and with the dynamic visualization tool. Results show that in easy situations (2 aircraft), performance is similar with both displays. However, as the complexity of the situations grows (from 3 to 5 aircraft), the dynamic visualization tool enables users to solve the conflicts more efficiently. Using the tool leads to fewer unsolved conflicts and shorter delays. No significant differences are found between the two test groups except for delays: ATC students give maneuvers that generate less delays than engineering students. These results suggest that humans are better able to manage complex situations with the help of our conflict visualization tool.

Autonomous air combat maneuver decision using Bayesian inference and moving horizon optimization

To reach a higher level of autonomy for unmanned combat aerial vehicle (UCAV) in air combat games, this paper builds an autonomous maneuver decision system. In this system, the air combat game is regarded as a Markov process, so that the air combat situation can be effectively calculated via Bayesian inference theory. According to the situation assessment result, adaptively adjusts the weights of maneuver decision factors, which makes the objective function more reasonable and ensures the superiority situation for UCAV. As the air combat game is characterized by highly dynamic and a significant amount of uncertainty, to enhance the robustness and effectiveness of maneuver decision results, fuzzy logic is used to build the functions of four maneuver decision factors. Accuracy prediction of opponent aircraft is also essential to ensure making a good decision; therefore, a prediction model of opponent aircraft is designed based on the elementary maneuver method. Finally, the moving horizon optimization strategy is used to effectively model the whole air combat maneuver decision process. Various simulations are performed on typical scenario test and close-in dogfight, the results sufficiently demonstrate the superiority of the designed maneuver decision method.

Decision Tree-Based Maneuver Prediction for Driver Rear-End Risk-Avoidance Behaviors in Cut-In Scenarios

Predicting driver rear-end risk-avoidance maneuvers in cut-in scenarios, especially dangerous precrash scenarios, benefits the customization of automatic driving, particularly automatic steering. This paper studies driver rear-end risk-avoidance behaviors in cut-in scenarios on a straight three-lane highway. Data from 24 participants in 1326 valid trials were collected using a motion-based driving simulator. An Eysenck Personality Questionnaire (revised for Chinese participants) was used to obtain the personality traits of the participants. Based on a statistical analysis, the candidate features used in the driver maneuver prediction were determined as a combination of objective risk indicators and driver characteristics. A decision tree-based model was constructed for maneuver prediction in cut-in scenarios. The prediction accuracy of the extracted classification rules was 79.2% for the training data set and 80.3% for the test data set. The most powerful predictive variables were extracted, and their effects on maneuver decisions were analyzed. The results show that driver characteristics strongly influence the prediction of maneuver decisions.

An Approach to Mathematical Modeling and Estimation of Probe-Drogue Docking Success Probability for UAV Autonomous Aerial Refueling

One of the keys to the success of aerial refueling for probe-drogue aerial refueling system (PDARS) is the successful docking between the probe and drogue. The study of probe-drogue docking success probability offers an important support to achieving successful docking. During the docking phase of PDARS, based on prior information and reasonable assumptions for the movements of the drogue under atmospheric disturbance, the probe-drogue docking success probability is converted to the probability of the drogue center located in a specific area. A model of the probe-drogue docking success probability is established with and without actuation error, respectively. The curves of the probe-drogue docking success probability with the standard deviation of the drogue central position, the maximum distance from the drogue center position to the equilibrium position, the actuation error, and the standard deviation of the actuation error are obtained through simulations. The study has referential value for the docking maneuver decision of aerial refueling for PDARS.