Uncertain Dynamic Environments Research Articles

Multilayer Decision-Based Fuzzy Logic Model to Navigate Mobile Robot in Unknown Dynamic Environments

The investigation into mobile robot navigation under uncertain dynamic environments is of great significance. This paper seeks to solve the current problems which are the difficulty to plan in indeterminate ever-changing environments, the problem of optimality, failure in complex situations, and the problem of predicting the obstacle velocity vector. The objective of this study is to propose a multilayer decision-based fuzzy logic model to find the solution for robot navigation through a safe path while preventing any types of barriers and to understand the non-collision mobile robots’ movement in an unknown dynamic environment. In this study, the prediction and priority rules of a multilayer decision are used by the fuzzy logic controller to improve the quality of the next position with regard to its path length, safety, and runtime. The results of comparison studies revealed a considerable improvement in failure rate and path length. Outcomes show that the suggested method displays attractive features, for instance, great stability, great optimality, zero failure rates, and low running time. The average path length for all test environments is 13.11 with 0.47 a standard deviation that provides 89% of an average optimality rate. The average running time is about 5.31 s with a 0.25 standard deviation.

Post-Pandemic Education: Falling or Preparing for a Jump?

Abstract. Relevance. The article provides an overview and analysis of the consequences of changes in the global educational space during the period of action of measures to counter the spread of the new COVID-19 coronavirus infection.The aim of the work was to understand the current state and identify possible trajectories of developmentof educational systems in the “post-pandemic” era. Research methodology and methods. The main methods used in the preparation of the work were methods of analysis and synthesis. The review and understanding of various standpoints on the educational organizations functioning in the situation of emergency introduction of quarantine measures and the uncertainty of their consequences allowed us to summarize the main challenges facing the education system today. The forecasting method was used to systematize the capabilities of universities in overcoming crisis phenomena and determine the strengths that can be used as the basis for the transformation of University management.Results. The analysis of the trajectories of the prospective organization of the educational process at the University of “post-pandemic” time is carried out. The article highlights the main ways of the educational system adaptation to work in the conditions of an unpredictable crisis consequences and presents the authors’ vision of the future of higher education in Russia, which is inextricably linked with the country’s attractiveness in the global educational space.Scientific novelty. The authors systematize the impact of the pandemic on various sectors of the economy, including the education sector. The results of the analysis of the practical experience of educational systems in the new conditions are presented. Scenarios of educational organizations development in an unfavorable dynamic uncertain environment are shown.Practical significance. The results obtained will be useful in adjusting University management systems and developing strategies for universities to overcome the crisis caused by COVID-19 and its consequences.

Scenario-Based Trajectory Optimization in Uncertain Dynamic Environments

We present an optimization-based method to plan the motion of an autonomous robot under the uncertainties associated with dynamic obstacles, such as humans. Our method bounds the marginal risk of collisions at each point in time by incorporating chance constraints into the planning problem. This problem is not suitable for online optimization outright for arbitrary probability distributions. Hence, we sample from these chance constraints using an uncertainty model, to generate “scenarios,” which translate the probabilistic constraints into deterministic ones. In practice, each scenario represents the collision constraint for a dynamic obstacle at the location of the sample. The number of theoretically required scenarios can be very large. Nevertheless, by exploiting the geometry of the workspace, we show how to prune most scenarios before optimization and we demonstrate how the reduced scenarios can still provide probabilistic guarantees on the safety of the motion plan. Since our approach is scenario based, we are able to handle arbitrary uncertainty distributions. We apply our method in a Model Predictive Contouring Control framework and demonstrate its benefits in simulations and experiments with a moving robot platform navigating among pedestrians, running in real-time.

Dynamic Multi-agent Real Time Scheduling Framework for Production Management

Production management is a complex problem. The problem becomes more complex in open, uncertain, dynamic environments and distributed productions. The existing static scheduling methods are not efficient to deal with this problem efficiently. This paper proposed multi-agent approach for real time scheduling in production management to efficiently manage production. The proposed scheduling optimizes the performance of production management in dynamic uncertain environment. Dynamic multi-agent approach in real time scheduling consist of the agent collect and analysis the production data in real time and ensure real time response to emergency events. Real time scheduling agents increase the resource utilization and success rate in production management by using real time feedback.

The Dynamic Extensions of Fuzzy Grey Cognitive Maps

Fuzzy Cognitive Map is recognized as an important model in soft computing. The Fuzzy Cognitive Map theory studies often treat uncertainty modeling and dynamic modeling separately. So far, there is no single Fuzzy Cognitive Map that can deal with uncertain data and dynamic environments simultaneously. This paper proposed an environment model to describe the link between changing weights and the dynamic environment. As the extensions of the classic Fuzzy Grey Cognitive Map, two dynamic models were designed and implemented in this work: Dynamic Fuzzy Grey Cognitive Map and Dynamic Fuzzy General Grey Map. In this work, we also analyze the characteristics of the two models and evaluate their performance using industrial control problems. The results showed that the proposed models could well handle uncertain data in dynamic environments.

UAV Dynamic Path Planning Based on Obstacle Position Prediction in an Unknown Environment

The application of unmanned aerial vehicle (UAV) has been increasingly popular for its advantages such as convenience and mobility. Thus, its application scenarios have been more and more complex. The UAV must avoid not only stationary obstacles but also dynamic obstacles. Typical UAV path planning algorithms perform well in avoiding static obstacles but poor in dynamic ones. A new dynamic path planning algorithm based on obstacles’ position prediction and modified artificial potential field - HOAP is proposed in this paper. The Markov prediction model is employed to predict the obstacles’ future position with an obstacle grid map. And to resolve the local minima of the typical APF algorithm, a new virtual obstacle method is put forward. What’s more, the attractive force gain coefficient gradient increase method is proposed to solve local oscillation. Simulation results show that the UAV can fly a safe path with high accuracy under static or uncertain dynamic obstacles environments.

Formation Control With Collision Avoidance Through Deep Reinforcement Learning Using Model-Guided Demonstration.

Generating collision-free, time-efficient paths in an uncertain dynamic environment poses huge challenges for the formation control with collision avoidance (FCCA) problem in a leader-follower structure. In particular, the followers have to take both formation maintenance and collision avoidance into account simultaneously. Unfortunately, most of the existing works are simple combinations of methods dealing with the two problems separately. In this article, a new method based on deep reinforcement learning (RL) is proposed to solve the problem of FCCA. Especially, the learning-based policy is extended to the field of formation control, which involves a two-stage training framework: an imitation learning (IL) and later an RL. In the IL stage, a model-guided method consisting of a consensus theory-based formation controller and an optimal reciprocal collision avoidance strategy is designed to speed up training and increase efficiency. In the RL stage, a compound reward function is presented to guide the training. In addition, we design a formation-oriented network structure to perceive the environment. Long short-term memory is adopted to enable the network structure to perceive the information of obstacles of an uncertain number, and a transfer training approach is adopted to improve the generalization of the network in different scenarios. Numerous representative simulations are conducted, and our method is further deployed to an experimental platform based on a multiomnidirectional-wheeled car system. The effectiveness and practicability of our proposed method are validated through both the simulation and experiment results.

A biologically-inspired reinforcement learning based intelligent distributed flocking control for Multi-Agent Systems in presence of uncertain system and dynamic environment

In this paper, we investigate the real-time flocking control of Multi-Agent Systems (MAS) in the presence of system uncertainties and dynamic environment. To handle the impacts from system uncertainties and dynamic environment, a novel reinforcement learning technique, which is appropriate for real-time implementation, has been integrated with multi-agent flocking control in this paper. The Brain Emotional Learning Based Intelligent Controller (BELBIC) is a biologically-inspired reinforcement learning-based controller relying on a computational model of emotional learning in the mammalian limbic system. The learning capabilities, multi-objective properties, and low computational complexity of BELBIC make it a very promising learning technique for implementation in real-time applications. Firstly, a novel brain emotional learning-based flocking control structure is proposed. Then, the real-time update laws are developed to tune the emotional signals based on real-time operational data. It is important to note that this data-driven reinforcement learning approach relaxes the requirement for system dynamics and effectively handle the uncertain impacts of the environment. Using the tuned emotional signals, the optimal flocking control can be obtained. The Lyapunov analysis has been used to prove the convergence of the proposed design. The effectiveness of the proposed design is also demonstrated through numerical and experimental results based on the coordination of multiple Unmanned Aerial Vehicles (UAVs).

A linear matrix inequality–based proportional-derivative sliding mode control tuning method in robotic systems subjected to external disturbance

The proportional-derivative sliding-mode control will be designed and tuned in the trajectory tracking of a robot manipulator which operates on uncertain dynamic environments. For achieving these goals, first, a linear matrix inequality–based framework is suggested to design a robust proportional-derivative sliding-mode control in the presence of external disturbances. Next, the parameters of the proportional-derivative sliding-mode control law will be tuned via another minimization problem subjected to some linear matrix inequality constraints. Thus, the controller parameters can be automatically updated via the solution of the optimization problem. The results are successfully used in the robot manipulator with considering two reference paths and some different loads. The simulation results show the effectiveness of the proposed method in comparison with the same technique.

Reactive Navigation Under Non-Parametric Uncertainty Through Hilbert Space Embedding of Probabilistic Velocity Obstacles

The probabilistic velocity obstacle (PVO) extends the concept of velocity obstacle (VO) to work in uncertain dynamic environments. In this paper, we show how a robust model predictive control (MPC) with PVO constraints under non-parametric uncertainty can be made computationally tractable. At the core of our formulation is a novel yet simple interpretation of our robust MPC as a problem of matching the distribution of PVO with a certain desired distribution. To this end, we propose two methods. Our first baseline method is based on approximating the distribution of PVO with a Gaussian Mixture Model (GMM) and subsequently performing distribution matching using Kullback Leibler (KL) divergence metric. Our second formulation is based on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to interpret our robust MPC as a problem of minimizing the distance between the desired distribution and the distribution of the PVO in the RKHS. Both the RKHS and GMM based formulation can work with any uncertainty distribution and thus allowing us to relax the prevalent Gaussian assumption in the existing works. We validate our formulation by taking an example of 2D navigation of quadrotors with a realistic noise model for perception and ego-motion uncertainty. In particular, we present a systematic comparison between the GMM and the RKHS approach and show that while both approaches can produce safe trajectories, the former is highly conservative and leads to poor tracking and control costs. Furthermore, RKHS based approach gives better computational times that are up to one order of magnitude lesser than the computation time of the GMM based approach.

An intelligent cooperative mission planning scheme of UAV swarm in uncertain dynamic environment

This paper presents an intelligent cooperative mission planning scheme for unmanned aerial vehicle (UAV) swarm, to search and attack the time-sensitive moving targets in uncertain dynamic environment, by using a hybrid artificial potential field and ant colony optimization (HAPF-ACO) method. In the search-attack mission environment of UAV swarm under the dynamic topology interaction, a time-sensitive target probability map is established. Based on the HAPF, the target attraction field, threat repulsive field and repulsive field are constructed for the environmental cognition. A distributed ACO algorithm is designed to improve the UAVs' global searching capability. For this mission planning problem, four time-sensitive moving target types and four constraint types of UAV swarm are considered, which will contribute to the practical applications of the HAPF-ACO. Several simulations are carried out to exhibit the superiority on the task execution efficiency and obstacle and collision avoidance performance of the proposed intelligent cooperative mission planning scheme.

Optimization in Uncertain and Complex Dynamic Environments with Evolutionary Methods

Optimization in Uncertain and Complex Dynamic Environments with Evolutionary Methods

Robust Motion Control for UAV in Dynamic Uncertain Environments Using Deep Reinforcement Learning

In this paper, a novel deep reinforcement learning (DRL) method, and robust deep deterministic policy gradient (Robust-DDPG), is proposed for developing a controller that allows robust flying of an unmanned aerial vehicle (UAV) in dynamic uncertain environments. This technique is applicable in many fields, such as penetration and remote surveillance. The learning-based controller is constructed with an actor-critic framework, and can perform a dual-channel continuous control (roll and speed) of the UAV. To overcome the fragility and volatility of original DDPG, three critical learning tricks are introduced in Robust-DDPG: (1) Delayed-learning trick, providing stable learnings, while facing dynamic environments; (2) adversarial attack trick, improving policy’s adaptability to uncertain environments; (3) mixed exploration trick, enabling faster convergence of the model. The training experiments show great improvement in its convergence speed, convergence effect, and stability. The exploiting experiments demonstrate high efficiency in providing the UAV a shorter and smoother path. While, the generalization experiments verify its better adaptability to complicated, dynamic and uncertain environments, comparing to Deep Q Network (DQN) and DDPG algorithms.

A cubic spline method combing improved particle swarm optimization for robot path planning in dynamic uncertain environment

This article considers a robot path planning problem originated from a robot factory inspection scenario. In the problem, the robot is in a dynamic uncertain environment, that is, a moving target object and several static and dynamic obstacles. An inertial positioning strategy is proposed to enable the robot to predict the position of the target in advance. From this predicted position, the robot path is generated by cubic spline interpolation, and then an improved particle swarm optimization algorithm with a random positive feedback factor in velocity updating optimizes the path. The experimental results show that the proposed method can successfully avoid the obstacles and reach the target object. In addition, the inertial positioning strategy and the improvement of particle swarm optimization can effectively shorten the path of the robot.

Robot Navigation among External Autonomous Agents through Deep Reinforcement Learning using Graph Attention Network

Abstract Finding collision-free and efficient paths in an uncertain dynamic environment is a challenge for robot navigation tasks, especially when there are external autonomous agents that also have decision-making abilities in the same environment. This paper develops a novel method based on DRL with graph attention network (GAT) to solve the problem of robot navigation among external autonomous agents (other agents). Specifically, GAT is adopted to describe the robot and other agents as a specific graph, and extract the spatial structural influence features of other agents on the robot from the graph. Multi-head attention mechanism is utilized to calculate the weights of interactions between the robot and other agents. This GAT uses observations of an arbitrary number of other agents in dynamic environments. Furthermore, the proposed method is combined with optimal reciprocal collision avoidance to improve its safety in new environments. Various simulations demonstrate that our method has good performance and robustness in different environments.

UAV Swarm Intelligence: Recent Advances and Future Trends

The dynamic uncertain environment and complex tasks determine that the unmanned aerial vehicle (UAV) system is bound to develop towards clustering, autonomy, and intelligence. In this article, we present a comprehensive survey of UAV swarm intelligence from the hierarchical framework perspective. Firstly, we review the basics and advances of UAV swarm intelligent technology. Then we look inside to investigate the research work by classifying UAV swarm intelligence research into five layers, i.e., decision-making layer, path planning layer, control layer, communication layer, and application layer. Furthermore, the relationship between each level is explicitly illustrated, and the research trends of each layer are given. Finally, limitations and possible technology trends of swarm intelligence are also covered to enable further research interests. Through this in-depth literature review, we intend to provide novel insights into the latest technologies in UAV swarm intelligence.

Robot Variable Impedance Skill Transfer and Learning Framework Based on a Simplified Human Arm Impedance Model

A variable impedance skill transfer framework is introduced and experimentally evaluated in this article. This framework mainly includes the following: 1. a real-time estimation algorithm of arm endpoint stiffness based on the human arm endpoint impedance model, and 2. the tele-impedance demonstration (tele-demonstration) method and Gaussian mixture model (GMM)-based learning from demonstration (LfD) method, which are designed according to the impedance model and the variable impedance skill transfer process. The modelling of the human arm endpoint impedance is inspired by human operation habits. The model can modify the task stiffness in three Cartesian directions based on the arm configuration and muscle co-contraction effect. The model parameters and their values are identified by performing a perturbation-based arm endpoint impedance identification experiment. The real-time impedance model is used for tele-demonstration and obtaining multimodality learning data, including human arm position and impedance information. These multimodality learning data are used for the variable impedance skill learning and generalization, and an electrical switch skill is learned and generalized by a compliant robotic arm in the experiments to execute a variable impedance interaction task in a dynamic uncertain environment.

A Q-Cube Framework of Reinforcement Learning Algorithm for Continuous Double Auction among Microgrids

Decision-making of microgrids in the condition of a dynamic uncertain bidding environment has always been a significant subject of interest in the context of energy markets. The emerging application of reinforcement learning algorithms in energy markets provides solutions to this problem. In this paper, we investigate the potential of applying a Q-learning algorithm into a continuous double auction mechanism. By choosing a global supply and demand relationship as states and considering both bidding price and quantity as actions, a new Q-learning architecture is proposed to better reflect personalized bidding preferences and response to real-time market conditions. The application of battery energy storage system performs an alternative form of demand response by exerting potential capacity. A Q-cube framework is designed to describe the Q-value distribution iteration. Results from a case study on 14 microgrids in Guizhou Province, China indicate that the proposed Q-cube framework is capable of making rational bidding decisions and raising the microgrids’ profits.

An Adaptive Iterative Learning Based Impedance Control for Robot-Aided Upper-Limb Passive Rehabilitation.

In this paper, an anthropomorphic arm is introduced and used to the upper-limb passive rehabilitation therapy. The anthropomorphic arm is constructed via pneumatic artificial muscles so that it may assist patients suffering upper-limb diseases to achieve mild therapeutic exercises. Due to the uncertain dynamic environment, external disturbances and model uncertainties, a combined control is proposed to stabilize and to enhance the adaptivity of the system. In the combined control, an iterative learning control is used to realize accurate position tracking. Meanwhile, an adaptive iterative learning based impedance control is proposed to execute the appropriate contact force during the therapy of the upper-limb. The advantage of the combined control is that it doesn't depend on the accurate model of systems and it may deal with highly nonlinear system which has strong coupling and redundancies. The convergence of the proposed control is analyzed in detail. Numerical simulations are performed to verify the proposed control method. In addition, real experiments are executed on the Southwest anthropomorphic arm.

An evolutionary approach for the target search problem in uncertain environment

Search path planning is critical to achieve efficient information-gathering tasks in dynamic uncertain environments. Given task complexity, most proposed approaches rely on various strategies to reduce computational complexity, from deliberate simplifications or ad hoc constraint relaxation to fast approximate global search methods utilization often focusing on a single objective. However, problem-solving search techniques designed to compute near-optimal solutions largely remain computationally prohibitive and are not scalable. In this paper, a new information-theoretic evolutionary anytime path planning algorithm is proposed to solve a dynamic search path planning problem in which a fleet of homogeneous unmanned aerial vehicles explores a search area to hierarchically minimize target zone occupancy uncertainty, lateness, and travel/discovery time respectively. Conditioned by new observation outcomes and request events, the evolutionary algorithm episodically solves an augmented static open-loop search path planning model over a receding time horizon incorporating any anticipated information feedback. The proposed approach has shown to outperform alternate myopic and greedy heuristics, significantly improving relative information gain at the expense of modest additional travel cost.