Uncertain Dynamic Environments Research Articles

Based on Deep Reinforcement Learning to path planning in uncertain ocean currents for Underwater Gliders

This paper studies the application of Deep Reinforcement Learning (DRL) in the field of underwater glider (UG) formation path planning and solves the path planning of glider formation in uncertain ocean currents. Facing the uncertain ocean current environment, based on the observation and collection of ocean current data by the overall ocean observation system, the glider formation needs to have the ability of path planning. The original DRL algorithm network architecture is simple, and the feature extraction ability is limited. According to the characteristics of ocean current and path planning, a Deep Convolutional Neural Network for uncertain ocean current environment (Duo-CNN) network architecture with a dual-branch structure is proposed. Based on the DRL, the glider model and the interaction between the glider and the environment are used to guide the path planning through the implicit prediction of the known ocean current data. A solution for formation construction, obstacle avoidance, and path planning of glider formation in a dynamic uncertain ocean current environment is provided. The simulation verifies the effectiveness of the path planning of the glider formation using the DRL of the dual-branch Duo-CNN architecture. Compared with other architectures with feature extraction ability, Duo-CNN has advantages.

POBO: Safe and Optimal Resource Management for Cloud Microservices

Resource management in microservices is challenging due to the uncertain latency-resource relationship, dynamic environment, and strict Service-Level Agreement (SLA) guarantees. This paper presents a Pessimistic and Optimistic Bayesian Optimization framework, named POBO, for safe and optimal resource configuration for microservice applications. POBO leverages Bayesian learning to estimate the uncertain latency-resource functions and combines primal-dual and penalty-based optimization to maximize resource efficiency while guaranteeing strict SLAs. We prove that POBO can achieve sublinear regret and SLA violation compared with the optimal resource configuration in hindsight. We have implemented a prototype of POBO and conducted extensive experiments on a real-world microservice application. Our results show that POBO can find the safe and optimal configuration efficiently, outperforming Kubernetes' built-in auto-scaling module and the state-of-the-art algorithm.

DEVELOPMENT OF AN INTELLIGENT ROBOT CONTROL SUBSYSTEM

The subsystem should provide the following functionalities: planning of trajectories of movement of an intelligent robot in an a priori uncertain dynamic environment of operation: representation of opposing objects of the environment and functional and executive nodes of a mobile robot using a fuzzy configuration space; formation of a movement trajectory with a fixed level of confidence; updating the environment map when exploring new areas of the operating environment; real-time operation of the scheduler; modularity and scalability of the subsystem. To successfully navigate in space, the robot control system must be able to build a route, control movement parameters, correctly interpret information about the world around it received from sensors, and constantly track its own coordinates. The paper investigates the development of control subsystems for an intelligent robot. To achieve this goal, the initial data were analyzed, the general principle of building a simulation model of the robot was described, and a structural diagram of the control system of an intelligent robot was developed.

Convex risk-bounded continuous-time trajectory planning and tube design in uncertain nonconvex environments

In this paper, we address the trajectory planning problem in uncertain nonconvex static and dynamic environments that contain obstacles with probabilistic location, size, and geometry. To address this problem, we provide a risk-bounded trajectory planning method that looks for continuous-time trajectories with guaranteed bounded risk over the planning time horizon. Risk is defined as the probability of collision with uncertain obstacles. Existing approaches to address risk-bounded trajectory planning problems either are limited to Gaussian uncertainties and convex obstacles or rely on sampling-based methods that need uncertainty samples and time discretization. To address the risk-bounded trajectory planning problem, we leverage the notion of risk contours to transform the risk-bounded planning problem into a deterministic optimization problem. Risk contours are the set of all points in the uncertain environment with guaranteed bounded risk. The obtained deterministic optimization is, in general, nonlinear and nonconvex time-varying optimization. We provide convex methods based on sum-of-squares optimization to efficiently solve the obtained nonconvex time-varying optimization problem and obtain the continuous-time risk-bounded trajectories without time discretization. The provided approach deals with arbitrary (and known) probabilistic uncertainties, nonconvex and nonlinear, static and dynamic obstacles, and is suitable for online trajectory planning problems. In addition, we provide convex methods based on sum-of-squares optimization to build the max-sized tube with respect to its parameterization along the trajectory so that any state inside the tube is guaranteed to have bounded risk.

OVERVIEW OF THE CONCEPTS AND APPROACHES FOR BUDGET PLANNING IN ENTERPRISES IN THE BULGARIAN SCIENTIFIC LITERATURE

In the conditions of a rapidly changing and uncertain dynamic environment, increasing competition, and limited resources, the global economic crisis brought planning and control to the fore as the leading functions of management. Modern practice is characterized by a significant variety of tools, approaches and methods in the implementation of planning in enterprises. The purpose of this publication is to review the concepts and tools for budget planning in the enterprise, which are known in the Bulgarian scientific literature, indexed in the scientific database Google Scholar at the end of March 2023. Methodologically, the review used a systematic approach. The review was conducted in three stages: developing a search protocol, conducting the search and reporting the results. The literature sample of review sources covers eleven scientific publications corresponding to the researched topic. The results of the review show that many Bulgarian authors recommend the application of quantitative methods and approaches for enterprise budget planning. As such, the balanced scorecard and tools of fuzzy set theory are indicated. A number of authors emphasize the need for the application of a flexible and dynamic toolkit for budget planning, which enables the assessment and successful management of internal and external risk. For crisis conditions, the authors most often recommend scenario planning and heuristic models. Some authors highlight the kaizen concept and controlling as attractive and successful planning concepts. Other authors point to expert systems based on artificial intelligence as a particularly valuable tool for planning. In the "discussion" part of the present study, some recommendations for future research are given. In the opinion of the author, this development has the character of a landmark for the state of the Bulgarian scientific literature about the concepts and tools for budget planning in the enterprise. Despite the theoretical nature of the study, it can be used by both researchers and management practice.

Online Weapon-target Assignment based on Distributed Auction Mechanism

To solve the problem of online weapon-target assignment (OWTA) in the integration of large-scale search and attack in unknown environment, an OWTA algorithm based on distributed auction mechanism is presented. Aiming at the problem that the traditional combinatorial optimization algorithm needs to set up the global battlefield situation in advance, considering the consumability of resources in the attack process, the integrated search and attack task flow is established. Considering the communication restricted environment, the unmanned aerial vehicles (uavs) are grouped, with centralized architecture within the group and distributed structure between the groups, and the corresponding distributed auction mechanism is constructed to achieve OWTA within the communication range limited. In order to solve the problem that it is difficult to ensure the time consistency of the coordinated attack target, a dubins cooperative path planning based on cooperative particle swarm optimization (CPSO) algorithm is proposed. Particle swarm optimization algorithm is used to adjust the radius of the dubins path of each bomb, so that the uav in the same group can hit the target simultaneously without collision and have the shortest flight range. The simulation results show that the designed distributed auction algorithm takes into account the consumption of attack resources, and quickly redistributes the firepower to the new targets in the dynamic uncertain environment, which ensures the maximization of the execution efficiency of the multi-machine cluster fire allocation task.

RAST: Risk-Aware Spatio-Temporal Safety Corridors for MAV Navigation in Dynamic Uncertain Environments

Autonomous navigation of Micro Aerial Vehicles (MAVs) in dynamic and unknown environments is a complex and challenging task. Current works rely on assumptions to solve the problem. The MAV's pose is precisely known, the dynamic obstacles can be explicitly segmented from static ones, their number is known and fixed, or they can be modeled with given shapes. In this letter, we present a method for MAV navigation in dynamic uncertain environments without making any of these assumptions. The method employs a particle-based dynamic map to represent the local environment and predicts it to the near future. Collision risk is defined based on the predicted maps and a series of risk-aware spatio-temporal (RAST) safety corridors are constructed, which are finally used to optimize a dynamically-feasible collision-free trajectory for the MAV. We compared our method with several state-of-the-art works in 12000 simulation tests in Gazebo with the physical engine enabled. The results show that our method has the highest success rate at different uncertainty levels. Finally, we validated the proposed method in real experiments.

When DSA Meets SWIPT: A Joint Power Allocation and Time Splitting Scheme Based on Multi-Agent Deep Reinforcement Learning

Dynamic spectrum access (DSA) and simultaneously wireless information and power transfer (SWIPT) are two promising approaches to address the spectrum and energy supply problems. However, most existing works studied these two techniques separately, and how to effectively realize them simultaneously is non-trivial, because the power allocation of DSA and time splitting of SWIPT will influence both data transmission and energy harvesting performance. Hence, a sophisticated joint design is necessary, which, however, is challenging considering the uncertain dynamic spectrum environment. In this paper, we propose a joint power allocation and time splitting scheme to facilitate devices to implement both DSA and SWIPT simultaneously, which aims at maximizing the long-term throughput while ensuring the interference limitation and energy supply requirements. Considering the environmental uncertainty, we develop a multi-agent deep reinforcement learning solution to make each device autonomously select the best decision that could optimize the whole network performance based on its local observations. Simulation results have shown the effectiveness of the proposed scheme.

Co-evolutionary hybrid intelligence is a key concept for the world intellectualization

PurposeThis study aims to show the inconsistency of the approach to the development of artificial intelligence as an independent tool (just one more tool that humans have developed); to describe the logic and concept of intelligence development regardless of its substrate: a human or a machine and to prove that the co-evolutionary hybridization of the machine and human intelligence will make it possible to reach a solution for the problems inaccessible to humanity so far (global climate monitoring and control, pandemics, etc.).Design/methodology/approachThe global trend for artificial intelligence development (has been) was set during the Dartmouth seminar in 1956. The main goal was to define characteristics and research directions for artificial intelligence comparable to or even outperforming human intelligence. It should be able to acquire and create new knowledge in a highly uncertain dynamic environment (the real-world environment is an example) and apply that knowledge to solving practical problems. Nowadays artificial intelligence overperforms human abilities (playing games, speech recognition, search, art generation, extracting patterns from data etc.), but all these examples show that developers have come to a dead end. Narrow artificial intelligence has no connection to real human intelligence and even cannot be successfully used in many cases due to lack of transparency, explainability, computational ineffectiveness and many other limits. A strong artificial intelligence development model can be discussed unrelated to the substrate development of intelligence and its general properties that are inherent in this development. Only then it is to be clarified which part of cognitive functions can be transferred to an artificial medium. The process of development of intelligence (as mutual development (co-development) of human and artificial intelligence) should correspond to the property of increasing cognitive interoperability. The degree of cognitive interoperability is arranged in the same way as the method of measuring the strength of intelligence. It is stronger if knowledge can be transferred between different domains on a higher level of abstraction (Chollet, 2018).FindingsThe key factors behind the development of hybrid intelligence are interoperability – the ability to create a common ontology in the context of the problem being solved, plan and carry out joint activities; co-evolution – ensuring the growth of aggregate intellectual ability without the loss of subjectness by each of the substrates (human, machine). The rate of co-evolution depends on the rate of knowledge interchange and the manufacturability of this process.Research limitations/implicationsResistance to the idea of developing co-evolutionary hybrid intelligence can be expected from agents and developers who have bet on and invested in data-driven artificial intelligence and machine learning.Practical implicationsRevision of the approach to intellectualization through the development of hybrid intelligence methods will help bridge the gap between the developers of specific solutions and those who apply them. Co-evolution of machine intelligence and human intelligence will ensure seamless integration of smart new solutions into the global division of labor and social institutions.Originality/valueThe novelty of the research is connected with a new look at the principles of the development of machine and human intelligence in the co-evolution style. Also new is the statement that the development of intelligence should take place within the framework of integration of the following four domains: global challenges and tasks, concepts (general hybrid intelligence), technologies and products (specific applications that satisfy the needs of the market).

Adaptive Tank-based Control for Aerial Physical Interaction with Uncertain Dynamic Environments Using Energy-Task Estimation

While aerial manipulation has witnessed noticeable growth as a field in the last decade, most works investigated forms of interaction with static and rigid environments only. Whenever dynamic environments were considered, the employed methods often relied on the knowledge of the model of the environment, which in most real applications cannot be obtained. In this work, we propose an adaptive controller for a fully actuated UAV performing stable and efficient physical interaction tasks with unmodeled and dynamic objects moving in unknown environments. We develop a passive time-varying impedance controller and wrench tracking controller, whose adaptable parameters allow us to minimize tracking error and instabilities during the execution of the interaction task. Robust stability is guaranteed by energy tanks, with the addition of a task-based formulation for adapting online the tank parameters in order to always provide the system with an adequate amount of energy. The control framework is validated both in simulations and experimentally by interacting with an unmodeled cart moving in passive time-varying environments, while subjected to unknown disturbances.

Risk-Aware Fast Trajectory Planner for Uncertain Environments Based on Probabilistic Surrogate Reliability and Risk Contours

This letter presents the <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">r</u> isk- <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">a</u> ware <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">f</u> ast <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">t</u> rajectory plann <underline xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">er</u> (RAFTER) for autonomous vehicles in dynamic uncertain environments, which is based on the probabilistic surrogate reliability of other traffic participants and risk contours. In contrast to the conventional risk metric, RAFTER not only provides the upper bound of the probability of constraint violation but deduces an infimum on the risk which the controlled plant can stand by the probabilistic reliable surrogate model. Such a risk-aware algorithm is capable of perceiving uncertainty and handling robustness. A series of covering disks are constructed utilizing a concise geometric configuration for a lower conservatism representation of the vehicle profile, which attains a desirable tradeoff between the quantity and area of occupation. Safe travel corridors are built on the dilated map via covering disks, significantly reducing the computational burden of a reliability-based optimization procedure for optimal trajectory planning. The effectiveness of the proposed method is confirmed by two numerical simulations derived from real scenarios.

PX4 Simulation Results of a Quadcopter with a Disturbance-Observer-Based and PSO-Optimized Sliding Mode Surface Controller

This work designed a disturbance-observer-based nonlinear sliding mode surface controller (SMC) and validated the controller using a simulated PX4-conducted quadcopter. To achieve this goal, this research (1) developed a dynamic mathematical model; (2) built a PX4-based simulated UAV following the model-based design process; (3) developed appropriate sliding mode control laws for each degree of freedom; (4) implemented disturbance observers on the proposed SMC controller to achieve finer disturbance rejection such as crosswind effect and other mutational disturbances; (5) optimized the SMC controller’s parameters based on particle swarm optimization (PSO) method; and (6) evaluated and compared the quadcopter’s tracking performance under a range of noise and disturbances. Comparisons of PID control strategies against the SMC were documented under the same conditions. Consequently, the SMC controller with disturbance observer facilitates accurate and fast UAV adaptation in uncertain dynamic environments.

Solving Chance-Constrained Optimization Under Nonparametric Uncertainty Through Hilbert Space Embedding

In this article, we present an efficient algorithm for solving a class of chance-constrained optimization under nonparametric uncertainty. Our algorithm is built on the possibility of representing arbitrary distributions as functions in Reproducing Kernel Hilbert Space (RKHS). We use this foundation to formulate chance-constrained optimization as one of minimizing the distance between a desired distribution and the distribution of the constraint functions in the RKHS. We provide a systematic way of constructing the desired distribution based on the notion of scenario approximation. Furthermore, we use the kernel trick to show that the computational complexity of our reformulated optimization problem is comparable to solving a deterministic variant of the chance-constrained optimization. We validate our formulation on two important robotic applications: 1) reactive collision avoidance of mobile robots in uncertain dynamic environments and 2) inverse-dynamics-based path-tracking of manipulators under perception uncertainty. In both these applications, the underlying chance constraints are defined over nonlinear and nonconvex functions of uncertain parameters and possibly also decision variables. We also benchmark our formulation with the existing approaches in terms of sample complexity and the achieved optimal cost highlighting significant improvements in both these metrics.

Reconfigurable Intelligent Surface Enabled Vehicular Communication: Joint User Scheduling and Passive Beamforming

Given its ability to control and manipulate wireless environments, reconfigurable intelligent surface (RIS), also known as intelligent reflecting surface (IRS), has emerged as a key enabler technology for the six-generation (6G) cellular networks. In the meantime, vehicular environment radio propagation is negatively influenced by a large set of objects that cause transmission distortion such as high buildings. Therefore, this work is devoted to explore the area of RIS technology integration with vehicular communications while considering the dynamic nature of such communication environment. Specifically, we provide a system model where RoadSide Unit (RSU) leverages RIS to provide indirect wireless transmissions to disconnected areas, known as dark zones. Dark zones are spots within RSU coverage where the communication links are blocked due to the existence of blockages. In details, a discrete RIS is utilized to provide communication links between the RSU and the vehicles passing through out-of-service zones. Therefore, the joint problem of RSU resource scheduling and RIS passive beamforming or phase-shift matrix is formulated as an optimization problem with the objective of maximizing the minimum average bit rate. The formulated problem is mixed integer non-convex program which is difficult to be solved and does not account for the uncertain dynamic environment in vehicular networks. Thereby, we resort to alternative methods based on Deep Reinforcement Learning to determine RSU wireless scheduling and Block Coordinate Descent (BCD) to solve for the phase-shift matrix, <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">i.e.,</i> passive beamforming, of the RIS. The Markov Decision Process (MDP) is defined and the complexity of the solution approach is discussed. Our numerical results demonstrate the superiority of our proposed approach over baseline techniques.

Three-Dimensional Obstacle Avoidance Strategy for Fixed-Wing UAVs Based on Quaternion Method

This work provides a generalization of the three-dimensional velocity obstacle (VO) collision avoidance strategy for nonlinear second-order underactuated systems in three-dimensional dynamic uncertain environments. A hierarchical architecture is exploited to deal with conflicting multiple subtasks, which are defined as several rotations and are parameterized by quaternions. An improved VO method considering the kinodynamic constraints of a class of fixed-wing unmanned aerial vehicles (UAV) is proposed to implement the motion planning. The position error and velocity error can be mapped onto one desired axis so that, only relying on an engine, UAVs can achieve the goal of point tracking without collision. Additionally, the performance of the closed-loop system is demonstrated through a series of simulations performed in a three-dimensional manner.

Research on information security interaction based on behavior trust measurement mechanism

Along with the computer, communication, control, the rapid development of automation technology, based on the analysis of the information security requirements, combined with the trusted computing provide security solution, the communication control information security interaction model and the key technology to realize information security interaction, put forward a real-time evaluation model based on credible global terminal entities. By introducing penalty factor and time factor, the evaluation method is improved from single evaluation to global evaluation. Taking the user information collection system as an example, the information security communication model of behavior trust measurement mechanism is simulated. The simulation results show that this model has the ability of continuous trust measurement in dynamic uncertain network environment, and can accurately determine the trust of terminal entities in real time, and it is more realistic, which lays a good foundation for the research of trust connection, trust management and trust decision based on trust measurement.

Self-Optimizing Path Tracking Controller for Intelligent Vehicles Based on Reinforcement Learning

The path tracking control system is a crucial component for autonomous vehicles; it is challenging to realize accurate tracking control when approaching a wide range of uncertain situations and dynamic environments, particularly when such control must perform as well as, or better than, human drivers. While many methods provide state-of-the-art tracking performance, they tend to emphasize constant PID control parameters, calibrated by human experience, to improve tracking accuracy. A detailed analysis shows that PID controllers inefficiently reduce the lateral error under various conditions, such as complex trajectories and variable speed. In addition, intelligent driving vehicles are highly non-linear objects, and high-fidelity models are unavailable in most autonomous systems. As for the model-based controller (MPC or LQR), the complex modeling process may increase the computational burden. With that in mind, a self-optimizing, path tracking controller structure, based on reinforcement learning, is proposed. For the lateral control of the vehicle, a steering method based on the fusion of the reinforcement learning and traditional PID controllers is designed to adapt to various tracking scenarios. According to the pre-defined path geometry and the real-time status of the vehicle, the interactive learning mechanism, based on an RL framework (actor–critic—a symmetric network structure), can realize the online optimization of PID control parameters in order to better deal with the tracking error under complex trajectories and dynamic changes of vehicle model parameters. The adaptive performance of velocity changes was also considered in the tracking process. The proposed controlling approach was tested in different path tracking scenarios, both the driving simulator platforms and on-site vehicle experiments have verified the effects of our proposed self-optimizing controller. The results show that the approach can adaptively change the weights of PID to maintain a tracking error (simulation: within ±0.071 m; realistic vehicle: within ±0.272 m) and steering wheel vibration standard deviations (simulation: within ±0.04°; realistic vehicle: within ±80.69°); additionally, it can adapt to high-speed simulation scenarios (the maximum speed is above 100 km/h and the average speed through curves is 63–76 km/h).

Multilayer Decision-Based Fuzzy Logic Model to Navigate Mobile Robot in Unknown Dynamic Environments

The investigation into mobile robot navigation under uncertain dynamic environments is of great significance. This paper seeks to solve the current problems which are the difficulty to plan in indeterminate ever-changing environments, the problem of optimality, failure in complex situations, and the problem of predicting the obstacle velocity vector. The objective of this study is to propose a multilayer decision-based fuzzy logic model to find the solution for robot navigation through a safe path while preventing any types of barriers and to understand the non-collision mobile robots’ movement in an unknown dynamic environment. In this study, the prediction and priority rules of a multilayer decision are used by the fuzzy logic controller to improve the quality of the next position with regard to its path length, safety, and runtime. The results of comparison studies revealed a considerable improvement in failure rate and path length. Outcomes show that the suggested method displays attractive features, for instance, great stability, great optimality, zero failure rates, and low running time. The average path length for all test environments is 13.11 with 0.47 a standard deviation that provides 89% of an average optimality rate. The average running time is about 5.31 s with a 0.25 standard deviation.

Post-Pandemic Education: Falling or Preparing for a Jump?

Abstract. Relevance. The article provides an overview and analysis of the consequences of changes in the global educational space during the period of action of measures to counter the spread of the new COVID-19 coronavirus infection.The aim of the work was to understand the current state and identify possible trajectories of developmentof educational systems in the “post-pandemic” era. Research methodology and methods. The main methods used in the preparation of the work were methods of analysis and synthesis. The review and understanding of various standpoints on the educational organizations functioning in the situation of emergency introduction of quarantine measures and the uncertainty of their consequences allowed us to summarize the main challenges facing the education system today. The forecasting method was used to systematize the capabilities of universities in overcoming crisis phenomena and determine the strengths that can be used as the basis for the transformation of University management.Results. The analysis of the trajectories of the prospective organization of the educational process at the University of “post-pandemic” time is carried out. The article highlights the main ways of the educational system adaptation to work in the conditions of an unpredictable crisis consequences and presents the authors’ vision of the future of higher education in Russia, which is inextricably linked with the country’s attractiveness in the global educational space.Scientific novelty. The authors systematize the impact of the pandemic on various sectors of the economy, including the education sector. The results of the analysis of the practical experience of educational systems in the new conditions are presented. Scenarios of educational organizations development in an unfavorable dynamic uncertain environment are shown.Practical significance. The results obtained will be useful in adjusting University management systems and developing strategies for universities to overcome the crisis caused by COVID-19 and its consequences.

Scenario-Based Trajectory Optimization in Uncertain Dynamic Environments

We present an optimization-based method to plan the motion of an autonomous robot under the uncertainties associated with dynamic obstacles, such as humans. Our method bounds the marginal risk of collisions at each point in time by incorporating chance constraints into the planning problem. This problem is not suitable for online optimization outright for arbitrary probability distributions. Hence, we sample from these chance constraints using an uncertainty model, to generate “scenarios,” which translate the probabilistic constraints into deterministic ones. In practice, each scenario represents the collision constraint for a dynamic obstacle at the location of the sample. The number of theoretically required scenarios can be very large. Nevertheless, by exploiting the geometry of the workspace, we show how to prune most scenarios before optimization and we demonstrate how the reduced scenarios can still provide probabilistic guarantees on the safety of the motion plan. Since our approach is scenario based, we are able to handle arbitrary uncertainty distributions. We apply our method in a Model Predictive Contouring Control framework and demonstrate its benefits in simulations and experiments with a moving robot platform navigating among pedestrians, running in real-time.