Uncertain Dynamic Environments Research Articles

UAV Motion Strategies in Uncertain Dynamic Environments: A Path Planning Method Based on Q-Learning Strategy

A solution framework for UAV motion strategies in uncertain dynamic environments is constructed in this paper. Considering that the motion states of UAV might be influenced by some dynamic uncertainties, such as control strategies, flight environments, and any other bursting-out threats, we model the uncertain factors that might cause such influences to the path planning of the UAV, unified as an unobservable part of the system and take the acceleration together with the bank angle of the UAV as a control variable. Meanwhile, the cost function is chosen based on the tracking error, then the control instructions and flight path for UAV can be achieved. Then, the cost function can be optimized through Q-learning, and the best UAV action sequence for conflict avoidance under the moving threat environment can be obtained. According to Bellman’s optimization principle, the optimal action strategies can be obtained from the current confidence level. The method in this paper is more in line with the actual UAV path planning, since the generation of the path planning strategy at each moment takes into account the influence of the UAV control strategy on its motion at the next moment. The simulation results show that all the planning paths that are created according to the solution framework proposed in this paper have a very high tracking accuracy, and this method has a much shorter processing time as well as a shorter path it can create.

A Reactive Management System for Reliable Power Supply in a Building Microgrid with Vehicle-to-Grid Interaction

We address a BEMS (Building Energy Management System) to guarantee reliability of electric-power supply in dynamic uncertain environments. The building microgrid as the target of BEMS has multiple distributed power sources including a photo-voltaic power system and Electric-Vehicle (EV). EV is regarded as an autonomously-moving battery due to the original means of transportation and is hence a cause of dynamic uncertainty of the building microgrid. The main objective of synthesis of BEMS in this paper is to guarantee the continuous supply of power to the most critical load in a building microgrid and to realize the power supply to the other loads according to a ranking of load importance. We synthesize the BEMS as a reactive control system that monitors changes of dynamic uncertain environment of the microgrid including departure and arrival of an EV, and determines a route of power supply to the most critical load. Also, we conduct numerical experiments of the reactive BEMS using models of power flows in the building and of charging states of the batteries. The experiments are incorporated with data measured in a practical office building and demonstration project of EMS at Osaka, Japan. We show that the BEMS works for extending the time duration of continuous power supply to the most critical load.

Enhancing Creativity and Change of Nursing Management Staff and Its Influencing on Their Performance at Benha University Hospital

Currently, organizations work in very competitive and uncertain dynamic environment, which requires organizations to be flexible, to acquire and adapt new knowledge, new technologies and new processes to introduce new products, services, cut costs, act faster than competitors and remain competitive. Purpose of the study: Enhance creativity and change of nursing management staff and it's influencing on their performance at Benha University Hospital. Design: A quasi-experimental design was utilized. The subjects were 98 head nurses, their assistants. Instruments for data collection: four instruments were used: (1) Knowledge Questionnaire Sheet, (2) Nursing Management Staff Observation Checklist (3) Nursing Management Staff Attitude toward Creativity and Change at work Questionnaire. (4) Creativity Questionnaire. The result showed that, There was improvement in the level of knowledge, performance, attitude toward creativity, change and level of creativity throughout post and follow up from the preprogram, There was significant positive statistical correlation between total knowledge scores with total performance scores and attitude toward creativity of the studied nursing management staff at preprogram and follow up phase of the study, there was positive statistical significant correlation between attitude toward change with performance and attitude toward creativity of the studied nursing management staff at post program and follow up program. The study concluded that, Enhance creativity and change of nursing management staff and it's influencing on their performance at Benha University Hospital. The study recommended the following: Hospital administration should provide program about enhancing creativity and change for nursing management staff, Replication of the study on a larger probability sample is highly recommended to achieve generalization of results.

Collision‐Free Path‐Planning for Six‐DOF Serial Harvesting Robot Based on Energy Optimal and Artificial Potential Field

Collision‐free autonomous path planning under a dynamic and uncertainty vineyard environment is the most important issue which needs to be resolved firstly in the process of improving robotic harvesting manipulator intelligence. We present and apply energy optimal and artificial potential field to develop a path planning method for six degree of freedom (DOF) serial harvesting robot under dynamic uncertain environment. Firstly, the kinematical model of Six‐DOF serial manipulator was constructed by using the Denavit‐Hartenberg (D‐H) method. The model of obstacles was defined by axis‐aligned bounding box, and then the configuration space of harvesting robot was described by combining the obstacles and arm space of robot. Secondly, the harvesting sequence in path planning was computed by energy optimal method, and the anticollision path points were automatically generated based on the artificial potential field and sampling searching method. Finally, to verify and test the proposed path planning algorithm, a virtual test system based on virtual reality was developed. After obtaining the space coordinates of grape picking point and anticollision bounding volume, the path points were drew out by the proposed method. 10 times picking tests for grape anticollision path planning were implemented on the developed simulation system, and the success rate was up to 90%. The results showed that the proposed path planning method can be used to the harvesting robot.

Spline-Based Motion Planning in an Obstructed 3D Environment

This paper presents a motion planning approach that steers systems in an optimal way through an obstructed 3D environment. The motion trajectories are parameterized as polynomial splines and by exploiting the properties of B-spline basis functions, constraints on the trajectories are efficiently enforced. The approach is applied on two relevant cases. The first one elaborates a pick and place task for a Cartesian robot which is validated experimentally on an industrial plate transportation system. Depending on the task, the proposed method can reduce the motion time with 10 - 30% with respect to the currently applied trajectories. In a second case the approach is applied on the navigation of Unmanned Aerial Vehicles (UAVs) flying in an uncertain dynamic environment. This problem is formulated in a receding-horizon fashion which can update trajectories with a rate of 2.5 Hz. A supporting software toolbox is provided that implements the proposed approach and facilitates its use.

New robot navigation algorithm for arbitrary unknown dynamic environments based on future prediction and priority behavior

Abstract This study focuses on existing drawbacks and inefficiencies of the available path planning approaches within unknown dynamic environments. The drawbacks are the inability to plan under uncertain dynamic environments, non-optimality, failure in crowded complex situations, and difficulty in predicting the velocity vector of obstacles. This study aims (1) to develop a new predictive method to avoid static and dynamic obstacles in planning the path of a mobile robot in unknown dynamic environments in which the obstacles are moving and their speed profiles are not pre-identified, to find a safe path and to react rapidly and (2) to integrate a decision-making process with the predictive behavior of the velocity vector of obstacles by using the sensory system information of the robot. Information on the locations, shapes, and velocities of static and dynamic obstacles is presumed to be unavailable. Such information is determined online using rangefinder sensors. Thus, the robot recognizes free directions that lead it toward its destination and keep it safe and prevent collision with obstacles. Extensive simulations confirm the efficiency of the suggested approach and its success in handling complex and extremely dynamic environments that contain various obstacle shapes. Findings indicate that the proposed method exhibits attractive features, such as high optimality, high stability, low running time, and zero failure rates. The failure rate is zero for all test problems. The average path length for all test environments is 16.51 with a standard deviation of 0.49, which provides an average optimality rate of 89.79%. The average running time is 4.74 s (the standard deviation is 0.26).

A Novel Dynamic Obstacle Avoidance Algorithm Based on Collision Time Histogram

Robot path planning in uncertain dynamic environment is a hot issue in the field of Unmanned ground vehicle (UGV). Starting from the practical demands of UGV, we propose a novel dynamic obstacle avoidance algorithm based on Collision time histogram (CTH). Given current steering angle, an effective collision check model, which is called Collision check circles (CCC), is firstly calculated. The local environment information is then combined with CCC to generate the proposed CTH. The nonholonomic nature of the vehicle is embedded in this process. Finally, the proposed algorithm calculates the executing steering angle by considering both the CTH and the target point. Extensive experiments and comparisons are conducted to evaluate the performance of the proposed algorithm. Simulation experiments are firstly conducted to verify its feasibility. Furthermore, real-world experiment is conducted to verify its effectiveness. Experimental results demonstrate the practical value of the proposed algorithm.

Situation Assessment and Decision Making Strategies of Emergency Medical Services Physicians in Routine and Non-Routine Situations

Naturalistic decision making (NDM) research is focused on macrocognition - cognitive functions and processes used by experienced people working in real-world settings (Crandall, Klein, & Hoffman, 2006). These settings characterize ill-structured problems; uncertain dynamic environments; shifting, ill-defined, or competing goals; action/feedback loops; time stress; high stakes; multiple players; and organizational goals and norms (Orasanu & Connoly, 1993). Exactly such setting, with multifactorial influences on cognition, is health care. In this field evidence based medicine (EBM; Rosenberg & Donald, 1995) is preferred, representing the normative approach, sometimes difficult to meet in real-world situations. By contrast, the main idea of NDM is that professionals rely heavily on their previous experience stored in memory, as described by the Recognition-Primed Decision model (RPD model; Klein, 1997). It is viewed as the experiential basis of intuition that enables rapid recognition of situations and decisions. Simply put, intuition is recognition (Simon, 1992).Two of the primary macrocognitive functions are situation assessment and decision making (Klein et al., 2003), included in the RPD model. Situation assessment (SA) is about understanding of what is going on in the course of dynamic unfolding situations, during which elaboration of an existing SA or shifting to a new SA may occur repeatedly (Kaempf, Wolf, & Miller, 1993). From the SA viewpoint, clear situations are usually immediately identified (recognized), and ambiguous situations need to be diagnosed. In NDM, the term diagnosing has not been used only in a medical context, it has been used to express deliberative SA regarding different situational problems as well. Based on the review (Zsambok, Beach, & Klein, 1992), there are the following SA strategies included in the RPD model: pattern matching - is about matching larger observed patterns with remembered patterns of previous situations; feature matching - is about matching observed features with remembered features of previous situations; and mental simulation - is about imagining the likely causal chain of events that preceded the observed situation. Adopting the theory of two processes (Evans, 2007), pattern matching, as an intuitive process, is experiencebased, fast and cognitively effortless and mental simulation, as a deliberative process, is slower and more cognitively demanding (see Klein, 1999). Feature matching is a dominant SA strategy (Kaempf et al., 1993) that can be viewed as an intuitive process, when a decision maker immediately recognizes the features, and as a deliberative process, when a decision maker consciously notices the features and uses them to retrieve a hypothesis or to contrast different hypotheses, concerning the particular unclear situation (Zsambok et al., 1992).SA is usually followed by decision making (DM), which is defined as the selection of one option from a set of options (Klein, Calderwood, & Clinton-Cirocco, 2010) and is interleaved with acting in complex realistic situations (Orasanu & Conolly, 1993), requiring more than one decision. One way to make a decision is a straightforward identification of the situation along with an identification of the typical single option, which is usually automatically implemented. This experience-based, intuitive DM strategy, which corresponds with the RPD model, prevails in experienced people (Klein, 1999). However, when it is not clear what action to take, more cognitively demanding strategies are necessary. There are distinguished two types of option evaluation (Klein et al., 2010; Klein, Calderwood, & MacGregor, 1989): sequential evaluation, when only one option is examined at a time, and concurrent evaluation, when several options are considered at the same time. The first one is characteristic for mental simulation, as the way of imagining the likely consequences of a single option, arriving at a satisficing decision. …

Sustaining performance under operational turbulence

PurposeMany studies have found that Lean practices provide better performance in a stable business environment. However, there is limited information on how Lean practices influence performance gains (defined in this paper as improvement and sustenance of performance) in an uncertain (complex and dynamic) environment. This study aims to investigate how the implementation of Lean helps to sustain performance in such context.Design/methodology/approachThe study draws on an in-depth investigation of two capital goods manufacturing engineer-to-order (ETO) cases, in which performance sustenance is discussed in relation to the extent, locus and extensiveness of implemented Lean practice bundles.FindingsFindings indicate that a higher extent of Lean practices’ implementation, covering both shop floor and transactional processes, increases the possibility of performance sustenance in ETO. Furthermore, a coherent approach in the pre-, during- and post-implementation phases of the Lean change process is required to foster performance sustenance. Lean practices in ETO are modified to suit context change from repetitive manufacturing.Research limitations/implicationsThis study proposes performance sustenance as a performance measure in a highly uncertain context, such as ETO, as a single reference cannot effectively measure performance improvements over diverse orders. From this perspective, appropriate Lean implementation contributes towards building capabilities for flexibly and proactively managing uncertain circumstances.Practical implicationEven companies operating in highly uncertain (complex and dynamic) contexts may benefit from significant performance gains, thanks to the Lean implementation. This can be achieved by a balanced implementation of practices at shop floor and transactional processes, and their mindful customisations.Originality/valueThe study compares Lean implementation in ETO with that of high-volume–low-variety systems established in the literature. It qualitatively discusses how Lean implementation as an overarching effort both in shop floor and transactional processes leads to better sustenance of achieved performance improvements in shop floor under high uncertainty.

Multi-agent reinforcement learning using ordinal action selection and approximate policy iteration

Multi-agent reinforcement learning (MARL) provides a useful and flexible framework for multi-agent coordination in uncertain dynamic environments. However, the generalization ability and scalability of algorithms to large problem sizes, already problematic in single-agent RL, is an even more formidable obstacle in MARL applications. In this paper, a new MARL method based on ordinal action selection and approximate policy iteration called OAPI (Ordinal Approximate Policy Iteration), is presented to address the scalability issue of MARL algorithms in common-interest Markov Games. In OAPI, an ordinal action selection and learning strategy is integrated with distributed approximate policy iteration not only to simplify the policy space and eliminate the conflicts in multi-agent coordination, but also to realize the approximation of near-optimal policies for Markov Games with large state spaces. Based on the simplified policy space using ordinal action selection, the OAPI algorithm implements distributed approximate policy iteration utilizing online least-squares policy iteration (LSPI). This resulted in multi-agent coordination with good convergence properties with reduced computational complexity. The simulation results of a coordinated multi-robot navigation task illustrate the feasibility and effectiveness of the proposed approach.

Applying probability navigation function in dynamic uncertain environments

This paper introduces a novel motion planning algorithm for stochastic dynamic scenarios. We apply a Probability Navigation Function (PNF), discussed in the authors’ previous research work, to dynamic environments. We first consider the ambient configuration space to be an n- dimensional ball; the robot and the obstacles loci are all known with a Gaussian probability distribution, and both the robot and the obstacles are assumed to have n-dimensional disc shapes. We fuse the geometries of the robot and the obstacles with the localization probability distribution using convolution. We then define a Probability Navigation Function (PNF) φ from the configuration space to R. We also provide a numerical method for the case where the obstacles and the robot shapes are non-symmetric and their probability distributions are non-Gaussian respectively. The PNF is applied to the dynamic case, where the obstacles are moving at different velocities, by calculating consecutive probability navigation functions according to a prediction of the obstacles’ positions and their estimation error covariance. We then apply a simulated annealing scheme on the sequence of motion directions to choose an optimal path. We demonstrate our algorithm for various scenarios.

A Receding Horizon Control Strategy for Autonomous Vehicles in Dynamic Environments

In this brief, a receding horizon control scheme for solving the obstacle avoidance motion planning problem of autonomous vehicles operating in uncertain dynamic environments is developed. By considering robots described by discrete-time linear time-invariant models, we propose a set-theoretic-based architecture to deal with disturbance effects, and input and state constraints. As one of its main merits, the strategy is capable of taking care of time-varying obstacle scenarios by jointly exploiting robust positively invariant region and one-step controllable set concepts. The resulting scheme guarantees uniformly ultimate boundedness and constraints fulfilment regardless of any admissible obstacle configuration.

Verifying Collision Avoidance Behaviours for Unmanned Surface Vehicles using Probabilistic Model Checking

Abstract: Collision avoidance is an essential safety requirement for unmanned surface vehicles (USVs). Normally, its practical verification is non-trivial, due to the stochastic behaviours of both the USVs and the intruders. This paper presents the probabilistic timed automata (PTAs) based formalism for three collision avoidance behaviours of USVs in uncertain dynamic environments, which are associated with the crossing situation in COLREGs. Steering right, acceleration, and deceleration are considered potential evasive manoeuvres. The state-of-the-art prism model checker is applied to analyse the underlying models. This work provides a framework and practical application of the probabilistic model checking for decision making in collision avoidance for USVs.

An online model-based method for state of energy estimation of lithium-ion batteries using dual filters

The state-of-energy of lithium-ion batteries is an important evaluation index for energy storage systems in electric vehicles and smart grids. To improve the battery state-of-energy estimation accuracy and reliability, an online model-based estimation approach is proposed against uncertain dynamic load currents and environment temperatures. Firstly, a three-dimensional response surface open-circuit-voltage model is built up to improve the battery state-of-energy estimation accuracy, taking various temperatures into account. Secondly, a total-available-energy-capacity model that involves temperatures and discharge rates is reconstructed to improve the accuracy of the battery model. An extended-Kalman-filter and particle-filter based dual filters algorithm is then developed to establish an online model-based estimator for the battery state-of-energy. The extended-Kalman-filter is employed to update parameters of the battery model using real-time battery current and voltage at each sampling interval, while the particle-filter is applied to estimate the battery state-of-energy. Finally, the proposed approach is verified by experiments conducted on a LiFePO4 lithium-ion battery under different operating currents and temperatures. Experimental results indicate that the battery model simulates battery dynamics robustly with high accuracy, and the estimates of the dual filters converge to the real state-of-energy within an error of ±4%.

Approaches to Learning to Control Dynamic Uncertainty

In dynamic environments, when faced with a choice of which learning strategy to adopt, do people choose to mostly explore (maximizing their long term gains) or exploit (maximizing their short term gains)? More to the point, how does this choice of learning strategy influence one’s later ability to control the environment? In the present study, we explore whether people’s self-reported learning strategies and levels of arousal (i.e., surprise, stress) correspond to performance measures of controlling a Highly Uncertain or Moderately Uncertain dynamic environment. Generally, self-reports suggest a preference for exploring the environment to begin with. After which, those in the Highly Uncertain environment generally indicated they exploited more than those in the Moderately Uncertain environment; this difference did not impact on performance on later tests of people’s ability to control the dynamic environment. Levels of arousal were also differentially associated with the uncertainty of the environment. Going beyond behavioral data, our model of dynamic decision-making revealed that, in actual fact, there was no difference in exploitation levels between those in the highly uncertain or moderately uncertain environments, but there were differences based on sensitivity to negative reinforcement. We consider the implications of our findings with respect to learning and strategic approaches to controlling dynamic uncertainty.

Structured Analysis of Critical Decision Method Data – Emergency Medicine Case Study

Abstract The critical decision method is a semi-structured retrospective interview used mainly in researching professional decision making within the naturalistic decision making paradigm. There are currently, two main methods of performing analysis: ETA - Emergent Themes Analysis and the structured approach (Wong, 2004). The structured approach consists of five steps: 1. decision chart, 2. incident summary, 3. decision analysis tables, 4. identification of items of interest and 5. collation and comparison of common items of interest across incidents. Naturalistic decision making can be time-consuming, involve high stakes, poorly-structured problems, uncertain dynamic environments and ill-defined or competing goals. For this reason we will demonstrate how to the structured approach to perform an analysis using the example of a decision making situation in emergency medicine.

Modeling and Analysis of the Obstacle-Avoidance Strategies for a Mobile Robot in a Dynamic Environment

Obstacle avoidance is a key performance of mobile robots. However, its experimental verification is rather difficult, due to the probabilistic behaviors of both the robots and the obstacles. This paper presents the Markov Decision Process based probabilistic formal models for three obstacle-avoidance strategies of a mobile robot in an uncertain dynamic environment. The models are employed to make analyses in PRISM, and the correctness of the analysis results is verified by MATLAB simulations. Finally, the minimum time and the energy consumption are determined by further analyses in PRISM, which prove to be useful in finding the optimal strategy. The present work provides a foundation for the probabilistic formal verification of more complicated obstacle-avoidance strategies.

Hybrid Motion Planning Method for Autonomous Robots Using Kinect Based Sensor Fusion and Virtual Plane Approach in Dynamic Environments

A new reactive motion planning method for an autonomous vehicle in dynamic environments is proposed. The new dynamic motion planning method combines a virtual plane based reactive motion planning technique with a sensor fusion based obstacle detection approach, which results in improving robustness and autonomy of vehicle navigation within unpredictable dynamic environments. The key feature of the new reactive motion planning method is based on a local observer in the virtual plane which allows the effective transformation of complex dynamic planning problems into simple stationary in the virtual plane. In addition, a sensor fusion based obstacle detection technique provides the pose estimation of moving obstacles by using a Kinect sensor and a sonar sensor, which helps to improve the accuracy and robustness of the reactive motion planning approach in uncertain dynamic environments. The performance of the proposed method was demonstrated through not only simulation studies but also field experiments using multiple moving obstacles even in hostile environments where conventional method failed.

A semi-autonomous motorized mobile hospital bed for safe transportation of head injury patients in dynamic hospital environments without bed switching

SUMMARYWe present a novel motorized semi-autonomous mobile hospital bed guided by a human operator and a reactive navigation algorithm. The proposed reactive navigation algorithm is launched when the sensory device detects that the hospital bed is in the potential danger of collision. The semi-autonomous hospital bed is able to safely and quickly deliver critical neurosurgery (head trauma) patients to target locations in dynamic uncertain hospital environments such as crowded hospital corridors while avoiding en-route steady and moving obstacles. We do not restrict the nature or the motion of the obstacles, meaning that the shapes of the obstacles may be time-varying or deforming and they may undergo arbitrary motions. The only information available to the navigation system is the current distance to the nearest obstacle. Performance of the proposed navigation algorithm is verified via theoretical studies. Simulation and experimental results also confirm the performance of the reactive navigation algorithm in real world scenarios.

Posterior Belief Clustering Algorithm For Energy-Efficient Tracking In Wireless Sensor Networksd

abstract In this paper, we propose a novel posterior belief clustering (PBC) algorithm to solve the tradeoff between target tracking performance and sensors energy consumption in wireless sensor networks. We model the target tracking under dynamic uncertain environment using partially observable Markov decision processes (POMDPs), and transform the optimization of the tradeoff between tracking performance and energy consumption into yielding the optimal value function of POMDPs. We analyze the error of a class of continuous posterior beliefs by Kullback–Leibler (KL) divergence, and cluster these posterior beliefs into one based on the error of KL divergence. So, we calculate the posterior reward value only once for each cluster to eliminate repeated computation. The numerical results show that the proposed algorithm has its effectiveness in optimizing the tradeoff between tracking performance and energy consumption.