Systems In Uncertain Environments Research Articles

Application of Bayesian Networks and Reinforcement Learning in Intelligent Control Systems in Uncertain Environments

<p>Reinforcement learning is a machine learning paradigm that focuses on how an agent can perform actions in an environment to achieve a certain goal. The agent learns through interaction with the environment, observing the state and making decisions to maximize its reward. Reinforcement learning has wide applications in intelligent control systems. However, one limitation of reinforcement learning is the uncertainty in handling the environment model. Usually, reinforcement learning is performed without a clear model, which requires estimating environmental uncertainty and state transitions. Bayesian Networks are effective in modeling uncertainty, which can aid in establishing a probabilistic model of environmental dynamics. This allows for the integration of uncertainty information into the environmental model, leading to a more accurate understanding of the dynamic characteristics of the environment. In this study, we propose a reinforcement learning algorithm based on Bayesian Networks. We utilize optimal generalized residual differentiation, parallel integration causal directional reasoning, and other modeling techniques to address reinforcement learning tasks. The main idea is to utilize the prior distribution to estimate the uncertainty of unknown parameters. Then, the obtained observation information is used to calculate the posterior distribution in order to acquire knowledge. Experiments demonstrate that this approach is feasible in intelligent control systems operating in uncertain environments.</p> <p> </p>

Exact and Approximate Moment Derivation for Probabilistic Loops With Non-Polynomial Assignments

Many stochastic continuous-state dynamical systems can be modeled as probabilistic programs with nonlinear non-polynomial updates in non-nested loops. We present two methods, one approximate and one exact, to automatically compute, without sampling, moment-based invariants for such probabilistic programs as closed-form solutions parameterized by the loop iteration. The exact method applies to probabilistic programs with trigonometric and exponential updates and is embedded in the Polar tool. The approximate method for moment computation applies to any nonlinear random function as it exploits the theory of polynomial chaos expansion to approximate non-polynomial updates as the sum of orthogonal polynomials. This translates the dynamical system to a non-nested loop with polynomial updates, and thus renders it conformable with the Polar tool that computes the moments of any order of the state variables. We evaluate our methods on an extensive number of examples ranging from modeling monetary policy to several physical motion systems in uncertain environments. The experimental results demonstrate the advantages of our approach with respect to the current state-of-the-art.

Extreme values of solution of Caputo–Hadamard uncertain fractional differential equation and applications

Uncertain fractional differential equation (UFDE) is a useful tool for studying complex systems in uncertain environments. The mathematical characteristics of solution of an UFDE are also widely used in various fields. In this paper, we give the extreme value theorems of solution of Caputo–Hadamard UFDE and applications. A numerical algorithm for obtaining the inverse uncertainty distributions (IUDs) for extreme values of solution of Caputo–Hadamard UFDE is presented; the stability and feasibility of the proposed algorithm are validated by numerical experiments. As an application of extreme value theorems in uncertain financial market, the pricing formulas of the American option based on the new uncertain stock model are given. Besides, the algorithms for computing the price of the American option without explicit pricing formulas based on the Simpson's rule are designed. Finally, the price fluctuation of the American option is illustrated by numerical experiments.

Editorial for the special issue “Control‐theoretic approaches for systems in the life sciences”

Editorial for the special issue “Control<scp>‐theoretic</scp> approaches for systems in the life sciences”

Adaptation Through Prediction: Multisensory Active Inference Torque Control

Adaptation to external and internal changes is of major importance for robotic systems in uncertain environments. Here, we present a novel multisensory active inference (AIF) torque controller for industrial arms that shows how prediction can be used to resolve adaptation. Our controller, inspired by the predictive brain hypothesis, improves the capabilities of current AIF approaches by incorporating learning and multimodal integration of low- and high-dimensional sensor inputs (e.g., raw images) while simplifying the architecture. We performed a systematic evaluation of our model on a 7DoF Franka Emika Panda robot arm by comparing its behavior with previous AIF baselines and classic controllers, analyzing both qualitatively and quantitatively adaptation capabilities and control accuracy. The results showed improved control accuracy in goal-directed reaching with high noise rejection due to multimodal filtering, and adaptability to dynamical inertial changes, elasticity constraints, and human disturbances without the need to relearn the model or parameter retuning.

The Robust Emergency Medical Facilities Location-Allocation Models under Uncertain Environment: A Hybrid Approach

In emergency medical facilities location, the hierarchical diagnosis and treatment system plays an obvious role in the rational allocation of medical resources and improving the use efficiency of medical resources. However, few studies have investigated the operational mechanism of hierarchical medical systems in uncertain environments. To address this research gap, this paper proposes a hybrid approach for emergency medical facilities’ location-allocation. In the first stage, in order to concentrate on the utilization of medical resources, we choose alternative facility points from the whole facilities through the entropy weight method (EWM). In the second stage, uncertainty sets are used to describe the uncertain number of patients at emergency medical points more accurately. We propose a robust model to configure large base hospitals based on the robust optimization method. Furthermore, the proposed robust models are applied to the emergency management of Huanggang City under COVID-19. The results show that the optimal emergency medical facility location-allocation scheme meets the actual treatment needs. Simultaneously, the disturbance ratio and uncertainty level have a significant impact on the configuration scheme.

Moment estimation for parameters in high-order uncertain differential equations

As a type of differential equations with high-order derivatives of uncertain processes, high-order uncertain differential equations are widely applied to modelling dynamic systems in uncertain environment, which usually involve unknown parameters to be estimated. Since observations are always discrete in practice, based on these discrete observations of solution processes, we propose moment estimations for unknown parameters by Euler method approximation of high-order uncertain differential equations. Matching sample moments with corresponding population moments, a system of equations whose solution is the moment estimation of the set of unknown parameters is derived. Finally, some examples illustrate our method in detail.

Optimal Scheduling of Integrated Demand Response-Enabled Community-Integrated Energy Systems in Uncertain Environments

The community integrated energy system (CIES) is an essential energy internet carrier that has recently been the focus of much attention. A scheduling model based on chance-constrained programming is proposed for integrated demand response (IDR)-enabled CIES in uncertain environments to minimize the system operating costs, where an IDR program is used to explore the potential interaction ability of electricity-gas-heat flexible loads and electric vehicles. Moreover, power to gas (P2G) and micro-gas turbine (MT), as links of multi-energy carriers, are adopted to strengthen the coupling of different energy subsystems. Sequence operation theory (SOT) and linearization methods are employed to transform the original model into a solvable mixed-integer linear programming model. Simulation results on a practical CIES in North China demonstrate an improvement in the CIES operational economy via the coordination of IDR and renewable uncertainties, with P2G and MT enhancing the system operational flexibility and user comprehensive satisfaction. The CIES operation is able to achieve a trade-off between economy and system reliability by setting a suitable confidence level for the spinning reserve constraints. Besides, the proposed solution method outperforms the Hybrid Intelligent Algorithm in terms of both optimization results and calculation efficiency.

Data-driven decision process for robust scheduling of remanufacturing systems

Robust scheduling problem is a major decision problem that is addressed in the literature, especially for remanufacturing systems; this problem is complex because of the high uncertainty and complex constraints involved. Generally, the existing approaches are dedicated to specific processes and do not enable the quick and efficient generation and evaluation of schedules. With the emergence of the Industry 4.0 paradigm, data availability is now considered an opportunity to facilitate the decision-making process. In this study, a data-driven decisionmaking process is proposed to treat the robust scheduling problem of remanufacturing systems in uncertain environments. In particular, this process generates simulation models based on a data-driven modeling approach. A robustness evaluation approach is proposed to answer several decision questions. An application of the decision process in an industrial case of a remanufacturing system is presented herein, illustrating the impact of robustness evaluation results on real-life decisions.

Hierarchical stochastic scheduling of multi-community integrated energy systems in uncertain environments via Stackelberg game

An operating entity utilizing community-integrated energy systems with a large number of small-scale distributed energy sources can easily trade with existing distribution markets. To solve the energy management and pricing problem of multi-community integrated energy systems (MCIESs) with multi-energy interaction, this study investigated a hierarchical stochastic optimal scheduling method for uncertain environments. To handle multiple uncertainties, a Wasserstein generative adversarial network with a gradient penalty was used to generate renewable scenarios, and the Kmeans++ clustering algorithm was employed to generate typical scenarios. A Stackelberg-based hierarchical stochastic schedule with an integrated demand response was constructed, where the MCIES operator acted as the leader pursuing the maximum net profit by setting energy prices, while the building users were followers who adjusted their energy consumption plans to minimize their total costs. Finally, a distributed iterative solution method based on a metaheuristic was designed. The effectiveness of the proposed method was verified using practical examples.

Planning polyphase behavior of autonomous intelligent mobile systems in uncertain environments

Introduction: We discuss the modern ways of developing intelligent problem solvers, focusing on their shortcomings in terms of the efficiency of their application for planning purposeful behavior of autonomous mobile intelligent systems in a priori undescribed conditions of a problem environment. Purpose: Developing a model of knowledge representation and processing which would provide the ways to organize purposeful activity of autonomous intelligent mobile systems in uncertain environment. Methods: Synthesis of frame-like behavior scenarios in the form of polyvariable conditionally dependent predicates whose structure includes complex variables as well as related variables of types “object”, “event” and “relationship”; synthesis of heuristic rules for knowledge representation in the process of purposeful behavior planning. In order to represent complex variables in polyvariable conditionally dependent predicates, fuzzy semantic networks are used which can represent knowledge of variously purposed intelligent systems without regard to particular knowledge domains, being adaptable to a priori undescribed operational conditions. Results: We have proposed a structure of various polyvariable conditionally dependent predicates. On their base, an autonomous intelligent mobile system can organize various activities in a priori undescribed and unstable problem environments. Specially developed knowledge processing tools allow such a system to automatically plan its purposeful behavior in a space of subtasks during the fulfilment of tasks formulated for it. Practical relevance: The obtained results can be efficiently used in building intelligent problem solvers for autonomous intelligent mobile systems of various purpose, capable of performing complex tasks in a priori undescribed operational conditions.

Multiobjective Modeling and Optimization for Scheduling a Stochastic Hybrid Flow Shop With Maximizing Processing Quality and Minimizing Total Tardiness

Currently, manufacturing enterprises attach great importance to improving processing quality and customer satisfaction. Hybrid flow shops have widespread applications in real-world manufacturing systems such as steel production and chemical industry. In a practical production process, uncertainty commonly arises due to the difficulty of knowing exact information of facilities and jobs beforehand. In order to improve processing quality and customer satisfaction of manufacturing systems in uncertain environments, this article proposes a stochastic multiobjective hybrid flow shop scheduling problem aiming at maximizing processing quality and minimizing total tardiness, where the processing time of jobs obeys a known random distribution. To describe jobs' processing quality mathematically, a quality-based cost function is presented, and further a chance-constrained programming approach is used to formulate this problem. Then, a multiobjective artificial bee colony algorithm incorporating a stochastic simulation approach is designed by considering its characteristics. Simulation experiments are performed on a set of instances and several state-of-the-art multiobjective optimization algorithms are chosen as peer approaches. Experiment results confirm that the proposed algorithm has an excellent performance in handling this problem.

A fuzzy multi-objective optimization model for recoverable manufacturing systems in uncertain environments

ABSTRACTThis study develops a fuzzy multi-objective linear programming (FMOLP) model with an interactive two-phase possibilistic linear programming (ITPPLP) approach for solving remanufacturing production system problems with multiple goals in a fuzzy environment. Because the related parameters are imprecise/fuzzy in nature, a triangular possibility distribution can be used. The proposed model aims to simultaneously minimize total production costs and total lead time, while integrating multiple suppliers, multiple components, multiple time periods, and multiple machines to realize a remanufacturing production system. An air-conditioning compressor case was used to test the proposed models. The analytical results presented in this work will allow decision-making managers to better understand the systematic analysis, potential for cost-effectiveness and lead time of recoverable remanufacturing planning.

Controller design scheme for stabilization and synchronization of a class of chaotic and hyperchaotic systems in uncertain environment using SMC approach

This paper presents sliding mode control (SMC) technique for stabilization and synchronization of a class of chaotic/hyperchaotic systems in master–slave configuration. Here, a detailed procedure for deriving a single controller using SMC technique based proportional-integral sliding surface is proposed. Adaptation laws have been derived for the system parameters when the systems are subject to parametric uncertainties. Further, explicit criteria to decide the minimum number of control inputs required to meet out the desired operation of stabilization and synchronization is also proposed. Lyapunov stability theory is utilized to accomplish the desired objective. The proposed controller ensures the occurrence of sliding motion and achieves synchronization for the addressed class of chaotic/hyperchaotic systems in master–slave configuration. To validate the analytical results, example of four dimensional Lorenz–Stenflo hyperchaotic system is considered. Finally, detailed simulation results are provided to illustrate the effectiveness of the proposed controller.

Distributed Fictitious Play for Multiagent Systems in Uncertain Environments

A multiagent system operates in an uncertain environment about which agents have different and time-varying probability distributions on the environment—beliefs—that, as time progresses, converge to a common probability distribution, i.e., an asymptotic belief. A global utility function that depends on the realized state of the environment and actions of all the agents determines the system's optimal behavior. We define the asymptotically optimal action profile as a Nash equilibrium of the potential game defined by considering the expected utility with respect to the asymptotic belief. At finite time, however, agents have not entirely congruous beliefs about the state of the environment and may select conflicting actions. This paper proposes a variation of the fictitious play algorithm, which is proven to converge to equilibrium actions if the state beliefs converge to a common distribution at a rate that is at least linear. In conventional fictitious play, agents infer about others’ future behavior by computing histograms of past actions and best respond to their expected payoffs integrated with respect to these histograms. In the variations developed here, histograms are built using knowledge of actions taken by nearby nodes and best responses are further integrated with respect to the local beliefs on the state of the environment. We exemplify the use of the algorithm in coordination games.

Event-Driven Cooperative Receding Horizon Control for Multi-Agent Systems in Uncertain Environments

We propose an event-driven Cooperative Receding Horizon (CRH) controller to solve maximum reward collection problems (MRCP) where multiple agents cooperate to maximize the total reward collected from a set of targets in a given mission space. In previous work, a CRH controller was developed and in this paper, we overcome several limitations of this controller, including potential instabilities in the agent trajectories and poor performance due to inaccurate estimation of a reward-to-go function. Rewards are non-increasing functions of time and the environment is uncertain with new targets detected by agents at random time instants. The controller sequentially solves optimization problems over a planning horizon and executes the control for a shorter action horizon, where both are defined by certain events associated with new information becoming available. In contrast to the earlier CRH controller, we reduce the originally infinite-dimensional feasible control set to a finite set at each control update event. We prove some properties of this new controller and include simulation results showing its improved performance.

Dynamical Scheduling and Robust Control in Uncertain Environments with Petri Nets for DESs

This paper is about the incremental computation of control sequences for discrete event systems in uncertain environments where uncontrollable events may occur. Timed Petri nets are used for this purpose. The aim is to drive the marking of the net from an initial value to a reference one, in minimal or near-minimal time, by avoiding forbidden markings, deadlocks, and dead branches. The approach is similar to model predictive control with a finite set of control actions. At each step only a small area of the reachability graph is explored: this leads to a reasonable computational complexity. The robustness of the resulting trajectory is also evaluated according to a risk probability. A sufficient condition is provided to compute robust trajectories. The proposed results are applicable to a large class of discrete event systems, in particular in the domains of flexible manufacturing. However, they are also applicable to other domains as communication, computer science, transportation, and traffic as long as the considered systems admit Petri Nets (PNs) models. They are suitable for dynamical deadlock-free scheduling and reconfiguration problems in uncertain environments.

Anxious relationships: The unmarked futures for post-normal scenarios in anticipatory systems

This article explores organisational anticipation in uncertain times. ‘Anticipation’ is interpreted as a mediating process between knowledge and action, where ‘feed-forward’ is causal. The context for examining organisational anticipation is one of ontological insecurity; raising issues of epistemological and therefore methodological uncertainty. The paper draws on re-emerging areas of study in the futures literature especially with respect to anticipatory systems and post-normal science. Rosen's 1985 theory of anticipatory systems is not well known, though has received recent attention as part of a growing discourse on Anticipation, for example as a possible discipline and as a form of governance. For Rosen, causality is mediated through a modelling relationship between actor and environment which entails causality, not by the direct effect of the environment on the actor. The paper discusses the implications of this perspective on the role of scenario planning in organisations, which is but one of multiple anticipatory systems at work in the organisation and hence often weak in power. The argument is further developed by considering ‘modelling relations’ which are inherent to active anticipatory systems. The conclusion is that in human social systems in uncertain environments require approaches to anticipation that recognise the multiplicity of modelling relations. One approach to this has been set out in earlier work by Funtowicz and Ravetz (1993), which they called post-normal science. The paper concludes by suggesting that the epistemology of anticipatory systems and methodology developed from PNS might be used to reduce Cartesian anxiety with respect to ontological insecurities of uncertain times. This has radical implications for scenario planning as it is currently conceived.

Risk Sensitivity of Price of Anarchy under Uncertainty

In game theory, the price of anarchy framework studies efficiency loss in decentralized environments. Optimization and decision theory, on the other hand, explore tradeoffs between optimality and robustness in the case of single-agent decision making under uncertainty. What happens when we combine both approaches?We examine connections between the efficiency loss due to decentralization and the efficiency loss due to uncertainty and establish tight performance guarantees for distributed systems in uncertain environments. We present applications based on novel variants of atomic congestion games with uncertain costs, for which we provide tight performance bounds under a wide range of risk attitudes. Our results establish that the individual’s attitude toward uncertainty has a critical effect on system performance and therefore should be a subject of close and systematic investigation.

Evolutionary algorithm based on dynamical structure of membrane systems in uncertain environments

In this paper, a new evolutionary algorithm based on a membrane system is proposed to solve the dynamic or uncertain optimization problems. The proposed algorithm employs objects, a dynamical membrane structure and several reaction rules of the membrane systems. The object represents a candidate solution of the optimization problems. The dynamical structure consists of the nested membranes where a skin membrane contains several membranes, which is useful for the proposed algorithm that finds optimal solutions. The reaction rules are designed to locate and track the optimal solutions of the dynamic optimization problems (DOPs), which are inspired by processing the chemical compounds in the region of cellular membranes. Experimental study is conducted based on the moving peaks benchmark to evaluate the performance of the proposed algorithm in comparison with three state-of-the-art dynamic optimization algorithms. The results indicate the proposed algorithm is effective to solve the DOPs.