Dimension Of State Space Research Articles

ADRL: A Hybrid Anomaly-Aware Deep Reinforcement Learning-Based Resource Scaling in Clouds

The virtualization concept and elasticity feature of cloud computing enable users to request resources on-demand and in the pay-as-you-go model. However, the high flexibility of the model makes the on-time resource scaling problem more complex. A variety of techniques such as threshold-based rules, time series analysis, or control theory are utilized to increase the efficiency of dynamic scaling of resources. However, the inherent dynamicity of cloud-hosted applications requires autonomic and adaptable systems that learn from the environment in real-time. Reinforcement Learning (RL) is a paradigm that requires some agents to monitor the surroundings and regularly perform an action based on the observed states. RL has a weakness to handle high dimensional state space problems. Deep-RL models are a recent breakthrough for modeling and learning in complex state space problems. In this article, we propose a Hybrid Anomaly-aware Deep Reinforcement Learning-based Resource Scaling (ADRL) for dynamic scaling of resources in the cloud. ADRL takes advantage of anomaly detection techniques to increase the stability of decision-makers by triggering actions in response to the identified anomalous states in the system. Two levels of global and local decision-makers are introduced to handle the required scaling actions. An extensive set of experiments for different types of anomaly problems shows that ADRL can significantly improve the quality of service with less number of actions and increased stability of the system.

Explainable AI in Internet of Control System Distributed at Edge-Cloud Architecture

Many current control systems are restricted to highly controlled environments. In complicated dynamic and unstructured environments such as autonomous vehicles, control systems must be able to deal with more and more complex state situations. In complex systems with large number of states, it is often too slow to use optimal planners and developing heuristic tactics for high level goals can be challenging. AI control is an attractive alternative to traditional control architectures due to their capability to approximate optimal solutions in high dimensional state spaces without requiring a human-designed heuristic. Explainable AI control attempts to produce a human readable control command which is both interpretable and manipulable. This paper is an attempt to propose an architecture for explainable AI control in edge-cloud environment in which there are connected autonomous agents that need to be controlled. In this architecture the designed controller is distributed across the edge and cloud platform using explainable AI. This architecture could be introduced as Internet of Control Systems (IoCS), which could be applied as distributed tactics to control of connected autonomous agents. The IoCS attempts to unleash AI services using resources at the edge near the autonomous agents and make intelligent edge for dynamic, adaptive, and optimized AI control.

Co-Learning of Task and Sensor Placement for Soft Robotics

Unlike rigid robots which operate with compact degrees of freedom, soft robots must reason about an infinite dimensional state space. Mapping this continuum state space presents significant challenges, especially when working with a finite set of discrete sensors. Reconstructing the robot's state from these sparse inputs is challenging, especially since sensor location has a profound downstream impact on the richness of learned models for robotic tasks. In this work, we present a novel representation for co-learning sensor placement and complex tasks. Specifically, we present a neural architecture which processes on-board sensor information to learn a salient and sparse selection of placements for optimal task performance. We evaluate our model and learning algorithm on six soft robot morphologies for various supervised learning tasks, including tactile sensing and proprioception. We also highlight applications to soft robot motion subspace visualization and control. Our method demonstrates superior performance in task learning to algorithmic and human baselines while also learning sensor placements and latent spaces that are semantically meaningful.

H2 suboptimal output synchronization of heterogeneous multi-agent systems

This paper deals with the H2 suboptimal output synchronization problem for heterogeneous linear multi-agent systems. Given a multi-agent system with possibly distinct agents and an associated H2 cost functional, the aim is to design output feedback based protocols that guarantee the associated cost to be smaller than a given upper bound while the controlled network achieves output synchronization. A design method is provided to compute such protocols. For each agent, the computation of its two local control gains involves two Riccati inequalities, each of dimension equal to the state space dimension of the agent. A simulation example is provided to illustrate the performance of the proposed protocols.

Weakly Jauch–Piron States in Effect Algebras

We study various types of weakly Jauch–Piron states and various types of state-space properties in effect algebras. We generalize several results of Tkadlec (Tatra Mt. Math. Publ. 10, 55–62 1997; Algebra Universalis 61, 187–194 2009) and of Matousek and Ptak (Internat. J. Theoret. Phys. 54, 4476–4481 2015). In particular, we show when an effect algebra is an orthomodular poset, when a unital set of states is strongly order determining, and we present some state space characterizations of Boolean algebras.

The Camouflage Machine: Optimizing protective coloration using deep learning with genetic algorithms.

Evolutionary biologists frequently wish to measure the fitness of alternative phenotypes using behavioral experiments. However, many phenotypes are complex. One example is coloration: camouflage aims to make detection harder, while conspicuous signals (e.g., for warning or mate attraction) require the opposite. Identifying the hardest and easiest to find patterns is essential for understanding the evolutionary forces that shape protective coloration, but the parameter space of potential patterns (colored visual textures) is vast, limiting previous empirical studies to a narrow range of phenotypes. Here, we demonstrate how deep learning combined with genetic algorithms can be used to augment behavioral experiments, identifying both the best camouflage and the most conspicuous signal(s) from an arbitrarily vast array of patterns. To show the generality of our approach, we do so for both trichromatic (e.g., human) and dichromatic (e.g., typical mammalian) visual systems, in two different habitats. The patterns identified were validated using human participants; those identified as the best for camouflage were significantly harder to find than a tried-and-tested military design, while those identified as most conspicuous were significantly easier to find than other patterns. More generally, our method, dubbed the "Camouflage Machine," will be a useful tool for identifying the optimal phenotype in high dimensional state spaces.

Solving High-Dimensional Dynamic Portfolio Choice Models with Hierarchical B-Splines on Sparse Grids

Discrete time dynamic programming to solve dynamic portfolio choice models has three immanent issues: firstly, the curse of dimensionality prohibits more than a handful of continuous states. Secondly, in higher dimensions, even regular sparse grid discretizations need too many grid points for sufficiently accurate approximations of the value function. Thirdly, the models usually require continuous control variables, and hence gradient-based optimization with smooth approximations of the value function is necessary to obtain accurate solutions to the optimization problem. For the first time, we enable accurate and fast numerical solutions with gradient-based optimization while still allowing for spatial adaptivity using hierarchical B-splines on sparse grids. When compared to the standard linear bases on sparse grids or finite difference approximations of the gradient, our approach saves an order of magnitude in total computational complexity for a representative dynamic portfolio choice model with varying state space dimensionality, stochastic sample space, and choice variables.

ENVELOPE FOLLOWING BASED METHOD FOR THE COMPUTATION OF LYAPUNOV EXPONENTS IN ELECTRONIC CIRCUITS

Lyapunov Exponents (LEs) can be considered as a statistical measure giving some approximate but signifficant insight in the behaviour of a dynamic system. If the systems we consider are models of realistic electronic circuits, we can reasonably assume them to be dissipative. LEs are useful to determine some general topological properties of the attractive region so that, even if the state space has a relatively large dimension, it is possible to foresee the presence of stable equilibrium points, periodic or quasi-periodic trajectories or, possibly, chaos. Lyapunov exponents and related measures such as the Lyapunov dimension are used to characterize the dynamics of complex systems and the geometrical properties of their trajectories in the state space. A few methods are available for the numerical computation of these measures and in most cases they are used only for systems that are normalized, well behaved and with a low dimensional state space. It is here proposed an approach that, being based on the envelope following method for the integration of stiff systems, can be efficiently used for the calculation of Lyapunov exponents and dimension in real electronic circuits.

ALIENs and Continuous Time Economies

I develop a new computational framework called Actively Learned and Informed Equilibrium Nets (ALIENs) to solve continuous time economic models with endogenous state variables and highly non-linear policy functions. I employ neural networks that are trained to solve supervised learning problems that respect the laws governed by the economic system in the form of general parabolic partial differential equations. The economic information is encoded as regularizers that disciplines the deep neural network in the learning process. The sub-domain of the high dimensional state space that carries the most economic information is learned actively in an iterative loop, enforcing the random training points to be sampled from areas that matter the most to ensure convergence. I utilize a state-of-the art distributed framework to train the network that speeds up computation time significantly. The method is applied to successfully solve a model of macro-finance that is notoriously difficult to handle using traditional finite difference schemes.

A light approach to quantum advantage

Quantum Computing Quantum computational advantage or supremacy is a long-anticipated milestone toward practical quantum computers. Recent work claimed to have reached this point, but subsequent work managed to speed up the classical simulation and pointed toward a sample size–dependent loophole. Quantum computational advantage, rather than being a one-shot experimental proof, will be the result of a long-term competition between quantum devices and classical simulation. Zhong et al. sent 50 indistinguishable single-mode squeezed states into a 100-mode ultralow-loss interferometer and sampled the output using 100 high-efficiency single-photon detectors. By obtaining up to 76-photon coincidence, yielding a state space dimension of about 1030, they measured a sampling rate that is about 1014-fold faster than using state-of-the-art classical simulation strategies and supercomputers. Science , this issue p. [1460][1] [1]: /lookup/doi/10.1126/science.abe8770

Quantum computational advantage using photons.

Quantum computers promise to perform certain tasks that are believed to be intractable to classical computers. Boson sampling is such a task and is considered a strong candidate to demonstrate the quantum computational advantage. We performed Gaussian boson sampling by sending 50 indistinguishable single-mode squeezed states into a 100-mode ultralow-loss interferometer with full connectivity and random matrix-the whole optical setup is phase-locked-and sampling the output using 100 high-efficiency single-photon detectors. The obtained samples were validated against plausible hypotheses exploiting thermal states, distinguishable photons, and uniform distribution. The photonic quantum computer, Jiuzhang, generates up to 76 output photon clicks, which yields an output state-space dimension of 1030 and a sampling rate that is faster than using the state-of-the-art simulation strategy and supercomputers by a factor of ~1014.

Generating series for networks of Chen–Fliess series

Consider a set of single-input, single-output nonlinear systems whose input-output maps are described only in terms of convergent Chen-Fliess series without any assumption that finite dimensional state space models are available. It is shown that any additive or multiplicative interconnection of such systems always has a Chen-Fliess series representation that can be computed explicitly in terms of iterated formal Lie derivatives.

A natural extension of Markov processes and applications to singular SDEs

On établit une méthode générale pour élargir l’espace d’états d’un processus de Markov, de telle façon que l’ensemble des points ajoutés est polaire. Cette extension est une amélioration de l’extension triviale, dans quel cas le processus est bloqué dans les points ajoutés, et elle produit une technique nouvelle de construction des solutions étendues pour des ED(P) stochastiques, à partir de tous les points de départ, telle qu’elles soient des solutions au moins après tout moment de temps strictement positif. Concrètement, on adopte cette stratégie pour étudier des ED stochastiques avec des coefficients singuliers, sur un espace d’états de dimension infinie (par example des EDP stochastiques de type evolution), pour lesquelles on rencontre des situations où tous les points de l’espace ne sont pas des conditions initiales autorisées. La même chose peut se passer dans la construction des solutions pour des problèmes de martingales ou pour des processus de Markov à partir de formes de Dirichlet (généralisées), pour lesquelles notre nouvelle technique s’applique aussi.

Deep Residual Networks for Flamelet/progress Variable Tabulation with Application to a Piloted Flame with Inhomogeneous Inlet

ABSTRACT In this work, a deep neural network is presented which is trained on flamelet/progress variable (FPV) tables and validated in a combustion large eddy simulation (LES) of the Sydney/Sandia flame with inhomogeneous inlets. Using data scaling and transformation techniques, as well as novel network architectures as the residual skip connection we are able to store all combustion relevant quantities in one single network, thus reducing effectively the memory footprint compared to the FPV tables, while keeping data retrieval times similar to table interpolation. The accuracy of the deep neural networks (DNN) is compared to its FPV counterpart and achieves excellent agreement. The DNN is also able to accurately predict the stiff progress variable source term in the thin reaction zone. Finally, the DNN thermochemistry representation is validated in combustion simulations of a 2D laminar premixed flame and a 3D LES of the Sydney/Sandia piloted jet flame with inhomogeneous inlet. The DNN results are in very good agreement with the conventional tabulated FPV simulations and show a promising way to efficiently reduce the storage size of high dimensional pretabulated thermochemical state space in reactive flow simulations. Abbreviations: ANN: Artificial neural network; API: Application programming interface; CFD: Computational fluid dynamics; CFL: Courant-Friedrichs-Lewy; CPU: Central processing unit; DL: Deep learning; DNN: Deep neural network; FGM: Flamelet generated manifold; FLOPS: Floating point operation per second; FPV: Flamelet/progress variable; GPU: Graphics processing unit; HPC: High-performance computing; ILDM: Intrinsic low-dimensional manifold; LES: Large eddy simulation; MAE: Mean absolute error; MPI: Message passing interface; MSE: Mean squared error; PDF: Probability density function; SGS: Subgrid scale; SOM: Self-organizing map; TCI: Turbulence–chemistry interaction.

A batch-wise LSTM-encoder decoder network for batch process monitoring

Process monitoring is essential to keep quality consistency and operation safety in the batch process. However, the existence of multiphase, nonlinearity and dynamic features in the batch process makes the batch process monitoring a complicated task. In this work, a multi-layer recurrent neural network in the encoder–decoder structure called batch-wise LSTM-encoder decoder network is proposed to solve the difficulties mentioned above in batch process monitoring. The LSTM-encoder extracts the nonlinear dynamic features in both between and within batch direction, then projects the high dimensional input space to a low dimensional hidden state space. The decoder part regenerates the samples from hidden states. Control statistics H2 and SPE are designed for process monitoring, and the corresponding control limits are estimated by kernel density estimation. A case study on an extensive reference penicillin fermentation dataset suggests that the proposed method can detect the fault samples more effectively than previous methods while keeping the same robustness in normal conditions.

Degeneracy and Complexity in Neuro-Behavioral Correlates of Team Coordination.

Team coordination—members of a group acting together rather than performing specific actions individually—is essential for success in many real-world tasks such as military missions, sports, workplace, or school interactions. However, team coordination is highly variable, which is one reason why its underlying neural processes are largely unknown. Here we used dual electroencephalography (EEG) in dyads to study the neurobehavioral dynamics of team coordination in an ecologically valid task that places intensive demands on joint performance. We present a novel conceptual framework to interpret neurobehavioral variability in terms of degeneracy, a fundamental property of complex biological systems said to enhance flexibility and robustness. We characterize degeneracy conceptually in terms of a manifold representing the geometric locus of the dynamics in the high dimensional state-space of neurobehavioral signals. The geometry and dimensionality of the manifold are determined by task constraints and team coordination requirements which restrict the manifold to trajectories that are conducive to successful task performance. Our results indicate that team coordination is associated with dimensionality reduction of the manifold as evident in increased inter-brain phase coherence of beta and gamma rhythms during critical phases of task performance where subjects exchange information. Team coordination was also found to affect the shape of the manifold manifested as a symmetry breaking of centro-parietal wavelet power patterns across subjects in trials with high team coordination. These results open a conceptual and empirical path to identifying the mechanisms underlying team performance in complex tasks.

The State Space of Artificial Intelligence

The goal of the paper is to develop and propose a general model of the state space of AI. Given the breathtaking progress in AI research and technologies in recent years, such conceptual work is of substantial theoretical interest. The present AI hype is mainly driven by the triumph of deep learning neural networks. As the distinguishing feature of such networks is the ability to self-learn, self-learning is identified as one important dimension of the AI state space. Another dimension is recognized as generalization, the possibility to go over from specific to more general types of problems. A third dimension is semantic grounding. Our overall analysis connects to a number of known foundational issues in the philosophy of mind and cognition: the blockhead objection, the Turing test, the symbol grounding problem, the Chinese room argument, and use theories of meaning. It shall finally be argued that the dimension of grounding decomposes into three sub-dimensions. And the dimension of self-learning turns out as only one of a whole range of “self-x-capacities” (based on ideas of organic computing) that span the self-x-subspace of the full AI state space.

A Turbo Q-Learning (TQL) for Energy Efficiency Optimization in Heterogeneous Networks

In order to maximize energy efficiency in heterogeneous networks (HetNets), a turbo Q-Learning (TQL) combined with multistage decision process and tabular Q-Learning is proposed to optimize the resource configuration. For the large dimensions of action space, the problem of energy efficiency optimization is designed as a multistage decision process in this paper, according to the resource allocation of optimization objectives, the initial problem is divided into several subproblems which are solved by tabular Q-Learning, and the traditional exponential increasing size of action space is decomposed into linear increase. By iterating the solutions of subproblems, the initial problem is solved. The simple stability analysis of the algorithm is given in this paper. As to the large dimension of state space, we use a deep neural network (DNN) to classify states where the optimization policy of novel Q-Learning is set to label samples. Thus far, the dimensions of action and state space have been solved. The simulation results show that our approach is convergent, improves the convergence speed by 60% while maintaining almost the same energy efficiency and having the characteristics of system adjustment.

Uncertainty quantification of bladed disc systems using data driven stochastic reduced order models

This study focusses on the development of stochastic reduced order model for probabilistic characterisation of bladed disc systems with random spatial inhomogeneities. High fidelity finite element modelling is used to mathematically model the system. A two step reduction strategy is applied involving reduction in the state space dimension and reduction in the stochastic dimensions. Information of the spatial inhomogeneities are assumed to be available from limited in situ measurements across the spatial extent and are modelled as non-Gaussian random fields. The stochastic version of the finite element matrices are developed using a polynomial chaos based framework, which optimizes the stochastic dimensionality of the problem. The uncertainties in the input propagates through the system into the response, which are also random. Surrogate models for these response quantities are obtained as PCE and are constructed using the method of stochastic collocations. Challenges involved in application of PCE on complex geometrically irregular spatial domains are addressed. The efficacy of the proposed framework is demonstrated through two numerical examples -an academic bladed disc system and an industrial turbine rotor blade.

Active Learning for Estimating Reachable Sets for Systems With Unknown Dynamics.

This article presents a data-driven method for computing reachable sets where active learning (AL) is used to reduce the computational burden. Set-based methods used to estimate reachable sets typically do not scale well with the state-space dimension, or rely heavily on the existence of a model. If such a model is not available, it is simple to generate state trajectory data by numerically simulating black-box oracles of systems (whose dynamics are unknown) from sampled initial conditions. Using these data samples, the estimation of reachable sets can be posed as a classification problem, wherein AL can intelligently select samples that are most informative and least similar to previously labeled samples. By exploiting submodularity, the actively learned samples can be selected efficiently, with bounded suboptimality. Our proposed framework is illustrated by estimating the domains of attractions of model predictive controllers (MPCs) and reinforcement learners. We also consider a scenario where there are two oracles that differ with respect to evaluation costs and labeling accuracy. We propose a framework to reduce the dependency of the expensive oracle in labeling samples using disagreement-based AL (DBAL). The potential of the DBAL algorithm is demonstrated on a solver selection problem for real-time MPC.