Fundamental Control Problem Research Articles

Stochastic Optimal Control for Multivariable Dynamical Systems Using Expectation Maximization.

Trajectory optimization is a fundamental stochastic optimal control (SOC) problem. This article deals with a trajectory optimization approach for dynamical systems subject to measurement noise that can be fitted into linear time-varying stochastic models. Exact/complete solutions to these kind of control problems have been deemed analytically intractable in literature because they come under the category of partially observable Markov decision processes (MDPs). Therefore, effective solutions with reasonable approximations are widely sought for. We propose a reformulation of stochastic control in a reinforcement learning setting. This type of formulation assimilates the benefits of conventional optimal control procedure, with the advantages of maximum likelihood approaches. Finally, an iterative trajectory optimization paradigm called as SOC-expectation maximization (SOC-EM) is put forth. This trajectory optimization procedure exhibits better performance in terms of reduction in cumulative cost-to-go which is proven both theoretically and empirically. Furthermore, we also provide novel theoretical work which is related to uniqueness of control parameter estimates. Analysis of the control covariance matrix is presented, which handles stochasticity through efficiently balancing exploration and exploitation.

Multiagent LQR-based Control Design and Gain tuning for Quadcopters Fleet

An LQR-based Control design and gain tuning strategies proposals for a multi-agent system are presented in this article, the agents are connected in an undirected graph. Controller gains tuning are adjusted by selecting the Q and R weighting matrices of the Linear Quadratic Regulator. Agreement (consensus) is one of the fundamental problems in multi-agent control, where a set of agents must agree on a joint state value. In the proposed design, first considering that the behavior of the agreement protocol is undirected and static, the main objective is to highlight the complexity of the relationship between the convergence properties of this protocol and the structure of adjacent interconnections. The effects on the formation due to static geometry are analyzed from the resulting data according to the proximity between the agents, where behavior and stability are analyzed based on the desired formation geometry through the construction of the Laplacian matrix.

Pinning Control of Boolean Networks via Injection Mode

The pinning control of Boolean networks with both controls and injection modes is proposed. The system concerned is converted into a classical Boolean network with mix-valued logical controls. Under this framework three other fundamental control problems, including controllability, observability, and stabilization, are investigated.

Triple-Mode Model Predictive Control Using Future Target Information

In this paper, we propose a triple-mode model predictive control (MPC) algorithm that uses future target information to improve tracking performance. To explicitly take into account the future target information in the MPC optimization, the proposed triple-mode control law encompasses three parts: (i) the future target information feedforward, (ii) the output feedback, and (iii) the extra degrees of freedom for constraint satisfaction. The first two parts of the control law are off-line designed through unconstrained MPC, and the optimal future trajectory horizon is obtained by golden section search based on the integral of squared error (ISE) criterion. The final part is calculated by the on-line MPC algorithm aiming to satisfy constraints. Furthermore, we analyze the feasibility and convergence properties of the proposed algorithm. The method is demonstrated by the simulation of the shell fundamental control problem and also tested on the coordinated control problem in the power plant. The test results show that the proposed algorithm can increase tracking performance dramatically due to the proper selection of future trajectory horizon.

Robust nonlinear burn control in ITER to handle uncertainties in the fuel-line concentrations

Tight regulation of the burn condition in ITER has been proven possible in simulations by the use of robustification techniques even under emulated time-dependent variations in the fuel concentration. One of the most fundamental control problems arising in ITER and future burning-plasma tokamaks is the regulation of the plasma temperature and density to produce a determined amount of fusion power while avoiding possible thermal instabilities. Such problem, known as burn control, requires the development of controllers that integrate all the available actuators. Moreover, the complex burning-plasma dynamics and the uncertain nature of some of its components suggest that nonlinear, robust, burn controllers are necessary. For instance, the deuterium–tritium concentration in the fueling lines may vary over time and the estimation of such variation during operation may be difficult or not even possible. Available actuators for the regulation of the burn condition are auxiliary power modulation, fueling rate modulation, and impurity injection. Also, recent experiments in the DIII-D tokamak have shown that in-vessel coil-current modulation can be used for burn control purposes. The in-vessel coils generate non-axisymmetric magnetic fields that have the capability to decrease the plasma-energy confinement time, which allows for regulation of the plasma energy during positive energy perturbations. In this work, all these actuators are used to design a nonlinear burn controller which is robust to unknown variations in the deuterium–tritium concentration of the fueling lines. Furthermore, fueling rate modulation is not only used to control the plasma density, but also to control the plasma energy, if necessary, by means of isotopic fuel tailoring. The model-based nonlinear controller is synthesized from a zero-dimensional model of the burning-plasma dynamics. A nonlinear simulation study is carried out to illustrate the successful controller performance in ITER-like scenarios in which unknown variations of the deuterium–tritium concentration of the fueling lines are emulated.

Control of neural systems at multiple scales using model-free, deep reinforcement learning

Recent improvements in hardware and data collection have lowered the barrier to practical neural control. Most of the current contributions to the field have focus on model-based control, however, models of neural systems are quite complex and difficult to design. To circumvent these issues, we adapt a model-free method from the reinforcement learning literature, Deep Deterministic Policy Gradients (DDPG). Model-free reinforcement learning presents an attractive framework because of the flexibility it offers, allowing the user to avoid modeling system dynamics. We make use of this feature by applying DDPG to models of low-level and high-level neural dynamics. We show that while model-free, DDPG is able to solve more difficult problems than can be solved by current methods. These problems include the induction of global synchrony by entrainment of weakly coupled oscillators and the control of trajectories through a latent phase space of an underactuated network of neurons. While this work has been performed on simulated systems, it suggests that advances in modern reinforcement learning may enable the solution of fundamental problems in neural control and movement towards more complex objectives in real systems.

Unmanned Combat Aerial Vehicle Path Planning by Brain Storm Optimization Algorithm

The use of the unmanned aerial vehicles is rapidly growing in ever wider range of applications where military use is among the oldest ones. One of the fundamental problems in the unmanned combat aerial vehicles control is the path planning problem that refers to establish the optimal route from the start position to the target, where optimality can be defined in numerous ways. Path planning represents a multi-objective constrained hard optimization problem. In this paper, we adjusted a recent swarm intelligence brain storm optimization algorithm for finding the unmanned combat aerial vehicle optimal path considering fuel consumption and safety degree. The proposed method was tested and compared to eleven different methods from literature. Based on the simulation results, it can be concluded that our proposed approach is robust, exhibits better performance in almost all cases and has potential for further improvements.

Chromatin Immunoprecipitation in Macrophages.

Macrophages are highly polymorphic depending upon their cellular origin and their tissue environment. The different forms that a macrophage can adopt fundamentally reflect different transcription patterns. In addition, macrophages are exquisitely sensitive to a wide variety of signals coming from either infectious agents or damaged tissues. Most of the responses to these signals involve rapid and massive modifications of transcription. The control of transcription relies on the one hand on the posttranslational modification of histones, and on the other hand on the binding on the chromatin of multiple protein complexes. Immunoprecipitation of cross-linked chromatin with specific antibodies will allow to identify the DNA regions bound by the targeted protein, or carrying the targeted histone modification. By taking a snapshot of the macrophage chromatin composition, this technique will be useful to address specific macrophage biology questions at the DNA level, but also to tackle fundamental problems in transcriptional control in a highly suited model cellular system. In this chapter we describe a protocol of chromatin immunoprecipitation in murine bone marrow-derived macrophages that can easily be adapted to other macrophage populations.

A Self-Organizing Architecture for Decentralized Smart Microgrids Synchronization, Control, and Monitoring

The concept of smart microgrid (SMG) is widely recognized as one of the most promising and enabling technologies for smart grids. In this context, the development of integrated and decentralized frameworks for executing complex control and monitoring applications is still in its infancy and needs to be researched. To address this issue, this paper conceptualizes a self-organizing computing framework, based on self-organizing agents, for solving the fundamental control and monitoring problems of an SMG without the need for centralized data acquisition and processing. Simulation results obtained for an 18-bus test network are presented and discussed in order to demonstrate the significance and validity of the proposed new framework.

Oops! I cannot do it again: Testing for recursive feasibility in MPC

One of the most fundamental problems in model predictive control (MPC) is the lack of guaranteed stability and feasibility. It is shown how Farkas’ Lemma in combination with bilevel programming and disjoint bilinear programming can be used to search for problematic initial states which lack recursive feasibility, thus invalidating a particular MPC controller. Alternatively, the method can be used to derive a certificate that the problem is recursively feasible. The results are initially derived for nominal linear MPC, and thereafter extended to the additive disturbance case.

A new control strategy for IKN-type coolers

This paper presents a new system for the optimized control of IKN-type cement coolers based on adaptive predictive methodology. The new system introduces the concepts of ’layer on grate’ and ’thickness of slope’, which allow a decoupled multivariable control of the layer of clinker over the grate, and the flow rate of the cooler fans, solving a fundamental control problem for this type of process and maximizing the heat transfer from the cooling process to the kiln. The experimental results obtained in the Gador mill of Holcim, in Almeria (Spain), are analyzed.

Just-in-time DNA replication.

Complete and timely replication of the genome is a prerequisite to fulfilling the “dream” of every cell to become two cells [1]. So far, biologists have been successful in identifying the processes involved in DNA replication, but they have not been able to explain a fundamental control problem that cells face, the so-called “random-completion” or “random-gap” problem: how do cells ensure that every last piece of the genome is replicated on time [2]? In a paper in Physical Review E, Scott Yang and John Bechhoefer of Simon Fraser University use insights from condensed-matter physics to answer this question [3]. Using a physical model originally developed to describe the kinetics of first-order phase transitions, they show that, despite the intrinsic stochasticity of the initiation of DNA replication, cells can still control the amount of time it takes to replicate the genome. The authors thus provide a rigorous solution to a long-standing problem in cell biology. The elegance of their formal approach bridging physics and biology, and the depth of their analysis, should inspire scientists from both disciplines.

Nonlinear ${\bf H}_{\bm 2}$ /${\bf H}_{\bm \infty}$Constrained Feedback Control: A Practical Design Approach Using Neural Networks (Abu-Khalaf, M. et al.; 2008) [Book reviews

This book examines some fundamental modern control problems of nonlinear systems. The book is organized into eight chapters and includes the following topics: basic concepts and background material; policy iterations and nonlinear state feedback control; and the solution of the Hamilton-Jacobi-Isaacs (HJI) equation for zero-sum games in continuous-time and discrete-time systems. The book is very well written. The topics chosen are all fundamental control problems and are treated in depth. The book is particularly a good text for a graduate or advanced undergraduate course.

Joint Optimal Access Point Selection and Channel Assignment in Wireless Networks

In wireless cellular networks or in other networks with single-hop communication, the fundamental access control problem pertains to access point (AP) selection and channel allocation for each user. For users in the coverage area of one AP, this involves only channel allocation. However, users that belong in the intersection of coverage areas of more than one AP can select the appropriate AP to establish connection and implicitly affect the channel assignment procedure. We address the joint problem of AP selection and channel assignment with the objective to satisfy a given user load vector with the minimum number of channels. Our major finding is that the joint problem reduces to plain channel allocation in a cellular network that emerges from the original one after executing an iterative and provably convergent clique load balancing algorithm. For linear cellular networks, our approach leads to minimum number of required channels to serve a given load vector. For 2D cellular networks, the same approach leads to a heuristic algorithm with a suboptimal solution due to the fact that clique loads cannot be balanced. Numerical results demonstrate the performance benefits of our approach in terms of blocking probability in a dynamic scenario with time-varying number of connection requests. The presented approach constitutes the basis for addressing more composite resource allocation problems in different context.

A novel extended Kalman filter for a class of nonlinear systems

Abstract Estimation of the state variables of nonlinear systems is one of the fundamental and significant problems in control and signal processing. A new extended Kalman filtering approach for a class of nonlinear discrete-time systems in engineering is presented in this paper. In contrast to the celebrated extended Kalman filter (EKF), there is no linearization operation in the design procedure of the filter, and the parameters of the filter are obtained through minimizing a proper upper bound of the mean-square estimation error. Simulation results show that this filter can provide higher estimation precision than that provided by the EKF. *Supported by the Key Project of Chinese Ministry of Education (Grant No. 104007)

How do strength, sensation, spasticity and joint individuation relate to the reaching deficits of people with chronic hemiparesis?

Hemiparetic subjects present with movement deficits including weakness, spasticity and an inability to isolate movement to one or a few joints. Voluntary attempts to move a single joint often result in excessive motion at adjacent joints. We investigated whether the inability to individuate joint movements is associated with deficits in functional reaching. Controls and hemiparetic subjects performed two different reaching movements and three individuated arm movements, all in the parasagittal plane. The reaching movements were a sagittal 'reach up' (shoulder flexion and elbow flexion) and 'reach out' (shoulder flexion and elbow extension). Joint individuation was assessed by getting each subject to perform an isolated flexion-extension movement at each of the shoulder, elbow and wrist joints. In addition, we measured strength, muscle tone and sensation using standard clinical instruments. Hemiparetic subjects showed varying degrees of impairment when performing reaching movements and individuated joint movements. Reaching impairments (hand path curvature, velocity) were worse in the reach out versus the reach up condition. Typical joint individuation abnormalities were excessive flexion of joints that should have been held fixed during movement of the instructed joint. Hemiparetic subjects tended to produce concurrent flexion motions of shoulder and elbow joints when attempting any movement, one explanation for why they were better at the 'reach up' than the 'reach out' task. Hierarchical regression analysis showed that impaired joint individuation explained most of the variance in the reach path curvature and end point error; strength explained most of the variance in reaching velocity. Sensation also contributed significantly, but spasticity and strength were not significant in the model. We conclude that the deficit in joint individuation reflects a fundamental motor control problem that largely explains some aspects of voluntary reaching deficits of hemiparetic subjects.

Computer Algebra Algorithms for Control Related Tests of Implicit Dynamic Systems

AbstractThis contribution is focused on control related tests for implicit dynamic systems, like accessibility, observability or input to output, input to state linearizability. Since the performance of these tests needs tedious symbolic calculations, computer algebra systems are the ideal tool to cope with this problem. Accessibility and observability are exemplarily used to present a new approach based on Lie groups. It is shown that non accessible or non observable systems admit Lie‐groups acting on their solutions such that distinguished parts remain unchanged. This fact allows us to apply this technique, as well as its realization by computer algebra algorithm, to several fundamental problems in control.

Three-dimensional eye-head coordination is implemented downstream from the superior colliculus.

How the brain transforms two-dimensional visual signals into multi-dimensional motor commands, and subsequently how it constrains the redundant degrees of freedom, are fundamental problems in sensorimotor control. During fixations between gaze shifts, the redundant torsional degree of freedom is determined by various neural constraints. For example, the eye- and head-in-space are constrained by Donders' law, whereas the eye-in-head obeys Listing's law. However, where and how the brain implements these laws is not yet known. In this study, we show that eye and head movements, elicited by unilateral microstimulations of the superior colliculus (SC) in head-free monkeys, obey the same Donders' strategies observed in normal behavior (i.e., Listing's law for final eye positions and the Fick strategy for the head). Moreover, these evoked movements showed a pattern of three-dimensional eye-head coordination, consistent with normal behavior, where the eye is driven purposely out of Listing's plane during the saccade portion of the gaze shift in opposition to a subsequent torsional vestibuloocular reflex slow phase, such that the final net torsion at the end of each head-free gaze shift is zero. The required amount of saccade-related torsion was highly variable, depending on the initial position of the eye and head prior to a gaze shift and the size of the gaze shift, pointing to a neural basis of torsional control. Because these variable, context-appropriate torsional saccades were correctly elicited by fixed SC commands during head-free stimulations, this shows that the SC only encodes the horizontal and vertical components of gaze, leaving the complexity of torsional organization to downstream control systems. Thus we conclude that Listing's and Donders' laws of the eyes and head, and their three-dimensional coordination mechanisms, must be implemented after the SC.

Modeling and control of MEMS with high speed synchronous micromotors and controllers/drivers-on-VLSI-chip ICs

This articles addresses and solves the modeling, analysis and control problems for microelectromechanical systems (MEMS) that integrate high speed permanent magnet synchronous micromotors (motion microdevices) and controllers/drivers-on-VLSI-chip ICs. High performance micromotors (2, 4 and 6 mm diameter) with and without Hall effect microsensors are fabricated using high aspect ratio and surface micromachining techniques, while the ICs are made using biCMOS technology. In addition to fabrication, a wide spectrum of fundamental problems in MEMS synthesis, modeling, analysis and control are solved. The MEMS synthesis is performed using basic principles with the ultimate goal to guarantee operationability, functionality, compatibility, integrity, controllability and synergy. Synchronous micromotors are designed by applying the MEMS Synthesis and Classification Solver. Signal processing, filtering, computing, interfacing and amplification are performed by the controllers/drivers-on-VLSI-chip ICs that control the micromotors by properly applying the phase voltages to the micromotor windings. Analysis, control and optimization are based on mathematical models. Mathematical modeling is done by applying basic principles and the fundamental laws of electromechanics. The derived lumped parameter mathematical model, which is given in the form of nonlinear differential equations, allows one to perform the data intensive analysis, heterogeneous simulations with outcome prediction and controller design. Based on micromotor electromagnetics–electromechanics and microelectronics capabilities, we derive control algorithms. An innovative synthesis procedure is reported to design soft switching continuous controllers with nonlinear switching surfaces. These control laws, which ensure robust tracking and disturbance rejection, are implemented. The algorithms designed are different from the existing variable structure control laws. Compared with hard switching discontinuous controllers, the advantages of the soft switching sliding mode control paradigm are that the singularity and sensitivity problems are avoided, robustness and stability are improved, chattering (high frequency switching) is eliminated, etc. These features expand the MEMS operating envelopes, enhance robustness and controllability, as well as improve efficiency and reliability. Experimental verification to assess the fundamental results is performed. Integrated MEMS and their components are fabricated and tested. We study the MEMS performance, verify mathematical models, analyze applicability and effectiveness of soft switching algorithms for micromotors controlled by ICs, etc.

Construction of a Hexapod Robot with Cockroach Kinematics Benefits both Robotics and Biology

Any device that moves by actuating several multi-segmented legs must solve fundamental problems in mechanics and control regardless of whether that device is made out of living tissue or metal. With this in mind, we believe that advances can be made both in the design of legged robots and in understanding how legged animals locomote by working on these issues in tandem. This basic philosophy has led us to build a hexapod robot with kinematics that are remarkably similar to those of the death head cockroach Blaberus discoidalis . This eff ort has required us to make detailed neurobiological and kinematic observations of cockroaches walking on a treadmill and climbing over barriers. The data acquired in this way were then input to our dynamic simulation tool to determine mechanically accurate parameters for the design of the robot. The resulting vehicle captures the cockroach kinematics so well that issues that are being addressed in controlling it are providing important notions regarding the neural control of locomotion in the animal. Thus, the project has led to benefits in both directions. Biological data inspired the initial design of the robot, while engineering eff orts in control are inspiring further neurobiological experiments in animal locomotion.