Control Theoretic Viewpoint Research Articles

On-line Reinforcement Learning for Nonlinear Motion Control: Quadratic and Non-Quadratic Reward Functions

Reinforcement learning (RL) is an active research area with applications in many fields. RL can be used to learn control strategies for nonlinear dynamic systems, without a mathematical model of the system being required. An essential element in RL is the reward function, which shows resemblance to the cost function in optimal control. Analogous to linear quadratic (LQ) control, a quadratic reward function has been applied in RL. However, there is no analysis or motivation in the literature, other than the parallel to LQ control. This paper shows that the use of a quadratic reward function in on-line RL may lead to counter-intuitive results in terms of a large steady-state error. Although the RL controller learns well, the final performance is not acceptable from a control-theoretic point of view. The reasons for this discrepancy are analyzed and the results are compared with non-quadratic functions (absolute value and square root) using a model learning actor-critic with local linear regression. One of the conclusions is that the absolute-value reward function reduces the steady-state error considerably, while the learning time is slightly longer than with the quadratic reward.

Finite-time synchronization of Lorenz chaotic systems: theory and circuits

This paper addresses the problem of finite-time master–slave synchronization of Lorenz chaotic systems from a control theoretic point of view. We propose a family of feedback couplings which accomplish the synchronization of Lorenz chaotic systems based on Lyapunov stability theory. These feedback couplings are based on non-periodic functions. A finite horizon can be arbitrarily established by ensuring that chaos synchronization is achieved at established time. An advantage is that some of the proposed feedback couplings are simple and easy to implement. Both mathematical investigations and numerical simulations followed by a Pspice experiment are presented to show the feasibility of the proposed method.

A Novel Trajectory Generation Method for Robot Control

This paper presents a novel trajectory generator based on Dynamic Movement Primitives (DMP). The key ideas from the original DMP formalism are extracted, reformulated and extended from a control theoretical viewpoint. This method can generate smooth trajectories, satisfy position- and velocity boundary conditions at start- and endpoint with high precision, and follow accurately geometrical paths as desired. Paths can be complex and processed as a whole, and smooth transitions can be generated automatically. Performance is analyzed for several cases and a comparison with a spline-based trajectory generation method is provided. Results are comparable and, thus, this novel trajectory generating technology appears to be a viable alternative to the existing solutions not only for service robotics but possibly also in industry.

An optimal regulator for stabilization of multi-joint reaching movements under DOF redundancy: a challenge to the Bernstein problem from a control-theoretic viewpoint

An optimal regulator problem for multi-joint reaching movements of redundant manipulators is solved from the non-linear control-theoretic viewpoint. Given a target point that the manipulator endpoint should reach, a task-space position feedback with joint damping is introduced, which enables stabilization of reaching movements even if the degrees-of-freedom of the manipulator is greater than the dimension of task space and the inverse kinematics is ill-posed. Usually, the speed of convergence is slow, depending on choice of feedback gains for joint damping. Hence, to speed up the convergence without incurring further energy consumption, an optimal control design for minimizing the integral of a quadratic form of task-space control input and velocity output for an interval [ t0, t1] plus a quadratic function at t = t1 are introduced. This leads to the derivation of a Hamilton–Jacobi–Bellman equation that is solvable in control variables. It is shown that, in the case of infinite time horizon [ t0, ∞), the optimal control reduces to a task-space velocity feedback, and the Hamilton–Jacobi equation becomes solvable in an explicit quadratic form. Finally, a functional relationship of the optimal regulator with the passivity of the closed-loop dynamics is presented.

Robust stability analysis of gene–protein regulatory networks with cyclic activation–repression interconnections

This paper investigates analytic robust stability criteria for large-scale cyclic gene–protein regulatory network systems consisting of genes with heterogeneous dynamics combined with parametric or unstructured uncertainties from a control-theoretic viewpoint. We first consider a class of gene expressions, which is described as an uncertain linear transcription–translation model (LTTM) with feedback loops from translation products to transcription. Next, we show that such a model belong to a class of large-scale dynamical linear network systems with a generalized frequency variable. Then, we derive robust stability criteria systematically for the following two types of biological uncertain LTTM: (i) LTTMs consisting of heterogeneous gene dynamics with parametric uncertainties and (ii) LTTMs consisting of homogeneous nominal gene dynamics with unstructured uncertainties. These criteria provide fairly simple analysis methodologies, which can be readily applied to the analysis for large-scale genetic regulatory networks and give some biological insight.

Random Access Game and Medium Access Control Design

Motivated partially by a control-theoretic viewpoint, we propose a game-theoretic model, called random access game, for contention control. We characterize Nash equilibria of random access games, study their dynamics, and propose distributed algorithms (strategy evolutions) to achieve Nash equilibria. This provides a general analytical framework that is capable of modeling a large class of system-wide quality-of-service (QoS) models via the specification of per-node utility functions, in which system-wide fairness or service differentiation can be achieved in a distributed manner as long as each node executes a contention resolution algorithm that is designed to achieve the Nash equilibrium. We thus propose a novel medium access method derived from carrier sense multiple access/collision avoidance (CSMA/CA) according to distributed strategy update mechanism achieving the Nash equilibrium of random access game. We present a concrete medium access method that adapts to a continuous contention measure called conditional collision probability, stabilizes the network into a steady state that achieves optimal throughput with targeted fairness (or service differentiation), and can decouple contention control from handling failed transmissions. In addition to guiding medium access control design, the random access game model also provides an analytical framework to understand equilibrium and dynamic properties of different medium access protocols.

Linear Quadratic Optimal Controller Design for Effective Price Mechanisms Using Limited Market Information

In this paper, a price mechanism for a competitive market possibly possessing a coupled dynamics of price and demand is considered from the control theoretic viewpoint. In particular, an application of Linear Quadratic controller is proposed as a price adjusting algorithm to regulate both price and demand to the unknown equilibrium. To design such a controller, the market operator needs to know the full characterization of the market, which is available only locally. Using a heuristics involving ℓ1 norm optimization, we propose a method for local agents to select “minimum but important” information describing themselves and send it to the market operator, with which a better controller design is possible. This gives a novel and practical way for the price mechanism design using reduced information about the market.

Synchronization of Wireless Sensor Networks

i-manager's Journal on Future Engineering and Technology, i-manager's Journal on Software Engineering, i-manager's Journal on Electrical Engineering, i-manager's Journal on Electronics Engineering, i-manager's Journal of Educational Technology, i-manager's Journal on Educational Psychology, i-manager's Journal on School Educational Technology, i-manager's Journal on English Language Teaching, i-manager's Journal on Mechanical Engineering, i-manager's Journal on Civil Engineering, i-manager's Journal on Embedded Systems, i-manager's Journal on Information Technology, i-manager's Journal on Structural Engineering, i-manager's Journal on Communication Engineering and Systems, i-manager's Journal on Wireless Communication Networks, i-manager's Journal on Instrumentation and Control Engineering, i-manager's Journal on Circuits and Systems, i-manager's Journal on Digital Signal Processing, i-manager's Journal on Power Systems Engineering, i-manager's Journal on Computer Science, i-manager's Journal on Material Science, i-manager's Journal on Cloud Computing, i-manager's Journal on Image Processing, i-manager's Journal on Mobile Applications and Technologies, i-manager's Journal on Pattern Recognition, i-manager's Journal on Mathematics, i-manager's Journal on Management, i-manager's Journal on Nursing, i-manager's Journal on Physical Sciences, i-manager's Journal on Chemical Sciences, i-manager's Journal on Economics & Commerce , i-manager's Journal on Humanities & Social Sciences, i-manager's Journal on Life Sciences , i-manager's Journal on Construction Management, i-manager's Oyla , ,India best Journal Publication Website,World Best Plication SIte,Free atrticle published website,Journal Publised,Published Journal,Research Paper Published,indian research paper publications,how to published your article/research paper in india for free,journal’s instructions for authors

Efficient Uplink Bandwidth Request with Delay Regulation for Real-Time Service in Mobile WiMAX Networks

The emerging broadband wireless access technology based on IEEE 802.16 is one of the most promising solutions to provide ubiquitous wireless access to the broadband service at low cost. This paper proposes an efficient uplink bandwidth request-allocation algorithm for real-time services in Mobile WiMAX networks based on IEEE 802.16e. In order to minimize bandwidth wastage without degrading quality of service (QoS), we introduce a notion of target delay and propose dual feedback architecture. The proposed algorithm calculates the amount of bandwidth request such that the delay is regulated around the desired level to minimize delay violation and delay jitter for real-time services. Also, it can increase utilization of wireless channel by making use of dual feedback, where the bandwidth request is adjusted based on the information about the backlogged amount of traffic in the queue and the rate mismatch between packet arrival and service rates. Due to the target delay and dual feedback, the proposed scheme can control delay and allocate bandwidth efficiently while satisfying QoS requirement. The stability of the proposed algorithm is analyzed from a control-theoretic viewpoint, and a simple design guideline is derived based on this analysis. By implementing the algorithm in OPNET simulator, its performance is evaluated in terms of queue regulation, optimal bandwidth allocation, delay controllability, and robustness to traffic characteristics.

Dynamics and stability of a class of low Reynolds number swimmers near a wall

We study the dynamic stability of low Reynolds number swimming near a plane wall from a control-theoretic viewpoint. We consider a special class of swimmers having a constant shape, focus on steady motion parallel to the wall, and derive conditions under which it is passively stable without sensing or feedback. We study the geometric structure of the swimming equation and highlight the relation between stability and reversing symmetry of the dynamical system. Finally, our numerical simulations reveal the existence of stable periodic motion. The results have implications for design of miniature robotic swimmers, as well as for explaining the attraction of micro-organisms to surfaces.

Stabilizing postural control for emulated human balancing systems

Human suffering from vertiginous might experience their body tilting and may eventually fall over. The analytical understanding and applications of human posture control could possibly be extended to understand the postural sway. On the basis of a simplified human body model with two stiff segments and articulation at the hip and ankle, this paper presents a two stage feedback control law for assisting balance in the human’s standing posture under interference of subjective verticals. A robust control law is developed from a control-theoretical point of view to guarantee posture stability under disturbances caused by vertigo. In understanding the task of posture control feedback, the control theory based on the human model could be a source of inspiration to physiologists for physiological modeling and understanding of complex mechanical and neurological interactions. The results developed are to be used as a basis for stance posture control of bionic robots and motion-aid robots.

Robust Synchronization of a Class of Nonlinear Systems: Applications to Chaotic Coupled Electromechanical Systems

This article treats the robust synchronization problem of a class of nonlinear systems from a control theoretical point of view. Because of the tremendous complexity of nonlinear systems, the problem is restricted to chaotic electromechanical devices. The results are discussed in the context of complete synchronization. A new dynamic output feedback is applied to perform synchronization in spite of master/slave mismatches. The main idea is to construct an augmented dynamical system from the synchronization error system, which is itself uncertain. The advantage of this method over the existing results is that the synchronization time is explicitly computed. Numerical simulations are provided to verify the operation of the proposed algorithm.

CONTROL PROBLEMS IN ANTIANGIOGENIC THERAPY – COMPARISON OF SIX MODELS

Six models of antiangiogenic therapy are compared and analyzed from control theoretic point of view. All of them consist of a model of tumor growth bounded by the capacity of a vascular network developed by the tumor in the process of angiogenesis and different model of dynamics of this network and they are based on the idea proposed by Hahnfeldt et al. Moreover we analyse optimal control problems resulted from their use to treatment protocols design.

Stabilization of a 2D-SpiderCrane Mechanism using Damping Assignment Passivity-based Control

In this paper, we present a modeling and control strategy for a cable suspended structure called the ‘SpiderCrane'. By avoiding heavy mobile components, the design of this crane makes it particularly useful for work requiring high speeds. The modeling of such a multiple cable mechanism is challenging due to the number of constraints arising from cable interactions. From a control theoretical point of view, such mechanical systems are underactuated, which gives rise to challenging control issues.

Nonminimum-phase zeros - much to do about nothing - classical control - revisited part II

The purpose of this article is to illuminate the critical role of system zeros in control-system performance for the benefit of a wide audience both inside and outside the control systems community. Zeros are a fundamental aspect of systems and control theory; however, the causes and effects of zeros are more subtle than those of poles. In particular, positive zeros can cause initial undershoot (initial error growth), zero crossings, and overshoot in the step response of a system, whereas nonminimum-phase zeros limit bandwidth. Both of these aspects have real-world implications in many applications. Nonminimum-phase zeros exacerbate the tradeoff between the robustness and achievable performance of a feedback control system. From a control-theoretic point of view, a nonminimum-phase zero in the loop transfer function L is arguably the worst feature a system can possess. Every feedback synthesis methodology must accept limitations due to the presence of open-right-half-plane zeros, and the mark of a good analysis tool is the ability to capture the performance limitations arising from nonminimum-phase zeros.

Statistical simulation of user behaviour in low-energy office buildings

A large number of design guidelines and tools are available for the design of passive cooling systems. Using the underlying thermodynamic models, a certain input (e.g. air change rate, internal heat gains or sun control) results in a certain output (i.e. room temperature). However, in real buildings the room temperature at a given outdoor temperature is a distribution rather than a single value. Therefore, the building engineer should take uncertainties into account, since the actual use of the building, the building physical properties or the user behaviour are statistically distributed. One promising approach to include these uncertainties in the design procedure is the use of statistical models: the design parameter is defined by a mean value and its deviation. From a control theoretical point of view, the deterministic controlled system responds to random disturbance variables by a statistically distributed response function. Considering the institute building of Fraunhofer ISE as example, this study shows how statistical simulations can be applied to the design process of passive cooling in low-energy office buildings.

An H∞ approach to congestion control design for AQM routers supporting TCP flows in wireless access networks

In this paper, we aim at developing an H∞ approach, from control-theoretic viewpoint, to the design of an active queue management (AQM) based congestion control algorithm for wireless networks supporting the Internet Protocol. We study networks in which the backbone is a traditional wired network supporting Internet TCP, while end user access is via wireless. First, a dynamic model for the congestion control problem of wireless networks is built up, which enables the application of modern control theory on time-delay systems to this problem. Second, an H∞ design approach for general time-delay systems is presented. Finally, the proposed approach is applied to the congestion control algorithm design of wireless networks, yielding an effective and systematic way for the design problem. Simulation results are provided to illustrate the design procedure and the effectiveness of the proposed method. Our design method is described by linear matrix inequalities (LMI), which can be solved very efficiently by LMI toolbox in Matlab.

Chaos synchronization and duration time of a class of uncertain chaotic systems

This paper addresses the problem of chaos synchronization from a control theoretic point of view. The main idea is to construct an augmented dynamical system from the synchronization error system, which is itself uncertain. A new dynamic output feedback is applied to perform synchronization in spite of master/slave mismatches. In this way, the nonidentical chaotic synchronization can be attained. The advantage of this method over the existing results is that the feedback controller has a predictable synchronization delay. The synchronization time is explicitly computed. Computer simulations are provided to verify the operation of the designed synchronization algorithm.

Discrete port-Hamiltonian systems

Either from a control theoretic viewpoint or from an analysis viewpoint it is necessary to convert smooth systems to discrete systems, which can then be implemented on computers for numerical simulations. Discrete models can be obtained either by discretizing a smooth model, or by directly modeling at the discrete level itself. One of the goals of this paper is to model port-Hamiltonian systems at the discrete level. We also show that the dynamics of the discrete models we obtain exactly correspond to the dynamics obtained via a usual discretization procedure. In this sense we offer an alternative to the usual procedure of modeling (at the smooth level) and discretization.

Control aspects of holonomic quantum computation

A unifying framework for the control of quantum systems with non-Abelian holonomy is presented. It is shown that, from a control theoretic point of view, holonomic quantum computation can be treated as a control system evolving on a principal fiber bundle. An extension of methods developed for these classical systems may be applied to quantum holonomic systems to obtain insight into the control properties of such systems and to construct control algorithms for two established examples of the computing paradigm.