Solving Control Problems Research Articles

Математическое моделирование мягкого экзоскелета из магнитно-реологических модулей с интеллектуальным управлением

The article considers a fundamentally new generation of soft exoskeletons based on magnetic rheological modules. A model of an exoskeleton link with adjustable rigidity is proposed, for which a system of differential equations of motion is compiled. A model of an exoskeleton for human lower extremities, composed of modules with magnetic rheological fluid, has been developed. The problems of kinematics and dynamics that arise when controlling the movement of an exoskeleton are presented. A block diagram of an algorithm for controlling the movement of an exoskeleton has been developed, solving control problems taking into account neural network methods. A qualitative study by analytical methods of the proposed mathematical model of an exoskeleton with links with adjustable stiffness is carried out. The use of regulation of the rigidity of the links of the exoskeleton modules in the necessary way will make it possible to create devices that are comfortable for the user, both from the physiological and psychological points of view. Soft exoskeletons will be in demand in many industries and medicine. The use of electromagnetic actuators, which, as a rule, have smaller dimensions and energy consumption, an almost instantaneous response speed in comparison with pneumatic drives, hydraulic drives and electric drives, is promising and technically, energetically, environmentally, economically more acceptable. Under the action of electric or magnetic fields, the stiffness of electroelastic and magneto-rheological materials can change by more than an order of magnitude. This makes it possible to create exoskeleton links with adjustable stiffness. Regulation of the rigidity of the elements of the exoskeleton modules in real time is a complex control task. Therefore, exoskeleton motion control requires the development of adaptive type algorithms based on a neuro-fuzzy approach, using information received from feedback. Neural network algorithms for self-learning in the process of functioning are a logically expedient part of a soft bionic exoskeleton. Also, as a necessary component of the exoskeleton, along with the actuator part, a sensor system is needed with sensitive sensors for the user’s neuromuscular impulses and various sensors for position, speed, acceleration, force, moment of force of the exoskeleton. A comprehensive study of scientific and technical problems of the functioning of the model of the magnetic-rheological medium, exoskeleton modules and their compilation into a single multi-link structure, its movement using the technology of modern mathematical modeling was carried out. The construction of neuro-fuzzy control systems for soft exoskeletons is described as part of the development of new mathematical methods for modeling complex technical objects using modern mechatronics and cyborgics.

Research on the Application of Neural Network PID in Quadcopter Aircraft

As a highly maneuverable and flexible unmanned aerial vehicle, quadcopters have broad application prospects in both civilian and military fields. However, due to their complex dynamic characteristics, nonlinearity, and strong coupling, achieving stable and precise control of quadcopters is a challenging task. Traditional control methods often fail to meet the control performance requirements for such complex nonlinear systems. Neural networks provide a new approach to solving control problems in complex systems due to their powerful learning and adaptive capabilities. Neural networks can improve the performance of control systems by learning from large amounts of data, capturing the dynamic characteristics of the system, and adjusting control strategies online. Combining neural networks with PID control is expected to fully leverage the advantages of both. Neural networks can adjust the parameters of PID controllers in real-time, enabling them to better adapt to the complex dynamic changes and external disturbances of quadcopter aircraft. This fusion control method brings new possibilities for improving the control performance of quadcopter aircraft. This study explores the application of neural network PID in quadcopter aircraft to achieve stable and precise control, laying the foundation for its widespread promotion in practical applications.

A review of dynamic optimization in aquaculture production economics

AbstractDynamic optimization (DO) has been applied to aquaculture industry to determine optimal management strategies of aquaculture production systems. Optimal control of feeding, optimal stocking successions, and harvesting time among other factors has been studied. This work shows a review of the application of optimal control theory (OCT) and dynamic programming in economics and management of aquaculture. The Pontriagyn's maximum principle and the Bellman's optimality are the most commonly used mechanisms to solve control problems that optimize the producer's benefit or cost. Recently, model‐based predictive control has also been applied. Such tools have been used in aspects involving the design, planning, and monitoring of variables relevant to the optimal management of the culture system. It was found that in aquaculture there is low scientific productivity in the application of DO in bioeconomic models. In this review, an example of OCT applied to the control of feeding of farmed tilapia using a bioeconomic model is shown. DO represents a useful tool for optimal decision‐making and this review discusses the implications of the use of DO in aquaculture and recommendations for its use in the future.

Controlling quantum many-body systems using reduced-order modeling

Quantum many-body control is among the most challenging problems in quantum science due to its outstanding computational complexity in a general case. We propose an efficient approach to a class of many-body quantum control problems, where time-dependent controls are applied to a sufficiently small subsystem. The method employs a tensor-network scheme to construct a reduced-order model of a subsystem's non-Markovian dynamics. The resulting reduced-order model serves as a digital twin of the original subsystem. Such twins allow significantly more efficient dynamics simulation, which enables the use of a gradient-based optimization toolbox in the control parameter space. This approach to building control protocols takes advantage of non-Markovian dynamics of subsystems by design. We validate the proposed method by solving control problems for quantum spin chains. In particular, the approach automatically identifies control sequences for exciting and guiding quasiparticles to recover and transmit quantum information across the system. In addition, we find generalized spin-echo sequences for a system in a many-body localized phase enabling significant revivals. We expect our approach can be useful for ongoing experiments with noisy intermediate-scale quantum devices. Published by the American Physical Society 2024

Reliability assessment of off-policy deep reinforcement learning: A benchmark for aerodynamics

Abstract Deep reinforcement learning (DRL) is promising for solving control problems in fluid mechanics, but it is a new field with many open questions. Possibilities are numerous and guidelines are rare concerning the choice of algorithms or best formulations for a given problem. Besides, DRL algorithms learn a control policy by collecting samples from an environment, which may be very costly when used with Computational Fluid Dynamics (CFD) solvers. Algorithms must therefore minimize the number of samples required for learning (sample efficiency) and generate a usable policy from each training (reliability). This paper aims to (a) evaluate three existing algorithms (DDPG, TD3, and SAC) on a fluid mechanics problem with respect to reliability and sample efficiency across a range of training configurations, (b) establish a fluid mechanics benchmark of increasing data collection cost, and (c) provide practical guidelines and insights for the fluid dynamics practitioner. The benchmark consists in controlling an airfoil to reach a target. The problem is solved with either a low-cost low-order model or with a high-fidelity CFD approach. The study found that DDPG and TD3 have learning stability issues highly dependent on DRL hyperparameters and reward formulation, requiring therefore significant tuning. In contrast, SAC is shown to be both reliable and sample efficient across a wide range of parameter setups, making it well suited to solve fluid mechanics problems and set up new cases without tremendous effort. In particular, SAC is resistant to small replay buffers, which could be critical if full-flow fields were to be stored.

Actor modeling of real-time cognitive systems: ontological basis and software-mathematical implementation

The article is devoted to the study of the problem of increasing the reliability of modeling of cognitive systems, to which the authors refer not only human intelligence, but also artificial intelligence systems, as well as intelligent control systems for production, technological processes and complex equipment. It is shown that the use of cognitive systems for solving control problems causes very high rapidity requirements for them. These requirements combined with the necessity to simplify modeling methods as the modeling object becomes more complex determine the choice of an approach to modeling cognitive systems. Models should be based on the use of simple algorithms in the form of trend detection, correlation, as well as (for solving intellectual problems) on the use of algorithms based on the application of various patterns of forms and laws. In addition, the models should be decentralized. An adequate representation of decentralized systems formed from a large number of autonomous elements can be formed within the framework of agent-based models. For cognitive systems, two models are the most elaborated: actor and reactor models. Actor models of cognitive systems have two possible realizations: as an instrumental model or as a simulation. Both implementations have the right to exist, but the possibilities of realizing a reliable description when using the tool model are higher, because it provides incommensurably higher rapidity, and also assumes variability of the modeled reality. The actor model can be realized by means of a large number of existing programming languages. The solution to the problem of creating simulative actor models is available in most languages that work with actors. Realization of instrumental actor models requires rapidity, which is unattainable in imperative programming. In this case, the optimal solution is to use actor metaprogramming. Such programming is realizable in many existing languages.

Оптимальное мультипликативное управление полулинейным параболическим уравнением

An analysis of optimal control problems for a nonlinear parabolic initial-boundary value problem that models the dynamics of the collective behavior of a bacterial community is presented. Estimates for the solution of the initial-boundary value problem are obtained, the solvability of control problems is proved, and optimality conditions are derived. The weak "bang-bang" principle is set for the problem with the final observation.

On Methodology for Solving Control Problems of One Class of Time-varying Systems

On Methodology for Solving Control Problems of One Class of Time-varying Systems

Inverted Pendulum Control: A Comparative Study from Conventional Control to Reinforcement Learning

The rise of deep reinforcement learning in recent years has led to its usage in solving various challenging problems, such as chess and Go games. However, despite its recent success in solving highly complex problems, a question arises on whether this class of method is best employed to solve control problems in general, such as driverless cars, mobile robot control, or industrial manipulator control. This paper presents a comparative study between various classes of control algorithms and reinforcement learning in controlling an inverted pendulum system to evaluate the performance of reinforcement learning in a control problem. A test was performed to test the performance of root locus-based control, state compensator control, proportional-derivative (PD) control, and a reinforcement learning method, namely the proximal policy optimization (PPO), to control an inverted pendulum on a cart. The performances of the transient responses (such as overshoot, peak time, and settling time) and the steady-state responses (namely steady-state error and the total energy) were compared. It is found that when given a sufficient amount of training, the reinforcement learning algorithm was able to produce a comparable solution to its control algorithm counterparts despite not knowing anything about the system’s properties. Therefore, it is best used to control plants with little to no information regarding the model where testing a particular policy is easy and safe. It is also recommended for a system with a clear objective function.

THE USE OF "SOFT COMPUTING" FOR THE DIFFERENTIAL DIAGNOSIS OF THE FUNCTIONAL STATE OF THE CARDIOVASCULAR SYSTEM

The application of soft computing technology for diagnosing the functional state of the cardiovascular system is considered. Soft computing technology uses fuzzy sets, fuzzy logic, fuzzy neural networks, genetic algorithms and evolutionary modeling as tools. Various methods of soft computing technology in solving various problems often complement each other when used in various combinations. This technology is focused on solving control problems with semi-structured control objects. The main informative indicators (indicator variables) characterizing the functional state of the cardiovascular system and obtained on the basis of statistical information are identified. These informative indicators include the tension index, the vegetative rhythm index, the indicator of the adequacy of regulatory processes, the tension index of regulatory systems, and also special indicators that are derivatives of classical statistical indicators: respiratory modulation index, functional arrhythmia index, cardiorespiratory synchrony index, parasympathetic control destabilization index. The quality of the classification of possible diseases is determined by indicators such as sensitivity, specificity, predictive value and diagnostic efficiency. Keywords: Neural networks, fuzzy inferences, diagnostic conclusion, confidence coefficient.

Numerical analysis of the stressed-deformed state of a tubular element under thermal loading

In articles [2-5] solving relations and the block iteration method algorithm for solving linear and nonlinear equations by the semi-analytical finite element method for curvilinear heterogeneous prismatic bodies are implemented. In article [1], a numerical study of the convergence of the solution was performed, a wide range of test problems for bodies with smoothly and abruptly changing physical and geometric characteristics in elastic and elastic-plastic settings were considered. In [6], to confirm the reliability of the results obtained on the basis of the semi-analytical method of finite elements, the effectiveness of the application of this approach for the calculation of curvilinear inhomogeneous prismatic objects is shown. Solving control problems of the theory of elasticity, thermoelasticity, and thermoplasticity, as well as shape change problems, makes it possible to draw a conclusion about the reliability of the results of the research of a selected class of objects based on the developed methodology and implements its application program package.
 In this paper, using the methodology outlined in the above works, the results of the numerical solution of the problem, which have an applied value, were presented. The stress-strain state of a tubular element with a rectangular cutout under conditions of thermoforce loading was studied. As a rule, solving similar problems is carried out in a flat setting without taking into account the bending load. This approach greatly simplifies the formulation of the problem, but leads to very significant errors in the results. At zero value of the bending load, a good agreement of the planar and spatial solutions is observed. The presence of a bending load leads to deviation of the curves from their initial position. The maximum discrepancy of the results is almost 50%.
 The analysis of this structure from the standpoint of the spatial problem of thermoplasticity, which ensures taking into account the dependence of the physical and mechanical characteristics of the material on temperature and taking into account the bending load on the section of the cut, allowed us to reveal the real features of its deformation.

About Some Aspects of Finite State Machine Models Application to Group Control

The article discusses using discrete-event models to control the coordinated behavior of groups of agents and proposes a mechanism for organizing behavior control based on a hierarchy of finite state machines (FSM). The basic level of agent movement control uses a continuous model of joint movement — the method of potentials, in which the direction of movement is determined by the total sum of the forces of "repulsion", "attraction" and "course alignment". The paper describes a set of independent rules for the group movement of agents — cohesion, distribution, leader following, movement along a chain of landmarks, pursuit, etc. Moreover, all of them are interpreted in terms of the results of the operation of FSM models. Mathematical modeling of controlling a group of agents was carried out using the independent rules of group movement, which confirmed their efficacy. Computational experiments were also carried out using the Gazebo 3D simulator. In addition, the paper describes an experimental group of three underwater vehicles that were used to test the independent rules of group movement: cohesion, distribution, leader following. Communication between the devices was carried out using a system of infrared transceivers via a pseudo-analog channel. The experiments were carried out in a laboratory pool and were successful.The paper shows that the technology for creating group control systems based on the hierarchy of FSM (meta-FSM) allows solving control problems not only at the level of abstract models and simulation modeling, but also at the level of creating real robotic complexes.

Использование дискриминантного анализа для агрегирования оценок знаний обучающихся в номинальных шкалах

A method for determining the parameters of a fuzzy discriminant function from a set of diagnostic test results is considered. These tests are the basis of the training sample for controlling the level of knowledge in nominal rating scales in automated learning systems. The fuzziness of measuring the results of solving control problems, as well as the classification of objects in the training sample, is taken into account. The method has a specific versatility and allows building similar diagnostic procedures for objects of different physical nature.

Implementation of mathematical models and algorithms in task control of the microalgae cultivation processes

This article considers the implementation of mathematical models and algorithms in the problems of controlling the process of chlorella cultivation. When building a control system, the following tasks are solved: collecting and primary processing of information, predicting the course of technological processes, optimizing regime parameters, managing a technological process, etc. One of the necessary conditions for the optimal conduct of the process of cultivating microorganisms is automatic control of the quality and composition of nutrients at the inlet, as well as control of the output indicators of the process. The frequency of solving control problems is set depending on the technological features, the control object, the mode of introducing the process, and also on the availability of a complex of technical means. As a result of the implementation of the proposed control system, the maximum ratio of the concentration of chlorella from its average value decreased by 23.8%. Thus, the control system for the cultivation of chlorella provides timely forecasting of the course of the technological process and the formation of control actions using a computing device, which contributes to an increase in the productivity of the cultivator by 8-12%.

Research of the stressed and deformed state of metal stripin the broaching process

In papers [2,4,5] the solution relations and the algorithm of the method of block iterations of solving linear and nonlinear equations by the semivanalytic finite element method for curvilinear inhomogeneous prismatic bodies are realized. In paper [1], a numerical study of the convergence of solutions was performed, and a wide range of test problems for bodies with smoothly and abruptly changing physical and geometric characteristics in elastic and resilient-plastic formulation was considered. In paper [6], to confirm the reliability of the results obtained on the basis of the semi-analytical finite element method, the effectiveness of this approach for the calculation of curvilinear inhomogeneous prismatic objects is shown. Solving control problems of the theory of elasticity, thermoelasticity and thermoplasticity, as well as problems of shape change makes it possible to draw conclusions about the reliability of the results of the study of a selected class of objects based on the developed methodology and implements its application package.
 In this work, using the methodology outlined in the above works, the solution of the spatial problem of plastic deformation of a prismatic body was solved on the basis of the semi-analytical method of finite elements, and the process of broaching a strip in the process of metal embossing was analyzed. The purpose of this study is to determine the spatial picture of the stress-strain state of the strip during drawing and to compare the spatial resolution with the flat one. A comparison of the parameters of the stress-strain state of the workpiece during the broaching process, obtained in flat and spatial settings, allows us to conclude that the research must be carried out on the basis of spatial calculation.

Near-optimal control of dynamical systems with neural ordinary differential equations

Optimal control problems naturally arise in many scientific applications where one wishes to steer a dynamical system from an initial state x 0 to a desired target state in finite time T. Recent advances in deep learning and neural network–based optimization have contributed to the development of numerical methods that can help solve control problems involving high-dimensional dynamical systems. In particular, the framework of neural ordinary differential equations (neural ODEs) provides an efficient means to iteratively approximate continuous-time control functions associated with analytically intractable and computationally demanding control tasks. Although neural ODE controllers have shown great potential in solving complex control problems, the understanding of the effects of hyperparameters such as network structure and optimizers on learning performance is still very limited. Our work aims at addressing some of these knowledge gaps to conduct efficient hyperparameter optimization. To this end, we first analyze how truncated and non-truncated backpropagation through time affect both runtime performance and the ability of neural networks to learn optimal control functions. Using analytical and numerical methods, we then study the role of parameter initializations, optimizers, and neural-network architecture. Finally, we connect our results to the ability of neural ODE controllers to implicitly regularize control energy.

PID controllers with delay in a problem of stabilization of robotic manipulators’ desired motions

Proportional-integro-differentiating (PID) controllers are widely used in solving control problems of technical systems, including mechanical ones. For this case, most of works are limited to the study of stabilization problem for steady motions and states; such studies are based on the analysis of model equations in a linear approximation. On the other hand, one of the urgent problems of controlled-motion mechanics is the problem of using PID controllers in tracking the trajectories of multi-link robotic manipulators with semi-global or global stabilization in a non-linear formulation. Practically little studied is the problem of justifying the applicability of such controllers taking into account possible delay in the feedback structure. This paper deals with such a problem. As an application of the theory developed in this paper, the control for a motion of a six-link manipulator is obtained.

Differential Graphical Games for Constrained Autonomous Vehicles Based on Viability Theory.

This article proposes an optimal-distributed control protocol for multivehicle systems with an unknown switching communication graph. The optimal-distributed control problem is formulated to differential graphical games, and the Pareto optimum to multiplayer games is sought based on the viability theory and reinforcement learning techniques. The viability theory characterizes the controllability of a wide range of constrained nonlinear systems; and the viability kernel and the capture basin are the pillars of the viability theory. The capture basin is the set of all initial states, in which there exist control strategies that enable the states to reach the target in finite time while remaining inside a set before reaching the target. In this regard, the feasible learning region is characterized by the reinforcement learner. In addition, the approximation of the capture basin provides the learner with prior knowledge. Unlike the existing works that employ the viability theory to solve control problems with only one agent and differential games with only two players, the viability theory, in this article, is utilized to solve multiagent control problems and multiplayer differential games. The distributed control law is composed of two parts: 1) the approximation of the capture basin and 2) reinforcement learning, which are computed offline and online, respectively. The convergence properties of the parameters' estimation errors in reinforcement learning are proved, and the convergence of the control policy to the Pareto optimum of the differential graphical game is discussed. The guaranteed approximation results of the capture basin are provided and the simulation results of the differential graphical game are provided for multivehicle systems with the proposed distributed control policy.

Adaptive tracking of double pendulum crane with payload hoisting/lowering

This paper focuses on the tracking control problem of double pendulum crane systems with payload hoisting/lowering. Some gravity parameters, such as payload masses, are difficult to be accurately measured, which may cause steady positioning errors. To deal with this difficult problem, we propose an effective adaptive tracking controller. By using the proposed method, accurate tracking and double pendulum swing suppression objectives are achieved, and unknown gravity parameters are also accurately estimated by an elaborately designed estimation method, which are verified by simulations and experiments. This work would improve the adaption of transporting payloads with different masses for double pendulum cranes. In our future work, this method would be extended to solve control problems of 3D (3 dimensional) double pendulum cranes.

Investigation of the influence of flange thickness on the nature of the development of zones of plasticity in casing detail

In papers [11, 15 18] the solution relations and the algorithm of the method of block iterations of solving linear and nonlinear equations by the semivanalytic finite element method for curvilinear inhomogeneous prismatic bodies are realized. In paper [1], a numerical study of the convergence of solutions was performed, and a wide range of test problems for bodies with smoothly and abruptly changing physical and geometric characteristics in elastic and resilient-plastic formulation was considered. In paper [21], to confirm the reliability of the results obtained on the basis of the semi-analytical finite element method, the effectiveness of this approach for the calculation of curvilinear inhomogeneous prismatic objects is shown. Solving control problems of the theory of elasticity, thermoelasticity and thermoplasticity, as well as problems of shape change makes it possible to draw conclusions about the reliability of the results of the study of a selected class of objects based on the developed methodology and implements its application package.
 In this work, using the method described in the above works, a numerical analysis of the stress-strain state of a spatial object was performed, namely the study of the influence of flange thickness on the nature of the development of plasticity zones in the casing detail. It should be noted that the use of a thickened flange allowed to localize the zone of plasticity and its length in this case does not exceed half the length of the bell. In this case, the additional cost of material for the manufacture of thickened flange is fully justified. This reduces the level of plastic deformations and stresses in the hazardous area and prolongs the life of the casing detail.