Bellman Equation Research Articles

Feedback Stabilization of Chain-Like MDOF Nonlinear Structural Systems Under Purely Parametric Gaussian White Noises

A feedback control method is proposed to asymptotically stabilize the chain-like multi-degree-of-freedom (MDOF) nonlinear structural system under purely parametric Gaussian white noises. First, the motion equation of a chain-like nonlinear structural system with [Formula: see text] coupled elements and an undetermined control law is derived. It is reduced to a one-dimensional partially averaged Itô stochastic differential equation of the controlled Hamiltonian by applying the stochastic averaging method, which bypasses the challenge of calculating multiple Lyapunov exponents. Next, based on the dynamical programming principle, the dynamical programming equation of the averaged system with an undetermined cost function is established and addressed to obtain the optimal control law. Then, the Lyapunov exponent of the completely averaged Itô equation is derived. For the given requirement of stabilizing the system, the Lyapunov exponent is entirely fixed and used to analyze the stability of the originally chain-like MDOF nonlinear structural system. A controlled chain-like 8-DOF nonlinear structural system is taken as an example to illustrate the application and effectiveness of the proposed procedure and the effect of control forces on stabilizing the system.

On penalized goal-reaching probability minimization under borrowing and short-selling constraints

Abstract We consider a robust optimal investment–reinsurance problem to minimize the goal-reaching probability that the value of the wealth process reaches a low barrier before a high goal for an ambiguity-averse insurer. The insurer invests its surplus in a constrained financial market, where the proportion of borrowed amount to the current wealth level is no more than a given constant, and short-selling is prohibited. We assume that the insurer purchases per-claim reinsurance to transfer its risk exposure to a reinsurer whose premium is computed according to the mean–variance premium principle. Using the stochastic control approach based on the Hamilton–Jacobi–Bellman equation, we derive robust optimal investment–reinsurance strategies and the corresponding value functions. We conclude that the behavior of borrowing typically occurs with a lower wealth level. Finally, numerical examples are given to illustrate our results.

Informational and Methodical Support of Out-Of-School Education Institutions: Basic Principles and Current State

The article reveals the essence of informational and methodical support of out-of-school education institutions. It was found that informational and methodological support is a system of necessary and sufficient conditions for the organization of the educational process, which includes a set of informational, educational, methodological and scientific materials to ensure the need for information sources. It was determined that when developing educational and methodological materials, certain principles should be followed: scientificity, availability and reliability of information, systematicity, relevance, adaptability and innovativeness of educational materials. The main principles of informational and methodical provision of out-of-school education institutions are characterized, in particular, the principle of legality and compliance of the regulatory and legal field; the principle of unity of purpose and goals; the principle of information security; the principle of functional structuring; optimization principle; principle of availability; the principle of sufficiency of informational and methodological support. An analysis of the current state of informational and methodological support of out-of-school education institutions in wartime conditions was carried out by interviewing leaders of clubs and other creative associations from all over Ukraine. The exception was Zaporizhzhia, Kharkiv and Kherson regions, which, due to the hostilities, could not participate. The level of provision of out-of-school education institutions with the necessary teaching and methodical materials for conducting classes has been determined; the quality of extracurricular education programs was analyzed; forms of improvement of the professional level of leaders of circles and other creative associations during the last five years have been clarified; the level of use of innovative forms and methods of work by teachers and which innovations are mainly used by teachers of extra-curricular education institutions is established. It has been proven that informational and methodological support plays a key role in the system of extracurricular education, contributing to the improvement of the quality of the educational process and the effectiveness of pedagogical activities.

Snow Resource Reutilization: Design of Snow Collection and Compression Equipment Based on Functional Analysis Method

High-latitude regions of the Earth are rich in natural snow resources; however, owing to their negative impact on daily life, they have not been effectively utilized for a long time and are instead viewed as obstacles that require substantial resources for clearance. This waste of resources contradicts the principles of sustainable development. With the rapid development of the ice and snow industry, the social and economic value of snow resources is gradually becoming apparent. Therefore, to promote sustainable social and economic development, this study explores new methods for processing snow resources to achieve their recycling and high-value transformation. This study employs the functional analysis method to analyze and solve the functions of the design system, utilizing the Analytic Hierarchy Process (AHP) and fuzzy comprehensive evaluation for the objective assessment and selection of schemes. The obtained relative optimal principles are utilized to guide the parameter setting and innovative design of the scheme in terms of functional structure. After the final scheme is output, its feasibility is verified through finite element simulation. Ultimately, to address the issue of snow resource clearing and recycling, this study designs a product scheme capable of collecting and compressing snow on road surfaces, proposing the use of snow in the form of standardized compressed snow blocks for large-scale snow sculpture construction and other fields. This method significantly reduces the cost of snow sculpture production, enhances efficiency, realizes the comprehensive utilization and high-value transformation of snow resources, and provides a reference for the sustainable development of the low-altitude ice and snow tourism industry.

Enhanced and robust contrast in CEST MRI: Saturation pulse shape design via optimal control.

To employ optimal control for the numerical design of Chemical Exchange Saturation Transfer (CEST) saturation pulses to maximize contrast and stability against inhomogeneities. We applied an optimal control framework for the design pulse shapes for CEST saturation pulse trains. The cost functional minimized both the pulse energy and the discrepancy between the corresponding CEST spectrum and the target spectrum based on a continuous radiofrequency (RF) pulse. The optimization is subject to hardware limitations. In measurements on a 7 T preclinical scanner, the optimal control pulses were compared to continuous-wave and Gaussian saturation methods. We conducted a comparison of the optimal control pulses with Gaussian, block pulse trains, and adiabatic spin-lock pulses. The optimal control pulse train demonstrated saturation levels comparable to continuous-wave saturation and surpassed Gaussian saturation by up to 50 % in phantom measurements. In phantom measurements at 3 T the optimized pulses not only showcased the highest CEST contrast, but also the highest stability against field inhomogeneities. In contrast, block pulse saturation resulted in severe artifacts. Dynamic Bloch-McConnell simulations were employed to identify the source of these artifacts, and underscore the robustness of the optimized pulses. In this work, it was shown that a substantial improvement in pulsed saturation CEST imaging can be achieved by using Optimal Control design principles. It is possible to overcome the sensitivity of saturation to B0 inhomogeneities while achieving CEST contrast close to continuous wave saturation.

Dynamic programming principle for optimal control of uncertain random differential equations and its application to optimal portfolio selection

This study aimed to examine an uncertain stochastic optimal control problem premised on an uncertain stochastic process. The proposed approach is used to solve an optimal portfolio selection problem. This paper’s research is relevant because it outlines the procedure for solving optimal control problems in uncertain random environments. We implement Bellman’s principle of optimality method in dynamic programming to derive the principle of optimality. Then the resulting Hamilton-Jacobi-Bellman equation (the equation of optimality in uncertain stochastic optimal control) is used to solve a proposed portfolio selection problem. The results of this study show that the dynamic programming principle for optimal control of uncertain stochastic differential equations can be applied in optimal portfolio selection. Also, the study results indicate that the optimal fraction of investment is independent of wealth. The main conclusion of this study is that, in Itô-Liu financial markets, the dynamic programming principle for optimal control of uncertain stochastic differential equations can be applied in solving the optimal portfolio selection problem.

Optimization Principles Applied in Planning and Operation of Active Distribution Networks

Optimization principles play an important role in the planning and operation of active distribution networks (ADNs), which are designed to handle the inrush of distributed generation (DG) resources like renewables and, nowadays, battery energy storage systems (BESSs) and electric vehicles (EVs) [...]

Evaluation of prototyping methods for interactive virtual systems based on fuzzy preference relation

The object of this study is the process of determining the optimal method for prototyping interactive virtual systems using the principle of multi-criteria optimization. The paper addresses the task to design tools for evaluating prototyping methods and introduces software developed to identify the best option according to priority criteria. The optimal prototyping method was chosen based on the maximum value of the membership function with indicators of 0.3, 0.7, and 1. Thus, the best method for the given task is rapid prototyping, with a membership function value equal to 1. These results have made it possible to solve the problem by evaluating fuzzy preference relations on a set of alternatives. Additionally, a relation convolution was constructed, from which a subset of non-dominated alternatives was identified. The key features of the proposed approach are obtaining clear quantitative evaluation parameters by processing descriptive, non-unified, and non-formalized input data. The fuzzy preference relations for the set of alternatives are explained by the analysis of the most common methods for prototyping interactive systems. The developed information system could be used for managerial decision-making regarding the selection of optimal prototyping option from several possibilities, by comparing methods based on predefined criteria with arbitrary weighting coefficients. The result of the research is a universal system adaptable to specific user tasks. The resulting data will contribute to improving the efficiency of prototyping and, consequently, reducing costs.

Optimizing Thermoplastic Starch Film with Heteroscedastic Gaussian Processes in Bayesian Experimental Design Framework

The development of thermoplastic starch (TPS) films is crucial for fabricating sustainable and compostable plastics with desirable mechanical properties. However, traditional design of experiments (DOE) methods used in TPS development are often inefficient. They require extensive time and resources while frequently failing to identify optimal material formulations. As an alternative, adaptive experimental design methods based on Bayesian optimization (BO) principles have been recently proposed to streamline material development by iteratively refining experiments based on prior results. However, most implementations are not suited to manage the heteroscedastic noise inherently present in physical experiments. This work introduces a heteroscedastic Gaussian process (HGP) model within the BO framework to account for varying levels of uncertainty in the data, improve the accuracy of the predictions, and increase the overall experimental efficiency. The aim is to find the optimal TPS film composition that maximizes its elongation at break and tensile strength. To demonstrate the effectiveness of this approach, TPS films were prepared by mixing potato starch, distilled water, glycerol as a plasticizer, and acetic acid as a catalyst. After gelation, the mixture was degassed via centrifugation and molded into films, which were dried at room temperature. Tensile tests were conducted according to ASTM D638 standards. After five iterations and 30 experiments, the films containing 4.5 wt% plasticizer and 2.0 wt% starch exhibited the highest elongation at break (M = 96.7%, SD = 5.6%), while the films with 0.5 wt% plasticizer and 7.0 wt% starch demonstrated the highest tensile strength (M = 2.77 MPa, SD = 1.54 MPa). These results demonstrate the potential of the HGP model within a BO framework to improve material development efficiency and performance in TPS film and other potential material formulations.

Professionally significant qualities of teaching producing in the context of digital development: Evaluating the effectiveness by education stakeholders

Introduction. The article is devoted to the problem of increasing the level of communicative culture in future economists in the digital era. The purpose of the research is to substantiate the effectiveness of professionally important qualities of teaching producing in universities of Economics, taking into account the digital format of interaction. Materials and Methods. The methodological basis of the study is the Pareto optimality principle (2080), which allows the authors to determine the necessary minimum of teaching resources in terms of modern communication knowledge for training economists in the digital age. During the study, an online survey was conducted in order to substantiate the effectiveness of professionally important qualities of teaching producing in the context of the digital specifics of the socio-economic space. The research sample consisted of students and academic staff of economics departments of the Financial University under the Government of the Russian Federation, Omsk State Pedagogical University, and Omsk State Technical University. Results. The theoretical analysis of the research problem revealed professionally important qualities of teaching producing (communication skills, digitals kills, knowledge of the principles of teaching ethics to create an atmosphere of trust and psychological safety), which facilitate the development of future economists’ communicative culture (motivational, cognitive, and functional components). According to the results of the empirical part of the study, the education stakeholders highly appreciated the effectiveness of professionally significant qualities of teaching producing and their positive correlation. Conclusions. The use of the Pareto rule allows the authors to conclude that the formation of professionally significant t qualities of teaching producing, which make up about 20% of the professional qualities composition for academic staff at universities of economics, determines 80% of the productivity of educational outcomes in terms of developing future economists’ communicative culture. This, in turn, determines the ability of graduates to build communication taking into account the career prospects at various levels of digitalization and interaction with artificial intelligence systems.

Optimal control of a queueing system

ABSTRACT We consider the M / M / k queueing model modified as follows: we assume that one can decide how many servers are working at any time. Suppose that at a given time instant, there are k + l customers in the system and that they are all waiting for service. Our aim is to determine how many servers should be used in order to reduce the number of customers to k + r, where 0 ≤ r < l , as rapidly as possible, while taking the control costs into account. The dynamic programming equation satisfied by the value function is obtained in the general case, and particular problems are solved explicitly.

Discretization of Fractional Fully Nonlinear Equations by Powers of Discrete Laplacians

AbstractWe study discretizations of fractional fully nonlinear equations by powers of discrete Laplacians. Our problems are parabolic and of order $$\sigma \in (0,2)$$ σ ∈ ( 0 , 2 ) since they involve fractional Laplace operators $$(-\Delta )^{\sigma /2}$$ ( - Δ ) σ / 2 . They arise e.g. in control and game theory as dynamic programming equations – HJB and Isaacs equation – and solutions are non-smooth in general and should be interpreted as viscosity solutions. Our approximations are realized as finite-difference quadrature approximations and are 2nd order accurate for all values of $$\sigma $$ σ . The accuracy of previous approximations of fractional fully nonlinear equations depend on $$\sigma $$ σ and are worse when $$\sigma $$ σ is close to 2. We show that the schemes are monotone, consistent, $$L^\infty $$ L ∞ -stable, and convergent using a priori estimates, viscosity solutions theory, and the method of half-relaxed limits. We also prove a second order error bound for smooth solutions and present many numerical examples.

Mathematical Modeling Unveils Optimization Strategies for Targeted Radionuclide Therapy of Blood Cancers.

Targeted radionuclide therapy is based on injections of cancer-specific molecules conjugated with radioactive nuclides. Despite the specificity of this treatment, it is not devoid of side-effects limiting its use and is especially harmful for rapidly proliferating organs well perfused by blood, like bone marrow. Optimization of radioconjugates administration accounting for toxicity constraints can increase treatment efficacy. Based on our experiments on disseminated multiple myeloma mouse model treated by 225Ac-DOTA-daratumumab, we developed a mathematical model which investigation highlighted the following principles for optimization of targeted radionuclide therapy. 1) Nuclide to antibody ratio importance. The density of radioconjugates on cancer cells determines the density of radiation energy deposited in them. Low labeling ratio as well as accumulation of unlabeled antibodies and antibodies attached to decay products in the bloodstream can mitigate cancer radiation damage due to excessive occupation of specific receptors by antibodies devoid of radioactive nuclides. 2) Cancer binding capacity-based dosing. The total number of specific receptors on cancer cells is a critical factor for treatment optimization, which estimation may allow increasing treatment efficacy close to its theoretical limit. Injection of doses significantly exceeding cancer binding capacity should be avoided since radioconjugates remaining in the bloodstream have negligible efficacy to toxicity ratio. 3) Particle range-guided multi-dosing. The use of short-range particle emitters and high-affinity antibodies can allow for robust treatment optimization via initial saturation of cancer binding capacity, enabling redistribution of further injected radioconjugates and deposited dose towards still viable cells that continue expressing specific receptors.

Discovering and Verifying the Optimization Principles for Carbon Neutrality in Human Society

Discovering and Verifying the Optimization Principles for Carbon Neutrality in Human Society

Discrete-Time Hybrid Control Processes with Unbounded Costs

This paper extends the results provided in Jasso-Fuentes et al. (Appl Math Optim 81(2):409–441, 2020b) and Jasso-Fuentes et al. (Pure Appl Funct Anal 9(3):675–704, 2024) regarding the study of discrete-time hybrid stochastic models with general spaces and total discounted payoffs. This extension incorporates the handling of negative and/or unbounded costs per stage. In particular, it encompasses interesting applications, such as scenarios where the controller optimizes net costs, social welfare costs, or distances between points. These situations arise when assumptions of both non-negativeness and boundedness on the cost per stage do not apply. Our proposal relies on Lyapunov-like conditions, enabling, among other aspects, the finiteness of the value function and the existence of solutions to the associated dynamic programming equation. This equation is crucial for deriving optimal control policies. To illustrate our theory, we include an example in inventory-manufacturing management, highlighting its evident hybrid nature.

Design of sliding mode controller for servo feed system based on generalized extended state observer with reinforcement learning

Nonlinear friction, system uncertainty, and external disturbances have a significant impact on the performance of high-precision servo feed systems. In order to achieve higher tracking accuracy, a sliding mode controller based on a generalized extended state observer with the double critic deep deterministic policy gradient algorithm is designed. Firstly, a flexible two mass drive model (FTMDM) is established for the two-axis differential micro-feed system (TDMS). Next, a generalized extended state observer (GESO) is designed to estimate matched interference and mismatched interference. And it is proved that the observation error of GESO is bounded. A sliding mode controller based on GESO is further proposed. The stability of the controller has been proven through Lyapunov theory, and the error is bounded and converges to zero in finite time. The tuning process of controller parameters is simplified by using quadratic optimal control principle. Furthermore, a double critic deep deterministic policy gradient algorithm (DCDDPG) is proposed to achieve dynamic optimization of parameters about GESO. The simulation results show that GESO with DCDDPG can reduce the observation error of step signal and sinusoidal signal, and improve the observation accuracy of nonlinear friction significantly. Finally, experimental results show that the proposed control method achieves more accurate position tracking performance on TDMS.

Discrete-time hybrid control with risk-sensitive discounted costs

AbstractThis paper studies discrete-time hybrid control problems where the controller’s payoff function follows a risk-sensitive discounted model. The associated discount factor can depend on state and action variables, which can even take values of zero or one. We prove the existence of optimal control policies under two different hypotheses. The following technique is the dynamic programming method, where we characterize the minimum cost as the solution of the so-called dynamic programming equations and find the above optimal policies through these equations. We illustrate our theory with practical examples involving inventory-manufacturing systems, and pollution management.

Dynamic Programming-Based Approach to Model Antigen-Driven Immune Repertoire Synthesis

This paper presents a novel approach to modeling the repertoire of the immune system and its adaptation in response to the evolutionary dynamics of pathogens associated with their genetic variability. It is based on application of a dynamic programming-based framework to model the antigen-driven immune repertoire synthesis. The processes of formation of new receptor specificity of lymphocytes (the growth of their affinity during maturation) are described by an ordinary differential equation (ODE) with a piecewise-constant right-hand side. Optimal control synthesis is based on the solution of the Hamilton–Jacobi–Bellman equation implementing the dynamic programming approach for controlling Gaussian random processes generated by a stochastic differential equation (SDE) with the noise in the form of the Wiener process. The proposed description of the clonal repertoire of the immune system allows us to introduce an integral characteristic of the immune repertoire completeness or the integrative fitness of the whole immune system. The quantitative index for characterizing the immune system fitness is analytically derived using the Feynman–Kac–Kolmogorov equation.

Environmental Management in Lithium Extraction: Optimization of the Hydrometallurgical Process for Lepidolite with Focus on Eco-Efficiency and Life Cycle Assessment

Objective: Develop and optimize an eco-efficient hydrometallurgical process for lithium extraction from lepidolite, enhancing environmental management in the lithium mining industry. Theoretical Framework: The study is grounded in principles of eco-efficiency, life cycle assessment (LCA), and optimization of chemical processes applied to lithium extraction within the context of organizational environmental management. Method: A fractional factorial design 2^(5-2) was employed for process optimization. Mineralogical characterization was conducted using XRD and ICP-MS. Extraction efficiency, product purity, and environmental parameters were evaluated through LCA. Results and Discussion: Extraction efficiency increased from 91.0 ± 1.5% to 94.5 ± 0.8%, with a product purity of 99.62 ± 0.04%. LCA revealed a carbon footprint of 5.9 ± 0.3 kg CO₂ eq / kg Li₂CO₃ and a water consumption of 38 ± 2 m³/t Li₂CO₃. Implications in relation to conventional extraction methods are discussed. Research Implications: The study provides a framework for implementing more sustainable processes in lithium mining, potentially applicable to the battery industry and natural resource management. Originality/Value: The research innovatively integrates process optimization, LCA, and eco-efficiency principles in lithium extraction, contributing to the sustainable transformation of the lithium mining industry.

Maximum Fourier spectrum cyclostationarity blind deconvolution and its application in structural health monitoring of power transformers

Abstract Power transformer is the most important equipment that affects whether the electric power system can be operated safely and normally, whose condition assessment problem has attracted considerable attention. Background noise frequently affects the effectiveness of nonintrusive techniques based on vibro-acoustic signals for structural health monitoring in power transformers. It is challenging to effectively eliminate background noise interference from time-frequency domain transformer defect detection techniques such as wavelet transformations, modal decomposition, etc. In this scenario, the Fourier spectrum cyclostationarity index (FSC) is designed based on the cyclostationarity index used for rotating machinery fault diagnosis to construct a maximum Fourier spectrum cyclostationarity blind deconvolution method (MFSCBD) for transformer fault detection in this paper. Firstly, the limitations of the traditional blind deconvolution (BD) in transformer fault detection are discussed in the mathematical principle. Then, a new BD framework based on Kepler optimization algorithm is proposed according to the principle of convex optimization to address the problems of difficulty in solving the complex blind objective function in the traditional differential BD framework and the ill-condition problem in the Rayleigh quotient BD framework. Subsequently, a synthetic nonstationary and nonlinear simulation signal is constructed for numerical verification, and a six-microphone array is designed to obtain the practical signals from the operating transformer to verify the performance of MFSCBD. Finally, the applications on the simulated and experimental signals of power transformers demonstrate that MFSCBD outperforms complete ensemble empirical mode decomposition with adaptive noise and successive variational mode decomposition to some extent for structural health monitoring.