Bellman Equation Research Articles

Surgical Strategies for Renal Transplantation: A Pictorial Essay.

This pictorial essay aims to navigate through the complexities and challenges of renal transplantation (RT), by weaving together visual imagery with clinical insights within a comprehensive illustrative surgical guide. Herein, we provide a detailed visual exploration of the intricate anatomy and surgical processes necessary for both renal graft retrieval from the donor and also for an adequate implantation in the recipient. Regarding graft retrieval, after reviewing the relevant retroperitoneal surgical anatomy, and donor nephrectomy techniques, graft preservation and optimal backbench graft dissection principles were meticulously analyzed. Thereafter, the recipient surgical strategy for graft implantation was addressed, focusing on preoperative preparations, the site of implantation selection, exposure, operative bed dissection, graft revascularization, and urinary tract reconstruction. Careful donor and recipient selection, meticulous surgical execution, and rigorous postoperative management clearly hold a pivotal role in optimizing patient outcomes. Fostering a deeper understanding of the surgical nuances and clinical management practices that contribute to successful results post-RT, we hope to provide a useful practical tool for clinicians about to embark on the treacherous road of RT surgery. Innovative technologies and surgical practices that have already significantly improved the safety and effectiveness of RT stand testament to the importance of further scientific inquiry, conceptual developments, and clinical integration. Moving forward, it is essential that the medical community continues to refine these strategies and advocate for equitable access to transplantation, ensuring that advancements in the field translate into real-world benefits for all patients grappling with ESRD. The collaborative efforts of multidisciplinary teams are essential in addressing the complex clinical challenges associated with RT, with the ultimate goal of improving patient survival, enhancing graft longevity, and reducing healthcare disparities.

Principles of Creating Multi-objective Quality Models for Software Systems

This article proposes an original approach at a mathematical level to the creation of multi- objective quality models for software systems. The research is based on the study and generalization of quality modeling trends of software systems and user needs in order to determine optimal principles for constructing such models. The article provides mathematical explanations that play a key role in identifying and formalizing the principles of creating multi-objective quality models of software. An important aspect is the consideration of quality model construction principles at the mathematical level, allowing for a more precise assessment and analysis of various aspects of software quality. The research results indicate that incorporating mathematical principles into the creation of multi-objective quality models for software systems can have a significant practical impact. It is established that on a practical level of developing multi-objective quality models for software systems, consideration of the principles of creating multi-objective quality models can have numerous practical implications. Specifically, the application of metrics within established principles allows for a comprehensive view of software quality and identifies areas requiring attention and improvement. This helps developers and software quality engineers make informed decisions regarding system improvement and optimization. The research has shown that creating high-quality models of quality requires attention to various aspects, from user needs to testing and continuous improvement, as well as the use of mathematical methods for their formalization and analysis. The developed principles of creating multi-objective quality models at the level of mathematical models allow the use of these models to assess and analyze various aspects of software quality, representing each model using a corresponding function that determines the relationship between quality metrics and the quality of the software itself. It is expected that further development and implementation of these principles will contribute to improving software development processes and ensure high quality of the resulting software.

A collaborative neurodynamic approach with two-timescale projection neural networks designed via majorization-minimization for global optimization and distributed global optimization

In this paper, two two-timescale projection neural networks are proposed based on the majorization-minimization principle for nonconvex optimization and distributed nonconvex optimization. They are proved to be globally convergent to Karush–Kuhn–Tucker points. A collaborative neurodynamic approach leverages multiple two-timescale projection neural networks repeatedly re-initialized using a meta-heuristic rule for global optimization and distributed global optimization. Two numerical examples are elaborated to demonstrate the efficacy of the proposed approaches.

Dynamic event-based cooperative control of virtually coupled high-speed trains: an ADP-based optimal control scheme

This paper is concerned with the cooperative control issue of virtually coupled high-speed trains (VCHSTs) under an adaptive dynamic programming (ADP) framework, where a dynamic event-triggered mechanism is employed to govern the communication between sensors and observers. A learning-based observer, equipped with a robust term, is exploited to mitigate the effect of the unknown nonlinear dynamics of the VCHSTs. Furthermore, to realise the distributed design, an auxiliary control signal is employed to simply the closed-loop system and also integrated into the cost function to handle the coupled nature of the investigated systems. The Hamilton–Jacobi–Bellman (HJB) equation is approximated by actor and critic networks in the ADP framework, which iteratively acquires the desired control strategy. By resorting to the common Lyapunov stability, some sufficient conditions are derived to achieve the desired observer gain and learning rates of NNs. Ultimately, the control scheme's efficiency is confirmed by simulations of VCHSTs.

Data-driven-based sliding-mode dynamic event-triggered control of unknown nonlinear systems via reinforcement learning

This paper investigates a data-driven-based dynamic event-triggered control problem for continuous-time unknown nonlinear systems using the reinforcement learning method and the sliding-mode surface technique. Initially, by constructing a cost function associated with sliding-mode surface variables for the nominal system, the original control problem is equivalently transformed into a problem of designing a dynamic event-triggered optimal control policy. To handle the unknown issue of system dynamics, a data-driven model is established to reconstruct the system dynamics. Then, under the framework of reinforcement learning, a critic network is employed to solve the event-triggered Hamilton–Jacobi–Bellman equation. The weight vector in the critic network is updated through the current data and historical data, such that the persistence of excitation condition is no longer needed. After that, it is strictly proven via Lyapunov stability theory that all the signals of the considered system are bounded in the sense of uniformly ultimately boundedness. Finally, the effectiveness of the developed control method is demonstrated by two simulation examples.

Modeling and Optimal Control of Infectious Diseases

We propose a stochastic model of infectious disease transmission that is more realistic than those found in the literature. The model is based on jump-diffusion processes. However, it is defined in such a way that the number of people susceptible to be infected decreases over time, which is the case for a population of fixed size. Next, we consider the problem of finding the optimal control of the proposed model. The dynamic programming equation satisfied by the value function is derived. Estimators of the various model parameters are obtained.

Approximation of the HJB viscosity solutions in differential inclusion optimal control with state constraint

This paper tries to investigate a differential inclusion optimal control problem with state constraint (P) through an approximation of Hamilton–Jacobi–Bellman viscosity solutions. By invoking a duality result between the control problem (P) and the upper hull of viscosity subsolutions of the HJB equation, we prove that value functions of the discrete problems are solutions of discrete Bellman equations and converge to the value function of (P). We also establish that the optimal trajectory sequence of discrete problems converges to the solution of the optimal control problem (P). The idea behind the approximation is based on Euler polygonal arcs and Subbotin's proximal aiming condition, which allows discrete trajectories to break the state constraint, evolve in its neighbourhood and then converge to it instead of staying in the state constraint set for all time.

REDUCING OF WATER AND ENERGY RESOURCES CONSUMPTION IN IRRIGATION BASED ON RESOURCE OPTIMISATION

The need for further development of irrigation practices on the basis of nature oriented and ecologically efficient solutions is considered primarily regarding the saving of water and energy resources for the adaptation of irrigated agricultural production to the current global challenges and threats, the achievement of sustainable development goals. It is shown that there is an objective need to change methodological approaches to the assessment of overall efficiency and justification of optimal solutions in design, reconstruction, and operation projects of irrigation systems based on the principles of resource optimization. On the basis of the indicator of the level of irrigation sufficiency (introduced by the authors), which reflects the reduction of the studied watering and irrigation rates in relation to their design values, a study of the impact of reducing the use of water and energy resources under different modes of sprinkler irrigation on the corresponding decrease in the level of cultivated crops productivity was carried out. At the same time, it was experimentally determined that the intensity (rate) of the decrease in the cultivated crops productivity, which occurs due to the decrease in the usage of water and energy resources during the application of irrigation, is significantly lower than the intensity (rate) of the decrease in the usage of the resources themselves. The studied options for reducing the consumption on water and energy resources as a whole turned out to be economically profitable when with a decrease in water and electricity usage by 27–48% there is more than two times lower decrease in the costs of gross products by 10,80–18,06% with the achievement of a net income of 11,4 to 5,7 thousand UAH. The influence of various options of reducing water and electricity consumption on the discounted investment payback period shows that several options may be acceptable, for which the investment payback period does not exceed 10 years, and the choice of the optimal solution requires taking into account the conditions of a specific object, limitations of water resources, and the interests of investors and land users. At the same time, the ecological component of the overall efficiency of irrigation practices consists in the decrease in the use of water and energy resources that a priori reflects the decrease in the negative impact of irrigation on the environment. Thus, reducing the usage of water and energy resources is a fully justified decision on the way to adapt irrigated agriculture production to the modern conditions and requirements, and the presented results can be a scientific basis for the implementation of this approach while practically applying irrigation.

Symmetry-Optimized Dynamical Analysis of Optical Soliton Patterns in the Flexibly Supported Euler–Bernoulli Beam Equation: A Semi-Analytical Solution Approach

This study employs spatial optimization principles to investigate the nonlinear vibration of a flexibly supported Euler–Bernoulli beam, a (1 + 1)-dimensional system subjected to axial loads. The modified Khater method, a crucial tool in mechanical engineering, is utilized to analyze analytical solutions, which include a symmetric spatial representation of the waveform as an integral part of each solution. Notably, periodic soliton solutions for the nonlinear model closely align with numerical and approximate analytical solutions, demonstrating the accuracy of our modeling approach. Density diagrams, contour diagrams, and Poincaré maps depicting the obtained analytical solutions are presented to elucidate their accuracy and provide visual confirmation of the optimized engineering model’s physical significance. The planar dynamical system is derived through the Galilean transformation by employing mathematical models and appropriate parameter values, thereby further refining problem understanding. Sensitivity analysis is conducted, and phase portraits with equilibrium points are illustrated by analyzing a special case of the investigated dynamical system, emphasizing its symmetrical properties. Lastly, we perform a global analysis to identify periodic, quasi-periodic, and chaotic behaviors, with an extra weak forcing term confirmed by Poincaré maps and a two-dimensional symmetric basin of the largest Lyapunov exponent.

Nutritional, Phytochemical, and Rheological Profiles of Different Oat Varieties and Their Potential in the Flour Industry

The purpose of this paper is to evaluate from a nutritional, phytochemical, and rheological point of view different varieties/lines of oats cultivated in Romania. In this sense, the proximate composition (proteins, lipids, mineral substances, starch, and carbohydrates), the total polyphenols content (TPC) of individual polyphenols and macro and microelements of 20 oat varieties/lines were determined. In order to evaluate the potential for use in the flour industry (pasta, bread, and biscuits)., a variety of oats with optimal nutritional properties and active principles was used to obtain oat/wheat composite flours in different proportions (10–30% oat flour) and subjected to rheological analysis using the Chopin Mixolab test (Chopin Technologies, Paris, France) The results showed a protein content between 14.36–17.08%, lipid content between 6.01–9.03%, starch between 44.13–52.95%, total mineral substances between 0.93–1.65%, carbohydrates between 65.16–69.32% and energy value between 389.81–409.86 kcal/100 g. The TPC content varied between 614.76–1244.58 µgGAE/g, and the average values of individual polyphenols were between 1.87 μg/g (rosmarinic acid) and 28.18 μg/g (gallic acid). Regarding the macroelements content, potassium (K) was the major element (203.19 μg/g), followed by calcium (Ca) (181.71 μg/g) and magnesium (Mg) (105.79 μg/g). The level of microelements varied between 3.02 μg/g for nickel (Ni) and 60.25 μg/g for iron (Fe). Multivariate data analysis PCA was used to correlate the obtained data. The rheological analysis using the MIXOLAB system proved that, from the point of view of technological properties, composite flours from a mixture of 10–30% oat and wheat flour are the best for obtaining biscuit dough, compared to pasta or pan bread. Following the study and based on nutritional, phytochemical and rheological results, line 2511 was selected as optimal for use in obtaining hypoglycemic flour foods for people with diabetes.

Применение принципа Понтрягина для оптимального отключения энергетического ядерного реактора

The theory of optimal control is based on two approaches, the dynamic programming method (Bellman equation) and Pontryagin maximum principle. Pontryagin maximum principle is applied in the physics of nuclear reactors when optimizing various transient processes. The mathematical justification of this theory is based on the elements of convex analysis, which is not used by physicists and engineers, so the physical issues are less studied in scientific literature. The subject of the study is a nuclear power reactor of the WWER type. The problem of minimizing its shutdown time, bypassing the iodine pit is solved and it is possible to start up the reactor at any moment after its shutdown. Analytical and numerical methods are used. The paper considers an example of applying the maximum principle for optimal control of the process of shutting down a nuclear reactor bypassing the iodine pit. A physical mathematical model of the Pontryagin principle is formulated. The process of optimal control of reactor shutdown for large and small reactivity margins is justified and calculated. Pontryagin principle does not contain an algorithm to find an optimization process; the stages of the process must be selected based on physical considerations, but these stages must satisfy the specified principle. Based on the Pontryagin principle, the results of the study make it possible to draw up a step-by-step action plan when shutting down a WWER-type reactor with any value of the reactivity margin and its switching is possible at any time after the transition process, which avoids downtime. The proposed plan can be used both in mathematical modeling of transient processes in a reactor and in reactor control systems to improve its controllability and, consequently, to improve safety.

Optimal Reinsurance and Derivative-Based Investment Decisions for Insurers with Mean-Variance Preference

In our study, we investigate reinsurance issues and optimal investment related to derivatives trading for a mean-variance insurer, employing game theory. Our primary objective is to identify strategies that are time-consistent. In particular, the insurer has the flexibility to purchase insurance in proportion to its needs, explore new business, and engage in capital market investments. This is under the assumption that insurance companies’surplus capital adheres to the classical Cramér-Lundberg model. The capital market is made up of risk-free bonds, equities, and derivatives, with pricing dependent on the underlying stock’s basic price and volatility. To obtain the most profitable expressions and functions for the associated investment strategies and time guarantees, we solve a system of expanded Hamilton–Jacobi–Bellman equations. In addition, we delve into scenarios involving optimal investment and reinsurance issues with no derivatives trading. In the end, we present a few numerical instances to display our findings, demonstrating that the efficient frontier in the case of derivative trading surpasses that in scenarios where derivative trading is absent.

Features of risk management in the enterprise under the conditions of martial law state

The paper is devoted to the strategic direction of risk management to improve the efficiency of the enterprise in wartime. Justification of the peculiarities of the formation and improvement of risk management at the enterprise in the conditions of martial law. The main aspects of risk management according to Strategy Execution are presented. The key principles of risk management optimization are considered, including features of strategy development, integration into business processes, improvement of management functions and control processes. In particular, the importance of introducing risk management into the main phases of business planning of an enterprise is presented. It has been proven that effective risk management in modern business is of great importance, especially for ensuring the success of the organization in a competitive environment. One of the key advantages of such management is the preservation of the company's reputation, by preventing possible incidents that may negatively affect the reputation, the organization protects its name and, accordingly, maintains the trust of consumers and partners. The main aspects and general approach to risk-oriented management in the economic security of the enterprise are presented. It has been proven that instability in the labor market and loss of confidence in the internal economic environment in the market can lead to a reduction in investment and lack of development of new sectors of the economy. It is justified that in order to improve the effective operation of the enterprise in the management system, it is recommended to carry out strategic direction of risk management based on such principles as: strategy development; integration of risk management into business processes; improvement of management functions; improvement of control processes and measures; Measures that will help enterprises reduce costs, increase risk coverage and ensure sustainable development are considered.

Construction of a Symbiotic Traffic Network of Traditional Villages in the Western Hunan Region of China Based on Circuit Theory

The concentrated and contiguous development of traditional Chinese villages is imminent, and the realization of their symbiotic and coordinated development has become both a priority and a challenge. Taking 370 traditional villages in Western Hunan as the research object, a GIS spatial analysis method was used to identify and extract the sources of traditional villages in Western Hunan, construct traffic resistance surfaces, identify traffic corridors and traffic nodes based on circuit theory, and construct and optimize the symbiotic traffic network of traditional villages in Western Hunan. The results show that the symbiotic traffic network of traditional villages in Western Hunan is composed of 47 traditional village sources, 77 traffic corridors, and 68 key traffic nodes, which are concentrated in the northwest and southeast of Western Hunan, showing a spider-like structure. The distribution of traditional village sources shows the characteristics of a high distribution in the north and a low distribution in the middle, and the traffic resistance surface shows spatial characteristics of being high in the north and low in the south. Four traffic corridor optimization principles and five key traffic node restoration strategies are proposed. This study provides a new concept underlying, and method for, the planning and construction of the symbiotic transport network of traditional villages in Western Hunan, promoting the centralized protection and utilization of traditional villages and the coordinated symbiotic development of regions and assisting in the implementation of the rural revitalization strategy.

Multi-agent Gradient-Based Off-Policy Actor-Critic Algorithm for Distributed Reinforcement Learning

This paper proposes a gradient-based multi-agent actor-critic algorithm for off-policy reinforcement learning using importance sampling. Our algorithm is incremental with full gradients, and its complexity per iteration scales linearly with the size of approximation features. Previous multi-agent actor-critic algorithms are limited to the on-policy setting or off-policy emphatic temporal difference (TD) learning and they do not take advantage of the advances in off-policy gradient temporal difference learning (GTD). As a theoretical contribution, we establish that the critic step of the proposed algorithm converges to the TD solution of the projected Bellman equation and the actor step converges to the set of asymptotically stable fixed points. Numerical experiments on the multi-agent generalization of the Boyan’s chain problem show that the proposed approach provides improved performances in terms of stability and convergence rate as compared with the state-of-the-art baseline algorithm.

Optimal control of linear cost networks

We present a method for optimal control with respect to a linear cost function for positive linear systems with coupled input constraints. We show that the Bellman equation giving the optimal cost function and resulting sparse state feedback for these systems can be stated explicitly, with the solution given by a linear program. Our framework admits a range of network routing problems with underlying linear dynamics. These dynamics can be used to model traditional graph-theoretical problems like shortest path as a special case, but can also capture more complex behaviors. We provide an asynchronous and distributed value iteration algorithm for obtaining the optimal cost function and control law.

Optimality principles in spacecraft neural guidance and control.

This Review discusses the main results obtained in training end-to-end neural architectures for guidance and control of interplanetary transfers, planetary landings, and close-proximity operations, highlighting the successful learning of optimality principles by the underlying neural models. Spacecraft and drones aimed at exploring our solar system are designed to operate in conditions where the smart use of onboard resources is vital to the success or failure of the mission. Sensorimotor actions are thus often derived from high-level, quantifiable, optimality principles assigned to each task, using consolidated tools in optimal control theory. The planned actions are derived on the ground and transferred on board, where controllers have the task of tracking the uploaded guidance profile. Here, we review recent trends based on the use of end-to-end networks, called guidance and control networks (G&CNets), which allow spacecraft to depart from such an architecture and to embrace the onboard computation of optimal actions. In this way, the sensor information is transformed in real time into optimal plans, thus increasing mission autonomy and robustness. We then analyze drone racing as an ideal gym environment to test these architectures on real robotic platforms and thus increase confidence in their use in future space exploration missions. Drone racing not only shares with spacecraft missions both limited onboard computational capabilities and similar control structures induced from the optimality principle sought but also entails different levels of uncertainties and unmodeled effects and a very different dynamical timescale.

Game-Theory-Based Design and Analysis of a Peer-to-Peer Energy Exchange System between Multi-Solar-Hydrogen-Battery Storage Electric Vehicle Charging Stations

As subsidies for renewable energy are progressively reduced worldwide, electric vehicle charging stations (EVCSs) powered by renewable energy must adopt market-driven approaches to stay competitive. The unpredictable nature of renewable energy production poses major challenges for strategic planning. To tackle the uncertainties stemming from forecast inaccuracies of renewable energy, this study introduces a peer-to-peer (P2P) energy trading strategy based on game theory for solar-hydrogen-battery storage electric vehicle charging stations (SHS-EVCSs). Firstly, the incorporation of prediction errors in renewable energy forecasts within four SHS-EVCSs enhances the resilience and efficiency of energy management. Secondly, employing game theory’s optimization principles, this work presents a day-ahead P2P interactive energy trading model specifically designed for mitigating the variability issues associated with renewable energy sources. Thirdly, the model is converted into a mixed integer linear programming (MILP) problem through dual theory, allowing for resolution via CPLEX optimization techniques. Case study results demonstrate that the method not only increases SHS-EVCS revenue by up to 24.6% through P2P transactions but also helps manage operational and maintenance expenses, contributing to the growth of the renewable energy sector.

ASSESSMENT OF THE EFFICIENCY OF MARKETING BUDGETS ON THE B2C E-COMMERCE MARKET

Introduction. In the conditions of the rapid development of e-commerce in the B2C market, it is important to ensure the effective use of marketing budgets to achieve competitive advantages. Evaluating the effectiveness of marketing spend allows businesses to optimize resources, increase return on investment, and reach a larger target audience. This requires the formation of methodological approaches for evaluating and improving the effectiveness of marketing budgets in this area. The purpose of the article. The purpose of the study is to study approaches to evaluating the effectiveness of marketing budgets in the B2C e-commerce market in order to determine the principles of increasing the effectiveness of their use. Methods. The research methodology includes a comprehensive approach combining quantitative and qualitative methods of analysis. In particular, for the in-depth study of specific examples of successful marketing strategies, such methods as economic modeling, the method of analysis and synthesis, and comparison were used. Results. The article examines approaches to evaluating the effectiveness of marketing costs in the B2C e-commerce sector, focusing on different channels, strategies and the corresponding profitability of marketing costs. The current market trends of the functioning of e-commerce platforms were analyzed and the main factors affecting the effectiveness of the use of marketing budgets of business entities were identified. The trends of the global growth of digital commerce volumes in the context of B2C sales and its functional specificity are clarified. It was determined that personalization, segmentation of customers and integration of multi-channel strategies significantly increase the efficiency of using marketing budgets. The principles of optimization of marketing costs based on the implementation of automated systems for data analysis and forecasting of consumer behavior were studied. The influence of the latest technologies, such as artificial intelligence and machine learning, on the effectiveness of using marketing budgets is considered. Conclusions. It has been proven that a reliable assessment of the effectiveness of marketing budgets in the B2C market determines a number of key aspects that ensure the maximum return on marketing investments of a business entity. At the same time, the concept of effectiveness of marketing budgets consists in a targeted approach implemented on the basis of personalization, segmentation and multi-channel integration.

Bioinspired Polarization‐Response Structure with Broadband Absorption based on a Multifrequency Point Response Mechanism

AbstractTo address the increasing demand for high microwave‐absorbing performance, new polarization‐response wave‐absorbing metamaterials are developed herein inspired by the microstructure of locust compound eyes by employing bionics and optimization principles. The materials overcome a common limitation of wave‐absorbing metamaterials, wherein the performance substantially decreases with changes in the polarization mode, enabling the broadband absorption for x and y polarizations. The designed bioinspired materials have various shapes of structural units, triggering a multifrequency point response mechanism that effectively improves the wave‐absorbing performance. Experimental and simulation results show that the designed materials can form different equivalent circuits based on structural and polarization differences, influencing the overall wave‐absorbing performance. They exhibit an effective absorption bandwidth of ≈12 GHz (reflection loss < −10 dB) for x or y polarizations at a thickness of 2 mm. This study provides a typical example for future research on wave‐absorbing metamaterials with polarization‐response characteristics.