Modified Cost Function Research Articles

UAV p­­­arcel Delivery System with Deep Reinforcement Learning Based Collision Avoidance and Route Optimization

Unmanned Aerial Vehicles (UAVs), or drones, have recently become a favorable solution for fast parcel delivery due to their maneuverability and advances in navigation technologies. With the limitation of battery capacity and payload of drones, it is crucial to consider both efficiency and cost while conducting the tasks. Meanwhile, UAVs should not collide with each other while traveling to customers. In this article, we propose a UAV parcel delivery system involving deep reinforcement learning (DRL) approach for collision avoidance and a genetic algorithm for route optimization. Specifically, a delivery center generates near-optimal routes, loading UAV with parcels according to demands. Each UAV takes charge of delivering packages in compliance with the assigned route while avoiding collision with each other. We utilize DRL to achieve collision avoidance without having prior knowledge about the trajectories of other UAVs. Additionally, we adopt a genetic algorithm to obtain the lowest energy cost path for each UAV. To find such an optimized path, we solve a capacitated vehicle routing problem (CVRP) with a modified cost function and extra constraints. Realistic simulations using a physics engine and software-in-the-loop (SITL) are conducted to evaluate the feasibility of the proposed methods.

Dynamic event-triggered optimal control for stochastic interconnected nonlinear systems with matched disturbances via adaptive dynamic programming

This paper presents a dynamic event-triggered adaptive dynamic programming (DETADP) algorithm to study the optimal control issue of stochastic interconnected nonlinear systems with matched disturbances. By constructing an improved cost function related to the actuator faults for each auxiliary subsystem, the original control issue is converted into finding an array of decentralized optimal control policies. In order to eliminate the effect of matching perturbations, an integral sliding mode control strategy is developed using the DETADP algorithm to ensure optimal performance for general nonlinear dynamics. When the system trajectories converge to a sliding-mode surface, equivalent sliding mode dynamics are transformed into reformulated auxiliary systems with a modified cost function. Then, a DETADP algorithm is introduced under an identifier-critic network framework, where the identifier aims to determine the stochastic dynamic, the critic aims to acquire the solutions of Hamilton–Jacobi–Bellman(HJB) equations. Furthermore, a novel dynamic event-triggering condition is designed to determine the occurrence of an event by introducing a dynamic variable. By using the Lyapunov stability theory, all signals in the closed-loop system are proved to be uniformly ultimately bounded. Finally, practical simulation examples are conducted to validate the effectiveness of the designed method.

A Simple Multistep Solution for Model Predictive Control in Multiphase Electric Drives

The multiphase electric drive field of research is rather young, and most of the efforts in this area have been focused on appropriately extending different techniques that were already popular in three-phase systems. This is the case of finite-control set model predictive control (FCS-MPC), which has been a hot topic since 2009. Despite the numerous contributions on FCS-MPC for multiphase systems, all of them assume a single-step horizon in the predictions. This paper shows the first attempt to include multistep predictions within the FCS-MPC that performs the current regulation. The extension from the three-phase case is not direct at all because the number of voltage vectors and subspaces is higher, this adding further complexity and eventually leading to issues related to the secondary current regulation. A smart selection of different subsets of voltage vectors is used to ease the implementation, and a modified cost function is developed to jeopardize the high instantaneous voltage values that significantly worsens the current quality at steady state. Experimental results confirm that the use of multistep FCS-MPC is feasible in multiphase electric drives, showing at the same time that some measures must be taken to minimize the distortion in the low-impedance secondary subspaces.

Event-Based Robust Optimal Consensus Control for Nonlinear Multiagent System With Local Adaptive Dynamic Programming.

This article investigates the robust optimal consensus for nonlinear multiagent systems (MASs) through the local adaptive dynamic programming (ADP) approach and the event-triggered control method. Due to the nonlinearities in dynamics, the first part defines a novel measurement error to construct a distributed integral sliding-mode controller, and the consensus errors can approximately converge to the origin in a fixed time. Then, a modified cost function with augmented control is proposed to deal with the unmatched disturbances for the event-based optimal consensus controller. Specifically, a single network local ADP structure with novel concurrent learning is presented to approximate the optimal consensus policies, which guarantees the robustness of the MASs and the uniform ultimate boundedness (UUB) of the neural network (NN) weights' estimation error and relaxes the requirement of initial admissible control. Finally, an illustrative simulation verifies the effectiveness of the method.

Optimal diagnosis interval for online quality control methods

Online quality control methods emphasize manufacturing processes to attain maximum conformance with respect to the specifications of the concerned quality characteristics of a product. One key factor that affects the effectiveness of these methods is the diagnosis interval. In this paper, the existing cost model along with its cost components for online quality control methods has been revisited and modified by incorporating new variables like the rate of production, the loss due to false alarm, the loss due to non-detection of process abnormalities, and considering a workable break-up of diagnosis cost for finding the optimal diagnosis interval from the perspective of present-day manufacturing engineering. As already mentioned, the proposed cost model has not ignored the loss due to the generation of defective items as well as the adjustment cost available in the pertinent literature. The modified cost function thus proposed has been appropriately minimized to obtain the corresponding optimal diagnosis interval. The proposed methodology has been compared numerically with other methodologies to establish its effectiveness. The cornerstone of the proposed methodology lies in reinforcing its effectiveness through a real-life case example in manufacturing. Sensitivity analysis has also been carried out for the real-life case example to fortify the proposed methodology.

Dynamic Performance Enhancement of a Triple Active Bridge With Power Decoupling-Based Configurable Model Predictive Control

The multiport active bridge (MAB) converter has been recently proposed aiming to increase the power density and the availability of electrical power distribution systems. However, the dynamic performance of proportional-integral (PI) controller for MAB voltage control is characterized by a relatively slow response and large overshoot. To address this issue, a power decoupling based configurable control (PDC-MPC) strategy inspired by the model predictive control (MPC) was developed. The proposed control strategy can achieve good transient performance and high control flexibility with good precision to comply to DC voltage regulations. In this paper, the PDC-MPC was investigated and implemented in a triple active bridge (TAB) converter with multi-winding high frequency transformers. The operating principle of the PDC-MPC is divided into two phases: prediction of the DC current through a binary search and decoupling of the desired power of each isolated virtual branch to its phase shift angles under the single phase shift (SPS) modulation strategy. Steady-state and dynamic performances of the proposed PDC-MPC for TAB converters were analysed using modified cost function and predictive models. Simulation and experiments were conducted to validate the theoretical analyses.

Event-triggered fault-tolerant control for input-constrained nonlinear systems with mismatched disturbances via adaptive dynamic programming

In this paper, the issue of event-triggered optimal fault-tolerant control is investigated for input-constrained nonlinear systems with mismatched disturbances. To eliminate the effect of abrupt faults and ensure the optimal performance of general nonlinear dynamics, an adaptive dynamic programming (ADP) algorithm is employed to develop a sliding mode fault-tolerant control strategy. When the system trajectories converge to the sliding-mode surface, the equivalent sliding mode dynamics is transformed into a reformulated auxiliary system with a modified cost function. Then, a single critic neural network (NN) is adopted to solve the modified Hamilton–Jacobi–Bellman (HJB) equation. In order to overcome the difficulty that arises from the persistence of excitation (PE) condition, the experience replay technique is utilized to update the critic weights. In this study, a novel control method is proposed, which can effectively eliminate the effects of abrupt faults while achieving optimal control with the minimum cost under a single network architecture. Furthermore, the closed-loop nonlinear system is proved to be uniformly ultimate boundedness based on Lyapunov stability theory. Finally, three examples are presented to verify the validity of the control strategy.

Mars powered descent phase guidance law based on reinforcement learning for collision avoidance

SummaryThis paper proposes a reinforcement learning‐based guidance law for Mars powered descent phase, which is an effective online calculation method that handles the nonlinearity caused by the mass variation and avoids collisions. The reinforcement learning method is designed to solve the constrained nonlinear optimization problem by using a critic neural network. Specifically, to cope with the position constraint (i.e., glide‐slope constraint) and the thrust force limit constraint, a modified cost function is proposed, and the associated Hamilton‐Jacobi‐Bellman equation is solved online without using an actor neural network, which significantly reduces the computational burden. The convergence of the critic neural network is proven. Simulation results show the effectiveness of the proposed method.

Optimal backstepping controller for trajectory tracking of a quadrotor UAV using ant colony optimisation algorithm

This paper concerns the design of an optimal stabiliser/tracker for the quadrotor UAV. Firstly, kinematic and dynamic equations of the aerial vehicle are yielded through Newton-Euler formalism. Then, the backstepping (BS) technique is adopted to sustain a controllable and stable behaviours of the quadrotor attitude. However, designing a high-precision flight controller essentially necessitates an optimal selection of its internal parameters. The improper configuration of these parameters can negatively influence the flight performances and sometimes leads to the instability of the quadrotor system. Ant colony optimisation algorithm (ACO) is used to configure the adopted stabilised technique, and a modified cost function is proposed to improve the response of the quadrotor system. In view to corroborate the accuracy of the suggested method, a series of numerical experiments were done and comparison with other control approaches is also given.

차량 능동 소음 제어 시스템에서 제어 위치 선택에 따른 소음 저감 성능에 관한 예비 평가

This study uses a modified cost function with weighted error signal channels to demonstrate that the vehicle active noise control system can focus control efforts on one specific seat. In a conventional active noise control system for road noise suppression inside a passenger car, the control algorithm targets the attenuation of all seats, hence affecting the maximum performance that can be obtained when applied to a single seat. To address this limitation, we employed weighting factors in the error signal to control the selected seat positions only. A practical active noise control system was implemented in a large sedan to investigate the effect of the weighting factor on the noise attenuation performance. Experimental results showed that the error weighting factor strategy increased attenuation of the target seats by up to 1.15 dB more than in the case of the entire seat control.

A Novel Generalized Predictive Control Strategy With Improved Time Domain Performance

Abstract Due to the ability of handling the coupling property of multivariable control system, generalized predictive control (GPC) has been widely employed in process industry. However, the existing GPC method normally utilizes the frequency domain indices in the design procedure, which is not intuitive to the industrial end users. Therefore, a new GPC method with improved time domain performance is proposed for multiple-input-multiple-output system. First, the slope and rate of change of the slope of the system outputs are considered as two new weighting terms in the cost function of GPC such that the dynamic characteristics of the system can be better considered in the optimal control problem of the controller. Second, given the modified cost function, the resultant GPC algorithm is developed to improve the time domain performance of the system. Third, the convergence and stability of the closed-loop system under the proposed method are theoretically proved. Finally, the experimental results based on the variable air volume (VAV) air conditioning system show that the time domain performance indices, including overshoot and settling time, of the proposed method, have been improved by 68.75% and 66.67%, compared with the traditional GPC, showing the advantages of the new GPC method.

3상4선식 컨버터의 독립전력제어 기반 능동수명제어 알고리즘

Along with using renewable energy sources such as photovoltaic (PV), wind, and biomass, the power system structure has evolved and grown more complex in recent years. As a crucial part of power distribution, the three-phase converter plays a significant role; nevertheless, this does not account for the fact that various phase legs may experience different aging conditions. This study offers an active thermal control (ATC) concept for a two-level, three-phase, four-wire converter with balance circuit in order to delay the failure of the aged leg and the entire converter. The first ATC approach using the model predictive control (MPC) strategy is investigated to regulate the output current and balance the capacitor voltage. A modified cost function is proposed to improve the capacitor voltages balancing control capability. Additionally, the second ATC approach using a carrier-based pulse-width-modulation (CBPWM) method based per-phase voltage and current control is introduced. The achieved simulation and experiment results of both CBPWM and MPC schemes are presented to verify the performance of the ATC based per-phase power control for the two-level three-phase four-wire converter with balance circuit.

Integrated Simulation-Optimization Framework for Assessing the Impact of I-66 Dynamic Toll Pricing on Pavement Deterioration and Maintenance Decisions

Tolled facilities are undoubtedly expected to alter the distribution of traffic across the transportation network. On the other hand, traffic volumes and loading have an impact on deteriorating pavement conditions. These traffic volumes are considered by Departments of Transportation (DOTs) while allocating annual budgets to maintain and rehabilitate roadway segments to sustain pavement performance targets. This research studies the specific site around I-66 inside the Beltway, which newly applied dynamic tolls during a.m. and p.m. peak hours. An integrated traffic-management/pavement-treatment framework was applied to address both the operational and the pavement performance of the network. Aimsun hybrid macro/meso dynamic user equilibrium experiments were used to simulate the network with the modified cost function taking care of the dynamic pricing along the I-66 tolled facility. An optimization was Python-coded into a Pyomo framework to specify the optimal maintenance and rehabilitation treatment plan, taking into account critical condition index (CCI) deterioration based on the traffic load distribution on the network. Finally, the results of the simulation showed the importance of having an optimized treatment schedule to achieve optimal pavement performance outcomes, with a difference in CCI index that could range all the way from 68 to 95.

Intensity-guided depth image estimation in long-range lidar

Long-range lidar systems usually record large but extremely sparse data cubes. It is a great challenge to estimate accurate depth images of the photon-starved regime with fewer memory requirements and lower computational complexity. An intensity-guided method is introduced to estimate the depth image by using temporal-spatial correlation of the reflected signals. Multi-scale superpixels and fast time-domain windows are established in the preprocessing step, leading to smaller data cubes with reduced empty and noisy pixels. To strike a balance between smoothness and preserving sharp edges, the fast-converging alternating direction method of multipliers (ADMM) is used in the modified cost function to estimate depth images. Experimental results show that the proposed method yields better depth estimates than other state-of-the-art methods, especially for long-rang targets of field experiments with a low signal return level of ∼1 photon per pixel.

Neural Predictor-Based Low Switching Frequency FCS-MPC for MMC With Online Weighting Factors Tuning

In this article, we propose a novel predictive control framework for modular multilevel converter, which makes use of neural predictor-based low switching frequency finite control-set model predictive control methodology with respect to online weighting factors tuning subject to robustness characteristics. The main objectives of this article are to maintain a low switching frequency operation and to enhance the system robustness. First, a modified cost function together with an online weighting factors auto-tuning technique, which has good potential to fulfill the low frequency operation, is presented under different operating conditions. Next, we provide a framework to explicitly address parameter mismatches and unknown nonlinear system dynamics in this proposal by introducing the neural network predictor control technique. The novelty of contribution of this article lies not only in incorporating the proposed cost function with a predictor-based neural network design solution for satisfying the requirement of the low switching frequency and robustness but also in the utilization of the auto-tuning technique for online determining the suitable weighting factors. Finally, we demonstrate the applicability of the proposed predictive control framework in comparison with the state-of-the-art finite control-set model predictive control approaches under various operating conditions.

Dynamic event-triggered distributed guaranteed cost FTC scheme for nonlinear interconnected systems via ADP approach

A novel dynamic event-triggered distributed fault tolerant control (FTC) scheme with guaranteed cost is proposed for uncertain nonlinear interconnected systems suffering from time-varying actuator faults. The event-triggered FTC problem of uncertain interconnected system is first converted to a series of equivalent event-triggered optimal control designs of isolated fault-free auxiliary subsystems by defining a modified cost function for each auxiliary subsystem. The equivalence of problem transformation is demonstrated meanwhile the upper bound of cost function of overall interconnected system is analyzed. Subsequently, for the isolated auxiliary subsystem, a dynamic event-triggered adaptive dynamic programming (DET-ADP) algorithm is proposed to obtain the optimal control policy, which can remove the restriction of initial admissible control, eliminate the actor network, and save more communication resources. Furthermore, we prove that the closed-loop auxiliary subsystems are ensured to be uniform ultimate bounded based on Lyapunov approach. Finally, simulation results are presented to validate the developed dynamic event-triggered distributed guaranteed cost FTC scheme.

Leader-Following Cluster Consensus as a Graphical Differential Game With a Nash Equilibrium Solution

This paper formulates the leader-following cluster consensus problem of a homogeneous linear time-invariant multi-agent system over a directed graph as an N-player graphical differential game. The agents utilize distributed state feedback control laws based on the available information to calculate their best response. In general, the absence of centralized information prevents the best responses of agents from constituting a Nash equilibrium solution. We show that the best responses depending on the unique solution of an algebraic Riccati equation constitute a Nash equilibrium for the proposed graphical differential game with a modified cost function. A numerical example illustrates the Nash equilibrium solution by showing that the agents can not improve their performance by altering their responses while solving the leader-following cluster consensus problem.

Optimal Energy Management Solutions Using Artificial Intelligence Techniques for Photovoltaic Empowered Water Desalination Plants Under Cost Function Uncertainties

Two modern methods of the energy management system (EMS) based on a modified cost function are addressed in this paper. Fuzzy logic (FL) and Harris Hawks Optimization (HHO) is implemented to achieve the optimal performance of seawater desalination plants (SWDP) within the minimum feed-in tariff (FiT). The technical difficulties involved in the variation of energy price from one time to another and the system parameters uncertainties. For example, the price of energy is higher at the peak time and the price is lower at normal times. Also, the peak time can change from one day to another day. The proposed management system can deal with these variations and uncertainty cases. The suggested EMS is achieved through a bidirectional electrical energy interchange approach (π-EEIA). The main concept behind the proposed π-EEIA is how and when to inject the excess generated energy of renewable energy into the utility grid or charge the battery depending on the minimum dynamic cost criterion and vice versa. To accomplish this study a 700 m <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">3</sup> /day SWDP located in Egypt fed on solar energy and a utility network has been constructed and analyzed. The system includes SWDP fed from a photovoltaic (PV) array as well as the utility grid in addition to a battery energy storage system (BESS). The main objective of this study is the management and coordination between the energy exchange process from the solar energy, the utility network, and BESS to provide sufficient electrical energy for SWDP within the minimum FiT. The system is constructed and validated using the MATLAB/SIMULINK <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">TM</sup> software package. The proposed FL and HHO-based EMSs are investigated in the presence of the system uncertainties such as the change in the energy (excess or shortage) as well as the change in the energy price in the utility network (high or low) with (normal or peak) time. The attained results demonstrate that the proposed FL and HHO-based EMSs provide high dynamic performance and accurate coordination between various energy resources and BESS. The results show that FL-based EMS achieves a profit of 10.28 $ but the HHO-based EMS achieves a profit of 10.11 $ in the same period.

A New Integral Critic Learning for Optimal Tracking Control with Applications to Boiler‐Turbine Systems

ABSTRACTOptimal control theory and reinforcement learning are gradually being used in the field of industrial control. In this article, a new optimal tracking control scheme for 160 MW boiler‐turbine systems is proposed based on an online policy iteration integral reinforcement learning (IRL) method. Firstly, a self‐learning state tracking control with a cost function is developed to deal with the optimal tracking control problems for the boiler‐turbine nonlinear system. Then with a modified cost function, a policy iteration‐based IRL method is introduced to obtain the optimal control law. Finally, the monotonicity and the convergence of the cost function is analyzed and the detailed implementation of the policy iteration‐based IRL method is provided via neural networks. The simulation results show that the control of the boiler‐turbine system can converge in a short time by using this online iterative method. Through a theoretical simulation case, it can be concluded that the proposed method is more advanced compared with the MPC method.

Bézier Curves-Based Optimal Trajectory Design for Multirotor UAVs with Any-Angle Pathfinding Algorithms.

Multirotor Unmanned Aerial Vehicles (UAVs) play an imperative role in many real-world applications in a variety of scenarios characterized by a high density of obstacles with different heights. Due to the complicated operation areas of UAVs and complex constraints associated with the assigned mission, there should be a suitable path to fly. Therefore, the most relevant challenge is how to plan a flyable path for a UAV without collisions with obstacles. This paper demonstrates how a flyable and continuous trajectory was constructed by using any-angle pathfinding algorithms, which are Basic Theta*, Lazy Theta*, and Phi* algorithms for a multirotor UAV in a cluttered environment. The three algorithms were modified by adopting a modified cost function during their implementation that considers the elevation of nodes. First, suitable paths are generated by using a modified version of the three algorithms. After that, four Bézier curves-based approaches are proposed to smooth the generated paths to be converted to flyable paths (trajectories). To determine the most suitable approach, particularly when searching for an optimal and collision-free trajectory design, an innovative evaluation process is proposed and applied in a variety of different size environments. The evaluation process results show high success rates of the four approaches; however, the approach with the highest success rate is adopted. Finally, based on the results of the evaluation process, a novel algorithm is proposed to increase the efficiency of the selected approach to the optimality in the construction process of the trajectory.