Dynamic Programming Scheme Research Articles

Scheduling decision model of liner shipping considering emission control areas regulations

Several Emission Control Areas (ECAs) have been established to limit sulfur emissions from ships, thus influencing ship voyage planning. In this study, we used a mixed integer nonlinear programming model (MINLP) to reschedule ports-of-call sequences, ship route, speed, and bunkering strategy to minimize total sailing costs, defined as the sum of bunkering costs, time costs, and carbon emissions costs. An improved genetic algorithm, which introduced many strategies (Multi-layer coding strategy, Pre-search and tabu search strategy, Crossover strategy with a correction method, Mutation strategy based on 2-opt method, Elite strategy and Dynamic programming strategy) is designed to solve the proposed model. The proposed model was then applied to ECAs in North America under different fuel prices. The results show that the proposed method can reduce total sailing costs and help shipping companies effectively cope with fuel price fluctuations.

A Fast Image Template Matching Algorithm Based on Normalized Cross Correlation

Image matching algorithms based on image gray value are commonly used, which can achieve high matching accuracy. Among them, the Normalized Cross Correlation (NCC) method has high accuracy and strong adaptability, however it has the disadvantages of high computational complexity and slow calculation speed. By using dynamic programming strategy, a fast template matching algorithm based on NCC is proposed in this paper. At the same time, by calculating the cumulative sum of pixel values and the sum of squares of pixel values, and using the down sampling process, the calculation time and complexity are greatly reduced. Experimental results show that the proposed method has the same matching accuracy as the existing algorithms, and further improves the image matching efficiency.

Lane’s Algorithm Revisited

In 1964, Kenneth Lane proposed an algorithm to optimize the production schedule of a single-metal, single-processor open pit mine. For this, he proposed a policy based on varying, over time, the so-called “cutoff grade”—or grade threshold used to determine if extracted material should be ore (processed material) or waste (thrown away). Lane’s algorithm had a profound impact on the mining industry. However, though it has been used in multiple commercial software systems and has traditionally been taught to every aspiring mining engineer, it is widely considered a heuristic, and little is known regarding the quality of the solutions it produces. In this paper, we formally study Lane’s problem. We show that Lane’s algorithm can be viewed as an approximate dynamic programming scheme and that Lane’s optimality conditions can be formally derived in two different ways: by considering a variant of the problem where the future value function is linearly approximated or by deriving the optimality conditions of a continuous-time version of the problem. We further show that Lane’s algorithm can naturally be extended to this continuous-time version of the problem and that when this algorithm converges, it converges to an optimal solution. Finally, through a reformulation, we show that Lane’s original problem can be solved using convex mixed-integer programming. Though hypothetical counterexamples can be constructed, computational experiments prove that Lane’s algorithm can produce the optimal solution in every real-world data set tested, thereby lending solid support for its practical application. This paper was accepted by Chung Piaw Teo, optimization.

Mixed-Integer Optimal Design and Energy Management of Hybrid Electric Vehicles With Automated Manual Transmissions

The energy efficiency improvement problems of hybrid electric vehicles (HEVs) with automated manual transmissions are investigated from both the design and control aspects for extended searching space in this work. To optimize the transmission, the gearshift commands and power distribution simultaneously, a combined mixed-integer optimal design and control problem (MIOCP) is formulated. The challenging MIOCP is then reformulated by introducing augmented terms in both objective function and the dynamic model to handle the integer control variables. The reformulated problem is solved with an optimal control software package developed by the authors, the performance of which is validated with dynamic programming and equivalent consumption minimization strategy. Optimal design and mixed-integer control results of the HEV are obtained for different cases, some existing confusions of transmission selection for the HEVs are clarified quantitatively. Results show that transmissions with fewer gear ratios can achieve competitive performances compared with the original one by using the proposed approach.

A comparison between mixed-integer linear programming and dynamic programming with state prediction as novelty for solving unit commitment

Recently, the increasing prevalence of renewable energies has faced the challenge of operating power supply systems to efficiently plan electricity generation on a daily basis, since renewable energies are generated intermittently and the decisions of the individual generation units are discrete. The Unit Commitment (UC) problem, which determines the dispatch of generation units, is one of the critical problems in the operation of power supply systems. A long list of formulation proposals have been made that claim to solve this problem. For this purpose, two established approaches, mixed-integer linear programming (MILP) and backward dynamic programming (DP), are used as basis for a deterministic single-generator unit with general convex cost function in this paper. The DP algorithm is enhanced by a so-called state prediction, which reduces the time to find the optimal solution. The proposed formulation is tested empirically on the basis of existing formulations at long-term profit based UC instance derived from real data. Finally, the calculation results show that the derived approach significantly shortens the computation time, which confirms the effectiveness of state prediction. The comparison of the approaches shows that the DP algorithm with state prediction delivers a satisfying solution in significantly less time than DP and MILP. Furthermore, the given linearity of the dependence of the computation time on number of steps is a superior advantage of the DP strategy. This superiority becomes even more evident when the planning horizon extends over a longer period of time.

Approximate dynamic programming for the military aeromedical evacuation dispatching, preemption-rerouting, and redeployment problem

Military medical planners must consider how aeromedical evacuation (MEDEVAC) assets will be utilized when preparing for and supporting combat operations. This research examines the MEDEVAC dispatching, preemption-rerouting, and redeployment (DPR) problem. The intent of this research is to determine high-quality DPR policies that improve the performance of United States Army MEDEVAC systems and ultimately increase the combat casualty survivability rate. A discounted, infinite-horizon Markov decision process (MDP) model of the MEDEVAC DPR problem is formulated and solved via an approximate dynamic programming (ADP) strategy that utilizes a support vector regression value function approximation scheme within an approximate policy iteration algorithmic framework. The objective is to maximize the expected total discounted reward attained by the system. The applicability of the MDP model is examined via a notional, representative planning scenario based on high-intensity combat operations to defend Azerbaijan against a notional aggressor. Computational experimentation is performed to determine how selected problem features and algorithmic features affect the quality of solutions attained by the ADP-generated DPR policies and to assess the efficacy of the proposed solution methodology. The results from the computational experiments indicate the ADP-generated policies significantly outperform the two benchmark policies considered. Moreover, the results reveal that the average service time of high-precedence, time-sensitive requests decreases when an ADP policy is adopted during high-intensity conflicts. As the rate at which requests enter the MEDEVAC system increases, the performance gap between the ADP policy and the first benchmark policy (i.e., the currently practiced, closest-available dispatching policy) increases substantially. Conversely, as the rate at which requests enter the system decreases, the ADP performance improvement over both benchmark policies decreases, indicating the ADP policy provides little-to-no benefit over a myopic approach (e.g., as utilized in the benchmark policies) when the intensity of a conflict is low. Ultimately, this research informs the development and implementation of future tactics, techniques, and procedures for military MEDEVAC operations.

Integral reinforcement learning based event-triggered control with input saturation

In this paper, a novel integral reinforcement learning (IRL)-based event-triggered adaptive dynamic programming scheme is developed for input-saturated continuous-time nonlinear systems. By using the IRL technique, the learning system does not require the knowledge of the drift dynamics. Then, a single critic neural network is designed to approximate the unknown value function and its learning is not subjected to the requirement of an initial admissible control. In order to reduce computational and communication costs, the event-triggered control law is designed. The triggering threshold is given to guarantee the asymptotic stability of the control system. Two examples are employed in the simulation studies, and the results verify the effectiveness of the developed IRL-based event-triggered control method.

Multiple Lane Detection via Combining Complementary Structural Constraints

Many studies have been conducted on single lane detection, but multi-lane detection is rarely addressed. The latter is more advantageous for applications such as autonomous navigation, unmanned vehicles, departure warning, and cruise control. In this paper, we propose a novel and robust multiple lane detection algorithm based on the road structure information, which contains five complementary constraints: length constraint, parallel constraint, distribution constraint, pair constraint and uniform width constraint. All the five constraints are incorporated into a Hough transform (HT) based unified framework to select lane candidates. Nearly 99% of the false alarm candidates in HT space can be removed. Moreover, a dynamic programming strategy is proposed to find the most rational solutions among the remaining candidates. This strategy can effectively deal with combination complexity and interferences introduced by multi-lane detection. Experimental results on the benchmark dataset and other collected data demonstrate that the proposed method can outperform the state-of-the-art approaches in both accuracy and efficiency.

A comparison study of battery size optimization and an energy management strategy for FCHEVs based on dynamic programming and convex programming

Aiming to address the hydrogen economy and system efficiency of a fuel cell hybrid electric vehicle, this paper proposes comparison research of battery size optimization and an energy management strategy. One approach is based on a bi-loop dynamic programming strategy, which selects the optimal one by initializing the battery parameters in the outer loop and performs energy distribution in the inner loop. The other approach is a framework based on convex programming, which can simultaneously design energy management strategies and optimize battery size. In the dynamic programming algorithm, the influence of the different discrete steps of state variables on the results is analysed, and a discrete step that can guarantee the accuracy of the algorithm and reduce computational time is selected. The results based on the above two algorithms and considering the transient response limitations of the fuel cell are analysed as well. Finally, two driving cycles are chosen to verify and compare the performance of the proposed methodology. Simulation results show that the dynamic programming-based energy management strategy and battery size provide more accurate results, and the transient response of the fuel cell has little effect on the optimization results of the battery size and energy management strategies.

Alignment of Cell Lineage Trees Elucidates Genetic Programs for the Development and Evolution of Cell Types

SummaryA full understanding of the developmental process requires fine-scale characterization of cell divisions and cell types, which are naturally organized as the developmental cell lineage tree (CLT). Technological breakthroughs facilitated determination of more CLTs, but complete comprehension of the data remains difficult without quantitative comparison among CLTs. We hereby quantified phenotypic similarity between CLTs using a novel computational method that exhaustively searches for optimal correspondence between individual cells meanwhile retaining their topological relationships. The revealed CLT similarities allowed us to infer functional similarity at the transcriptome level, identify cell fate transformations, predict functional relationships between mutants, and find evolutionary correspondence between cell types of different species. By allowing quantitative comparison between CLTs, our work is expected to greatly enhance the interpretability of relevant data and help answer the myriad of questions surrounding the developmental process.

Resource Allocation in One-dimensional Distributed Service Networks with Applications

We consider assignment policies that allocate resources to users, where both resources and users are located on a one-dimensional line [0,∞). First, we consider unidirectional assignment policies that allocate resources only to users located to their left. We propose the Move to Right (MTR) policy, which scans from left to right assigning nearest rightmost available resource to a user, and contrast it to the Unidirectional Gale–Shapley (UGS) matching policy. While both policies among all unidirectional policies, minimize the expected distance traveled by a request (request distance), MTR is fairer. Moreover, we show that when user and resource locations are modeled by statistical point processes, and resources are allowed to satisfy more than one user, the spatial system under unidirectional policies can be mapped into bulk service queueing systems, thus allowing the application of many queueing theory results that yield closed form expressions. As we consider a case where different resources can satisfy different numbers of users, we also generate new results for bulk service queues. We also consider bidirectional policies where there are no directional restrictions on resource allocation and develop an algorithm for computing the optimal assignment which is more efficient than known algorithms in the literature when there are more resources than users. Numerical evaluation of performance of unidirectional and bidirectional allocation schemes yields design guidelines beneficial for resource placement. Finally, we present a heuristic algorithm, which applies the optimal dynamic programming scheme for one-dimensional inputs to obtain approximate solutions to the optimal assignment problem for the two-dimensional scenario and empirically yields request distances within a constant factor of the optimal solution.

A study of the differences among representative investment strategies

This study compares the differences and efficiencies of investment strategies among anticipative and adaptive models using three representative decision approaches: the static approach (SA), semidynamic strategy (or re-assess by static approach, Re-SA), and dynamic programming (DP). We show that each approach has individual merits and weaknesses. A DP strategy may allow for relatively aggressive decisions because of opportunities to adapt the decisions later. However, that strategy may result in a serious downside risk. The suboptimal adaptive strategy, Re-SA, acts as a good proxy for the DP strategy. Therefore, both SA and Re-SA are important tools for addressing asset allocation problems.

An Adaptive Dynamic Programming Scheme for Nonlinear Optimal Control With Unknown Dynamics and Its Application to Turbofan Engines

In this article, a novel adaptive dynamic programming (ADP) approach is proposed for the optimal control problem of nonlinear continuous control systems with unknown dynamics. First, an alternating iteration algorithm based on Hamilton-Jacobi-Bellman equation is proposed for the optimal control of known nonlinear control systems. Then, the convergence results of the alternating iteration algorithm are obtained by using mathematical induction and monotone bounded convergence theorem. Moreover, the global asymptotic stability of the nonlinear closed-loop system is proved. Second, based on the scheme of alternating iteration algorithm, an ADP algorithm for the optimal control problem with unknown nonlinear dynamic model is developed by using the basis function approximation method and Newton-Leibniz formula, which can update the control strategy online by utilizing input and output information of the system. In addition, the convergence analysis of the proposed ADP algorithm is derived. Finally, the feasibility of the established results is verified by two examples, and the ADP method is applied to the optimal tracking fuel control problem of turbofan engines.

A cascaded energy management optimization method of multimode power-split hybrid electric vehicles

A novel multimode power-split hybrid electric vehicle demonstrates significant advantages in energy conservation. The complicated structure and multiple operation modes, however, have brought significant challenges to energy management. The problems of operation mode selection, power allocation, and operating point selection need to be solved simultaneously. Traditionally, existing energy management strategies first have determined operation mode and then have solved power allocation and operating points. Moreover, traditional dynamic programming usually has not considered electric energy consumption and has deviated from the optimal solution. To solve these problems, a cascaded energy management strategy that integrates dynamic programming and equivalent consumption minimization strategy (DP + ECMS) is proposed. An iterative method is used to optimize the equivalence factor. A vehicle model based on actual control strategy is established and validated by test results. Then fuel economy simulations are conducted. Results showed that DP + ECMS could achieve 19.9% fuel economy improvement compared with the rule-based strategy for a new European driving cycle. In addition, simulation results of a Worldwide Harmonized Light-Duty Test Cycle demonstrated that DP + ECMS performed well in real road driving conditions. This study introduces a novel multimode power-split hybrid system and provides a global energy management optimization method.

Simultaneous Identification and Control Using Active Signal Injection for Series Hybrid Electric Vehicles Based on Dynamic Programming

Hybrid electric vehicles (HEVs) have an over-actuated system by including two power sources, a battery pack and an internal combustion engine. This feature of HEV is exploited in this paper to simultaneously achieve accurate identification of battery parameters/states. By actively injecting current signals, state of charge, state of health, and other battery parameters can be estimated in a specific sequence to improve the identification performance when compared to the case where all parameters and states are estimated concurrently using the baseline current signals. A dynamic programming strategy is developed to provide the benchmark results about how to balance the conflicting objectives corresponding to identification and system efficiency. The tradeoff between different objectives is presented to optimize the current profile so that the richness of signal can be ensured and the fuel economy can be optimized. In addition, simulation results show that the Root-Mean-Square error of the estimation can be decreased by up to 100% at a cost of less than 2% increase in fuel consumption. With the proposed simultaneous identification and control algorithm, the parameters/states of the battery can be monitored to ensure safe and efficient application of the battery for HEVs.

Approximate Optimal Sliding Mode Tracking Control for Modular Reconfigurable Robots Based on Critic-only Structure

Abstract In this paper, an approximate optimal sliding mode tracking control (SMTC) strategy is investigated for modular reconfigurable robots (MRRs) through critic-only structure-based adaptive dynamic programming (ADP) scheme. The SMTC is achieved by three parts, i.e., the optimal control of the nominal system, the sliding mode-based iterative control and an adaptive robust term. The sliding mode-based iterative controller suppresses the error caused by the trajectory tracking, and the adaptive robust term is employed to ensure the reachable condition of sliding mode surface. By solving the Hamilton-Jacobi-Bellman equation with the critic neural network only, the sliding mode-based iterative control can be derived. The SMTC strategy can drive the MRR to present achieve a faster control action based on the approximate optimal control. The closed-loop MRR system is guaranteed to be asymptotically stable under the developed SMTC policy. At last, the effectiveness of the presented strategy was validated via the comparative simulation.

Economic Nonlinear Predictive Control for Real-Time Optimal Energy Management of Parallel Hybrid Electric Vehicles

This article presents an economic nonlinear hybrid model predictive control strategy for optimal energy management of parallel hybrid electric vehicles. Hybrid electric vehicles are controlled for operation in various driveline modes and the associated optimal control problem involves both continuous and discrete control variables. To solve the resultant mixed-integer nonlinear optimal control problem, we propose a hierarchical supervisory control architecture that consists of demand prediction, driveline mode determination, and real-time optimization. These three modules are designed independently and connected in series to perform computer-aided control. The demand prediction module uses a times series model to forecast the mechanical traction power requests of the driver over a prediction horizon based on vehicle speed, road grade, acceleration pedal scale, brake pedal scale, and past and current power demands. For a given forecasted power demand profile, the mode determination module decides a sequence of driveline modes that are presumed to be operated over the prediction horizon. The model-based real-time optimization corresponding to nonlinear model predictive control computes the optimal motor power over a prediction horizon, and the receding horizon scheme as feedback control is applied to repeat the processes of the three control modules. A dedicated case study with real driving data obtained from Hyundai IONIQ PHEV 2018 is presented to demonstrate the effectiveness in fuel economy and emission reduction offered by the proposed optimal energy management strategy. The proposed hierarchical real-time predictive optimization-based strategy is competitive with any exiting power management strategies such as dynamic programming and equivalent consumption minimization strategy in fuel economy and emission reduction while showing better charge-sustaining capability. This trade-off between fuel economy and charge-sustainability can be further improved by tuning the hyper-parameters in the proposed optimal control problem.

Efficient energy harvesting scheme with power optimization strategies over cognitive radio networks

A wide growth of wireless communication and it's associated technologies innovates many innovative and robust systems. These kinds of innovations over wireless technologies highly depend on energy efficiency and power saving procedures over communication scenarios. These wirless technologies update its nature day-by-day based on the user needs, so a dynamic wireless communication medium is essential to tolerate and resolve this kind of robust needs. Cognitive Radio Networks, an intelligent dynamic programming scheme as well as which has powerful network configurations to choose the efficient wireless network channel for proceeding the communications without any interruptions. This kind of dynamicity and robustness requires more energy to manipulate the sensor node and routing details as well as their presence in every time to check the node availability for further communications. In this paper, we concentrate more on energy harvesting issues and power optimization procedures occurred over Cognitive Radio Networks. In this proposed system, the energy efficiency is preserved and power optimization over CR Networks is handled by means of three intelligent factors/strategies such as: (i) Efficient Route Reporter (ERR), (ii) Dynamic Channel Optimizer (DCO) and (iii) Power Efficiency Handler (PEH). These three strategies are collectively called as Dynamic Optimization Law over CR Networks (DOL-CRNs). By using these DOL-CRNs, we can easily manipulate the issues and provide best resolution to optimize the CR Networks to enhance the network lifetime, throughput, reduce the delay over CR Networks communication and maximize the frame success ratio. For all the proposed DOL-CRNs assures the efficiency of Cognitive Radio Networks as well as its communication robustness with the help of three associated strategies.

A Novel way out to Unit Commitment Problem utilizing Evolutionary Particle Swarm Optimization

This paper differentiates the shows of three Unit Commitment strategies, three of which are the essential arrangement techniques for taking care of the Unit Commitment Problem named Priority List and Dynamic Programming strategies. The third strategy is the Evolutionary Particle Swarm Optimization method which has been applied productively to a plentiful blend of streamlining issues. Various regions in control frameworks require understanding at least one nonlinear streamlining emergencies. In spite of the way that systematic techniques may experience moderate intermingling and the scourge of dimensionality, heuristics-based swarm knowledge can be a capable substitute. Evolutionary Particle Swarm Optimization (EPSO), some portion of the swarm insight family, is known to adequately take care of enormous scale nonlinear improvement issues. This paper introduces the exit plan for Unit Commitment Problem by methods for EPSO system. A calculation was created to achieve an exit plan to the Unit Commitment Problem utilizing EPSO procedure. The adequacy of the calculation is tried on three generating units and the cultivated results utilizing the three techniques are thought about for complete working expense.

An adaptive dynamic programming method for torque ripple minimization of PMSM

<p style='text-indent:20px;'>The imperfect sinusoidal flux distribution, cogging torque, and current measurement errors can cause periodic torque ripple in the permanent magnet synchronous motor (PMSM). These ripples are reflected in the periodic oscillation of the motor speed and torque, causing vibration at low speeds and noise at high speeds. As a high-precision tracking application, ripple degrades the application performance of PMSM. In this paper, an adaptive dynamic programming (ADP) scheme is proposed to reduce the periodic torque ripples. An optimal controller is designed by iterative control algorithm using robust adaptive dynamic programming theory and strategic iterative technique. ADP is combined with the existing Proportional-Integral (PI) current controller and generates compensated reference current iteratively from cycle to cycle so as to minimize the mean square torque error. As a result, an optimization problem is constructed and an optimal controller is obtained. The simulation results show that the robust adaptive dynamic programming achieves lower torque ripple and shorter dynamic adjustment time during steady-state operation, thus meeting the requirements of steady speed state and the dynamic performance of the regulation system.