Dynamic Programming Algorithm Research Articles

Directed capacity-preserving subgraphs: hardness and exact polynomial algorithms

We introduce and discuss the Minimum Capacity-Preserving Subgraph (MCPS) problem: given a directed graph with edge capacities cap and a retention ratio α∈(0,1), find the smallest subgraph that, for each pair of vertices (u, v), preserves at least a fraction α of a maximum u-v-flow’s value. This problem originates from the practical setting of reducing the power consumption in a computer network: it models turning off as many links as possible, while retaining the ability to transmit at least α times the traffic compared to the original network. First we prove that MCPS is NP-hard already on a restricted set of directed acyclic graphs (DAGs) with unit edge capacities. Our reduction also shows that a closely related problem (which only considers the arguably most complicated core of the problem in the objective function) is NP-hard to approximate within a sublogarithmic factor already on DAGs. In terms of positive results, we present two algorithms that solve MCPS optimally on directed series-parallel graphs (DSPs): a simple linear-time algorithm for the special case of unit edge capacities and a cubic-time dynamic programming algorithm for the general case of non-uniform edge capacities. Further, we introduce the family of laminar series-parallel graphs (LSPs), a generalization of DSPs that also includes cyclic and very dense graphs. Their properties allow us to solve MCPS on LSPs by employing our DSP-algorithms as subroutines. In addition, we give a separate quadratic-time algorithm for MCPS on LSPs with unit edge capacities that also yields straightforward quadratic time algorithms for several related problems such as Minimum Equivalent Digraph and Directed Hamiltonian Cycle on LSPs.

Research on ship trajectory compression based on a Dynamic Programming algorithm

The Automatic Identification System (AIS) is extensively used in monitoring vessel traffic, and ship navigation related information can be obtained from the AIS data. However, AIS data contain extensive redundant information, which leads to the general need to compress the data when applying it in practice or conducting research. In this paper, a three-dimensional compression of ship trajectories using the Dynamic Programming algorithm has been proposed. The AIS data near the ports of Long Beach and San Francisco in the United States were used to test and compare the Dynamic Programming algorithm with the Top-down Time-ratio algorithms. The experimental results show that the proposed algorithm can better retain the position and time information at low compression ratio such as 1%, 20% and 40%. Moreover, the algorithm is applicable to ship trajectories with different motion modes such as steering, mooring and straight ahead. The results show that the proposed algorithm can reasonably solve the problem of AIS data redundancy and ensure the quality of data, which is of practical significance for water transport, transport planning and other related research.

Nonzero-sum game-based decentralized approximate optimal control of modular robot manipulators with coordinate operation tasks using value iteration

Abstract Accurate trajectory tracking and appropriate contact force are crucial for the coordinated operation-oriented control of modular robot manipulators (MRMs). Considering the practical need for precision in system control, resource optimization, and disturbance compensation within the context of the coordinated operation tasks (COTs) of MRMs, this paper employs a value iteration (VI) technique to devise a decentralized approximate optimal control strategy grounded in nonzero-sum game (NZSG) theory. To obtain more accurate, reliable, and safe control, a dynamic model of the MRM is established using joint torque feedback technology; then, the problem of optimal control for MRM systems focused on coordinated operation-oriented control is reformulated as an NZSG involving multiple subsystems. The present study, grounded in the theoretical framework of the adaptive dynamic programming (ADP) algorithm, employs an event-triggered NZSG strategy, utilizing VI to resolve the coupled Hamilton–Jacobian equations, culminating in the derivation of the Nash equilibrium solutions. Through stringent stability analysis, it is established that the trajectory tracking error for the closed-loop MRM system engaged in COTs is uniformly ultimately bounded. The proposed method’s efficacy is subsequently corroborated through experimental validation.

Application of Video Game Algorithm Based on Deep Q-Network Learning in Music Rhythm Teaching

The difficulty of music rhythm teaching makes its teaching effect limited. Some game algorithms show problems of stuttery and unclear picture when they are introduced into teaching, which greatly restricts the development of teaching content. Therefore, it is proposed to activate the deep Q network and improve the filter to improve the processing performance of game image data and the integrity of information retention. The model fuses the extracted features and processes them through the neural network. According to the performance characteristics and forms of music rhythm, constant Q transformation, dynamic programming algorithm and hidden Markov model are proposed to analyze and design music signal, beat point analysis and signal traveling speed, so as to realize the identification and matching of music rhythm. The performance test and application analysis of the proposed fusion improvement algorithm are carried out: the video game algorithm can effectively extract data feature information, and its accuracy rate is basically 90% or above. The improved Deep Q-network (DQN) and Artificial Fish Swarms Algorithm (AFSA) algorithms can maintain an accuracy of over 75%, followed by the ABC algorithm, which maintains an accuracy of over 70%, but exhibits significant fluctuations in the curve changes and nodes. The recognition accuracy of music signal feature extraction is over 70%, and it is less affected by signal-to-noise ratio. The improved activation function showed that the highest game score exceeded 10 points, while the scores of Rectified Linear Unit (ReLU) function and Hyperbolic tangent function (Tanh) function were basically hovering below 5 points, and the overall score curve fluctuated significantly. The recognition accuracy of music signal features extraction is above 70% and is less affected by Signal to Noise Ratio. The algorithm has a good effect of music rhythm teaching, the students' interest in classroom learning is more than 45%, and the learning weakness is less. The average score of students in music style judgment, rhythm division and index is 85 points and 90 points respectively, which is much higher than the 40 points and 63 points of traditional teaching. The proposed algorithm can better combine video games with music teaching and help students improve their learning effect under the gamification teaching method, and has good application value

Floor space optimization of Molds in Boat Manufacturing using Dynamic programming and optimization algorithm

Floor space management in manufacturing of boats requires to be optimized as much as possible and particularly with reference to molds, which are large fixed assets. This article uses dynamic programming to solve the problem of floor space allocation, with particular emphasis on the minimization of unused space and the maximization of mold placement. By specifying the problem as a resource allocation problem, the article naturally considers spatial configurations and operational conditions such as mold size, production timeline, and material transportation. The dynamic programming approach is used to provide a scalable and cost-efficient solution as it is a decomposition of the problem into sub-problems and the global optimum is achieved for layouts. The analysis shows a potential of achieving a high space utilization, the elimination of operational constraints and an improvement of flow rates leading to the formulation of smart manufacturing solutions in the boat industry. This article explains about the analysis of Optimizing Floor Space in Boat Manufacturing, Formulation of the Mold Arrangement Challenge, Key Constraints and Variables in Space Optimization, Dynamic Programming for Mold Layout Optimization, Significance of Efficient Floor Space Utilization and Practical Applications in Boat Manufacturing Facilities were also discussed. In this review article, a total number of 60 papers were choosen from the year 2019 to 2024 accordingly. Moreover, 11 papers were selected from the year 2019, 18 papers were selected from 2020, 13 were selected from the year 2021, 9 papers from the year 2022, 6 papers were selected from 2023 and 3 were from 2024 respectively

Electric Vehicle Charging Route Planning for Shortest Travel Time Based on Improved Ant Colony Optimization

Electric vehicles (EVs) are gaining significant attention as an environmentally friendly transportation solution. However, limitations in battery technology continue to restrict EV range and charging speed, resulting in range anxiety, which hampers widespread adoption. While there has been increasing research on EV route optimization, personalized path planning that caters to individual user needs remains underexplored. To bridge this gap, we propose the electric vehicle charging route planning based on user requirements (EVCRP-UR) problem, which aims to integrate user preferences and multiple constraints. Our approach utilizes topology optimization to reduce computational complexity and improve path planning efficiency. Furthermore, we introduce an improved ant colony optimization (IACO) algorithm incorporating novel heuristic functions and refined probability distribution models to select optimal paths and charging stations. To further enhance charging strategies, we develop a discrete electricity dynamic programming (DE-DP) algorithm to determine charging times at efficiently chosen stations. By combining these methods, the proposed IACO algorithm leverages the strengths of each approach, overcoming their individual limitations and delivering superior performance in EV routing and charging optimization.

An Exploration of Crop Planting Based on Simulated Annealing Algorithm and Mixed Integer Linear Programming

This paper is dedicated to exploring the optimal planting scheme of crops through the comprehensive use of various computer optimization algorithms, and is expected to provide a scientific decision-making basis for crop planting from 2024 to 2030. Firstly, based on Mixed Integer Linear Programming (MILP) and Simulated Annealing Algorithm, this paper analyzes the planting area and benefits of different crops based on the existing plot types and crop cultivation in rural arable land to find the optimal planting plan. Secondly, when facing the uncertainty factors of the future market, this paper quantifies the risk by dynamic programming and genetic algorithm, combining the parameters such as the sales price, planting cost and expected sales volume of the crops, to optimize the planting decision, and considering the substitutability and complementary relationship between the crops. Then, Pearson correlation coefficient and cross-elasticity coefficient are introduced to analyze the correlation between variables such as crop sales price and planting cost, and constrained optimization. Finally, integrating multiple model algorithms, this paper proposes a crop planting strategy that maximizes the comprehensive benefits.

CParty: hierarchically constrained partition function of RNA pseudoknots.

Biologically relevant RNA secondary structures are routinely predicted by efficient dynamic programming algorithms that minimize their free energy. Starting from such algorithms, one can devise partition function algorithms, which enable stochastic perspectives on RNA structure ensembles. As the most prominent example, McCaskill's partition function algorithm is derived from pseudoknot-free energy minimization. While this algorithm became hugely successful for the analysis of pseudoknot-free RNA structure ensembles, as of yet there exists only one pseudoknotted partition function implementation, which covers only simple pseudoknots and comes with a borderline-prohibitive complexity of O(n5) in the RNA length n. Here, we develop a partition function algorithm corresponding to the hierarchical pseudoknot prediction of HFold, which performs exact optimization in a realistic pseudoknot energy model. In consequence, our algorithm CParty carries over HFold's advantages over classical pseudoknot prediction in characterizing the Boltzmann ensemble at equilibrium. Given an RNA sequence S and a pseudoknot-free structure G, CParty computes the partition function over all possibly pseudoknotted density-2 structures G∪G' of S that extend the fixed G by a disjoint pseudoknot-free structure G'. Thus, CParty follows the common hypothesis of hierarchical pseudoknot formation, where pseudoknots form as tertiary contacts only after a first pseudoknot-free "core" G and we call the computed partition function hierarchically constrained (by G). Like HFold, the dynamic programming algorithm CParty is very efficient, achieving the low complexity of the pseudoknot-free algorithm, i.e. cubic time and quadratic space. Finally, by computing pseudoknotted ensemble energies, we unveil kinetics features of a therapeutic target in SARS-CoV-2. CParty is available at https://github.com/HosnaJabbari/CParty.

Research on Production Process Decision Making Based on Dynamic Programming and Simulated Annealing Algorithm

In this paper, a comprehensive decision model based on dynamic programming and simulated annealing algorithm is constructed to solve the decision optimization problem of an enterprise in the production of popular electronic products. By analyzing each stage of the production process in detail, including parts purchase, inspection, assembly and finished product inspection, and combining the defective rate, inspection cost, market price and return loss and other key parameters of each stage, a scheme for optimizing production decision is formed. In this paper, the dynamic programming method is used to optimize the production path, and the simulated annealing algorithm is introduced to solve the larger scale production planning problem, and the detection and treatment strategy in the production process of enterprises is optimized. Finally, it provides an effective decision-making basis for enterprises to obtain the maximum production profit under uncertain conditions.

Joint Optimization of Condition‐Based Maintenance and Production Rate Using Reinforcement Learning Algorithms

ABSTRACTMaintenance has always been a crucial aspect of the manufacturing and industrial sectors. There is a recent surge of interest in utilizing advanced machine learning, and reinforcement learning models to enhance maintenance strategies. In this regard, this paper focuses on the development of a joint optimization model of maintenance and production for a special type of production system that has an adjustable production rate, where the system's deterioration is closely related to the production rate. When the production rate is increased, the expected deterioration of the system also increases. To control the deterioration of the system, the paper proposes two main actions or policies: maintenance policy and production policy. These policies involve scheduling and conducting maintenance actions on the system and adjusting the production rate, respectively. To solve the optimization problem of minimizing the expected costs of the system during a finite planning horizon, the paper develops a Markov decision process and employs reinforcement learning algorithms such as Q‐learning and SARSA. The hyperparameters of the algorithms are tuned using a value‐iteration algorithm of dynamic programming. The developed optimal actions given the state of the system ensure efficient management of the production system while controlling the deterioration of the system.

Research on Speed Planning and Energy Management Strategy for Fuel Cell Hybrid Bus in Green Wave Scenarios at Traffic Light Intersections Based on Deep Reinforcement Learning

In the context of intelligent and connected transportation, obtaining the real-time vehicle status and comprehensive traffic data is crucial for addressing challenges related to speed optimization and energy regulation in intricate transportation situations. This paper introduces a control method for the speed optimization and energy management of a fuel cell hybrid bus (FCHB) based on the Deep Deterministic Policy Gradient (DDPG) algorithm. The strategy framework is built on a dual-objective optimization deep reinforcement learning (D-DRL) architecture, which integrates traffic signal information into the energy management framework, in addition to conventional state spaces to guide control decisions. The aim is to achieve “green wave” traffic while minimizing hydrogen consumption. To validate the effectiveness of the proposed strategy, simulation tests were conducted using the SUMO platform. The results show that in terms of speed planning, the difference between the maximum and minimum speeds of the FCHB was reduced by 21.66% compared with the traditional Intelligent Driver Model (IDM), while the acceleration and its variation were reduced by 8.89% and 13.21%, respectively. In terms of the hydrogen fuel efficiency, the proposed strategy achieved 95.71% of the performance level of the dynamic programming (DP) algorithm. The solution proposed in this paper is of great significance for improving passenger comfort and FCHB economy.

Optimal Commitment Strategy of the Wind-battery-hydrogen Hybrid System in the Spot Market

In this paper, we consider the problem of making the advance power commitments for the wind farms and the hydrogen supply plan for the hydrogen spot market, in the presence of the hybrid energy storage system with battery based and hydrogen based, mean-reverting price pocesses of power and hydrogen and the auto regressive energy generation process from winds, which extends the model in (Kim & Powell, 2011) and (Finnah & Gönsch, 2021). The problem is solved with dynamic programming algorithm with backward induction and the infinite horizon analysis. We obtained an optimal energy commitment policy and an optimal hydrogen supply plan under the above assumptions, which are indeed the extensions of the conclusion in (Kim & Powell, 2011) and (Finnah & Gönsch, 2021). Finally the property of stationary process of the hybrid storage levels corresponding to the optimal policy and plan are proved.

Development of a DDP-Based Trajectory Generation Method Considering the Manipulability Measure for 6-DOF Collaborative Robots

Abstract This paper presents a trajectory generation method for 6-DOF collaborative robots using the Differential Dynamic Programming algorithm, integrating the Manipulability Measure to effectively avoid singularities. Traditional trajectory generation methods often neglect the robot’s dynamics, leading to non-optimized trajectories and potential singularity issues. Our approach integrates Manipulability Measure into the Differential Dynamic Programming algorithm, ensuring consideration of the robot’s dynamics while effectively avoiding singularities. The proposed algorithm is applied to the 6-Degrees of freedom collaborative robot RB1-500e and validated on the actual robot. The results demonstrate that the proposed method significantly improves trajectory generation and singularity avoidance compared to traditional methods. This study offers a robust solution that enhances both efficiency and safety in dynamic trajectory planning for collaborative robots.

Robust Optimal Consensus Control for Nonlinear Multi‐agent Systems: Two Hybrid Dynamic Event‐Triggered‐Based Approach

ABSTRACTIn this paper, the dynamic event‐triggered robust optimal consensus control problem is investigated for nonlinear multi‐agent systems (MASs) with matched and mismatched disturbances via integral sliding mode (ISM) control and the adaptive dynamic programming (ADP) algorithm. First, a dynamic event‐triggered fixed‐time ISM controller is designed to guarantee that the system state converges to the predefined ISM manifold, thereby eliminating the matched disturbances. Next, the consensus control problem is transformed into an optimal control problem by constructing neighborhood error dynamics and a new modified cost function; thus, an event‐triggered‐based coupled Hamilton‐Jacobi‐Bellman equation (HJBE) is established. Then, an ADP‐based single‐critic neural network (NN) is constructed to solve coupled HJBE to obtain the event‐triggered optimal controller, in which the NN weight is updated only at the triggering instants. Through the implementation of these two dynamic event‐triggered mechanisms, resources and controller execution time can be saved. It is proved that the whole closed‐loop system signals are uniformly ultimately bounded by the Lyapunov technique. Finally, two illustrative examples verify the effectiveness and superiority of the proposed control scheme.

UKF-Based Optimal Tracking Control for Uncertain Dynamic Systems With Asymmetric Input Constraints.

To enhance system robustness in the face of uncertainty and achieve adaptive optimization of control strategies, a novel algorithm based on the unscented Kalman filter (UKF) is developed. This algorithm addresses the finite-horizon optimal tracking control problem (FHOTCP) for nonlinear discrete-time (DT) systems with uncertainty and asymmetric input constraints. An augmented system is constructed with asymmetric control constraints being considered. The augmented problem is addressed with a DT Hamilton-Jacobi-Bellman equation (DTHJBE). By analyzing convergence with regard to the cost function and control law, the UKF-based iterative adaptive dynamic programming (ADP) algorithm is proposed. This algorithm approximates the solution of the DTHJBE, ensuring that the cost function converges to its optimal value within a bounded range. To execute the UKF-based iterative ADP algorithm, the actor-estimator-critic framework is built, in which the estimator refers to system state estimation through the application of UKF. Ultimately, simulation examples are presented to show the performance of the proposed method.

Dynamic Event-Triggered Strategy-Based Optimal Control of Modular Robot Manipulator: A Multiplayer Nonzero-Sum Game Perspective.

Due to the limited computing and processing ability of modular robot manipulator (MRM) components, such as sensors and controllers, event-triggered mechanisms are considered a crucial communication paradigm shift in resource constrained applications. Dynamic event-triggered mechanism is developing into a new technology by reason of its higher resource utilization efficiency and more flexible system design requirements than traditional event-triggered. Therefore, an optimal control scheme of multiplayer nonzero-sum game based on dynamic event-triggered is developed for MRM systems with uncertain disturbances. First, dynamic model of the MRM is established according to joint torque feedback technique and model uncertainty is estimated by data-driven-based neural network identifier. In the framework of differential game, the tracking control problem of MRM system is transformed into the optimal control problem for multiplayer nonzero-sum game with the control input of each joint module as the player. Then, the static event-triggered control problem of MRM system is studied based on adaptive dynamic programming algorithm. On this basis, the internal dynamic variable describing the previous state of the system is introduced, and the characteristics of dynamic trigger rule and its relationship with static rule are revealed theoretically. By designing an exponential attenuation signal, the minimum sampling interval of the system is always positive, so that Zeno behavior is excluded. Lyapunov theory proves that the system is asymptotically stable and the experimental results verify the validity of the proposed method.

Logic-based benders decomposition algorithm for robust parallel drone scheduling problem considering uncertain travel times for drones

The integration of trucks and drones in last-mile delivery has introduced new capabilities to the transportation industry. These two vehicles simultaneously offer unique features, which have improved performance and efficiency in the process of delivering products. This paper investigates a robust parallel drone scheduling traveling salesman problem with supporting drone, where drone travel times are uncertain and products gradually arrive at a depot over time. In this problem, a truck, a supporting drone, and service drones are located in the depot to deliver products with the goal of minimizing the total completion time. A mathematical model is proposed which is improved using the earliest release dates rule, followed by the development of an exact logic-based benders decomposition algorithm to solve the problem. In this algorithm, customers are initially assigned to the service drones or the truck in a master problem, and subsequent auxiliary problems are addressed utilizing the earliest release dates rule and a dynamic programming algorithm. Finally, various cuts are enhanced through strengthening techniques and sequentially added into the master problem. Numerical experiments demonstrate the efficiency of the improved mathematical model and the proposed algorithm. Furthermore, sensitivity analysis has provided several managerial recommendations for enhancing the delivery system performance.

Co-Optimization of Speed Planning and Energy Management for Plug-In Hybrid Electric Trucks Passing Through Traffic Light Intersections

To tackle the energy-saving optimization issue of plug-in hybrid electric trucks traversing multiple traffic light intersections continuously, this paper presents a double-layer energy management strategy that utilizes the dynamic programming–twin delayed deep deterministic policy gradient (DP-TD3) algorithm to synergistically optimize the speed planning and energy management of plug-in hybrid electric trucks, thereby enhancing the vehicle’s passability through traffic light intersections and fuel economy. In the upper layer, the dynamic programming (DP) algorithm is employed to create a speed-planning model. This model effectively converts the nonlinear constraints related to the position, phase, and timing information of each traffic signal on the road into time-varying constraints, thereby improving computational efficiency. In the lower layer, an energy management model is constructed using the twin delayed deep deterministic policy gradient (TD3) algorithm to achieve optimal allocation of demanded power through the interaction of the TD3 agent with the truck environment. The model’s validity is confirmed through testing on a hardware-in-the-loop test machine, followed by simulation experiments. The results demonstrate that the DP-TD3 method proposed in this paper effectively enhances fuel economy, achieving an average fuel saving of 14.61% compared to the dynamic programming–charge depletion/charge sustenance (DP-CD/CS) method.

Research on the Optimization Strategy of Crop Planting Combining Dynamic Programming and Monte Carlo Simulation

This paper explores the optimization of crop planting strategies by combining dynamic programming with Monte Carlo simulation. In the multi-stage decision-making for planting optimization, we decomposed the optimal planting plans for crops from 2024 to 2030 into annual best planting plans. We established an objective function aimed at maximizing total profit, taking into account factors such as planting area, crop yield, sales price, and planting cost. A series of constraints were also introduced, including that the planting area should not exceed the cultivated land area and that the total crop production should not surpass the expected sales volume. By employing dynamic programming and greedy algorithm models, we utilized the PuLP linear programming library to define problems, add constraints, and invoke solvers to find the optimal solution. The model considered not only the planting costs and sales prices but also treated excess sales volume as waste. In the result analysis phase, we analyzed the cyclical changes in data across different years and observed that the total profit exhibited a certain fluctuation trend. We also conducted error analysis on the potential errors introduced by the greedy algorithm, running the model multiple times to verify the stability of the results and ensure the reliability of the model. Finally, we discussed the complementarity and substitutability between crops, which significantly impact the total income in the actual planting process.

Contextual bandits learning-based branch-and-price-and-cut algorithm for the two-dimensional vector packing problem with conflicts and time windows

A two-dimensional vector packing problem with conflicts and time windows (2DVPPCTW) is investigated in this study. It consists of packing items into the minimum number of bins, and items are characterized by different weights, volumes, and time windows. Items also have conflicts and cannot be packed in the same bin. We formulate the 2DVPPCTW as an integer programming model and reformulate it to the master problem and the subproblem based on the Danzig–Wolfe decomposition. An exact algorithm, contextual bandits learning-based branch-and-price-and-cut algorithm (CBL-BPC), is proposed for the 2DVPPCTW with reinforcement learning technique. In particular, we provide a CBL framework for the subproblem, which usually poses considerable computational challenges. Five heuristic algorithms, namely, adaptive large neighborhood search (ALNS), ant colony optimization heuristic (ACO), heuristic dynamic programming (DP), a combination of ALNS and heuristic DP, and a combination of ACO and heuristic DP, are developed as bandit arms in the CBL framework. The CBL framework adaptively chooses one of five heuristics algorithms to solve the subproblem by learning from previous experiences. An exact dynamic programming algorithm is invoked to guarantee optimality once the CBL fails to find a better solution to the subproblem. Rounded capacity inequalities and accelerating strategies are introduced to accelerate the solution. An extensive computational study shows that the CBL-BPC can solve all 800 instances optimally within a reasonable time frame and is highly competitive with state-of-the-art exact and heuristics methods.