Dynamic Programming Procedure Research Articles

Dynamic Programming Approach for Arterial Signal Optimization

Link performance functions (LPFs) are essential components of macroscopic models for signal optimization, traffic assignment, or combined control-assignment models. Numerous studies have developed a variety of performance functions for signal optimization. Most of these studies attempted to use simplified LPFs that did not capture the important relationships and dependencies among the variables. Dependent LPFs are introduced; in these LPFs the performance on any link depends on the flow pattern not only on that link alone but also on the feeder links. Such functions represent with greater fidelity the performance characteristics of the link and can lead to better control and assignment. An optimization model explicitly considers flow interactions among successive links and applies it in a dynamic programming procedure to determine optimal signal settings. The procedure, which is an offshoot of the combination method for offset optimization, is illustrated for an arterial street and demonstrates its efficacy in comparison with existing optimization models.

An algorithm for robust explicit/multi-parametric model predictive control

A new algorithm for robust explicit/multi-parametric Model Predictive Control (MPC) for uncertain, linear discrete-time systems is proposed. Based on previous work on Dynamic Programming (DP), multi-parametric Programming and Robust Optimization, the proposed algorithm features, (i) a DP reformulations of the MPC optimization problem, (ii) a robust reformulation of the constraints, and (iii) a multi-parametric programming step, where the control variables are obtained as explicit functions of the state variable, such that the state and input constraints are satisfied for all admissible values of the uncertainty. A key feature of the proposed procedure is that, as opposed to previous methods, it only solves a convex multi-parametric programming problem for each stage of the DP procedure.

A Modified Computational Dynamic Programming Approach To a Production and Inventory Control Problem

A production and inventory control problem is solved by a modified computational dynamic programming procedure. Specifically a two dimensional problem described by differential system equations with an integral objective function is solved by the proposed scheme. The numerical results are examined for different number of stages. The advertising rate, inventory, sales, and profit are illustrated through tables and diagrams.

The Fixed-Charge Shortest-Path Problem

Consider a network 𝒩 =(N, A) and associate with each arc e ∈ A a fixed cost ce for using arc e, an interval [le, ue] (le, ue ∈ ℤ) specifying the range of allowable resource consumption quantities along arc e, and a per-unit cost [Formula: see text] for resource consumed along e. Furthermore, for each node n ∈ N, let Un ∈ ℤ be the maximum amount of resource consumption a path can accumulate before visiting node n. Given a source node ns ∈ N and sink node nt ∈ N, the fixed-charge shortest-path problem (FCSPP) seeks to find a minimum-cost-feasible path from ns to nt. When resource consumption is simply additive, the resource-constrained shortest-path problem (RCSPP) is a special case of FCSPP. We develop a new dynamic programming algorithm for FCSPP. The algorithm uses solutions from labeling and dominance techniques for standard RCSPPs on slightly modified problems, and it combines these solutions by exploiting the structure provided by certain classes of knapsack problems to efficiently construct an optimal solution to FCSPP. Computational experiments demonstrate that our algorithm is often several orders of magnitude faster than naive dynamic programming procedures.

A new dynamic programming procedure for three-staged cutting patterns

Three-staged patterns are often used to solve the 2D cutting stock problem of rectangular items. They can be divided into items in three stages: Vertical cuts divide the plate into segments; then horizontal cuts divide the segments into strips, and finally vertical cuts divide the strips into items. An algorithm for unconstrained three-staged patterns is presented, where a set of rectangular item types are packed into the plate so as to maximize the pattern value, and there is no constraint on the frequencies of each item type. It can be used jointly with the linear programming approach to solve the cutting stock problem. The algorithm solves three large knapsack problems to obtain the optimal pattern: One for the item layout on the widest strip, one for the strip layout on the longest segment, and the third for the segment layout on the plate. The computational results indicate that the algorithm is efficient.

Linear–quadratic switching control with switching cost

We study in this paper the linear–quadratic (LQ) optimal control problem of discrete-time switched systems with a constant switching cost for both finite and infinite time horizons. We reduce these problems into an auxiliary problem, which is an LQ optimal switching control problem with a cardinality constraint on the total number of switchings. Based on the solution structure derived from the dynamic programming (DP) procedure, we develop a lower bounding scheme by exploiting the monotonicity of the Riccati difference equation. Integrating such a lower bounding scheme into a branch and bound (BnB) framework, we offer an efficient numerical solution scheme for the LQ switching control problem with switching cost.

Example-Based Automatic Music-Driven Conventional Dance Motion Synthesis

We introduce a novel method for synthesizing dance motions that follow the emotions and contents of a piece of music. Our method employs a learning-based approach to model the music to motion mapping relationship embodied in example dance motions along with those motions' accompanying background music. A key step in our method is to train a music to motion matching quality rating function through learning the music to motion mapping relationship exhibited in synchronized music and dance motion data, which were captured from professional human dance performance. To generate an optimal sequence of dance motion segments to match with a piece of music, we introduce a constraint-based dynamic programming procedure. This procedure considers both music to motion matching quality and visual smoothness of a resultant dance motion sequence. We also introduce a two-way evaluation strategy, coupled with a GPU-based implementation, through which we can execute the dynamic programming process in parallel, resulting in significant speedup. To evaluate the effectiveness of our method, we quantitatively compare the dance motions synthesized by our method with motion synthesis results by several peer methods using the motions captured from professional human dancers' performance as the gold standard. We also conducted several medium-scale user studies to explore how perceptually our dance motion synthesis method can outperform existing methods in synthesizing dance motions to match with a piece of music. These user studies produced very positive results on our music-driven dance motion synthesis experiments for several Asian dance genres, confirming the advantages of our method.

Water network operational optimization: Utilizing symmetries in combinatorial problems by dynamic programming

This paper introduces a dynamic programming (DP) approach for solving deterministic combinatorial operational optimization problem of water distribution networks. The implementation of dynamic programming over control domain using permutational symmetries is suggested to replace the state space based DP procedures. To enhance the understanding an application on a ub-network of the water supply and distribution network of the city of Sopron (Hungary) is presented which is sufficiently small to track the (pseudo) state space and approach related quantities.

An efficient hybrid genetic algorithm to solve assembly line balancing problem with sequence-dependent setup times

In this paper the setup assembly line balancing and scheduling problem (SUALBSP) is considered. Since this problem is NP-hard, a hybrid genetic algorithm (GA) is proposed to solve the problem. This problem involves assigning the tasks to the stations and scheduling them inside each station. A simple permutation is used to determine the sequence of tasks. To determine the assignment of tasks to stations, the algorithm is hybridized using a dynamic programming procedure. Using dynamic programming, at any time a chromosome can be converted to an optimal solution (subject to the chromosome sequence).Since population diversity is very important to prevent from being trapped in local optimum solutions some diversity maintaining schemes are used to overcome this issue. Operators and parameters of the algorithm is calibrated using design of experiments (DOEs) method. The computational results show that the proposed GA outperforms all of the algorithms presented to solve SUALBSP so far.

Finding cliques of maximum weight on a generalization of permutation graphs

We propose a dynamic programming procedure for computing the clique of maximum weight on a class of graphs arising in the solution of train timetabling problems. These graphs generalize, in two ways permutation graphs, defined as the intersection graphs of segments joining two parallel lines. First, two segments are joined by an edge not only if they intersect but also if their endpoints are sufficiently close. Second, two parallel segments may be joined by an edge even if they are arbitrarily far away from each other.

Fast Newton-CG Method for Batch Learning of Conditional Random Fields

We propose a fast batch learning method for linear-chain Conditional Random Fields (CRFs) based on Newton-CG methods. Newton-CG methods are a variant of Newton method for high-dimensional problems. They only require the Hessian-vector products instead of the full Hessian matrices. To speed up Newton-CG methods for the CRF learning, we derive a novel dynamic programming procedure for the Hessian-vector products of the CRF objective function. The proposed procedure can reuse the byproducts of the time-consuming gradient computation for the Hessian-vector products to drastically reduce the total computation time of the Newton-CG methods. In experiments with tasks in natural language processing, the proposed method outperforms a conventional quasi-Newton method. Remarkably, the proposed method is competitive with online learning algorithms that are fast but unstable.

Cardinality Constrained Linear-Quadratic Optimal Control

As control implementation often incurs not only a variable cost associated with the magnitude or energy of the control, but also a setup cost, we consider a discrete-time linear-quadratic (LQ) optimal control problem with a limited number of control implementations, termed in this technical note the cardinality constrained linear-quadratic optimal control (CCLQ). We first derive a semi-analytical feedback policy for CCLQ problems using dynamic programming (DP). Due to the exponential growth of the complexity in calculating the action regions, however, DP procedure is only efficient for CCLQ problems with a scalar state space. Recognizing this fact, we develop then two lower-bounding schemes and integrate them into a branch-and-bound (BnB) solution framework to offer an efficient algorithm in solving general CCLQ problems. Adopting the devised solution algorithm for CCLQ problems, we can solve efficiently the linear-quadratic optimal control problem with setup costs.

The Cloud Hunter’s Problem: An Automated Decision Algorithm to Improve the Productivity of Scientific Data Collection in Stochastic Environments

AbstractA decision algorithm is presented that improves the productivity of data collection activities in stochastic environments. The algorithm was developed in the context of an aircraft field campaign organized to collect data in situ from boundary layer clouds. Required lead times implied that aircraft deployments had to be scheduled in advance, based on imperfect forecasts regarding the presence of conditions meeting specified requirements. Given an overall cap on the number of flights, daily fly/no-fly decisions were taken traditionally using a discussion-intensive process involving heuristic analysis of weather forecasts by a group of skilled human investigators. An alternative automated decision process uses self-organizing maps to convert weather forecasts into quantified probabilities of suitable conditions, together with a dynamic programming procedure to compute the opportunity costs of using up scarce flights from the limited budget. Applied to conditions prevailing during the 2009 Routine ARM Aerial Facility (AAF) Clouds with Low Optical Water Depths (CLOWD) Optical Radiative Observations (RACORO) campaign of the U.S. Department of Energy’s Atmospheric Radiation Measurement Program, the algorithm shows a 21% increase in data yield and a 66% improvement in skill over the heuristic decision process used traditionally. The algorithmic approach promises to free up investigators’ cognitive resources, reduce stress on flight crews, and increase productivity in a range of data collection applications.

Optimizing Batch Size in a Flow-Oriented Synchronized Production

This study was prepared for a leading company, Miele GmbH, the global premium brand of domestic appliancesand commercial machines in the field of laundry care, dishwashing and disinfection in Germany. The productionline of Miele GmbH in Bielefeld, Germany was analyzed to develop a model that can be used for all the firms inthe group.Dynamic programming models are widely used by companies to efficiently meet the demand for a variety ofproducts. In a flow shop, each product has to be processed by a number of machines in synchronized lines. Theproduction smoothing problems under the presence of setup and processing times vary among the products. Themaster production-inventory problem of Miele GmbH was divided into two sub-problems which were concernedwith determining the batch sizes and production sequences of products, respectively. A dynamic programmingprocedure was developed to solve the batching problem for the current problem. A dynamic computational studyfor the first case was conducted so that the solution method is effective in meeting the goals of the firm andefficient in its computational requirements.Scientific problem –the firm has a problem of high logistics. Moreover, they want to decrease the cost ofproduction in order to compete with their competitors. Their competitors start to make some of their productionin low labour countries such as China. Miele is a special brand for upper level. However, they now have a morestrict competition with new global players and one way to stay competitive in the market is to decrease the costsand find new market segments.The aim of the research –The dynamic programming algorithm is suggested to them to decrease the costs. Thenumbers of products are decreased to explain the algorithm. An example with calculations of this algorithm wasexplained in this study. The number of variables and constraints can be increased. After the logic of algorithm isunderstood, it can be applied many similar problems. The algorithm can be developed by using differentsoftware such as Java. Then, the variables and values of algorithm can be input for the algorithm and the resultscan be gotten in a short time.The object of the research – Inventory and Production mix projects.The methods of the research –Dynamic Programming.

On Cooperative Settlement Between Content, Transit, and Eyeball Internet Service Providers

Internet service providers (ISPs) depend on one another to provide global network services. However, the profit-seeking nature of the ISPs leads to selfish behaviors that result in inefficiencies and disputes in the network. This concern is at the heart of the “network neutrality” debate, which also asks for an appropriate compensation structure that satisfies all types of ISPs. Our previous work showed in a general network model that the Shapley value has several desirable properties, and that if applied as the profit model, selfish ISPs would yield globally optimal routing and interconnecting decisions. In this paper, we use a more detailed and realistic network model with three classes of ISPs: content, transit, and eyeball. This additional detail enables us to delve much deeper into the implications of a Shapley settlement mechanism. We derive closed-form Shapley values for more structured ISP topologies and develop a dynamic programming procedure to compute the Shapley values under more diverse Internet topologies. We also identify the implications on the bilateral compensation between ISPs and the pricing structures for differentiated services. In practice, these results provide guidelines for solving disputes between ISPs and for establishing regulatory protocols for differentiated services and the industry.

Are People Successful at Learning Sequences of Actions on a Perceptual Matching Task?

We report the results of an experiment in which human subjects were trained to perform a perceptual matching task. Subjects were asked to manipulate comparison objects until they matched target objects using the fewest manipulations possible. An unusual feature of the experimental task is that efficient performance requires an understanding of the hidden or latent causal structure governing the relationships between actions and perceptual outcomes. We use two benchmarks to evaluate the quality of subjects' learning. One benchmark is based on optimal performance as calculated by a dynamic programming procedure. The other is based on an adaptive computational agent that uses a reinforcement-learning method known as Q-learning to learn to perform the task. Our analyses suggest that subjects were successful learners. In particular, they learned to perform the perceptual matching task in a near-optimal manner (i.e., using a small number of manipulations) at the end of training. Subjects were able to achieve near-optimal performance because they learned, at least partially, the causal structure underlying the task. In addition, subjects' performances were broadly consistent with those of model-based reinforcement-learning agents that built and used internal models of how their actions influenced the external environment. We hypothesize that people will achieve near-optimal performances on tasks requiring sequences of action-especially sensorimotor tasks with underlying latent causal structures-when they can detect the effects of their actions on the environment, and when they can represent and reason about these effects using an internal mental model.

Dynamic programming procedure for searching optimal models to estimate substitution rates based on the maximum-likelihood method

The substitution rate in a gene can provide valuable information for understanding its functionality and evolution. A widely used method to estimate substitution rates is the maximum-likelihood method implemented in the CODEML program in the PAML package. A limited number of branch models, chosen based on a priori information or an interest in a particular lineage(s), are tested, whereas a large number of potential models are neglected. A complementary approach is also needed to test all or a large number of possible models to search for the globally optional model(s) of maximum likelihood. However, the computational time for this search even in a small number of sequences becomes impractically long. Thus, it is desirable to explore the most probable spaces to search for the optimal models. Using dynamic programming techniques, we developed a simple computational method for searching the most probable optimal branch-specific models in a practically feasible computational time. We propose three search methods to find the optimal models, which explored O(n) (method 1) to O(n(2)) (method 2 and method 3) models when the given phylogeny has n branches. In addition, we derived a formula to calculate the number of all possible models, revealing the complexity of finding the optimal branch-specific model. We show that in a reanalysis of over 50 previously published studies, the vast majority obtained better models with significantly higher likelihoods than the conventional hypothesis model methods.

Optimal search path for service in the presence of disruptions

We consider the following basic search path problem: a customer residing at a node of a network needs to obtain service from one of the facilities; facility locations are known and fixed. Facilities may become inoperational with certain probability; the state of the facility only becomes known when the facility is visited. Customer travel stops when the first operational facility is found. The objective is to minimize the expected total travel distance.We show that this problem is NP-hard and develop a forward dynamic programming procedure. The efficiency of this procedure is improved by utilizing the special structure of the network and features of the optimal search paths. Polynomial procedures are developed for path and cycle networks. Through a set of computational experiments we show that the “visit closest unvisited facility” heuristic is quite efficient, especially when the failure probability is small. Our results substantiate optimal location models that rely on the behavioral assumptions of this heuristic.

A time-indexed LP-based approach for min-sum job-shop problems

In this paper we propose two time-indexed IP formulations for job-shop scheduling problems with a min-sum objective. The first model has variables associated to job scheduling patterns. The exponential number of variables calls for a column generation scheme which is carried out by a dynamic programming procedure. The second model is of network flow type with side constraints. This model can be strengthened by adding cutting inequalities of clique type. It turns out that the two models are equivalent, since the dual of the second formulation is equivalent to the compact dual of the first model. However, they require significantly different solution approaches and may behave differently in terms of computing time and memory usage. Good upper bounds are found by a heuristic procedure that randomly generates schedules from fractional solutions. These features allow for an effective pruning of the branch-and-bound tree and narrowing the gap between lower and upper bounds. However, the size of both models is critically affected by the time-indexed formulation which may heavily slow down the computation.

Hierarchical control for discrete large-scale complex systems by intelligent controllers

This paper presents a new method to approximate optimal control strategies of discrete time large-scale nonlinear systems using intelligent approaches. The idea is based on the decomposition principle of the global system into interconnected subsystems for which nonlinearities are located in the interconnection terms. Then, the mixed coordination procedure between different subsystems is formulated as a hierarchical method for the solution of large-scale optimal control problems. So, for each subsystem, local optimal feedback gains are expressed in terms of the interconnection vector. For this purpose, neural networks and fuzzy logic controllers have been constructed in order to identify these gains. A comparison with the differential dynamic programming procedure as a reference method is done. Simulation results of two numerical examples show that the proposed method yields to satisfactory performances, and the robustness of the proposed approaches has been tested.