Dynamic Programming-based Approach Research Articles

COALA: Concurrently Assigning Wire Segments to Layers for 2-D Global Routing

Two-dimensional (2-D) global routing followed by layer assignment is a common and popular strategy to obtain a good tradeoff between runtime and routing performance. Yet, the huge gap between 2-D routing patterns and the final 3-D routing paths often results in inevitable overflow after layer assignment. State-of-the-art (SOTA) studies on layer assignment usually adopt dynamic programming-based approaches to sequentially find an optimal solution for each net in terms of overflow or/and the number of vias. However, a fixed assignment ordering severely restricts the solution space, and the distributed overflows can hardly be resolved with any existing refinement approach. This article proposes a novel layer assignment framework that concurrently considers all the wire segments of nets and iteratively assigns them from the lowest available layer to the highest one. The concurrent scheme facilitates the maximal utilization of routing resource on each layer, contributing to an effective rerouting procedure that greatly reduces inevitable overflows. Based on the proposed framework, we further propose an obstacle-aware strategy that can mitigate obstacle-induced inevitable overflows in the original framework. Experimental results show that compared to an implemented sequential layer assignment approach based on SOTA techniques and refined by the well-known overflow/congestion reduction rip up and rerouting procedure, the proposed concurrent layer assignment framework (COALA) brings great improvements in the overflow reduction and runtime efficiency, which shows the significant advantage of the concurrent layer assignment scheme over sequential methods. The improvement is also verified in detailed routing, where the proposed COALA framework contributes to sparser routing results with fewer vias and design rule violations (DRVs).

Toward Efficient Similarity Search under Edit Distance on Hybrid Architectures

Edit distance is the most widely used method to quantify similarity between two strings. We investigate the problem of similarity search under edit distance. Given a collection of sequences, the goal of similarity search under edit distance is to find sequences in the collection that are similar to a given query sequence where the similarity score is computed using edit distance. The canonical method of computing edit distance between two strings uses a dynamic programming-based approach that runs in quadratic time and space, which may not provide results in a reasonable amount of time for large sequences. It advocates for parallel algorithms to reduce the time taken by edit distance computation. To this end, we present scalable parallel algorithms to support efficient similarity search under edit distance. The efficiency and scalability of the proposed algorithms is demonstrated through an extensive set of experiments on real datasets. Moreover, to address the problem of uneven workload across different processing units, which is mainly caused due to the significant variance in the size of the sequences, different data distribution schemes are discussed and empirically analyzed. Experimental results have shown that the speedup achieved by the hybrid approach over inter-task and intra-task parallelism is 18 and 13, respectively.

A dynamic programming-based approach for cloud instance type selection and optimisation

With the advantages of cloud computing gradually highlighted, users increasingly want to deploy their applications and services on the cloud to reduce costs and obtain high computing capacity. Nowadays, cloud providers (e.g., Amazon, Microsoft) at home and abroad provide a large amount of cloud instance types optimised to fit different use cases, such as compute optimised and memory optimised. Due to the potentially large quantity of cloud instance types in the public cloud market, it is often a challenge for users to select an optimal set of cloud instance types subject to limited resource capacity. In this paper, a dynamic programming-based approach is proposed for cloud instance type selection, which can provide optimal combination of cloud instance types to users. Experiments are performed based on real-world cloud information to evaluate the proposed method.

A Decision Trees-based knowledge mining approach for controlling a complex production system

In this study, we use decision trees constructed by learning-with-supervision techniques for representing the best policies found by a Reinforcement Learning algorithm applied to an optimization problem of a complex production system. Until now, the relative scientific literature includes studies that mainly propose dynamic programming-based approaches for treating such kind of combinatorial problems. Decision trees are used to approximate functions of multiple variables with discrete values. In this case the “leaves” of the tree correspond to the set of function values while the non-terminal nodes to its independent variables. In the present research, the parameters of the optimization problem and the corresponding optimal policies found by the Reinforcement Learning algorithm applied, will be used as the training data set. Representing the best found policies using decision trees will support more effective qualitative analysis and further understanding of their properties.

A dynamic programming-based approach for cloud instance type selection and optimisation

A dynamic programming-based approach for cloud instance type selection and optimisation

Allocating resources via price management systems: a dynamic programming-based approach

In this paper, a novel model for price management systems in resource allocation problems is proposed. Stochastic customer requests for resource allocations and releases are modelled as constrained parallel Birth–Death Processes (BDP). We address both instant (i.e. the customer requires a resource to be allocated immediately) and advance (i.e. the customer books a resource for future use) reservation requests, the latter with both bounded and unbounded time interval options. Algorithms based on Dynamic Programming (DP) principles are proposed for the calculation of suitable price profiles. At the core of such algorithms, there is the resolution of stochastic optimisation problems. In particular, the maximisation of the expected total revenue is formulated via a constrained Stochastic Dynamic Programming (SDP) approach, which becomes time-variant in case of advance reservation requests. Approximate Dynamic Programming (ADP) techniques are adopted in case of large state spaces. Simulations are performed to show the effectiveness of the proposed models and the related algorithms.

Modeling M Warehouse N Manpower-Team Allocation Problem Using Dynamic Programming Approach

In this research, a dynamic programming-based approach is deployed to model and solve the manpower allocation problem for warehouses. The authors specifically evolve the detailed model for M warehouses and N teams (available for allocation to these warehouses). Profitability is considered as a performance measure for the allocation problem. The warehouses and manpower-team are modelled as stages and states respectively within the dynamic programming problem structure. Owing to the rather abstract nature of such allocation problems possessing Markovian properties and having similarities with stage-gate type of a problem, dynamic programming approach is deployed. The study results in recommending key decisions in workforce allocation for organizations such as retailers operating multiple warehouses.

De novo glycan structural identification from mass spectra using tree merging strategy

MotivationGlycans are large molecules with specific tree structures. Glycans play important roles in a great variety of biological processes. These roles are primarily determined by the fine details of their structures, making glycan structural identification highly desirable. Mass spectrometry (MS) has become the major technology for elucidation of glycan structures. Most de novo approaches to glycan structural identification from mass spectra fall into three categories: enumerating followed by filtering approaches, heuristic and dynamic programming-based approaches. The former suffers from its low efficiency while the latter two suffer from the possibility of missing the actual glycan structures. Thus, how to reliably and efficiently identify glycan structures from mass spectra still remains challenging. ResultsIn this study we propose an efficient and reliable approach to glycan structure identification using tree merging strategy. Briefly, for each MS peak, our approach first calculated monosaccharide composition of its corresponding fragment ion, and then built a constraint that forces these monosaccharides to be directly connected in the underlying glycan tree structure. According to these connecting constraints, we next merged constituting monosaccharides of the glycan into a complete structure step by step. During this process, the intermediate structures were represented as subtrees, which were merged iteratively until a complete tree structure was generated. Finally the generated complete structures were ranked according to their compatibility to the input mass spectra. Unlike the traditional enumerating followed by filtering strategy, our approach performed deisomorphism to remove isomorphic subtrees, and ruled out invalid structures that violates the connection constraints at each tree merging step, thus significantly increasing efficiency. In addition, all complete structures satisfying the connection constraints were enumerated without any missing structure. Over a test set of 10 N-glycan standards, our approach accomplished structural identification in minutes and gave the manually-validated structure first three highest score. We further successfully applied our approach to profiling and subsequent structure assignment of glycans released from glycoprotein mAb, which was in perfect agreement with previous studies and CE analysis.

Approximate Optimal Distributed Control of Nonlinear Interconnected Systems Using Event-Triggered Nonzero-Sum Games.

In this paper, approximate optimal distributed control schemes for a class of nonlinear interconnected systems with strong interconnections are presented using continuous and event-sampled feedback information. The optimal control design is formulated as an N -player nonzero-sum game where the control policies of the subsystems act as players. An approximate Nash equilibrium solution to the game, which is the solution to the coupled Hamilton-Jacobi equation, is obtained using the approximate dynamic programming-based approach. A critic neural network (NN) at each subsystem is utilized to approximate the Nash solution and novel event-sampling conditions, that are decentralized, are designed to asynchronously orchestrate the sampling and transmission of state vector at each subsystem. To ensure the local ultimate boundedness of the closed-loop system state and NN parameter estimation errors, a hybrid-learning scheme is introduced and the stability is guaranteed using Lyapunov-based stability analysis. Finally, implementation of the proposed event-based distributed control scheme for linear interconnected systems is discussed. For completeness, Zeno-free behavior of the event-sampled system is shown analytically and a numerical example is included to support the analytical results.

MrDP: Multiple-Row Detailed Placement of Heterogeneous-Sized Cells for Advanced Nodes

As very large-scale integration technology shrinks to fewer tracks per standard cell, e.g., from 10 to 7.5-track libraries (and lesser for 7 nm), there has been a rapid increase in the usage of multiple-row cells like two- and three-row flip-flops, buffers, etc., for design closure. Additionally, the usage of multibit flip-flops or flop trays to save power creates large cells that further complicate critical design tasks, such as placement. Detailed placement happens to be a key optimization transform, which is repeatedly invoked during the design closure flow to improve design parameters, such as wirelength, timing, and local wiring congestion. Advanced node designs, with hundreds of thousands of multiple-row cells, require a paradigm change for this critical design closure transform. The traditional approach of fixing multiple-row cells during detailed placement and only optimizing the locations of single-row standard cells can no longer obtain appreciable quality of results. It is imperative to have new techniques that can simultaneously optimize both multiple- and single-row height cell locations during detailed placement. In this paper, we propose a new density-aware detailed placer for heterogeneous-sized netlists. Our approach consists of a chain move scheme that generalizes the movement of heterogeneous-sized cells, a nested dynamic programming-based approach for ordered double-row placement and a network flow-based formulation to solve ordered multiple-row placement for wirelength and density optimization. Experimental results demonstrate the effectiveness of these techniques in wirelength minimization and density smoothing compared with the most recent detailed placers for designs with heterogeneous-sized cells.

An approximate dynamic programming method for the multi-period technician scheduling problem with experience-based service times and stochastic customers

In this paper, we study how an organization can recognize that individuals learn when assigning employees to tasks. By doing so, an organization can meet current demands and position the capabilities of their workforce for the yet unknown demands in future days. Specifically, we study a variant of the technician and task scheduling problem in which the tasks to be performed in the current day are known, but there is uncertainty regarding the tasks to be performed in subsequent days. To solve this problem, we present an Approximate Dynamic Programming-based approach that incorporates into daily assignment decisions estimates of the long-term benefits associated with experience accumulation. We benchmark this approach against an approach that only considers the impact of experience accumulation on just the next day's productivity and show that the ADP approach outperforms this one-step lookahead approach. Finally, based on the results from an extensive computational study we derive insights into how an organization can schedule their employees in a manner that enables meeting both near and long-term demands.

Optimal Stopping Under Probability Distortions

In this paper we study optimal stopping problems with respect to distorted expectations with concave distortion functions. Our starting point is a seminal work of Xu and Zhou in 2013, who gave an explicit solution of such a stopping problem under a rather large class of distortion functionals. In this paper, we continue this line of research and prove a novel representation, which relates the solution of an optimal stopping problem under distorted expectation to the sequence of standard optimal stopping problems and hence makes the application of the standard dynamic programming-based approaches possible. Furthermore, by means of the well-known Kusuoka representation, we extend our results to optimal stopping under general law invariant coherent risk measures. Finally, based on our representations, we develop several Monte Carlo approximation algorithms and illustrate their power for optimal stopping under absolute semideviation risk measures.

A semi-automatic and an automatic segmentation algorithm to remove the internal organs from live pig CT images

Removal of internal organs such as lungs, liver, and kidneys is a key step required to compute the lean meat percentage from Computed Tomography (CT) scans of live animals. In this paper, we propose two segmentation techniques to remove these organs focusing on pigs. The first method is semi-automatic, and it starts with the first CT slice and a manually defined mask with internal organs. Then, it applies a four-step iterative process that computes the masks of the next CT slices by using the information of the previous one. To find the best boundary it uses a Dynamic Programming-based approach. At each iteration the user can check the correctness of the new computed mask. The second method is fully automatic, and segments each slice individually by using distance maps and morphological operators, such as dilation. It is composed of three main steps which detect the pig’s torso, pre-classify the voxels in different tissues, and segment the internal organs using the information of such classification. Although it has some parameters, user interaction is not required to obtain the results. The proposed approaches have been tested on CT data sets from 9 pigs, and compared with a manual segmentation. To evaluate the results, the precision, recall, and F-score measures have been used. From our test, we can observe that the performance of both methods is very high according to their average F-score. We also analyse how the accuracy of the results in the semi-automatic approach increases when more user interaction is applied. For the automatic approach, we evaluate the dependence of the results on the algorithm’s parameters. If robustness is enough, and high accuracy is not required, the automatic algorithm can be used to segment a whole pig in less than 50s. However, if the user wants to control the level of accuracy, the semi-automatic algorithm is preferred. Both methods are useful to reduce the time needed to segment the internal organs of a pig from hours (manual segmentation) to minutes or seconds.

Cooperation of Storage Operation in a Power Network With Renewable Generation

In this paper, we seek to properly schedule the operation of multiple storage devices so as to minimize the expected total cost (of conventional generation) in a power network with intermittent renewable generation. Since the power network constraints make it intractable to compute optimal storage operation policies through dynamic programming-based approaches, we propose a Lyapunov optimization-based online algorithm (LOPN), which makes decisions based only on the current state of the system (i.e., the current demand and renewable generation). The proposed algorithm is computationally simple and achieves asymptotic optimality (as the capacity of energy storage grows large). To improve the performance of the LOPN algorithm for the case with limited storage capacity, we propose a threshold-based energy storage management (TESM) algorithm that utilizes the forecast information (on demand and renewable generation) over the next a few time slots to make storage operation decisions. Numerical experiments are conducted on IEEE 6- and 9-bus test systems to validate the asymptotic optimality of LOPN, and compare the performance of LOPN and TESM. Numerical results show that TESM significantly outperforms LOPN when the storage capacity is relatively small.

Distributed Dynamic Programming-Based Approach for Economic Dispatch in Smart Grids

In this paper, the discrete economic dispatch problem is formulated as a knapsack problem. An effective distributed strategy based on distributed dynamic programming algorithm is proposed to optimally allocate the total power demand among different generation units considering the generation limits and ramping rate limits. The proposed distributed strategy is implemented based on a multiagent system framework which only requires local computation and communication among neighboring agents. Thus, it enables the sharing of computational and communication burden among distributed agents. In addition, the proposed strategy can be implemented with asynchronous communication, which may lead to simpler implementation and faster convergence speed. Simulation results with a four-generator system and the IEEE 162-bus system are presented to demonstrate the effectiveness of the proposed distributed strategy.

Real-time implementable optimal control strategy for hybrid electric vehicles energy management: application to medium-duty commercial vehicles

This paper presents a model predictive control approach called predictive power management to optimise the hybrid electric vehicle energy management controls over a horizon for medium-duty commercial applications. The two challenges associated with optimisation over a horizon including: a) reliable vehicle speed duty cycle prediction; b) fast computation to make optimal control decisions in real-time, are addressed. A novel dynamic programming-based approach, where the cost function is decoupled into a computationally expensive offline part and a fast computing online part to enable real-time implementation, forms the optimisation algorithm. A vehicle speed prediction algorithm based on duty cycle learning is developed for commercial applications with repetitive duty cycles. The simulation results of implementing the predictive power management demonstrate significant fuel economy improvement over a baseline energy management strategy. The impact of length of vehicle speed prediction horizon on the fuel savings is discussed.

Product low-carbon design using dynamic programming algorithm

Greenhouse gas emission has become a recent global concern for green manufacturing. As product low-carbon design is an essential approach to achieve low-carbon manufacturing, which has a profound effect on the product carbon footprint, many researches have been focused on it in recent years with a result of valuable contributions. This paper is devoted to presenting a dynamic programmingbased approach to product low-carbon design. After product low-carbon design is characterized by a multi-stage decision process with interaction effects on each other in the product life cycle, a dynamic programming method is used to optimize the total carbon footprint of each stage while considering interaction effects of solutions at each stage in product life cycle. The low-carbon design of a cold heading machine is used to demonstrate the proposed methodology.

A heuristic, dynamic programming-based approach for a two-dimensional cutting problem with defects

This paper deals with a two-dimensional cutting problem in which small rectangular items of given types are to be cut from a rectangular large object which contains several defects. It is assumed that the number of pieces of each small item type which can be cut from the large object is not limited. In addition, all cuts are restricted to be of the guillotine-type and the number of stages necessary to perform all cuts is not limited. Furthermore, no small item must overlap with a defective region. The objective is to maximize the value of the cut small items. For the solution of the above-described problem, a heuristic, dynamic programming-based approach is presented which overcomes the structural and computational limitations of previously-proposed methods. In the presence of defects, the representation of the defective regions and the definition of discretization sets are revisited. This allows for an improvement of the computational efficiency as well as of the storage space requirements for solving the given problem with any number of defects in this approach. We further analyze the computational complexity of the algorithm and identify the factors which affect its running time. The proposed method is evaluated by means of a series of detailed numerical experiments which are performed on problem instances extracted from the literature, as well as on randomly generated instances. The experiments do not only illustrate how the suggested method is able to identify optimal solutions of the test problem instances, but they also explain why already existing methods fail to do so. Furthermore, the computational results indicate that the proposed method, equipped with the newly-proposed discretization sets, is capable of efficiently generating a high percentage of optimal solutions to the corresponding problem with defects.

Divergence Behaviour of the Successive Geometric Mean Method of Pairwise Comparison Matrix Generation for a Multiple Stage, Multiple Objective Optimization Problem

This paper focuses on the divergence behaviour of the successive geometric mean (SGM) method used to generate pairwise comparison matrices while solving a multiple stage, multiple objective (MSMO) optimization problem. The SGM method can be used in the matrix generation phase of our three-phase methodology to obtain pairwise comparison matrix at each stage of an MSMO optimization problem, which can be subsequently used to obtain the weight vector at the corresponding stage. The weight vectors across the stages can be used to convert an MSMO problem into a multiple stage, single objective (MSSO) problem, which can be solved using dynamic programming-based approaches. To obtain a practical set of non-dominated solutions (also referred to as Pareto optimal solutions) to the MSMO optimization problem, it is important to use a solution approach that has the potential to allow for a better exploration of the Pareto optimal solution space. To accomplish a more exhaustive exploration of the Pareto optimal solution space, the weight vectors that are used to scalarize the MSMO optimization problem into its corresponding MSSO optimization problem should vary across the stages. Distinct weight vectors across the stages are tied directly with distinct pairwise comparison matrices across the stages. A pairwise comparison matrix generation method is said to diverge if it can generate distinct pairwise comparison matrices across the stages of an MSMO optimization problem. In this paper, we demonstrate the SGM method's divergence behaviour when the three-phase methodology is used in conjunction with an augmented high-dimensional, continuous-state stochastic dynamic programming method to solve a large-scale MSMO optimization problem. Copyright © 2013 John Wiley & Sons, Ltd.

Dynamic programming approaches to solve the shortest path problem with forbidden paths

In this paper, the shortest path problem with forbidden paths is addressed. The problem under consideration is formulated as a particular instance of the resource-constrained shortest path problem. Different versions of a dynamic programming-based solution approach are defined and implemented. The proposed algorithms can be viewed as an extension of the node selection approach proposed by Desrochers in 1988. An extensive computational test is carried out on a meaningful number of random instances with the purpose of assessing the behaviour of the developed solution approaches. A comparison with the state-of-the-art method proposed to address the problem under study is also made. The computational results are very encouraging and highlight that the proposed algorithms are very efficient.