Dynamic Programming-based Algorithms Research Articles

An FPGA Based Energy-Efficient Read Mapper With Parallel Filtering and In-Situ Verification.

In the assembly pipeline of Whole Genome Sequencing (WGS), read mapping is a widely used method to re-assemble the genome. It employs approximate string matching and dynamic programming-based algorithms on a large volume of data and associated structures, making it a computationally intensive process. Currently, the state-of-the-art data centers for genome sequencing incur substantial setup and energy costs for maintaining hardware, data storage and cooling systems. To enable low-cost genomics, we propose an energy-efficient architectural methodology for read mapping using a single system-on-chip (SoC) platform. The proposed methodology is based on the q-gram lemma and designed using a novel architecture for filtering and verification. The filtering algorithm is designed using a parallel sorted q-gram lemma based method for the first time, and it is complemented by an in-situ verification routine using parallel Myers bit-vector algorithm. We have implemented our design on the Zynq Ultrascale+ XCZU9EG MPSoC platform. It is then extensively validated using real genomic data to demonstrate up to 7.8× energy reduction and up to 13.3× less resource utilization when compared with the state-of-the-art software and hardware approaches.

Decision models for personal shopper platform operations optimization

The ‘lazy economy’ gives rise to an emerging business mode, called personal shopper platforms (PSPs). A customer who needs some goods urgently can release an order on a PSP, which is then assigned by the platform to a personal shopper, who will buy the goods at a nearby retail store and deliver them to the customer within a short time interval. Since the development of PSP is relatively new, the decision mechanisms and policies are at an early stage. The operations of the PSPs can be optimized through operations research methodologies. This study proposes a series of mixed integer programming (MIP) models and improved dynamic programming-based algorithms to support operational decisions on order assignment and shopper routing, as well as strategic decisions on the PSP mode adoption and territory planning. Some intuitive but practical criteria are also designed to accelerate the proposed algorithms so that they can be applied to large-scale realistic instances. The proposed algorithm can solve the basic case with 1000 orders and 1050 shoppers (about 107 variables and 107 constraints in the MIP models) in half a minute. A realistic case in the Changning district of Shanghai is also used to validate the effectiveness of the proposed models and the efficiency of the algorithms. An extended model for considering the uncertain arrival of future orders is also presented. This study provides a comprehensive model-driven decision methodology for this emerging service industry mode.

Linear-Time Algorithms for RNA Structure Prediction.

RNA secondary structure prediction is widely used to understand RNA function. Existing dynamic programming-based algorithms, both the classical minimum free energy (MFE) methods and partition function methods, suffer from a major limitation: their runtimes scale cubically with the RNA length, and this slowness limits their use in genome-wide applications. Inspired by incremental parsing for context-free grammars in computational linguistics, we designed linear-time heuristic algorithms, LinearFold and LinearPartition, to approximate the MFE structure, partition function and base pairing probabilities. These programs are orders of magnitude faster than Vienna RNAfold and CONTRAfold on long sequences. More interestingly, LinearFold and LinearPartition lead to more accurate predictions on the longest sequence families for which the structures are well established (16S and 23S Ribosomal RNAs), as well as improved accuracies for long-range base pairs (500 + nucleotides apart). This chapter provides protocols for using LinearFold and LinearPartition for secondary structure prediction.

Constrained Q-Learning for Batch Process Optimization

Chemical process optimization and control often require satisfaction of constraints for safe operation. Reinforcement learning (RL) has been shown to be a powerful control technique that can handle nonlinear stochastic optimal control problems. Despite this promise, RL has yet to see significant translation to industrial practice due to its inability to satisfy state constraints. This work aims to address this challenge. We propose an “oracle”-assisted constrained Q-learning algorithm that guarantees the satisfaction of joint chance constraints with high probability, which is required for safety critical tasks. To that end, constraint tightening (backoffs) are introduced, which are adjusted using Broyden’s method, hence making the backoffs self-tuned. This results in a general methodology that can be integrated into approximate dynamic programming-based algorithms to guarantee constraint satisfaction with high probability. Finally, a case study is presented to compare the performance of the proposed approach with that of model predictive control (MPC). The superior performance of the proposed algorithm, in terms of constraint handling, signifies a step toward the use of RL in real world optimization and control of systems, where constraints are critical in ensuring safety.

LCSS-Based Algorithm for Computing Multivariate Data Set Similarity: A Case Study of Real-Time WSN Data.

Multivariate data sets are common in various application areas, such as wireless sensor networks (WSNs) and DNA analysis. A robust mechanism is required to compute their similarity indexes regardless of the environment and problem domain. This study describes the usefulness of a non-metric-based approach (i.e., longest common subsequence) in computing similarity indexes. Several non-metric-based algorithms are available in the literature, the most robust and reliable one is the dynamic programming-based technique. However, dynamic programming-based techniques are considered inefficient, particularly in the context of multivariate data sets. Furthermore, the classical approaches are not powerful enough in scenarios with multivariate data sets, sensor data or when the similarity indexes are extremely high or low. To address this issue, we propose an efficient algorithm to measure the similarity indexes of multivariate data sets using a non-metric-based methodology. The proposed algorithm performs exceptionally well on numerous multivariate data sets compared with the classical dynamic programming-based algorithms. The performance of the algorithms is evaluated on the basis of several benchmark data sets and a dynamic multivariate data set, which is obtained from a WSN deployed in the Ghulam Ishaq Khan (GIK) Institute of Engineering Sciences and Technology. Our evaluation suggests that the proposed algorithm can be approximately 39.9% more efficient than its counterparts for various data sets in terms of computational time.

Narrowband Internet of Things Systems With Opportunistic D2D Communication

Narrowband Internet of Things (NB-IoT) is a new cellular technology introduced by the third generation partnership (3GPP) providing low-power and wide-area coverage for IoT. In this paper, we consider the scenario that NB-IoT is deployed in an heterogeneous network and the quality of the direct link from the NB-IoT user equipment (UE) to the serving base station (BS) cannot satisfy the quality of service requirement for transmission of vital sensing data. Thereupon, device-to-device (D2D) communication is adopted as a routing extension to NB-IoT systems, and thus, enables two-hop routes between NB-IoT UE and the serving BS via a set of D2D relays. As the candidate UE relays work in duty cycle to save energy, we derive a model to select a set of UE relays and perform opportunistic D2D communication according to a working schedule. Two optimization problems are formulated aiming at achieving optimal expected delivery ratio (EDR) and expected two-hop delay, respectively. Dynamic programming-based algorithms are proposed to solve the optimization problems and obtain the optimal working schedule of the relays. Simulation results demonstrate that our proposed maxEDR and minEED algorithms improves the system performance compared with other state-of-the-art algorithms.

Trajectory Mining Using Uncertain Sensor Data

Trajectory mining is an interesting data mining problem. Traditionally, it is either assumed that the time-ordered location data recorded as trajectories are either deterministic or that the uncertainty, e.g., due to equipment or technological limitations, is removed by incorporating some pre-processing routines. Thus, the trajectories are processed as deterministic paths of mobile object location data. However, it is important to understand that the transformation from uncertain to deterministic trajectory data may result in the loss of information about the level of confidence in the recorded events. Probabilistic databases offer ways to model uncertainties using possible world semantics. In this paper, we consider uncertain sensor data and transform this to probabilistic trajectory data using pre-processing routines. Next, we model this data as tuple level uncertain data and propose dynamic programming-based algorithms to mine interesting trajectories. A comprehensive empirical study is performed to evaluate the effectiveness of the approach. The results show that the trajectories could be modeled and worked as probabilistic data and that the results could be computed efficiently using dynamic programming.

Flexible Packed Stencil Design With Multiple Shaping Apertures and Overlapping Shots for E-beam Lithography

Electron-beam (e-beam) lithography has long been employed for mask writing but the write time is increasing due to the escalating mask pattern complexity. Besides, e-beam direct write is being pursued as an alternative solution for chip production in the sub-22 nm regime. To improve the throughput of e-beam lithography, character projection method is commonly employed and a critical problem is to pack as many useful characters as possible onto the stencil. In this paper, we consider three enhancements in packed stencil design over previous works. First, the fact that the pattern of a character can be located anywhere within its enclosing projection region is exploited to facilitate flexible blank space sharing. Second, the use of multiple shaping apertures with different sizes is explored. Third, the use of overlapping shots for printing some characters is investigated. For the packed stencil design problem with flexible blank space sharing and multiple shaping apertures, two dynamic programming-based algorithms are proposed, one allows overlapping shots and the other does not. Experimental results show that the proposed enhancement and the associated algorithms can significantly reduce the total shot count and hence improve the throughput of e-beam lithography.

An Efficient Scheduling Technique for Biological Sequence Alignment

Computing alignment matrix score to search for regions of homology between biological sequences is time consuming task. This is due to the recursive nature of the dynamic programming-based algorithms such as the Smith-Waterman and the Needleman-Wunsch algorithmns. Typical FPGA-based protein sequencer comprises of two main logic blocks. One for computing alignment scores i.e. the processing element (PE), while another logic block for configuring the PE with coefficients. During alignment matrix computation, the logic block for configuring the PE are left unused until the time consuming alignment matrix computation finished. Therefore, a new technique, known as overlap computation and configuration (OCC) is proposed to minimize the time overhead for performing biological sequence alignment. The OCC technique simultaneously updating substitution matrix in a processing element (PE) systolic array, while computing alignment matrix scores. Results showed that, the sequencer achieves more than two order of magnitude speed-up higher compared to the state of the art, at negligible area overhead, if any.

Efficient interscanline consistency enforcing method for dynamic programming-based dense stereo matching algorithms

The dynamic programming-based dense stereo matching algorithms are faster than other global ones, but they usually suffer from the tangible horizontal “streaks” in their generated disparity maps. This problem takes place due to the independent processing of scanlines, ignoring vertical consistency between them. In this paper the path remembering method, which uses the information of the best paths of previous rows, is employed with new developments to establish more depth similarity between rows, especially along the slanted edges of objects in the scene. Also, a new way for estimating matching costs in the left border of the image is presented based on the information of the right-hand neighboring pixels. Mentioned improvements were applied to a well known and standard dynamic programming-based algorithm called “DP.” Evaluation of the new proposed algorithm on standard stereo images shows considerable improvements in the depth estimation along the slanted edges and in the left border of the final results. As a more precise report, the average error rate reduces from 14.2% to 10.4%.

Fast algorithms for finding a minimum repetition representation of strings and trees

A string with many repetitions can be represented compactly by replacing h-fold contiguous repetitions of a string r with (r)h. We present a compact representation, which we call a repetition representation (of a string) or RRS, by which a set of disjoint or nested tandem arrays can be compacted. In this paper, we study the problem of finding a minimum RRS or MRRS, where the size of an RRS is defined by the sum of the length of component letters and the description length of the component repetitions (⋅)h which is defined by wR(h) using a repetition weight function wR. We develop two dynamic programming-based algorithms to solve this problem: CMR, which works for any type of wR, and CMR-C, which is faster but can be applied to a constant wR only. CMR-C is an O(n2logn)-time O(nlogn)-space algorithm, which is more efficient in both time and space than CMR by a ((logn)/n)-factor, where n is the length of the given string. The problem of finding an MRRS for a string can be extended to that of finding a minimum repetition representation (of a tree) or MRRT for a given labeled ordered tree. For this problem, we present two algorithms, CMRT and CMRT-C, by using CMR and CMR-C, respectively, as a subroutine. As well as the theoretical analysis, we confirmed the efficiency of the proposed algorithms by experiments, which consist of the following three parts: First we demonstrated that CMR-C and CMRT-C are fast enough for large-scale data by using synthetic strings and trees, respectively. The size of an MRRS for a given string can be a measure of how compactly the string can be represented, meaning how well the string is structurally organized. This is also true of trees. To check such ability of MRRS-size, second we measured the size of an MRRS for chromosomes of nine different species. We found that all the chromosomes of the same species have a similar compression rate when realized by an MRRS. Run length encoding (RLE) was also shown to have species-specific compression rate, but species were separated more clearly by MRRS than by RLE. Third we examined the size of an MRRT for web pages of world-leading companies by using the tag trees, showing a consistency between the compression rate by an MRRT and visual web page structures.

Template-based program verification and program synthesis

Program verification is the task of automatically generating proofs for a program’s compliance with a given specification. Program synthesis is the task of automatically generating a program that meets a given specification. Both program verification and program synthesis can be viewed as search problems, for proofs and programs, respectively. For these search problems, we present approaches based on user-provided insights in the form of templates. Templates are hints about the syntactic forms of the invariants and programs, and help guide the search for solutions. We show how to reduce the template-based search problem to satisfiability solving, which permits the use of off-the-shelf solvers to efficiently explore the search space. Template-based approaches have allowed us to verify and synthesize programs outside the abilities of previous verifiers and synthesizers. Our approach can verify and synthesize difficult algorithmic textbook programs (e.g., sorting and dynamic programming-based algorithms) and difficult arithmetic programs.

Finding Anchors for Genomic Sequence Comparison

Recent sequencing of the human and other mammalian genomes has brought about the necessity to align them, to identify and characterize their commonalities and differences. Programs that align whole genomes generally use a seed-and-extend technique, starting from exact or near-exact matches and selecting a reliable subset of these, called anchors, and then filling in the remaining portions between the anchors using a combination of local and global alignment algorithms, but their choices for the parameters so far have been primarily heuristic. We present a statistical framework and practical methods for selecting a set of matches that is both sensitive and specific and can constitute a reliable set of anchors for a one-to-one mapping of two genomes from which a whole-genome alignment can be built. Starting from exact matches, we introduce a novel per-base repeat annotation, the Z-score, from which noise and repeat filtering conditions are explored. Dynamic programming-based chaining algorithms are also evaluated as context-based filters. We apply the methods described here to the comparison of two progressive assemblies of the human genome, NCBI build 28 and build 34 (www.genome.ucsc.edu), and show that a significant portion of the two genomes can be found in selected exact matches, with very limited amount of sequence duplication.

An integrated logical and physical design flow for deep submicron circuits

This paper presents a set of techniques and a new design flow to be used in the synthesis of high-performance deep-submicron logic circuits. The design flow consists of circuit partitioning into tree like clusters, floorplanning, global routing, and timing analysis/budgeting steps, followed by simultaneous technology mapping and linear placement of each cluster. The strength of this approach lies in the dynamic programming-based algorithms used in performing simultaneous technology mapping and linear placement of the logic clusters. The two algorithms we propose, one for exact total (gate plus routing) area minimization and the other for total (gate plus routing) delay minimization, generate a set of noninferior solutions for each cluster enabling designers to perform tradeoffs between total-area and total-delay. Experimental results on large benchmarks prove the effectiveness of the proposed flow.

An Objective-Based Framework for Motion Planning under Sensing and Control Uncertainties

The authors consider the problem of determining robot motion plans under sensing and control uncertainties. Traditional approaches are often based on methodology known as preimage planning, which involves worst-case analysis. The authors have developed a gen eral framework for determining feedback strategies by blending ideas from stochastic optimal control and dynamic game theory with traditional preimage planning concepts. This generalizes classical preimages to performance preimages and preimage planning for de signing motion strategies with information feedback. For a given problem, one can define a performance criterion that evaluates any executed trajectory of the robot. The authors present methods for selecting a motion strategy that optimizes this criterion under either nondeterministic uncertainty (resulting in worst-case analysis) or probabilistic uncertainty (resulting in expected-case analysis). The authors present dynamic programming-based algorithms that nu merically compute performance preimages and optimal strategies; several computed examples of forward projections, performance preimages, and optimal strategies are presented.

Efficient Algorithms for Scheduling Semiconductor Burn-In Operations

In this paper, we study the problem of scheduling semiconductor burn-in operations, where burn-in ovens are modeled as batch processing machines. A batch processing machine is one that can process up to B jobs simultaneously. The processing time of a batch is equal to the largest processing time among all jobs in the batch. We present efficient dynamic programming-based algorithms for minimizing a number of different performance measures on a single batch processing machine. We also present heuristics for a number of problems concerning parallel identical batch processing machines and we provide worst case error bounds.