Standard Dynamic Programming Algorithm Research Articles

Track-before-detect in distributed sensor applications

In this article, we propose a new extension to a Dynamic Programming Algorithm (DPA) approach for Track-before-Detect challenges. This extension enables the DPA to process time-delayed sensor data directly. Such delay might appear because of delays in communication networks. The extended DPA is identical to the recursive standard DPA in case of all sensor data appear in the timely correct order. Furthermore, an intense evaluation of the Accumulated State Density (ASD) filter is given on simulation data. Last but not least, we apply a combination of DPA and ASD on data of a real radar system and present the resulting tracks. Our experience concerning this combination is a seamless cooperation between the track initialization by DPA and a track maintenance by ASD filter.

STOCHASTIC MODEL OF LAKE SYSTEM INVASION AND ITS OPTIMAL CONTROL: NEURODYNAMIC PROGRAMMING AS A SOLUTION METHOD

Abstract We develop a stochastic model for the process of spread of an aquatic invader in a lake system. The invader is transported along with recreation boats, and treatment of boats allows one to control the invader spread. Optimization of invasion and control costs leads to a dynamic programming problem. However, standard stochastic dynamic programming (SDP) algorithms allow us to solve the problem for at most 13–14 lakes, which is significantly less than the size of systems arising in applications. We have developed a new technique, which is based upon the ideas of reinforcement learning and neurodynamic programming. This approach allows us to obtain approximate but reasonable control policy for essentially greater lake systems. The resulting numerical technique is comparatively easy to implement, and it can be applied to other spatially extended optimal control problems as well.

Partition function and base pairing probabilities of RNA heterodimers.

BackgroundRNA has been recognized as a key player in cellular regulation in recent years. In many cases, non-coding RNAs exert their function by binding to other nucleic acids, as in the case of microRNAs and snoRNAs. The specificity of these interactions derives from the stability of inter-molecular base pairing. The accurate computational treatment of RNA-RNA binding therefore lies at the heart of target prediction algorithms.MethodsThe standard dynamic programming algorithms for computing secondary structures of linear single-stranded RNA molecules are extended to the co-folding of two interacting RNAs.ResultsWe present a program, RNAcofold, that computes the hybridization energy and base pairing pattern of a pair of interacting RNA molecules. In contrast to earlier approaches, complex internal structures in both RNAs are fully taken into account. RNAcofold supports the calculation of the minimum energy structure and of a complete set of suboptimal structures in an energy band above the ground state. Furthermore, it provides an extension of McCaskill's partition function algorithm to compute base pairing probabilities, realistic interaction energies, and equilibrium concentrations of duplex structures.AvailabilityRNAcofold is distributed as part of the Vienna RNA Package, .ContactStephan H. Bernhart – berni@tbi.univie.ac.at

Cache efficient simple dynamic programming

New cache-oblivious and cache-aware algorithms for simple dynamic programming based on Valiant's context-free language recognition algorithm are designed, implemented, analyzed, and empirically evaluated with timing studies and cache simulations. The studies show that for large inputs the cache-oblivious and cache-aware dynamic programming algorithms are significantly faster than the standard dynamic programming algorithm.

Neural Network Based Algorithm for Dynamic System Optimization

ABSTRACTA class of artificial neural networks with a two‐layer feedback topology to solve nonlinear discrete dynamic optimization problems is developed. Generalized recurrent neuron models are introduced. A direct method to assign the weights of neural networks is presented. The method is based on Bellmann's Optimality Principle and on the interchange of information which occurs during the synaptic chemical processing among neurons. A comparative analysis of the computational requirements is made. The analysis shows advantages of this approach as compared to the standard dynamic programming algorithm. The technique has been applied to several important optimization problems, such as shortest path and control optimal problems.

Stochastic dynamic programming with factored representations

Markov decision processes (MDPs) have proven to be popular models for decision-theoretic planning, but standard dynamic programming algorithms for solving MDPs rely on explicit, state-based specifications and computations. To alleviate the combinatorial problems associated with such methods, we propose new representational and computational techniques for MDPs that exploit certain types of problem structure. We use dynamic Bayesian networks (with decision trees representing the local families of conditional probability distributions) to represent stochastic actions in an MDP, together with a decision-tree representation of rewards. Based on this representation, we develop versions of standard dynamic programming algorithms that directly manipulate decision-tree representations of policies and value functions. This generally obviates the need for state-by-state computation, aggregating states at the leaves of these trees and requiring computations only for each aggregate state. The key to these algorithms is a decision-theoretic generalization of classic regression analysis, in which we determine the features relevant to predicting expected value. We demonstrate the method empirically on several planning problems, showing significant savings for certain types of domains. We also identify certain classes of problems for which this technique fails to perform well and suggest extensions and related ideas that may prove useful in such circumstances. We also briefly describe an approximation scheme based on this approach.

A simple algorithm for detecting circular permutations in proteins.

Circular permutation of a protein is a genetic operation in which part of the C-terminal of the protein is moved to its N-terminal. Recently, it has been shown that proteins that undergo engineered circular permutations generally maintain their three dimensional structure and biological function. This observation raises the possibility that circular permutation has occurred in Nature during evolution. In this scenario a protein underwent circular permutation into another protein, thereafter both proteins further diverged by standard genetic operations. To study this possibility one needs an efficient algorithm that for a given pair of proteins can detect the underlying event of circular permutations. A possible formal description of the question is: given two sequences, find a circular permutation of one of them under which the edit distance between the proteins is minimal. A naive algorithm might take time proportional to N3 or even N4, which is prohibitively slow for a large-scale survey. A sophisticated algorithm that runs in asymptotic time of N2 was recently suggested, but it is not practical for a large-scale survey. A simple and efficient algorithm that runs in time N2 is presented. The algorithm is based on duplicating one of the two sequences, and then performing a modified version of the standard dynamic programming algorithm. While the algorithm is not guaranteed to find the optimal results, we present data that indicate that in practice the algorithm performs very well. A Fortran program that calculates the optimal edit distance under circular permutation is available upon request from the authors. ron@biocom1.ls.biu.ac.il.

Sequence to structure alignment in comparative modeling using PrISM

PrISM (Protein Informatics System for Modeling) is a protein analysis and modeling system in which informatics, alignment, modeling, and assessment modules are integrated in a computational environment where protein analysis and modeling protocols can be designed and assessed interactively. It can then be used automatically and repetitively in response to a variety of protein analysis and modeling problems. PrISM was used to predict a single model for each of the 43 targets in the CASP3 experiment. In this paper, we present results for 13 target sequences, which we consider to be comparative modeling targets with clearly related structural templates. We emphasize the problem of aligning a target sequence to a template structure with various alignment methods. When more than one alignment method and/or parameter set are applied, the final alignment is chosen on the basis of a model ranking system also used in PrISM's fold recognition module. Advanced sequence–template alignment procedures in PrISM are useful in some cases when standard pairwise dynamic programming algorithm fail to make any reasonable global alignment. The same procedures, however, failed in other cases, corresponding to remotely related query-template pairs that involved extensive insertions and deletions. Proteins Suppl 1999;3:66–72. © 1999 Wiley-Liss, Inc.

On Robust Dynamic Programming

Dynamic programming achieves optimum control for known deterministic and stochastic systems. There is a need, however, to apply dynamic programming ideas to real-world uncertain systems. The standard dynamic programming algorithm is ineffective in the case of shedding an optimal decision path due to uncertainty in modelling. A robust variation on dynamic programming in which a minimization operation is replaced by an averaging operation is proposed in this paper. A simple example to illustrate the effectiveness of the algorithm is presented.

A non-local gap-penalty for profile alignment

The length of an alignment of biological sequences is typically longer than the mean length of its component sequences. (This arises from the insertion of gaps in the alignment.) When such an alignment is used as a profile for the alignment of further sequences (or profiles), it will have a bias toward additional sequences that match the length of the profile, rather than the mean length of sequences in the profile, as the alignment of these will entail fewer (or smaller) insertions (so avoiding gap-penalties). An algorithm is described to correct this bias that entails monitoring the correspondence, for every pair of positions, of the mean separations in both profiles as they are aligned. The correction was incorporated into a standard dynamic programming algorithm through a modification of the gap-penalty, but, unlike other approaches, this modification is not local and takes into consideration the overall alignment of the sequences. This implies that the algorithm cannot guarantee to find the optimal alignment, but tests suggest that close approximations are obtained. The method was tested on protein families by measuring the area in the parameter space of the phase containing the correct multiple alignment. No improvement (increase in phase area) was found with a family that required few gaps to be aligned correctly. However, for highly gapped alignments, a 50% increase in area was obtained with one family and the correct alignment was found for another that could not be aligned with the unbiased method.

CASM: a VLSI chip for approximate string matching

The edit distance between two strings a1, ..., a/sub m/ and b/sub 1/, ..., b/sub n/ is the minimum cost s of a sequence of editing operations (insertions, deletions and substitutions) that convert one string into the other. This paper describes the design and implementation of a linear systolic array chip for computing the edit distance between two strings over a given alphabet. An encoding scheme is proposed which reduces the number of bits required to represent a state in the computation. The architecture is a parallel realization of the standard dynamic programming algorithm proposed by Wagner and Fischer (1974), and can perform approximate string matching for variable edit costs. More importantly, the architecture does not place any constraint on the lengths of the strings that can be compared. It makes use of simple basic cells and requires regular nearest neighbor communication, which makes it suitable for VLSI implementation. A prototype of this array has been built at the University of South Florida. >

Discovering simple DNA sequences by the algorithmic significance method

A new method, 'algorithmic significance', is proposed as a tool for discovery of patterns in DNA sequences. The main idea is that patterns can be discovered by finding ways to encode the observed data concisely. In this sense, the method can be viewed as a formal version of the Occam's Razor principle. In this paper the method is applied to discover significantly simple DNA sequences. We define DNA sequences to be simple if they contain repeated occurrences of certain 'words' and thus can be encoded in a small number of bits. Such definition includes minisatellites and microsatellites. A standard dynamic programming algorithm for data compression is applied to compute the minimal encoding lengths of sequences in linear time. An electronic mail server for identification of simple sequences based on the proposed method has been installed at the Internet address pythia/anl.gov.

Stochastic dynamic programming for reservoir optimal control: Dense discretization and inflow correlation assumption made possible by parallel computing

The solution via dynamic programming (DP) of a reservoir optimal control problem is often computationally prohibitive when the proper description of the inflow process leads to a system model having several state variables and/or when a sufficiently dense state discretization is required to achieve numerical accuracy. Thus, to simplify, the inflow correlation is usually neglected and/or a coarse state discretization is adopted. However, these simplifications may significantly affect the reliability of the solution of the optimization problem. Nowadays, the availability of very powerful computers based on innovative architectures (vector and parallel machines), even in the domain of personal computers (transputer architectures), stimulates the reformulation of the standard dynamic programming algorithm in a form able to exploit these new machine architectures. The reformulated DP algorithm and new machines enable faster and less costly solution of optimization problems involving a system model having two state variables (storage and previous period inflow, then taking into account the inflow correlation) and a number of states (of the order of 104) such as to guarantee a high numerical accuracy.

Utilization of graph theory and dynamic programming as substitutes for decision trees

A methodology is presented to select from two marketing or pricing plans with the objective of maximizing profit. The decision structure is portrayed in graphical form, with alternate stages of decision and uncertainty. A standard dynamic programming algorithm is used to solve the graph and aid in final decision making. A decision path is plotted at each decision vertex on the graph such that the decision maker can determine, at any combination of sales volume and time period, which of the two marketing plans to follow from that vertex so as to achieve the greatest expected profit for the remaining span of time.