Dynamic Programming Solution Research Articles

Joint relay and jammer selection in Internet of Things systems

In this article, we consider the physical layer security issue in Internet of Things systems, in which there exist a sensing transceiver pair, a number of candidate nodes, and an eavesdropper. The transceiver pair needs to select a jammer node and a relay node among the candidate nodes so as to preserve the secrecy of the communications. Considering the diversity of candidate channels and the limited available power, it is infeasible to scan all the nodes and find the optimal one. We formulate this jammer and relay selection problem as an optimal stopping problem under a fixed sensing order. Then, through applying dynamic programming solution, we propose a low-complexity approach to obtain the optimal sensing order. The performance of the proposed selection scheme is evaluated through numerical results.

AJSR: an Efficient Multiple Jumps Forwarding Scheme in Software-Defined WAN

Software Defined Networks (SDN) is an emerging network architecture. It is shown that SDN can be deployed in a variety of small size networks, such as personal area networks and local area networks. Nevertheless, the deployment of SDN in large scale networks, such as wide area networks, raises a lot of concerns. In these scenarios, SDN relies on traditional hop-by-hop forwarding scheme which may lead to performance degradation due to heavy control traffic incurred and flow tables over-consumption. Source routing can be utilized to reduce control traffic and flow table consumption by installing only one flow table entry in the ingress switch and encoding complete routing path of a particular flow in the packet header. However, it will lead to severe bandwidth overhead which may also degrade network performance. In this paper, we propose an efficient forwarding scheme called Arbitrary Jump Source Routing (AJSR), which makes use of MPLS-based source routing. AJSR aims to achieve a trade-off between the control traffic overhead and the bandwidth overhead by dividing the complete routing path of a particular flow into arbitrary length sections and distributing these sections at different switches along the flow’s routing path. To find the most suitable flow entry placement scheme, we formulate the flow entry placement problem as an optimization problem and present a dynamic programming solution with the objective of maximizing the overall cost saving. Through simulation experiments, we find that AJSR can effectively achieve a trade-off between the control traffic overhead and the bandwidth overhead.

Constrained energy-efficient routing in time-aware road networks

Routing problems find applications in many location-based services. Existing works which answer route queries mainly focus on static road networks rather than time-aware road networks. Observe the fact that (i) the same road segment may be driven with different speeds during different time intervals, and (ii) users usually prefer driving on a route that consumes minimum energy within a travel time budget. Motivated by this, this paper proposes the Constrained Energy-Efficient Time-Aware Routing problem, denoted as CEETAR. We take time factor into consideration and utilize a time-aware speed model and a time-aware polynomial energy cost model. To solve CEETAR, we propose a dynamic programming solution, and then propose an approximation algorithm which uses the branch and bound, and scaling strategy with provable approximation bounds to answer the query of CEETAR in real-world dense road networks. In addition, we also propose a greedy route planning algorithm. Experimental results demonstrate that our approximation algorithms can efficiently answer CEETAR queries with high accuracy.

Velocity profile optimization of on road vehicles: Pontryagin's Maximum Principle based approach

Driving profile of on road vehicles has shown to have significant effect on fuel economy. This paper discusses the development of Pontryagin's Maximum Principle (PMP) based solution to determine the energy optimal velocity profile by incorporating the gear shifting, speed limit and road grade constraints simultaneously. In the proposed approach the real world road grade profile and speed limits are approximated by a set of piece-wise constant functions and the corresponding first order necessary conditions are derived. By solving a number of differential equations an analytical solution is generated. Therefore, the computation time of the solution is extremely fast. To verify the global optimality of the solution, the results are compared with dynamic programming (DP) solution that solves the complex and non-linear representative model of the actual test vehicle. The comparison results prove that the generated optimal speed trajectories are very close to global optimal solution.

Cost-efficient traffic-aware data collection protocol in VANET

Data from running vehicles are invaluable to numerous ITS and urban computing applications. This paper studies the issue of collecting data from multiple vehicles to a roadside base station via vehicle-to-vehicle and vehicle-to-infrastructure communications. Existing data collection approaches in VANET have mainly focused on the network problems, such as packet loss, network clustering or data aggregation, but the impact of real-time traffic condition is barely considered. In this paper, we investigate the data collection problem in VANETs under rapid evolving traffic conditions. Our approach can adaptively choose to carry or forward the data packet, based on current traffic information. The objective is to minimize the network communication overhead while satisfying the data collection time constraint. We formulate the data collection problem as a scheduling optimization problem and prove it is NP-complete. An optimal dynamic programming solution and a genetic algorithm based heuristic solution are developed to solve the problem under different application scenarios. Extensive evaluations validate that our proposed solution outperforms some existing ones in terms of effectiveness and efficiency.

Study of HEV Power Management Control Strategy Based on Driving Pattern Recognition

In this work, an optimized HEV power management fuzzy control strategy is proposed with the aim to further improve the fuel efficiency of the rule-based control strategy and overcome the drawbacks of the conventional control strategies. The driving pattern recognition method is used to classify the driving condition into one of the driving patterns to select proper control algorithm. The dynamic programming solution is used to design the fuzzy control strategies for each driving pattern. The simulation results indicate that by adopting the proposed strategy the fuel efficiency of HEV is improved, especially under complex driving conditions.

Finding optimal solutions for vehicle routing problem with pickup and delivery services with time windows: A dynamic programming approach based on state–space–time network representations

Optimization of on-demand transportation systems and ride-sharing services involves solving a class of complex vehicle routing problems with pickup and delivery with time windows (VRPPDTW). This paper first proposes a new time-discretized multi-commodity network flow model for the VRPPDTW based on the integration of vehicles’ carrying states within space–time transportation networks, so as to allow a joint optimization of passenger-to-vehicle assignment and turn-by-turn routing in congested transportation networks. Our three-dimensional state–space–time network construct is able to comprehensively enumerate possible transportation states at any given time along vehicle space–time paths, and further allows a forward dynamic programming solution algorithm to solve the single vehicle VRPPDTW problem. By utilizing a Lagrangian relaxation approach, the primal multi-vehicle routing problem is decomposed to a sequence of single vehicle routing sub-problems, with Lagrangian multipliers for individual passengers’ requests being updated by sub-gradient-based algorithms. We further discuss a number of search space reduction strategies and test our algorithms, implemented through a specialized program in C++, on medium-scale and large-scale transportation networks, namely the Chicago sketch and Phoenix regional networks.

Performance Analysis and Optimization of Distributed Workflows in Heterogeneous Network Environments

Large-scale e-science features complex DAG-structured workflows comprised of computing modules with intricate inter-module dependencies. Mapping such workflows in heterogeneous network environments and optimizing their end-to-end performance are crucial to the success of scientific collaborations that require fast system response and smooth data flow. We construct analytical cost models and formulate workflow mapping as optimization problems for minimum end-to-end delay and maximum frame rate. The difficulty of these problems essentially arises from the topological matching nature in the spatial domain, which is further compounded by the resource sharing complicacy in the temporal dimension. For unitary processing applications, we develop a workflow mapping algorithm based on a recursive critical path optimization procedure to minimize the latency; while for streaming applications, we conduct a rigorous workflow stability analysis and develop a layer-oriented dynamic programming solution based on topological sorting to identify and minimize the global bottleneck time. The accuracy of the proposed exact delay calculation algorithm is verified in comparison with an approximate solution, a dynamic distributed system simulation program, and a real network deployment, and the performance superiority of the proposed mapping approaches are illustrated by extensive simulation-based comparisons with existing algorithms and verified by large-scale experiments on real-life scientific workflows through effective system implementation and deployment in real networks.

Gap Filling as Exact Path Length Problem

One of the last steps in a genome assembly project is filling the gaps between consecutive contigs in the scaffolds. This problem can be naturally stated as finding an s-t path in a directed graph whose sum of arc costs belongs to a given range (the estimate on the gap length). Here s and t are any two contigs flanking a gap. This problem is known to be NP-hard in general. Here we derive a simpler dynamic programming solution than already known, pseudo-polynomial in the maximum value of the input range. We implemented various practical optimizations to it, and compared our exact gap-filling solution experimentally to popular gap-filling tools. Summing over all the bacterial assemblies considered in our experiments, we can in total fill 76% more gaps than the best previous tool, and the gaps filled by our method span 136% more sequence. Furthermore, the error level of the newly introduced sequence is comparable to that of the previous tools. The experiments also show that our exact approach does not easily scale to larger genomes, where the problem is in general difficult for all tools.

Risk minimising strategies for revenue management problems with target values

Consider a risk-averse decision maker in the setting of a single-leg dynamic revenue management problem with revenue controlled by limiting capacity for a fixed set of prices. Instead of focussing on maximising the expected revenue, the decision maker has the main objective of minimising the risk of failing to achieve a given target revenue. Interpreting the revenue management problem in the framework of finite Markov decision processes, we augment the state space of the risk-neutral problem definition and change the objective function to the probability of failing a certain specified target revenue. This enables us to obtain a dynamic programming solution that generates the policy minimising the risk of not attaining this target revenue. We compare this solution with recently proposed risk-sensitive policies in a numerical study and discuss advantages and limitations.

Ground Delay Program Planning Using Markov Decision Processes

This paper compares three approaches for selecting planned airport acceptance rates in the single-airport ground-holding problem: the Ball et al. model, the Richetta–Odoni dynamic model, and an approach based on approximate dynamic programming. Selecting planned airport acceptance rates is motivated by current practice of ground delay program planning under collaborative decision making. The approaches were evaluated using real flight schedules and landing capacity data from Newark Liberty International and San Francisco International Airports. It is shown that planned airport acceptance rates can be determined from the decision variables of the Richetta–Odoni dynamic model. The approximate dynamic programming solution, introduced by the authors, is found by posing a model that evaluates planned airport acceptance as a Markov decision process. The dynamic Richetta–Odoni and approximate dynamic programming approaches were found to produce similar solutions, and both dominated the Ball et al. model. The Richetta–Odoni dynamic model is computationally more efficient, finding solutions 10 times faster than approximate dynamic programming, although the approximate dynamic programming approach can more easily incorporate complex objectives. Surprisingly, the performance of all three approaches did not change significantly when evaluated using different models for collaborative decision-making procedures. This observation suggests that modeling collaborative decision making may not be important for selecting near-optimal planned airport acceptance rates.

Resource-Aware Mobile-Based Health Monitoring.

Monitoring heart diseases often requires frequent measurements of electrocardiogram (ECG) signals at different periods of the day, and at different situations (e.g., traveling, and exercising). This can only be implemented using mobile devices in order to cope with mobility of patients under monitoring, thus supporting continuous monitoring practices. However, these devices are energy-aware, have limited computing resources (e.g., CPU speed and memory), and might lose network connectivity, which makes it very challenging to maintain a continuity of the monitoring episode. In this paper, we propose a mobile monitoring solution to cope with these challenges by compromising on the fly resources availability, battery level, and network intermittence. In order to solve this problem, first we divide the whole process into several subtasks such that each subtask can be executed sequentially either in the server or in the mobile or in parallel in both devices. Then, we developed a mathematical model that considers all the constraints and finds a dynamic programing solution to obtain the best execution path (i.e., which substep should be done where). The solution guarantees an optimum execution time, while considering device battery availability, execution and transmission time, and network availability. We conducted a series of experiments to evaluate our proposed approach using some key monitoring tasks starting from preprocessing to classification and prediction. The results we have obtained proved that our approach gives the best (lowest) running time for any combination of factors including processing speed, input size, and network bandwidth. Compared to several greedy but nonoptimal solutions, the execution time of our approach was at least 10 times faster and consumed 90% less energy.

EDISON-WMW: Exact Dynamic Programing Solution of the Wilcoxon–Mann–Whitney Test

In many research disciplines, hypothesis tests are applied to evaluate whether findings are statistically significant or could be explained by chance. The Wilcoxon–Mann–Whitney (WMW) test is among the most popular hypothesis tests in medicine and life science to analyze if two groups of samples are equally distributed. This nonparametric statistical homogeneity test is commonly applied in molecular diagnosis. Generally, the solution of the WMW test takes a high combinatorial effort for large sample cohorts containing a significant number of ties. Hence, P value is frequently approximated by a normal distribution. We developed EDISON-WMW, a new approach to calculate the exact permutation of the two-tailed unpaired WMW test without any corrections required and allowing for ties. The method relies on dynamic programing to solve the combinatorial problem of the WMW test efficiently. Beyond a straightforward implementation of the algorithm, we presented different optimization strategies and developed a parallel solution. Using our program, the exact P value for large cohorts containing more than 1000 samples with ties can be calculated within minutes. We demonstrate the performance of this novel approach on randomly-generated data, benchmark it against 13 other commonly-applied approaches and moreover evaluate molecular biomarkers for lung carcinoma and chronic obstructive pulmonary disease (COPD). We found that approximated P values were generally higher than the exact solution provided by EDISON-WMW. Importantly, the algorithm can also be applied to high-throughput omics datasets, where hundreds or thousands of features are included. To provide easy access to the multi-threaded version of EDISON-WMW, a web-based solution of our algorithm is freely available at http://www.ccb.uni-saarland.de/software/wtest/.

On the generalized bin packing problem

AbstractThe generalized bin packing problem (GBPP) is a novel packing problem arising in many transportation and logistic settings, characterized by multiple items and bins attributes and the presence of both compulsory and non‐compulsory items. In this paper, we study the computational complexity and the approximability of the GBPP. We prove that the GBPP cannot be approximated by any constant, unless . We also study the particular case of a single bin type and show that when an unlimited number of bins is available, the GBPP can be reduced to the bin packing with rejection (BPR) problem, which is approximable. We also prove that the GBPP satisfies Bellman's optimality principle and, exploiting this result, we develop a dynamic programming solution approach. Finally, we study the behavior of standard and widespread heuristics such as the first fit, best fit, first fit decreasing, and best fit decreasing. We show that while they successfully approximate previous versions of bin packing problems, they fail to approximate the GBPP.

Active set solver for min-max robust control with state and input constraints

This paper proposes an online active set strategy for computing the dynamic programming solution to a min-max robust optimal control problem with quadratic H1 stage cost for linear systems with linear state and input constraints in the presence of bounded disturbances. The solver determines the optimal active constraint set for a given plant state using an iterative procedure which computes the optimal sequence of feedback laws for a candidate active set and updates the active set by performing a line search in state space. The computational complexity of each iteration depends linearly on the length of the prediction horizon. The main contribution of the paper is its treatment of degeneracy caused by linearly dependent state and input constraints and its efficient handling is a crucial step in formulating the active set algorithm. The proposed approach ensures the continuity of optimal control laws along the line-of-search, thus enabling an efficient solution method based on homotopy. Conditions for global optimality are given and the convergence of the active set solver is established using the geometric properties of an associated multi-parametric programming problem. A receding horizon control strategy is proposed, which ensures a specified l2-gain from the disturbance input to the state and control inputs in the presence of linearly dependent constraints

Optimal Operation of Energy Storage with Random Renewable Generation and AC/DC Loads

We study the optimal operation of energy storage owned by a consumer who has intermittent renewable generation and faces time-varying (random) electricity prices and different types of ac/dc loads. We formulate the optimal storage operation problem as a finite-horizon dynamic program, with an objective of minimizing the expected energy cost. The incorporation of different types of ac/dc energy sources and appliances complicates the sequential decision making problem on storage operation. We provide a complete characterization on an optimal threshold policy, and implement the characterized optimal policy in realistic settings with random renewable generation and electricity prices. Numerical results demonstrate that the optimal dynamic programming (DP) solution saves 5%–12% of the total cost resulting from conventional DP solutions (that ignore the difference in ac/dc conversion efficiencies). Further, the value of storage (the consumer’s net benefit obtained by optimally operating the storage) increases with renewable generation. For example, as the (average) solar generation increases from 40% to 100% of the average (total) demand, the value of a 32 kWh (80 kWh) storage increases from 2.0% to 5.4% (7.2% to 14.9%, respectively) of the maximum energy cost.

Information Space Receding Horizon Control for Multisensor Tasking Problems.

In this paper, we present a receding horizon solution to the problem of optimal scheduling for multiple sensors monitoring a group of dynamical targets. The term target is used here in the classic sense of being the object that is being sensed or observed by the sensors. This problem is motivated by the space situational awareness (SSA) problem. The multisensor optimal scheduling problem can be posed as a multiagent Markov decision process on the information space which has a dynamic programming (DP) solution. We present a simulation-based stochastic optimization technique that exploits the structure inherent in the problem to obtain variance reduction along with a distributed solution. This stochastic optimization technique is combined with a receding horizon approach which uses online solution of the control problems to obviate the need to solve the computationally intractable multiagent information space DP problem and hence, makes the technique computationally tractable. The technique is tested on a moderate scale SSA example which is nonetheless computationally intractable for existing solution techniques.

Mining Frequent Itemsets in Correlated Uncertain Databases

Recently, with the growing popularity of Internet of Things (IoT) and pervasive computing, a large amount of uncertain data, e.g., RFID data, sensor data, real-time video data, has been collected. As one of the most fundamental issues of uncertain data mining, uncertain frequent pattern mining has attracted much attention in database and data mining communities. Although there have been some solutions for uncertain frequent pattern mining, most of them assume that the data is independent, which is not true in most real-world scenarios. Therefore, current methods that are based on the independent assumption may generate inaccurate results for correlated uncertain data. In this paper, we focus on the problem of mining frequent itemsets over correlated uncertain data, where correlation can exist in any pair of uncertain data objects (transactions). We propose a novel probabilistic model, called Correlated Frequent Probability model (CFP model) to represent the probability distribution of support in a given correlated uncertain dataset. Based on the distribution of support derived from the CFP model, we observe that some probabilistic frequent itemsets are only frequent in several transactions with high positive correlation. In particular, the itemsets, which are global probabilistic frequent, have more significance in eliminating the influence of the existing noise and correlation in data. In order to reduce redundant frequent itemsets, we further propose a new type of patterns, called global probabilistic frequent itemsets, to identify itemsets that are always frequent in each group of transactions if the whole correlated uncertain database is divided into disjoint groups based on their correlation. To speed up the mining process, we also design a dynamic programming solution, as well as two pruning and bounding techniques. Extensive experiments on both real and synthetic datasets verify the effectiveness and efficiency of the proposed model and algorithms.

Operations scheduling under electricity time-varying prices

Global climate change requires stakeholders to consider energy elements in their decision-making. Electricity costs, in particular, constitute a significant portion of operational costs in most manufacturing systems. The electricity bills can be lowered if electricity-consuming operations are correctly scheduled. We consider a manufacturing operations control problem with known time-varying electricity prices in a finite planning horizon. Each operation is unique and has its own concave electricity consumption function. Pre-emptions of operations are allowed, yet postponing an operation incurs a cumulative penalty for each time period. In addition, each pre-emption is considered a new operation. The electricity cost in each time period is exogenous and there exists a capacity constraint on the total electricity amount consumed in each period due to infrastructure and provider’s limitations. There is a fixed start-up cost incurred for switching on the machine and a fixed reservation cost incurred for keeping the machine ‘On’. The system also includes a rechargeable battery. The customer has to determine when to process each operation within the time horizon so as to minimise total electricity consumption and operations postponement penalty costs. A dynamic programming solution is proposed and the complexity of the models is analysed. After examining several special cases of the model, the optimum times to charge and discharge the rechargeable battery are determined. A polynomial time algorithm for a special case of a single operation with uniform capacity is proposed.

Adaptive Search and Tracking of Sparse Dynamic Targets Under Resource Constraints

This paper considers the problem of resource-constrained and noise-limited localization and estimation of dynamic targets that are sparsely distributed over a large area. We generalize an existing framework [Bashan , 2008] for adaptive allocation of sensing resources to the dynamic case, accounting for time-varying target behavior such as transitions to neighboring cells and varying amplitudes over a potentially long time horizon. The proposed adaptive sensing policy is driven by minimization of a surrogate function for mean squared error within locations containing targets. We provide theoretical upper bounds on the performance of adaptive sensing policies by analyzing solutions with oracle knowledge of target locations, gaining insight into the effect of target motion and amplitude variation as well as sparsity. Exact minimization of the multistage objective function is infeasible, but myopic optimization yields a closed-form solution. We propose a simple non-myopic extension, the Dynamic Adaptive Resource Allocation Policy (D-ARAP), that allocates a fraction of resources for exploring all locations rather than solely exploiting the current belief state. Our numerical studies indicate that D-ARAP has the following advantages: (a) it is more robust than the myopic policy to noise, missing data, and model mismatch; (b) it performs comparably to well-known approximate dynamic programming solutions but at significantly lower computational complexity; and (c) it improves greatly upon nonadaptive uniform resource allocation in terms of estimation error and probability of detection.