Optimal Dynamic Programming Algorithm Research Articles

Optimized Energy Management Control of a Hybrid Electric Locomotive

Hybrid electric propulsion, using batteries for energy storage, is making significant inroads into railway transportation because of its potential for notable fuel savings and the related reductions in greenhouse gases emissions of hybrid railway traction over non-electrified railway lines. Due to the inherent complexity of hybridized powertrains, combining different power conversions and energy storage capabilities, the corresponding operation of their energy management needs to be precisely optimized in order to achieve the minimum possible fuel consumption. Having this in mind, this paper proposes a real-time energy management control strategy for a diesel–electric hybrid locomotive based on the optimization results obtained by means of a dynamic programming optimization algorithm aimed at fuel consumption minimization while honoring the battery state-of-charge constraints and powertrain physical constraints. The final optimization result, expressed in terms of the optimal battery state-of-charge reference (target), is used as an additional input into the state-of-charge controller within the real-time energy management system. The subsequent simulation analysis shows clear fuel economy improvement with 22.9% of fuel savings obtained for the locomotive featuring a hybrid powertrain equipped with batteries over the conventional one.

Channel Allocation for Mega LEO Satellite Constellations in the MEO–LEO Networked Telemetry System

Recently, numerous mega LEO satellite constellation plans have emerged as an indispensable part supporting the future 6G satellite communications. Since the traditional telemetry systems are inappropriate for monitoring the operation status of all satellites, a networked telemetry system is adopted to achieve full time, low delay telemetry for mega LEO satellite constellations, which is a significant scenario of satellite Internet of Things. Furthermore, this article formulates a channel allocation problem to maximize the overall transmitted data amount. Adopting the dual decomposition method, this problem can be decomposed and transformed into a dual problem and multiple path scheduling subproblems for each LEO satellite. This article develops an optimal dynamic programming algorithm to solve the path scheduling subproblems and an iterative algorithm to solve the channel allocation problem. Numerical simulations show that the proposed channel allocation algorithm improves the transmitted data amount and approximates the upper bound performance.

Collaborative Energy Beamforming for Wireless Powered Fog Computing Networks

Beam-based wireless power transfer and Fog/edge computing are promising dual technologies for realizing wireless powered Fog computing networks to support the upcoming B5G/6G IoT applications, which require latency-aware and intensive computing, with a limited energy supply. In such systems, IoT devices can either offload their computing tasks to the proximal Fog nodes or execute local computing with replenishing energy from the dedicated beamforming. However, effective integration of these techniques is still challenging, where two new issues arise: energy-aware task offloading and signal interferences from spillovers of wireless beamforming. In this paper, we observe that the beam-ripple phenomenon, which takes advantage of beamformer defects to transfer energy to IoT devices, is the key to jointly addressing these two issues. Different from traditional SWIPT technology, as in our approach the stream is not separately divided into data/energy streams, but target IoT devices can potentially harvest the whole stream. Inspired by this phenomenon, we treat the collaborative energy beamforming and edge computing design as a strongly <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {NP}$ </tex-math></inline-formula> -hard optimization problem. The proposed solution is an iterative algorithm to cascadingly integrate a polynomial-time <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\left({1 - \frac {1}{e}}\right)$ </tex-math></inline-formula> -approximation algorithm, which achieves the theoretical upper bound in approximation ratio unless <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathcal {P} = \mathcal {NP}$ </tex-math></inline-formula> , and an optimal dynamic programming algorithm. The numerical results show that the energy minimization goal among IoT devices can achieve, and the developed harvest-when-interfered protocol is practical in the wireless powered Fog computing networks.

Shift Strategy Optimization for Automatic Transmission of Heavy Trucks Based on Dynamic Programming Algorithm

The shift strategy of automatic transmission plays a vital role in the smoothness and economy of heavy-duty mining trucks. In this paper, an engine fuel consumption model, a 6 + 2 speed automatic transmission model, and a vehicle resistance model are built in MATLAB. Combined with the development of smart mining technology, the traditional two-parameter economic shift strategy is corrected based on vehicle load and road slope. The dynamic programming optimization algorithm is used to extract the best economical shift strategy under known working conditions to reduce the fuel consumption and the number of shifts. Finally, simulation experiments of the optimized shift strategy by dynamic programming in a typical mine work cycle are carried out. The simulation results show that the engine speed and output torque are maintained in a relatively stable and efficient working range by using the proposed shift strategy. Compared with the traditional two-parameter shift strategy, dynamic programming has advantages in reducing fuel consumption and shift numbers.

Application Conditions of Bounded Rationality and a Microgrid Energy Management Control Strategy Combining Real-Time Power Price and Demand-Side Response

Microgrid energy management is a typical discrete non-linear optimization problem that is usually solved by off-line optimization, day-ahead demand-side management, and long-term dynamic optimization scheduling strategy. However, due to the intermittent distributed generation and time-varying load in microgrids, more attention should be paid to the real-time optimal scheduling of the overall operation of energy to ensure the dynamic balance of supply and demand in microgrids. Combining demand-side response with real-time power price, this paper applies the strategy to microgrid energy management and proposes a distributed energy real-time management model of microgrid based on demand-side response function. A deep adaptive dynamic programming optimization algorithm is also proposed for the model. The real-time interaction between microgrid operators and users is realized. The closed-loop feedback control structure of the proposed model ensures the real-time optimization control strategy. Therefore, the proposed energy management model and control strategy can realize intra-day dispatching in microgrids. The real-time performance and effectiveness of the proposed energy management model and control strategy are also verified by numerical simulation. Finally, since the proposed model is approximate, whether the solution obtained by the algorithm is the optimal or satisfactory solution of the optimization strategy set is a lack of theoretical support. Therefore, according to the approximation theorem of bounded rationality, the application conditions of the model in power markets are proposed. It is proved that the proposed model meets the application conditions, and is a specific application of bounded rationality approaching complete rationality in the power market. It is also proved that the best solution is involved in the satisfactory solution set of the model. Thus, the control strategy is a rational and feasible optimal management control strategy, which provides a theoretical basis for its further implementation.

Development, analysis and application of a predictive controller to a small-scale district heating system

District heating and cooling networks have great potential for energy saving, efficient thermal energy distribution and renewable energy source integration. Currently, heating systems are managed on the basis of operator experience or by using adaptive controllers, however these solutions are not suitable when there are remarkable variations in boundary conditions. In this context, Model Predictive Control is a promising strategy as it optimizes control based on the prediction of the future behavior of system dynamics and disturbances by means of simplified models. This paper presents the development of a predictive controller based on a novel Dynamic Programming optimization algorithm and aimed to supply the thermal energy to entire buildings within district heating networks. The controller is exploited to operate the district heating network of a school complex in a simulation environment (i.e. Model-in-the-Loop). Each branch connected to the network is optimized by a dedicated controller according to a multi-agent strategy. The performance of the innovative controller is compared to the results obtained by using a conventional PID controller. Conservative results show that, with the innovative controller, a reduction in fuel consumption of up to more than 7% is obtained together with up to 5 h of avoided failures of the indoor comfort constraints, depending on the season. Overall, the Model-based Predictive Controller is able to fulfill comfort requirements adequately while minimizing energy consumption. Moreover, the multi-agent approach allows these results to be extended to larger networks in future studies.

QoE-Aware Computation Offloading to Capture Energy-Latency-Pricing Tradeoff in Mobile Clouds

Computation offloading in mobile clouds helps mobile users save energy and enhance performance via mobile-to-cloud migration of processing. Although there exist many approaches to computation offloading, they have not explicitly considered the energy-latency-pricing tradeoff from the viewpoint of mobile users’ context, e.g., user tendency, the remaining battery level. This paper tries to capture the user-centric perspective via quality-of-experience (QoE), and formulates two important problems: mobile-to-cloud transmission scheduling of the offloaded task's data and offloading service class selection. Regarding transmission scheduling, we introduce a database-assisted optimal dynamic programming (DP) algorithm and then propose two suboptimal but computationally-efficient approximate DP algorithms, ADP and ADPe, based on the limited lookahead technique. Regarding service class selection, we consider multiple service classes with different computing power and service charge, and formulate an optimization problem to minimize the overall cost incurred during offloading. An extensive numerical analysis has revealed that ADP and ADPe achieve near-optimal performance incurring only 0.35 and 2.1 percent extra cost than the optimum on average, and enhances the QoE-aware cost by up to 2.38 times compared to the energy-only scheduling. In addition, our service class selection algorithm is shown to choose the best class according to user tendency and the remaining battery level.

Integrated Scheduling of Production and Two-Stage Delivery of Make-to-Order Products: Offline and Online Algorithms

We study integrated production- and delivery-scheduling problems that arise in practical make-to-order settings in several industries. In these problems, make-to-order products are first processed in a plant and then delivered to customer sites through two stages of shipping: first, from the plant to a pool point (e.g., a port, a distribution, or a consolidation center) and, second, from the pool point to customer sites. The objective is to obtain a joint schedule of job processing at the plant and two-stage shipping of completed jobs to customer sites to optimize a performance measure that takes into account both delivery timeliness and total transportation costs. We consider two problems in which delivery timeliness is measured by total or maximum lead time of the jobs and study both offline and online versions of these problems. For the offline problems involving a single production line at the plant, we provide optimal dynamic programming algorithms. For the more general offline problems involving multiple production lines at the plant, we propose fast heuristics and analyze their worst-case and asymptotic performances. For the online problems, we propose online algorithms and analyze their competitive ratios. By comparing our offline heuristics with lower bounds using randomly generated test instances, it is shown that these heuristics are capable of generating near-optimal solutions quickly. Using real data from Baosteel’s Meishan plant, we also show that our corresponding offline heuristic generates significantly better solutions than Baosteel’s rule-based approach. In addition, our computational results on the performance of the online algorithms relative to the offline heuristics generate important methodological insights that can be used by practitioners in choosing a specific solution approach.

Numerical analyses of a hydraulic hybrid powertrain system for a transit bus

A dynamic programming optimization algorithm has been applied on a transit bus model in MATLAB in order to assess the fuel economy improvement potential by implementing a hydraulic hybrid powertrain system. The numerical model parameters have been calibrated using experimental data obtained on a Belgrade?s public transport bus. This experiment also provided the representative driving cycle on which to conduct simulation analyses. Various functional parameters of a hydraulic hybrid system have been evaluated for obtaining the best possible fuel economy. Dynamic programming optimization runs have been completed for various hydraulic accumulator sizes, preload values and accumulator foam quantities. It has been shown that a fuel economy improvement of 28% can be achieved by implementing such a system.

Development of a Model-based Predictive Controller for a heat distribution network

Research studies concerning heating and cooling systems have increased in recent years, pledging great potential for energy saving, efficient thermal energy distribution and renewable energy source integration. Currently, heating systems are managed on the basis of operator experience or by using adaptive controllers, however these solutions are not suitable when there are remarkable variations in boundary conditions (e.g. weather changes). In this context, Model Predictive Control is a promising strategy as it optimizes the control action based on the prediction of the future behavior of both system dynamics and disturbances by means of simplified models. This paper presents the control of a building heating system through a Model-in-the-Loop Model Predictive Control approach. A detailed model that replicates the behavior of the real system is controlled with a predictive controller based on a novel Dynamic Programming optimization algorithm implemented in Matlab®. The performance of the innovative controller is compared to the results obtained with a PID controller. Overall, the Model Predictive Control strategy is able to fulfill comfort requirements properly while minimizing energy consumption.

Optimizing Retention-Aware Caching in Vehicular Networks

Caching is an effective way to address the challenges due to explosive data traffic growth and massive device connectivity in fifth-generation (5G) networks. Currently, few works on caching pay attention to the impact of the time duration for which content is stored, called retention time, on caching optimization. The research on retention time is motivated by two practical issues, i.e., flash memory damage and storage rental cost in cloud networks, together giving rise to the storage cost. How to optimize caching contents taking the storage cost into consideration is a challenging problem, especially for the scenarios with cache-enabled mobile nodes. In this paper, a retention-aware caching problem (RACP) in vehicular networks is formulated, considering the impact of the storage cost. The problem’s complexity analysis is provided. For symmetric cases, an optimal dynamic programming (DP) algorithm with polynomial time complexity is derived. For general cases, a low complexity and effective retention aware multi-helper caching algorithm (RAMA) is proposed. Numerical results are used to verify the effectiveness of the algorithms.

Cost allocation in rescheduling with machine unavailable period

We study a rescheduling problem faced by multiple job owners sharing a single machine, where jobs need to be rescheduled when the machine becomes unavailable for a period of time. The disruption caused by any new schedule is restricted such that the difference between the completion times in the initial and the new schedules of any job is no more than a given threshold. A natural way to reschedule is to process the jobs in the initial sequence, each as early as possible. This defines a feasible schedule over which cost saving can potentially be achieved by optimal rescheduling as long as the cost saving can be fairly shared by job owners. We define a cooperative game for job owners accordingly, to share the cost saving. Given that the optimization problem is computationally intractable, we find several optimal properties and develop an optimal pseudopolynomial time dynamic programming algorithm for rescheduling. We provide a simple closed form core allocation of the total cost saving for all the jobs, and also provide the Shapley value of the game in a computable form. Then we computationally evaluate the extra cost caused by machine unavailability, the cost saving from optimization relative to the naturally constructed schedule, the likelihood for the Shapley value to be a core allocation, and how the Shapley value allocates cost saving among job owners. Managerial insights are derived from the computational studies. This work contributes to the literature by explicitly incorporating two classic scheduling topics: sequencing game and rescheduling.

Mobility-Aware Caching in D2D Networks

Caching at mobile devices can facilitate device-to-device (D2D) communications, which may significantly improve spectrum efficiency and alleviate the heavy burden on backhaul links. However, most previous works ignored user mobility, thus having limited practical applications. In this paper, we take advantage of the user mobility pattern by the inter-contact times between different users, and propose a mobility-aware caching placement strategy to maximize the data offloading ratio, which is defined as the percentage of the requested data that can be delivered via D2D links rather than through base stations (BSs). Given the NP-hard caching placement problem, we first propose an optimal dynamic programming (DP) algorithm to obtain a performance benchmark, but with much lower complexity than exhaustive search. We will then prove that the problem falls in the category of monotone submodular maximization over a matroid constraint, and propose a time-efficient greedy algorithm, which achieves an approximation ratio as 1/2. Simulation results with real-life data-sets will validate the effectiveness of our proposed mobility-aware caching placement strategy. We observe that users moving at either a very low or very high speed should cache the most popular files, while users moving at a medium speed should cache less popular files to avoid duplication.

Long-term assessment of economic plug-in hybrid electric vehicle battery lifetime degradation management through near optimal fuel cell load sharing

This work evaluates the performance of a plug-in hybrid electric vehicle (PHEV) energy management process that relies on the active management of the degradation of its energy carriers – in this scenario, a lithium-ion battery pack and a polymer electrolyte membrane fuel cell (PEMFC) – to produce a near economically-optimal vehicle operating profile over its entire useful lifetime. This solution is obtained through experimentally-supported PHEV models exploited by an optimal discrete dynamic programming (DDP) algorithm designed to efficiently process vehicle usage cycles over an extended timescale. Our results demonstrate the economic and component lifetime gains afforded by our strategy when compared with alternative rule-based PHEV energy management benchmarks.

Versatile Size-&lt;formula formulatype="inline"&gt;&lt;tex Notation="TeX"&gt;$l$&lt;/tex&gt;&lt;/formula&gt; Object Summaries for Relational Keyword Search

The Object Summary (OS)is a recently proposed tree structure, which summarizes all data held in a relational database about a data subject. An OS can potentially be very large in size and therefore unfriendly for users who wish to view synoptic information about the data subject. In this paper, we investigate the effective and efficient retrieval of concise and informative OS snippets (denoted as size-l OSs). We propose and investigate the effectiveness of two types of size- l OSs, namely size- l OS (t)s and size-l OS (a)s that consist of l tuple nodes and l attribute nodes respectively. For computing size-l OSs, we propose an optimal dynamic programming algorithm, two greedy algorithms and preprocessing heuristics. By collecting feedback from real users (e.g., from DBLP authors), we assess the relative usability of the two different types of snippets, the choice of the size- l parameter, as well as the effectiveness of the snippets with respect to the user expectations. In addition, via thorough evaluation on real databases, we test the speed and effectiveness of our techniques.

An Optimal Approximate Dynamic Programming Algorithm for Concave, Scalar Storage Problems With Vector-Valued Controls

We prove convergence of an approximate dynamic programming algorithm for a class of high-dimensional stochastic control problems linked by a scalar storage device, given a technical condition. Our problem is motivated by the problem of optimizing energy flows for a power grid supported by grid-level storage. The problem is formulated as a stochastic, dynamic program, where we estimate the value of resources in storage using a piecewise linear value function approximation. Given the technical condition, we provide a rigorous convergence proof for an approximate dynamic programming algorithm, which can capture the presence of both the amount of energy held in storage as well as other exogenous variables. Our algorithm exploits the natural concavity of the problem to avoid any need for explicit exploration policies.

An integrated guaranteed- and stochastic-service approach to inventory optimization in supply chains

Multi-echelon inventory optimization literature distinguishes stochastic- (SS) and guaranteed-service (GS) approaches as mutually exclusive frameworks. While the GS approach considers flexibility measures at the stages to deal with stockouts, the SS approach only relies on safety stock. Within a supply chain, flexibility levels might differ between stages rendering them appropriate candidates for one approach or the other. The existing approaches, however, require the selection of a single framework for the entire supply chain instead of a stage-wise choice. We develop an integrated hybrid-service (HS) approach which endogenously determines the overall cost-optimal approach for each stage and computes the required inventory levels. We present a dynamic programming optimization algorithm for serial supply chains that partitions the entire system into subchains of different types. From a numerical study we find that, besides implicitly choosing the better of the two pure frameworks, whose cost differences can be considerable, the HS approach enables additional pipeline and on-hand stock cost savings. We further identify drivers for the preferability of the HS approach.

Energy-Efficient, Heterogeneous Sensor Selection for Physical Activity Detection in Wireless Body Area Networks

In this paper, the problem of efficient operation of an energy-constrained, heterogeneous Wireless Body Area Network (WBAN) to optimize an activity detection application is addressed. WBANs constitute a new class of wireless sensor networks that enable diverse applications in healthcare, entertainment, sports, military and emergency situations. A typical WBAN consists of a few, heterogeneous sensors wirelessly coupled to an energy-constrained fusion center which, according to observations of a real-world prototype WBAN, imposes critical restrictions on system lifetime. To address this issue, a novel stochastic control framework is introduced, which considers both sensor heterogeneity and application requirements, for achieving the two-fold goal: energy savings with satisfactory detection performance. An optimal dynamic programming algorithm for the sensor selection problem is also derived. Important properties of the cost functionals are derived and used to design three approximation algorithms, which offer near optimal performance with significant complexity reduction. Simulations on real-world data show energy gains as high as 68% in comparison to an equal allocation scheme with probability of detection error on the order of 10-4.

Optimizing the Reliability of Streaming Applications Under Throughput Constraints

Mapping a pipelined application onto a distributed and parallel platform is a challenging problem. The problem becomes even more difficult when multiple optimization criteria are involved, and when the target resources are heterogeneous (processors and communication links) and subject to failures. This paper investigates the problem of mapping pipelined applications, consisting of a linear chain of stages executed in a pipeline way, onto such platforms. The objective is to optimize the reliability under a performance constraint, i.e., while guaranteeing a threshold throughput. In order to increase reliability, we replicate the execution of stages on multiple processors. We compare interval mappings, where the application is partitioned into intervals of consecutive stages, with general mappings, where stages may be partitioned without any constraint, thereby allowing a better usage of processors and communication network capabilities. However, the price to pay for general mappings is a dramatic increase in the problem complexity. We show that computing the period of a given general mapping is an NP-complete problem, and we give polynomial bounds to determine a (conservative) approximated value. On the contrary, the period of an interval mapping obeys a simple formula, and we provide an optimal dynamic programming algorithm for the bi-criteria interval mapping problem on homogeneous platforms. On the more practical side, we design a set of efficient heuristics, and we compare the performance of interval and general mapping strategies through extensive simulations.

FINDING MANY OPTIMAL PATHS WITHOUT GROWING ANY OPTIMAL PATH TREES

Many algorithms for applications such as pattern recognition, computer vision, and computer graphics seek to compute actual optimal paths in weighted directed graphs. The standard approach for reporting an actual optimal path is based on building a single-source optimal path tree. A technique by Chen et al.2 was given for a class of problems such that a single actual optimal path can be reported without maintaining any single-source optimal path tree, thus significantly reducing the space bound of those problems with no or little increase in their running time. In this paper, we extend the technique by Chen et al.2 to the generalized problem of reporting many actual optimal paths with different starting and ending vertices in certain directed graphs, and show how this new technique yields improved results on several application problems, such as reconstructing a 3-D surface band bounded by two simple closed curves, finding various constrained segmentation of 2-D medical images, and circular string-to-string correction. We also correct an error in the time/space complexity for the well-cited circular string-to-string correction algorithm12 and give an improved result for this problem. Although the generalized many-path problem seems more difficult than the single-path cases, our algorithms have nearly the same space and time bounds as those of the single-path cases. Our technique is likely to help improve many other optimal paths or dynamic programming algorithms.