Dynamic Control Policy Research Articles

Admission and Termination Control of a Two Class Loss System

We consider dynamic admission and termination control policies in a Markovian loss system with two classes, each with a fixed reward, a termination cost, an arrival and service rate. The system may admit or reject an arriving job or admit it by terminating a job in the system to maximize its total expected discounted reward. We prove that (1) when there is an idle server, it is never optimal to terminate a job, (2) there exists an optimal threshold policy for both admission and termination decisions. Furthermore, we identify the conditions which ensure that a class is “preferred” or “strongly preferred.”

Security Extensible Access Control Markup Language Policy Integration Based on Role-Based Access Control Model in Healthcare Collaborative Environments

Recently research is focused on security policy integration and conflict reconciliation among various healthcare organizations. Problem statement: However, challenging security and privacy risk issues still arisen during sharing sensitive patient data in different large distributed organizations. Though eXtensible Access Control Markup Language (XACML) has a powerful capacity of expression, it does not support all the elements character of RBAC. Thus, it has not been built to manage security in large distributed systems in healthcare domain since each organization may join or leave at runtime. The policy redundancy and conflict resolution are important to resolve redundancy and inconsistencies before security policies can be integrated for healthcare collaboration. Existing approaches did not look at policy redundancy and conflict resolution process based on the types of redundancy and conflict for dynamic set of organizations collaboration. Besides that, a policy integration mechanism in order to generate actual security policy integration is not in well studied. Approach: In this study, we proposed an approach for integrating security XACML policies based on RBAC policy model considering both constraints and meta data information. Besides that, an approach to filter and collect only the required policies from different organizations based on user’s integration requirements is investigated. It is important for us to resolve policy redundancy and conflicts based on the types of policy redundancy and conflicts. Results: From the observation and literature analysis, it can be concluded that our work could provide the maximum confidence for pre-compile a large amount of policies and only return the most similar policies for policy integration. Besides that, our approach proved that the more restrict policy will be generated during the policy integration. Conclusion: Our work can guarantee the completeness as well as consistency of the access control policy. It is recommended that the dynamic constraints such as dynamic Separation Of Duty (SOD) should be considered because we believe this consideration can support dynamic updates and control policies in collaborative environments.

Dynamic nonpoint-source pollution control policy: Ambient transfers and uncertainty

When a regulator cannot observe or infer individual emissions, corrective policy must rely on ambient pollution data. Assuming this kind of environment, we study a class of differential games of pollution control with profit functions that are polynomial in the global pollution stock. Given an open-loop emissions strategy satisfying mild regularity conditions, an ambient transfer scheme is exhibited that induces it in Markov-perfect equilibrium (MPE). Proposed transfers are a polynomial function of the difference between actual and desired pollution levels; moreover, they are designed so that in MPE no tax or subsidy is ever levied. Their applicability under stochastic pollution dynamics is studied for a symmetric game of polluting oligopolists with linear demand. We discuss a quadratic scheme that induces agents to adopt Markovian emission strategies that are stationary and linearly decreasing in total pollution. Total expected ambient transfers are non-positive and their magnitude is linearly increasing in physical volatility, the size of the economy, and the absolute value of the slope of the inverse demand function. However, if the regulator is interested in inducing a constant emissions strategy then, in expectation, transfers vanish. The total expected ambient transfer is compared to its point-source equivalent.

Real time production improvement through bottleneck control

Variability is a key characteristic for evaluating the performance of a process. Small variability for a bottleneck machine can generate high production variability. Short-term production analysis and bottleneck identification are imperative for enabling optimal response to dynamic changes within the system. In comparison to the rich and abundant literature available on long-term analysis, only a small section of the literature addresses the dynamic bottleneck control policies, which may be used to maximise sustainable benefits. In this paper, a real time bottleneck control method is introduced to efficiently utilise the finite manufacturing resources and to mitigate the short-term production constraints by using two practical approaches: initial buffer adjustment and maintenance task prioritisation. The objective for real time bottleneck control is to obtain a continuous production improvement towards a balanced-line status to increase the throughput efficiently. The benefits of this method are presented by considering an industrial case study of an automotive assembly line. The results obtained from this case study show significant production improvements as compared to traditional approaches.

Self-adaptive admission control policies for resource-sharing systems

We consider the problem of admission control in resource sharing systems, such as web servers and transaction processing systems, when the job size distribution has high variability, with the aim of minimizing the mean response time. It is well known that in such resource sharing systems, as the number of tasks concurrently sharing the resource is increased, the server throughput initially increases, due to more efficient utilization of resources, but starts falling beyond a certain point, due to resource contention and thrashing. Most admission control mechanisms solve this problem by imposing a fixed upper bound on the number of concurrent transactions allowed into the system, called the Multi-Programming-Limit (MPL), and making the arrivals which find the server full queue up. Almost always, the MPL is chosen to be the point that maximizes server efficiency. In this paper we abstract such resource sharing systems as a Processor Sharing (PS) server with state-dependent service rate and a First-Come-First-Served (FCFS) queue, and we analyze the performance of this model from a queueing theoretic perspective. We start by showing that, counter to the common wisdom, the peak efficiency point is not always optimal for minimizing the mean response time. Instead, significant performance gains can be obtained by running the system at less than the peak efficiency. We provide a simple expression for the static MPL that achieves near-optimal mean response time for general distributions. Next we present two traffic-oblivious dynamic admission control policies that adjust the MPL based on the instantaneous queue length while also taking into account the variability of the job size distribution. The structure of our admission control policies is a mixture of fluid control when the number of jobs in the system is high, with a stochastic component when the system is near-empty. We show via simulations that our dynamic policies are much more robust to unknown traffic intensities and burstiness in the arrival process than imposing a static MPL.

DYNAMIC PRICING AND INVENTORY CONTROL FOR A PRODUCTION SYSTEM WITH AVERAGE PROFIT CRITERION

In this article we study the joint optimization of finished goods inventory and pricing in a make-to-stock production system with long-run average profit criterion. The production time is random with controllable rate and the demand is Markovian with rate depending on the sale price. The objective is to dynamically adjust the production rate and the sale price to maximize the long-run average profit. We obtain the optimal dynamic pricing and production control policy and present an efficient bisection algorithm for computing the policy parameters.

Critically loaded queueing models that are throughput suboptimal

This paper introduces and analyzes the notion of throughput suboptimality for many-server queueing systems in heavy traffic. The queueing model under consideration has multiple customer classes, indexed by a finite set $\mathcal{I}$, and heterogenous, exponential servers. Servers are dynamically chosen to serve customers, and buffers are available for customers waiting to be served. The arrival rates and the number of servers are scaled up in such a way that the processes representing the number of class-i customers in the system, $i\in\mathcal{I}$, fluctuate about a static fluid model, that is assumed to be critically loaded in a standard sense. At the same time, the fluid model is assumed to be throughput suboptimal. Roughly, this means that the servers can be allocated so as to achieve a total processing rate that is greater than the total arrival rate. We show that there exists a dynamic control policy for the queueing model that is efficient in the following strong sense: Under this policy, for every finite T, the measure of the set of times prior to T, at which at least one customer is in the buffer, converges to zero in probability as the arrival rates and number of servers go to infinity. On the way to prove our main result, we provide a characterization of throughput suboptimality in terms of properties of the buffer-station graph.

Dynamic re-order point inventory control with lead-time uncertainty: analysis and empirical investigation

A new forecast-based dynamic inventory control approach is discussed in this paper. In this approach, forecasts and forecast uncertainties are assumed to be exogenous data known in advance at each period over a fixed horizon. The control parameters are derived by using a sequential procedure. The merits of this approach as compared to the classical one are presented. We focus on a single-stage and single-item inventory system with non-stationary demand and lead-time uncertainty. A dynamic re-order point control policy is analysed based on the new approach and its parameters are determined for a given target cycle service level (CSL). The performance of this policy is assessed by means of empirical experimentation on a large demand data set from the pharmaceutical industry. The empirical results demonstrate the benefits arising from using such a policy and allow insights to be gained into other pertinent managerial issues.

Dynamic versus static control policies in single stage production-inventory systems

In this paper, two approaches of production-inventory control are considered: the future requirements-based approach and the inventory consumption-based approach. We focus on a single-stage and single-item inventory system under non-stationary demand, information about which is provided by means of forecasts. Forecasts and forecast uncertainties are assumed to be exogenous data known in advance at each period over a forecasting horizon. Three forecast-based dynamic inventory control policies are analysed. These dynamic policies are extensions of the standard static re-order point and order-up-to-level policies that have been developed under the stationary demand assumption. The policies’ parameters are derived by using a sequential optimization procedure. A numerical investigation is conducted based on theoretical generated data; the results demonstrate the benefit of using the dynamic policies when compared with the static ones. Connections between the analysed policies are also highlighted.

Optimal Resource Allocation for Multimedia Applications over Multiaccess Fading Channels

We study the problem of optimal resource allocation for multi-user multiaccess wireless video communication from an information-theoretic point of view. We derive the optimal resource allocation policies by directly maximizing at the application layer the weighted sum of video qualities of all users, subject to information-theoretic multiaccess capacity region constraints in the MAC-PHY layers. We solve this problem for three multiaccess capacity regions: (1) non-fading channel, (2) fading channel with a given power control policy, and (3) fading channel with dynamic power control policies. The optimal resource allocation policy is referred as largest quality improvement highest possible rate (LQIHPR). We propose simple greedy algorithms to implement this policy. Since the capacity region is the fundamental characterization of achievable rates, the solutions developed in this paper provide the operational upper bound of achievable video quality in a multiaccess fading channel.

On the Choice-Based Linear Programming Model for Network Revenue Management

Gallego et al. [Gallego, G., G. Iyengar, R. Phillips, A. Dubey. 2004. Managing flexible products on a network. CORC Technical Report TR-2004-01, Department of Industrial Engineering and Operations Research, Columbia University, New York.] recently proposed a choice-based deterministic linear programming model (CDLP) for network revenue management (RM) that parallels the widely used deterministic linear programming (DLP) model. While they focused on analyzing “flexible products”—a situation in which the provider has the flexibility of using a collection of products (e.g., different flight times and/or itineraries) to serve the same market demand (e.g., an origin-destination connection)—their approach has broader implications for understanding choice-based RM on a network. In this paper, we explore the implications in detail. Specifically, we characterize optimal offer sets (sets of available network products) by extending to the network case a notion of “efficiency” developed by Talluri and van Ryzin [Talluri, K. T., G. J. van Ryzin. 2004. Revenue management under a general discrete choice model of consumer behavior. Management Sci. 50 15–33.] for the single-leg, choice-based RM problem. We show that, asymptotically, as demand and capacity are scaled up, only these efficient sets are used in an optimal policy. This analysis suggests that efficiency is a potentially useful approach for identifying “good” offer sets on networks, as it is in the case of single-leg problems. Second, we propose a practical decomposition heuristic for converting the static CDLP solution into a dynamic control policy. The heuristic is quite similar to the familiar displacement-adjusted virtual nesting (DAVN) approximation used in traditional network RM, and it significantly improves on the performance of the static LP solution. We illustrate the heuristic on several numerical examples.

An observer based switching control strategy for the stabilization of uncertain multi-input–multi-output plants

The paper proposes a stabilizing dynamic output switching control policy, based on variable structure control, which extends recently published results to a wider class of systems, and further improves the controller performances in terms of transient duration. It is worthy of note that the proposed observer based variable structure controller, exploiting the features of the coupled switching control, is based on less restrictive conditions with respect to those traditionally assumed. The proposed controller has been tested by simulation on IFAC benchmark problem no. #90-09.

Energy-delay efficient data memory subsystems: suitable for embedded media "processing"

This article presents a novel special-purpose data memory subsystem, called Xtream-Fit, suitable for embedded media processing, and demonstrates how it achieves high energy-delay efficiency across a wide range of media devices, including systems concurrently executing multiple applications under synchronization constraints. Experimental results show that Xtream-Fit delivers a substantial improvement in energy-delay product, as compared to general-purpose memory subsystems enhanced with state of-the-art cache decay and SDRAM dynamic power mode control policies. Xtreams-Fit's performance is predicted on a novel, task-based execution model that enhances opportunities for efficient stream granularity prefetching and aggressive software-based energy conservation techniques.

A multi-agent reinforcement learning approach to obtaining dynamic control policies for stochastic lot scheduling problem

This paper presents a methodology that, for the problem of scheduling of a single server on multiple products, finds a dynamic control policy via intelligent agents. The dynamic (state dependent) policy optimizes a cost function based on the WIP inventory, the backorder penalty costs and the setup costs, while meeting the productivity constraints for the products. The methodology uses a simulation optimization technique called Reinforcement Learning (RL) and was tested on a stochastic lot-scheduling problem (SELSP) having a state–action space of size 1.8 × 10 7. The dynamic policies obtained through the RL-based approach outperformed various cyclic policies. The RL approach was implemented via a multi-agent control architecture where a decision agent was assigned to each of the products. A Neural Network based approach (least mean square (LMS) algorithm) was used to approximate the reinforcement value function during the implementation of the RL-based methodology. Finally, the dynamic control policy over the large state space was extracted from the reinforcement values using a commercially available tree classifier tool.

Optimal Power and Retransmission Control Policies for Random Access Systems

We consider in this study dynamic control policies for slotted Aloha random access systems. New performance bounds are derived when random access is combined with power control for system optimization, and we establish the existence of optimal control approaches for such systems. We analyze throughput and delay when the number of backlogged users is known, where we can explicitly obtain optimal policies and analyze their corresponding performance using Markov Decision Process (MDP) theory with average cost criterion. For the realistic unknown-backlog case, we establish the existence of optimal backlog-minimizing policies for the same range of arrival rates as the ideal known-backlog case by using the theory of MDPs with Borel state space and unbounded costs. We also propose suboptimal control policies with performance close to the optimal without sacrificing stability. These policies perform substantially better than existing Certainty Equivalence controllers.

Dynamic optimal control policy in advertising price and quality

Dynamic optimal control policy in advertising price and quality

Intelligent dynamic control policies for serial production lines

Heuristic production control policies such as CONWIP, kanban, and other hybrid policies have been in use for years as better alternatives to MRP-based push control policies. It is a fact that these policies, although efficient, are far from optimal. Our goal is to develop a methodology that, for a given system, finds a dynamic control policy via intelligent agents. Such a policy while achieving the productivity (i.e., demand service rate) goal of the system will optimize a cost/reward function based on the WIP inventory. To achieve this goal we applied a simulation-based optimization technique called Reinforcement Learning (RL) on a four-station serial line. The control policy attained by the application of a RL algorithm was compared with the other existing policies on the basis of total average WIP and average cost of WIP. We also develop a heuristic control policy in light of our experience gained from a close examination of the policies obtained by the RL algorithm. This heuristic policy named Behavior-Based Control (BBC), although placed second to the RL policy, proved to be a more efficient and leaner control policy than most of the existing policies in the literature. The performance of the BBC policy was found to be comparable to the Extended Kanban Control System (EKCS), which as per our experimentation, turned out to be the best of the existing policies. The numerical results used for comparison purposes were obtained from a four-station serial line with two different (constant and Poisson) demand arrival processes.

A stochastic control model and dynamic control policy for bribery involving a dynamic pay

Aimed at preventing bribery a stochastic control model and a dynamic control policy is proposed in which a prescription is set to challenge root causes of bribery. In DS ε Nρ, defining D as a Public department of S bounded by a finite public official of size Nρ it assumes that in the time interval (t, t + ∂t) of a month in the year Y the chance for each public official to expose bribery is (bρp -aρ)δt, where bρ (bρ > aρ) is defined to be each public official’s dynamic pay a month in the year Y needs to challenge basic needs of life, aρ is each public official’s inadequate pay a month in the year Y, which creates a stage for receiving bribe, bρ -aρ = λρ (λρ > 0) is called bribery force attached with each public official, a root cause of public official’s receiving, and ρ (ρ =1, 2, 3, … , 10) is the grade of public officials. The quantity P(λρ/xρ/t) is called intensity of bribery which represents the distribution of the total size of public officials xρ(xρ < Np) who have caught bribery at time t(t> 0) of a month in the year Y. The intensity can be a useful guide in determining policy aimed at preventing bribery. The quantity, P(λρ/xρ/t)→ 0 is called zero intensity which can be used as indicator of control The quantity of interest P(λρ/xρ/t) is derived from the formula where f(z, t) is called probability generating function of ρ(λρ/xρ/t), Z is called giver in circulation, and Z =0 is called a necessary condition for the termination of bribery. A policy to get Z = 0 and ρ(λρ/xρ/t) → 0 are concluded to gain control

Intelligent dynamic control policies for serial production lines

Heuristic production control policies such as CONWIP, kanban, and other hybrid policies have been in use for years as better alternatives to MRP-based push control policies. It is a fact that these policies, although efficient, are far from optimal. Our goal is to develop a methodology that, for a given system, finds a dynamic control policy via intelligent agents. Such a policy while achieving the productivity (i.e., demand service rate) goal of the system will optimize a cost/reward function based on the WIP inventory. To achieve this goal we applied a simulation-based optimization technique called Reinforcement Learning (RL) on a four-station serial line. The control policy attained by the application of a RL algorithm was compared with the other existing policies on the basis of total average WIP and average cost of WIP. We also develop a heuristic control policy in light of our experience gained from a close examination of the policies obtained by the RL algorithm. This heuristic policy named Behavior-Based Control (BBC), although placed second to the RL policy, proved to be a more efficient and leaner control policy than most of the existing policies in the literature. The performance of the BBC policy was found to be comparable to the Extended Kanban Control System (EKCS), which as per our experimentation, turned out to be the best of the existing policies. The numerical results used for comparison purposes were obtained from a four-station serial line with two different (constant and Poisson) demand arrival processes.

Dynamic optimal control policy in advertising price and quality

This study presents an algorithm for deriving the long-term polices of quality level, price and advertisement for a product. The diffusion models and cost functions are combined to formulate profit functions capable of determining future profit trends. The algorithm first implements the optimal control theory to derive the optimal conditions of the profit function. Then the genetic algorithm is employed to search for the approximate solutions of quality level, price and advertising expenditure at each period on the planning horizon (life cycle). Examples of different scenarios of the model parameters are presented to describe the results obtained herein. Sensitivity analysis for the major parameters is performed to specify their effects on profits. Results in this study allow us, firstly, to obtain explicit solutions simultaneously with respect to quality, price and advertising policies, secondly, to propose an appropriate algorithm for solving the different scenarios of the dynamic profit function, which consists of the diffusion function and cost function, and thirdly, to enhance the long-term profit performance via the polices proposed herein, that is the approximation of the best solution.