Dynamic Control Policy Research Articles

Quality Control Based Tool Condition Monitoring

Quality control and tool condition monitoring are two most important aspects of machining process. This paper studies the correlation between tool wear and surface roughness to explore the possibility of modelling the interdependencies between these two aspects. An experimental study is presented in this paper to model the relationship between product quality parameter i.e. average surface roughness and tool wear. Current study reveals that there is a strong positive correlation between surface roughness and tool wear. To map this relationship an ensemble (random forest) fault estimation model is developed for identification and estimation of cutting tool health state. The results from fault estimation model are then used to provide guidelines for future process monitoring and developing dynamic quality control policy.

Scheduling in Networks With Time-Varying Channels and Reconfiguration Delay

We consider the optimal control problem for networks subjected to time-varying channels, reconfiguration delays, and interference constraints. We show that the simultaneous presence of time-varying channels and reconfiguration delays significantly reduces the system stability region and changes the structure of optimal policies. We first consider memoryless channel processes and characterize the stability region in closed form. We prove that a frame-based Max-Weight scheduling algorithm that sets frame durations dynamically, as a function of the current queue lengths and average channel gains, is throughput-optimal. Next, we consider arbitrary Markov-modulated channel processes and show that memory in the channel processes can be exploited to improve the stability region. We develop a novel approach to characterizing the stability region of such systems using state-action frequencies, which are stationary solutions to a Markov Decision Process (MDP) formulation. Moreover, we develop a dynamic control policy using the state-action frequencies and variable frames whose lengths are functions of queue sizes and show that it is throughput-optimal. The frame-based dynamic control (FBDC) policy is applicable to a broad class of network control systems, with or without reconfiguration delays, and provides a new framework for developing throughput-optimal network control policies using state-action frequencies. Finally, we propose Myopic policies that are easy to implement and have better delay properties as compared to the FBDC policy.

Grid Power-Delay Tradeoff for Energy Harvesting Wireless Communication Systems with Finite Renewable Energy Storage

In this paper, we study the grid power-delay tradeoff in a point-to-point energy harvesting wireless communication system with finite energy storage capacity serving delay-sensitive applications. This communication system is powered by both grid and renewable power sources. First, we consider the average grid power consumption minimization subject to the data queue stability and renewable energy availability constraints. By exploring the optimality property and using the theory of random walks, we transform the grid power minimization problem to an asymptotically equivalent problem. Using the Lyapunov drift approach, we obtain an online dynamic power control policy to solve the asymptotically equivalent problem. Then, we introduce a novel analysis framework to study the grid power-delay tradeoff relationship of the online power control in the small delay regime. Specifically, using continuous-time approximation, dynamic programming and sample-path approach, we obtain bounds on the average delay and grid power consumption, which are asymptotically tight in the small delay regime. Based upon the derived closed-form expressions, we quantify the impacts of energy storage capacity and some other system parameters on the grid power-delay tradeoff.

Dynamic Control Policy for Delay Guarantees in Multi-hop Wireless Networks

This paper considers the dynamic control of stochastic networks for the guaranteed delay with network utility maximization in multi-hop wireless networks. On contrary to single hop case, the delay is difficult to predict in multi-hop scenarios due to their unpredictable traffic pattern and variable queuing time in intermediate nodes. We present here a control policy for joint flow control and scheduling which can achieve high network utility and guarantees delay constraints. In addition, we derive the capacity region of the network under a given delay condition. Simulation results show that the proposed policy achieves near optimal network utility with delay guaranteed compare to exiting algorithms.

On Maximally Stabilizing Traffic Signal Control with Unknown Turn Ratios

This paper designs distributed dynamic traffic signal control policies for urban traffic networks. Vehicles at the end of an approach to an intersection queue up in separate lanes corresponding to different possible turn maneuvers at the upcoming intersection, according to fixed turn ratios. The departure rate of vehicles from the queue is governed by traffic signal control at the intersections. We consider traffic signal control architectures under which, at every intersection, only a subset of non-conflicting approaches get green light simultaneously. We propose a class of cooperative green light policies, under which traffic signal control at an intersection requires information only about the occupancy levels on the lanes incoming at that intersection. In particular, such a policy does not require information about turn ratios, external arrival rates or departure rates. We show that such minimalist policies are maximally stabilizing for acyclic network topologies under some scenarios, and, when the network is stabilizable, the network admits a globally asymptotically stable equilibrium under these policies. Simulations to illustrate the applicability of our results to cyclic network topologies are also presented.

Optimal Control of a Server Farm

We consider a server farm consisting of ample exponential servers, that serve a Poisson stream of arriving customers. Each server can be either busy, idle or off. An arriving customer will immediately occupy an idle server, if there is one, and otherwise, an off server will be turned on and start servicing this customer. Upon service completion, it can be decided to keep the server idle or to turn it off. It costs per unit time to keep servers idle, and it costs to turn an off server on. We derive structural properties of the dynamic control policy minimizing the expected total discounted cost. The structural results are extended to the average cost criterion. We also propose and analyze a simple control policy, and numerically investigate its performance.

Dynamic Lot Optimization of Assembly Supply Chain with Delivery Time Constraint

In this paper, we investigate operational decision optimization problem of assembly manufacturing capacitated supply chain. Open equations of this system in all planning periods are set up. Delivery time to clients form a time window that limits point to assembly manufacturer delivery start and end time. The system model divide planning time into many equally periods, and decisions can vary with time. Ability constraint in the process of supply chain operation, time constraints and assembly production constrain are all considered in the inventory control dynamic batch optimization model. We solve the optimization problem by hybrid mixed integral optimization and SQP algorithm. At last, we examine a set of numerical examples that reveal the insights into the dynamic inventory control policy and the performance of such an assembly type inventory mode.

Robust active vibration control of piezoelectric flexible structures using deterministic and probabilistic analysis

The main objective of this article is to propose a general and systematic robust control methodology for active vibration control of piezoelectric flexible structures such that the probabilistic information of parametric uncertainties can be investigated, and the robustness properties of the closed-loop system can be quantitatively ensured. For the purpose, the modal parameter identification is performed based on the finite element analysis to have the reduced nominal dynamical models. The generalized polynomial chaos framework is employed for uncertainty analysis to obtain the probabilistic information of parametric uncertainties. In the presence of probabilistic parametric and dynamic uncertainties, phase and gain control policies–based [Formula: see text] output feedback control is used for the controller design to ensure a set of control objectives simultaneously, and then deterministic and probabilistic robustness analyses are conducted to quantitatively verify the robustness properties of the closed-loop system both in the deterministic sense and in the probabilistic one. The design process and the effectiveness of the proposed control methodology are illustrated via active vibration control of a piezoelectric cantilever beam.

Capacity region and dynamic control of wireless networks under per-link queueing

The capacity region of wireless networks with per-destination PD queueing model has been studied extensively in the literature. However, the PD queueing structure is not scalable because the number of queues in a node can be as large as the number of all possible source-destination pairs. In this work, we study the capacity region of wireless networks with per-link PL queueing model. The advantage of the PL queueing structure is that the number of queues in a node can be reduced significantly to the number of its neighboring nodes. In this paper, the capacity region of a wireless network with PL queueing structure is characterized, and a dynamic routing and power control policy, namely, DRPC-PL, is proposed to stabilize the network whenever the input rate is within the capacity region. Copyright © 2013 John Wiley & Sons, Ltd.

POMDP-based control of workflows for crowdsourcing

Crowdsourcing, outsourcing of tasks to a crowd of unknown people (“workers”) in an open call, is rapidly rising in popularity. It is already being heavily used by numerous employers (“requesters”) for solving a wide variety of tasks, such as audio transcription, content screening, and labeling training data for machine learning. However, quality control of such tasks continues to be a key challenge because of the high variability in worker quality. In this paper we show the value of decision-theoretic techniques for the problem of optimizing workflows used in crowdsourcing. In particular, we design AI agents that use Bayesian network learning and inference in combination with Partially-Observable Markov Decision Processes (POMDPs) for obtaining excellent cost-quality tradeoffs.We use these techniques for three distinct crowdsourcing scenarios: (1) control of voting to answer a binary-choice question, (2) control of an iterative improvement workflow, and (3) control of switching between alternate workflows for a task. In each scenario, we design a Bayes net model that relates worker competency, task difficulty and worker response quality. We also design a POMDP for each task, whose solution provides the dynamic control policy. We demonstrate the usefulness of our models and agents in live experiments on Amazon Mechanical Turk. We consistently achieve superior quality results than non-adaptive controllers, while incurring equal or less cost.

A Market Based Scheme to Integrate Distributed Wind Energy

Efficiently integrating wind energy into the smart grid is gaining momentum under renewable portfolio standard (RPS) with deep wind penetration. Due to the randomness of wind energy production, ancillary service (AS) is needed in large amount to regulate wind power for system stability and reliability. As a result, the cost of wind power depends on the AS market and may be, quite higher than that of conventional power. Therefore, it is challenging to economically integrate wind energy with current power system to satisfy RPS. With the communication, sensing and advanced control features incorporated into the smart grid, the interactions among the grid components will facilitate solving this problem. In this paper, we consider the wind energy integration of small-scale utilities installed with wind turbines and acted as distributed energy resources (DERs). Since wind energy can be integrated to serve customer load or enter a separate green energy market, we propose a theoretical framework to dynamically determine the role of wind energy and provide long-term RPS guarantee. This approach results in a simple dynamic threshold control policy which maximizes the expectation of the profit for a green utility and is easily implemented online.

Blind Fair Routing in Large-Scale Service Systems with Heterogeneous Customers and Servers

In a call center, arriving customers must be routed to available servers, and servers that have just become available must be scheduled to help waiting customers. These dynamic routing and scheduling decisions are very difficult, because customers have different needs and servers have different skill levels. A further complication is that it is preferable that these decisions are made blindly; that is, they depend only on the system state and not on system parameter information such as call arrival rates and service speeds. This is because this information is generally not known with certainty. Ideally, a dynamic control policy for making routing and scheduling decisions balances customer and server needs by keeping customer delays low but still fairly dividing the workload amongst the various servers. In this paper, we propose a blind dynamic control policy for parallel-server systems with multiple customer classes and server pools that is based on the number of customers waiting and the number of agents idling.We show that in the Halfin-Whitt many-server heavy-traffic limiting regime, our proposed blind policy performs extremely well when the objective is to minimize customer holding costs subject to “server fairness,” as defined by how the system idleness is divided among servers. To do this, we formulate an approximating diffusion control problem (DCP) and compare the performance of the nonblind DCP solution to a feasible policy for the DCP that is blind. We establish that the increase in the DCP objective function value is small over a wide range of parameter values. We then use simulation to validate that a small increase in the DCP objective function value is indicative of our proposed blind policy performing very well.

Fixed task zone chaining: worker coordination and zone design for inexpensive cross-training in serial CONWIP lines

This work introduces a new canonical model of worker cross-training, called a Fixed Task Zone Chain (FTZC), as a special type of zone-based cross-training and develops a methodology to employ it in U-shaped CONstant Work In Process (CONWIP) lines. The FTZC approach is intended to address lines with more stations than workers in environments where extensive cross-training is prohibitive. It incorporates a two-skill chain to cross-train one skill at each end of each zone; however, tasks at the interior of a zone are not cross-trained. A heuristic dynamic control policy is developed that maximizes the line’s throughput given a zone structure. Some useful properties of the optimal policy are provided as a basis for a heuristic control policy that yields high throughput. The performance of an FTZC system is contingent upon the choice of zone structure; therefore, the zone assignment (ZonA) algorithm is created to design the zone structure to achieve high throughput levels. Sufficient conditions which guarantee that the line is balanceable through ZonA are derived. Benchmarking over a test suite supports the effectiveness of the proposed heuristic worker control policy as well as the ZonA algorithm, and its performance is compared with other paradigms.

Dynamic Server Allocation Over Time-Varying Channels With Switchover Delay

We consider a dynamic server allocation problem over parallel queues with randomly varying connectivity and server switchover delay between the queues. At each time slot, the server decides either to stay with the current queue or switch to another queue based on the current connectivity and the queue length information. Switchover delay occurs in many telecommunications applications and is a new modeling component of this problem that has not been previously addressed. We show that the simultaneous presence of randomly varying connectivity and switchover delay changes the system stability region and the structure of optimal policies. In the first part of this paper, we consider a system of two parallel queues, and develop a novel approach to explicitly characterize the stability region of the system using state-action frequencies which are stationary solutions to a Markov decision process formulation. We then develop a frame-based dynamic control (FBDC) policy, based on the state-action frequencies, and show that it is throughput optimal asymptotically in the frame length. The FBDC policy is applicable to a broad class of network control systems and provides a new framework for developing throughput-optimal network control policies using state-action frequencies. Furthermore, we develop simple myopic policies that provably achieve more than 90% of the stability region. In the second part of this paper, we extend our results to systems with an arbitrary finite number of queues. In particular, we show that the stability region characterization in terms of state-action frequencies and the throughput optimality of the FBDC policy follows for the general case. Furthermore, we characterize an outer bound on the stability region and an upper bound on sum throughput and show that a simple myopic policy can achieve this sum-throughput upper bound in the corresponding saturated system. Finally, simulation results show that the myopic policies may achieve the full stability region and are more delay efficient than the FBDC policy in most cases.

Robust Design and Control of Call Centers with Flexible Interactive Voice Response Systems

We consider the robust design and control of call center systems with flexible interactive voice response (IVR) systems. In a typical call center equipped with an IVR system, customers are handled first by the IVR system, which is a computerized system that often enables customers to self-serve. Those customers whose service cannot be handled by the IVR system are sent to the second stage to be served by agents. To address the design of a call center with a flexible IVR system, we consider a call center whose IVR system can operate in two different service delivery modes that have statistically different outcomes. We formulate a stochastic program to determine the number of agents needed in the second stage as well as the proportion of customers that should be selected for each IVR service mode under different demand levels with the objective of minimizing the total staffing and abandonment costs. We also propose dynamic control policies to achieve the optimal proportions without the knowledge of the exact arrival rate. We show that the staffing levels found by the stochastic program combined with the proposed routing policies are asymptotically optimal in large systems. We present extensive numerical and simulation results to illustrate the effectiveness of the proposed solutions.

Stability and Control of Acyclic Stochastic Processing Networks With Shared Resources

We consider a general model framework for acyclic stochastic processing networks with shared resources that has many applications in telecommunication, computer, and manufacturing systems. A dynamic control policy that utilizes the maximal matching (for scheduling) and the join-the-shortest-queue (for routing) discipline, is shown to maximize the throughput and stabilize the system in a sense called “uniform mean recurrence time property” under fairly mild stochastic assumptions. Owing to the non-Markovian nature of the states, system stability is established using a perturbed Lyapunov function method.

Linear probability based admission control policy for video on demand system

In this paper, we propose a dynamic admission control policy for distributed video on demand system, based on linear probability function. The probability of accepting a video request is calculated by using a liner mathematical formula. Quality of service (QoS) is granted by restricting the requests on the basis of popularity of request and availability of resources. The popular video request will get better chance to be accepted. The distinction of this paper from the others is dynamic admission control policy instead of static and threshold. The evaluation of proposed system is done by simulation model, number of rejected requests and total generated revenue is calculated for performance measurement, and it is observed that proposed admission control policy not only improves the performance of all the request classes but also improves a lot the overall system.

Robust Design and Control of Call Centers with Flexible IVR Systems

We consider the robust design and control of call center systems with flexible interactive voice response (IVR) systems. In a typical call center equipped with an IVR system, customers are handled first by the IVR system, which is a computerized system that often enables customers to self-serve. Those customers whose service cannot be handled by the IVR system are sent to the second stage to be served by agents. In order to address the design of a call center with a flexible IVR system, we consider a call center whose IVR system can operate in two different service delivery modes that have statistically different outcomes. We formulate a stochastic program to determine the number of agents needed in the second stage as well as the proportion of customers that should be selected for each IVR service mode under different demand levels with the objective of minimizing the total staffing and abandonment costs. We also propose dynamic control policies to achieve the optimal proportions without the knowledge of the exact arrival rate. We show that the staffing levels found by the stochastic program combined with the proposed routing policies are asymptotically optimal in large systems. We present extensive numerical and simulation results to illustrate the effectiveness of the proposed solutions.

Dynamic sampling control policy for soil moisture measurements using wireless sensor networks

Dynamic sampling control policy for soil moisture measurements using wireless sensor networks

Dynamic Pricing and Inventory Control in a Make-to-Stock Queue With Information on the Production Status

This paper addresses the dynamic pricing problem of a single-item, make-to-stock production system. Demand arrives according to Poisson processes with changeable arrival rate dependent on the selling price. Item processing times follow an Erlang distribution, which allows to use the information on the production status in a tractable way. The objective is to identify a dynamic control policy that decides production and adjusts the price to maximize the long-run total discounted profit. An optimal policy is based on the so-called work-storage level that captures the information of the inventory level and the status of ongoing production process. Specifically, we show that: 1) the finished goods inventory is optimally managed by a critical stage level policy: when the inventory is below a certain work-storage level, production is started if the system is currently idle and 2) the price is optimally set by threshold levels: a certain price is posted when the work-storage level is at or below a threshold corresponding to that level of price. Moreover, we develop an efficient algorithm to compute the optimal policy.