Index Policy Research Articles

Throughput control and revenue optimization of a poultry product batcher

This article studies the optimal control of batching equipment in the poultry processing industry. The problem is to determine a control policy that maximizes the long-term average revenue while achieving a target throughput. This problem is formulated as a Markov Decision Process (MDP). The developed MDP model captures the unique characteristics of poultry processing operations, such as, the trade-off between giveaway and throughput. Structural properties of the optimal policy are derived for small-sized problems where batching equipment utilizes a single bin. Since the MDP model is numerically intractable for industry-sized problems, we propose a heuristic index policy with a Dynamic Rejection Threshold (DRT). The DRT heuristic is constructed based on the salient characteristics of the problem setting and is easy to implement in practice. Numerical experiments demonstrate that DRT performs well. We present an industry case study where DRT is benchmarked against current practice, which shows that the expected revenue can be potentially increased by over 2% (yielding an additional revenue between 750,000 and 2,270,000 Euros per year) through the use of DRT.

Evolving Trends in Risk Profiles and Outcomes in Older Adults Undergoing Kidney Retransplantation.

In older adults (≥65), access to and outcomes following kidney transplantation (KT) have improved over the past 3 decades. It is unknown if there were parallel trends in re-KT. We characterized the trends, changing landscape, and outcomes of re-KT in older adults. Among the 44,149 older kidney-only recipients (1995-2016) in the Scientific Registry of Transplant Recipients, we identified 1743 who underwent re-KT. We analyzed trends and outcomes (mortality, death-censored graft failure [DCGF]) by eras (1995-2002, 2003-2014, and 2015-2016) that were defined by changes to the expanded criteria donors and Kidney Donor Profile Index policies. Among all older kidney-only recipients during 1995-2002, 2003-2014, 2015-2016 the proportion that were re-KTs increased from 2.7% to 4.2% to 5.7%, P < 0.001, respectively. Median age at re-KT (67-68-68, P = 0.04), years on dialysis after graft failure (1.4-1.5-2.2, P = 0.003), donor age (40.0-43.0-43.5, P = 0.04), proportion with panel reactive antibody 80-100 (22.0%-32.7%-48.7%, P < 0.001), and donation after circulatory death (1.1%-13.4%-19.5%, P < 0.001) have increased. Despite this, the 3-y cumulative incidence for mortality (22.3%-19.1%-11.5%, P = 0.002) and DCGF (13.3%-10.0%-5.1%, P = 0.01) decreased over time. Compared with deceased donor retransplant recipients during 1995-2002, those during 2003-2014 and 2015-2016 had lower mortality hazard (aHR = 0.78, 95% confidence interval, 0.63-0.86 and aHR = 0.55, 95% confidence interval, 0.35-0.86, respectively). These declines were noted but not significant for DCGF and in living donor re-KTs. In older retransplant recipients, outcomes have improved significantly over time despite higher risk profiles; yet they represent a fraction of the KTs performed. Our results support increasing access to re-KT in older adults; however, approaches to guide the selection and management in those with graft failure need to be explored.

Reward-Biased Maximum Likelihood Estimation for Linear Stochastic Bandits

Modifying the reward-biased maximum likelihood method originally proposed in the adaptive control literature, we propose novel learning algorithms to handle the explore-exploit trade-off in linear bandits problems as well as generalized linear bandits problems. We develop novel index policies that we prove achieve order-optimality, and show that they achieve empirical performance competitive with the state-of-the-art benchmark methods in extensive experiments. The new policies achieve this with low computation time per pull for linear bandits, and thereby resulting in both favorable regret as well as computational efficiency.

A Restless Bandit Model for Resource Allocation, Competition, and Reservation

In “A Restless Bandit Model for Resource Allocation, Competition and Reservation,” J. Fu, B. Moran, and P. G. Taylor study a resource allocation problem with varying requests and with resources of limited capacity shared by multiple requests. This problem is modeled as a set of heterogeneous restless multi-armed bandit problems (RMABPs) connected by constraints imposed by resource capacity. Following Whittle’s idea of relaxing the constraints and Weber and Weiss’s proof of asymptotic optimality, the authors propose an index policy and establish conditions for it to be asymptotically optimal in a regime where both arrival rates and capacities increase. In particular, they provide a simple sufficient condition for asymptotic optimality of the policy and, in complete generality, propose a method that generates a set of candidate policies for which asymptotic optimality can be checked. Via numerical experiments, they demonstrate the effectiveness of these results even in the pre-limit case.

Cluster based area yield scheme for crop insurance policy in Java

Zone based area yield scheme becomes a potential in improving the development of area yield index for crop insurance policy in Indonesia. The characteristics of productivity varies among municipalities in Indonesia. Therefore, it will be challenging to apply area yield index policy without developing clusters, especially in Java which is considered as one of the central productivities of paddy in Indonesia. Clustering the productivity is one of the solutions to reduce the heterogeneous basis risk within municipalities of a province. Clustering areas is recommended in order to determine accurate critical yield index leading to a more precise ratemaking in the process of developing area yield index policy. In order to define the clusters, we have not only considered average and volatility of productivity over the past ten years, but also integrated the spatial effect of municipalities. Ward-like hierarchical method shows that clustering of municipalities within each province provides better homogeneity. It has been found that there are 12 clusters in Java tend to have low (8.33%), middle (50%), and high (41.67%) productivity. Therefore, we suggest that the Ministry of Agriculture design the area yield index policy based on these clusters and each cluster will have different critical productivity index and adjusted premium.

On the Optimality of the Whittle’s Index Policy for Minimizing the Age of Information

In this article, we consider the average age minimization problem where a central entity schedules M users among the N available users for transmission over unreliable channels. It is well-known that obtaining the optimal policy, in this case, is a difficult task. Accordingly, the Whittle's index policy has been suggested in earlier works as a heuristic for this problem. However, the analysis of its performance remained elusive. In the sequel, we overcome these difficulties and provide rigorous results on its asymptotic optimality in the many-users regime. Specifically, we first establish its optimality in the neighborhood of a specific system's state. Next, we extend our proof to the global case under a recurrence assumption, which we verify numerically. These findings showcase that the Whittle's index policy has analytically provable optimality in the many-users regime for the AoI minimization problem. Finally, numerical results that showcase its performance and corroborate our theoretical findings are presented.

Asymptotically Optimal Lagrangian Priority Policy for Deadline Scheduling With Processing Rate Limits

We study the deadline scheduling problem for multiple deferrable jobs that arrive in a random manner and are to be processed before individual deadlines. The processing of the jobs is subject to a time-varying limit on the total processing rate at each stage. We formulate the scheduling problem as a restless multiarmed bandit problem. Relaxing the scheduling problem into multiple independent single-arm scheduling problems, we define the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Lagrangian priority value</i> as the greatest tax under which it is optimal to activate the arm. We propose a Lagrangian priority policy that processes jobs in the order of their Lagrangian priority values, and establish its asymptotic optimality as the system scales. Numerical results show that the proposed Lagrangian priority policy achieves 22%–49% higher average reward than the classical Whittle index policy (that does not take into account the processing rate limits).

A General Theory of MultiArmed Bandit Processes with Constrained Arm Switches

This paper proposes a general framework of multi-armed bandit (MAB) processes by introducing a type of restrictions on the switches among arms evolving in continuous time. The Gittins index process is constructed for any single arm subject to the restrictions on switches and then the optimality of the corresponding Gittins index rule is established. The Gittins indices defined in this paper are consistent with the ones for MAB processes in continuous time, integer time, semi-Markovian setting as well as general discrete time setting, so that the new theory covers the classical models as special cases and also applies to many other situations that have not yet been touched in the literature. While the proof of the optimality of Gittins index policies benefits from ideas in the existing theory of MAB processes in continuous time, new techniques are introduced which drastically simplify the proof.

A Whittle Index Approach to Minimizing Age of Multi-Packet Information in IoT Network

Age of information (AoI) captures the freshness of information and has been used broadly as an important performance metric in big data analytics in the Internet of Things (IoT). We consider a general scenario where a meaningful piece of information consists of multiple packets and the information is not complete until all related packets have been correctly received. Minimizing AoI in this general scenario is challenging in both scheduling algorithm design and theoretical analysis, because we need to track the history of received packets before a complete piece of information can be updated. We first analyse the necessary condition for optimal scheduling, based on which we present an optimal scheduling method. The optimal solution, however, has high time complexity. To address the problem, we investigate the problem with a special type of learning, i.e., learning in restless multi-armed bandits (RMAB), and propose a Whittle index-based scheduling method. We also propose a new transmission strategy based on erasure codes to improve the performance of scheduling policies in lossy networks. Performance evaluation results demonstrate that our solution outperforms other baseline policies such as greedy policy and naive Whittle index policy in both lossless and lossy networks.

Fast Two-Stage Computation of an Index Policy for Multi-Armed Bandits with Setup Delays

We consider the multi-armed bandit problem with penalties for switching that include setup delays and costs, extending the former results of the author for the special case with no switching delays. A priority index for projects with setup delays that characterizes, in part, optimal policies was introduced by Asawa and Teneketzis in 1996, yet without giving a means of computing it. We present a fast two-stage index computing method, which computes the continuation index (which applies when the project has been set up) in a first stage and certain extra quantities with cubic (arithmetic-operation) complexity in the number of project states and then computes the switching index (which applies when the project is not set up), in a second stage, with quadratic complexity. The approach is based on new methodological advances on restless bandit indexation, which are introduced and deployed herein, being motivated by the limitations of previous results, exploiting the fact that the aforementioned index is the Whittle index of the project in its restless reformulation. A numerical study demonstrates substantial runtime speed-ups of the new two-stage index algorithm versus a general one-stage Whittle index algorithm. The study further gives evidence that, in a multi-project setting, the index policy is consistently nearly optimal.

Factors of Financial Imbalance in the Compulsory Pension Insurance System: the Case of Russia

The aim of the article is to study the impact of tax and tariff policy on the financial balance of the compulsory pension insurance system in Russia, as well as to substantiate proposals for its financial stabilization. The authors analyzed the consequences of regression applied on wages while forming tariffs for pensions. The methods used in this study were economic, statistical and empirical methods, analysis and synthesis, abstraction, systemic-structural approach. The authors analyzed the influence of preferential tariffs for certain types of economic activity and the use of simplified tax systems for organizations on the income of the Pension Fund of the Russian Federation. The work contains the calculation of income shortfalls in the compulsory pension insurance system resulting from the use of regression and tax benefits when calculating insurance payments to the budget of the Pension Fund of Russia. The authors estimated the financial consequences of the pension valorization in the Soviet period and the pension indexation policy in Russia resulted in the rising costs of the Pension Fund of Russia. The conclusion is that the current procedure for determining insurance contributions to the Pension Fund of the Russian Federation, which provides for a large number of benefits, does not correspond to generally accepted world practice and creates major risks for the financial balance of the pension system. This may cause increasing poverty among seniors. Raising the retirement age did not solve the problem of balancing the pension system, but exacerbated it. The regression in the payment of insurance premiums led to a conundrum in Russia, when an increase in wages resulted in the decrease of the Pension Fund of the Russian Federation’s income. This is associated with the precarization of wages and the desire of business to minimize tax payments. The prospect of further research is to study the conditions for financial stabilization of compulsory pension insurance during the Russian economic crisis. The analysis showed that there is a need to create an independent actuarial center for setting tariffs for deductions to the Pension Fund of the Russian Federation, as well as the redistribution of areas of responsibility between the Pension Fund of the Russian Federation and the Ministry of Finance of Russia.

Resource competition in virtual network embedding

We consider a virtual network (VN) embedding problem on a large-scale substrate physical network. We permit varying capacities and cost rates, and reservation of resources (physical links and nodes) for more profitable later VN requests. Our aim is to maximize the long-run average revenue by controlling the allocation of physical components to arriving VN requests. We propose an index policy that selects, according to state-dependent indices, a set of available physical components for each arrival. The indices are calculated in closed form requiring only intrinsic off-line information about physical components and physical resource requirements. Under reasonable assumptions related to rapidly increasing demands for VNs and resource competition, we show that this proposed policy is asymptotically optimal when the lifespans of embedded virtual components are exponentially distributed. Extensive numerical results demonstrate that the long-run average revenue earned by the proposed index policy rapidly approaches optimality: performance deviations between the index policy and the optimal solution are less than 5% in most of our simulations even for relatively small systems. Our numerical results also indicate that the long-run performance of this policy is relatively insensitive to the form of the lifespan distributions.

On the computation of Whittle’s index for Markovian restless bandits

The multi-armed restless bandit framework allows to model a wide variety of decision-making problems in areas as diverse as industrial engineering, computer communication, operations research, financial engineering, communication networks etc. In a seminal work, Whittle developed a methodology to derive well-performing (Whittle’s) index policies that are obtained by solving a relaxed version of the original problem. However, the computation of Whittle’s index itself is a difficult problem and hence researchers focused on calculating Whittle’s index numerically or with a problem dependent approach. In our main contribution we derive an analytical expression for Whittle’s index for any Markovian bandit with both finite and infinite transition rates. We derive sufficient conditions for the optimal solution of the relaxed problem to be of threshold type, and obtain conditions for the bandit to be indexable, a property assuring the existence of Whittle’s index. Our solution approach provides a unifying expression for Whittle’s index, which we highlight by retrieving known indices from literature as particular cases. The applicability of finite rates is illustrated with the machine repairmen problem, and that of infinite rates by an example of communication networks where transmission rates react instantaneously to packet losses.

Inflation Variability Across Australian Households: Implications for Inequality and Indexation Policy*

We examine the consumer price inflation experiences of Australian households across the income distribution using Australian Bureau of Statistics data and prices and methodology from the national consumer price index. Across the period 2011–18, we find that the lowest‐income households have experienced the largest inflation and those at the top have experienced the least inflation. These differences in inflation experience are completely driven by alcohol and tobacco. When we remove alcohol and tobacco, we find that there is little variation in the inflation experiences of households across the income distribution.

Política de Indexação em bibliotecas universitárias da área da saúde

A indexação possui variáveis que envolvem os usuários, a organização, e o próprio procedimento de análise conceitual e tradução, com o uso das linguagens documentárias, por esse motivo a política de indexação torna-se uma discussão essencial para esta atividade, partindo da premissa de se padronizar o processo de indexação que ocorre a partir das etapas de análise de assunto e tradução dos termos. A pesquisa tem o objetivo de investigar os elementos de política de indexação em bibliotecas universitárias da área da saúde de um sistema integrado de bibliotecas de uma Universidade Federal brasileira. O percurso metodológico caracteriza-se como exploratório com estudo de caso a partir da análise qualitativa de questionários aplicados com quatro catalogadores de quatro bibliotecas setoriais da área da saúde. Identificaram-se na análise dos resultados alguns elementos da política de indexação em comum entre as bibliotecas setoriais, no entanto, esses elementos aparecem sem uma padronização, reforçando que não há um diálogo efetivo entre essas bibliotecas. Conclui-se que a ausência de uma política de indexação nas bibliotecas setoriais da saúde causa a não sistematização dos procedimentos de indexação e a reflexão sobre essa atividade para a representação da informação reforçando a necessidade da elaboração de uma política de indexação.

Uniform Loss Algorithms for Online Stochastic Decision-Making With Applications to Bin Packing

We consider a general class of finite-horizon online decision-making problems, where in each period a controller is presented a stochastic arrival and must choose an action from a set of permissible actions, and the final objective depends only on the aggregate type-action counts. Such a framework encapsulates many online stochastic variants of common optimization problems including bin packing, generalized assignment, and network revenue management. In such settings, we study a natural model-predictive control algorithm that in each period, acts greedily based on an updated certainty-equivalent optimization problem. We introduce a simple, yet general, condition under which this algorithm obtains uniform additive loss (independent of the horizon) compared to an optimal solution with full knowledge of arrivals. Our condition is fulfilled by the above-mentioned problems, as well as more general settings involving piece-wise linear objectives and offline index policies, including an airline overbooking problem.

On the management of retirement age indexed to life expectancy: a scenario analysis of the Italian longevity experience

PurposeThis paper aims to measure the financial impact on social security system of a recently proposed indexation mechanism for retirement age by considering the Italian longevity experience. The analysis is motivated by the progressive increase in life expectancy at advanced age, which is rapidly bringing to the fore noticeable socio-economic consequences in most industrialized countries. Among those, the impact on National Social Security systems is particularly relevant if people live longer than expected; this will lead to greater financial exposure for pension providers.Design/methodology/approachReferring to the Italian population for illustrative purposes, the authors contemplate different scenarios for mortality projection methods and for the implementation of pension age shift while accounting for gender and cohort gaps and model risk. Synthetic indicators to measure the impact of the indexation mechanism on social security system are introduced on the basis of pension cash flows.FindingsAn indexation policy that manages gender gap while adjusting retirement age for varying life expectancy is proposed. As a result, sustainability of public retirement expenditure is improved.Originality/valueThe paper is a concise scenario analysis of the reduction of costs and risks that pension providers would have if the system resorted to link retirement age to life expectancy. The ideas fostered by the paper follow a recent proposal of the Authors on a flexible retirement scheme that deals with model risk for mortality projection and accounts for gender gap in mortality rates.

Index Policies and Performance Bounds for Dynamic Selection Problems

We consider dynamic selection problems, where a decision maker repeatedly selects a set of items from a larger collection of available items. A classic example is the dynamic assortment problem with demand learning, where a retailer chooses items to offer for sale subject to a display space constraint. The retailer may adjust the assortment over time in response to the observed demand. These dynamic selection problems are naturally formulated as stochastic dynamic programs (DPs) but are difficult to solve because the optimal selection decisions depend on the states of all items. In this paper, we study heuristic policies for dynamic selection problems and provide upper bounds on the performance of an optimal policy that can be used to assess the performance of a heuristic policy. The policies and bounds that we consider are based on a Lagrangian relaxation of the DP that relaxes the constraint limiting the number of items that may be selected. We characterize the performance of the Lagrangian index policy and bound and show that, under mild conditions, these policies and bounds are asymptotically optimal for problems with many items; mixed policies and tiebreaking play an essential role in the analysis of these index policies and can have a surprising impact on performance. We demonstrate these policies and bounds in two large scale examples: a dynamic assortment problem with demand learning and an applicant screening problem. This paper was accepted by Yinyu Ye, optimization.

Exchange Market Pressure and Monetary Policy: The Turkish Case

The purpose of this study is to determine the relationship between monetary policy and the exchange market pressure index in Turkey for the 2002–2018 period with monthly data. To obtain the foreign exchange market pressure index, this study uses the model developed by L. Girton and D.E. Roper and is based fundamentally on the monetary approach to exchange rate determination and the balance of payments. The calculated exchange market pressure index is in accordance with the developments lived in financial markets and changes in monetary policy during the period under investigation. As for the relation between exchange market pressure index and monetary policy, a VAR model was set up and a Granger type causality analysis was carried out. According to Granger causality test results, there is a unidirectional causality running from domestic credit expansion to exchange market pressure and from domestic credit expansion to interest rate differential while there is a bidirectional causality between exchange market pressure and interest rate differential. Since increasing exchange market pressure means a depreciation of the Turkish Lira, the estimated VAR model’s results support the view that the Central Bank will increase the interest rate to temper the exchange market pressure.

Inflation Variability Across Australian Households: Implications for Inequality and Indexation Policy

We examine the consumer price inflation experiences of Australian households across the income distribution using Australian Bureau of Statistics data and prices and methodology from the national consumer price index. Across the period 2011 – 2018, we find that the lowest income households have experienced the largest inflation and those at the top have experienced the least inflation. These differences in inflation experience are completely driven by alcohol and tobacco. When we remove alcohol and tobacco, we find that there is little variation in the inflation experiences of households across the income distribution.