Large-scale Service Systems Research Articles

Distributed Rate Scaling in Large-Scale Service Systems

Distributed Rate Scaling in Large-Scale Service Systems

Enhancing Fault Injection Testing of Service Systems via Fault-Tolerance Bottleneck

Modern large-scale service systems are usually deployed with redundant components to ensure high dependability in distributed and volatile environments. Fault Injection Testing (FIT) is a popular technique for testing such systems, while the application of FIT to validating the correctness of redundant components remains a challenging task, especially when the system's structural information is unavailable when testing starts. In this study, we refer to a minimum set of faults that, when injected, will cut off all execution paths in a service system as a <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">fault-tolerance bottleneck</i> , and we propose a novel Fault-tolerance Bottleneck driven Fault Injection (FBFI) approach to the exploration and validation of redundant components without prior knowledge of the system's business structure. The core idea of FBFI is to iteratively infer and inject bottlenecks of the business structure constructed so far. In this way, FBFI is able to discover and test redundant components by repeatedly triggering new system behaviors. The effectiveness and efficiency of FBFI is evaluated using two microservice benchmark systems with different deployment scales. The results reveal that FBFI is more practical and cost-effective than random and lineage-driven FIT approaches in testing service systems of high redundancy levels.

A Review of Cooking of Potatoes (Solanum tuberosum L.) Served in Large-Scale Food-Service Systems, Including Industrial Pre-Treatments

Potatoes (Solanum tuberosum L.) often constitute a meal’s main carbohydrate source. When consumed outside the home, dishes are often prepared in large-scale food service systems, like school canteens and hospitals. To manage the logistics of serving the required quantities of potatoes, raw tubers must be prepared by washing, industrial peeling, preservative actions, and packaging to stand transportation and storage before cooking. There are several steps of pre-treatment, packaging, transportation, and cooking techniques that differ from traditional preparation at home, and each of these steps— or more likely a combination of several steps— might contribute to reduced quality in terms of enzymatic discoloration, microbiological failure, and subsurface hardening. In this review, the effect of each of these steps on the potato tuber; from industrial peeling to steam-cooking in the large-scale food service system, has been studied to understand where the most significant quality changes occur, and to understand the combined impact of different actions.

To Pool or Not to Pool: Queueing Design for Large-Scale Service Systems

In large-scale service systems, it is a common practice to organize customers with similar service requirements into a single queue served by a group of servers. This pooled queue structure is deemed highly efficient because the servers’ idleness will be minimized. In “To Pool or Not to Pool: Queueing Design for Large-Scale Service Systems,” Cao, He, Huang, and Liu demonstrate that the dedicated queue structure, under which each server has her own queue, could be more advantageous for improving the system’s service level. Moreover, the servers’ additional idleness induced by the dedicated queue structure will be negligible when the system scale is large. By solving a staffing problem, this study also intends to help service system designers answer the following question: To achieve a specified service-level objective in a more efficient manner, should the servers have a common queue or separate queues?

Perils and Benefits of Free Trials in Large Scale Service Systems: An Operations Perspective

Despite their pervasive use in practice, the operational implications of offering free trials are not well understood. Motivated by this, we consider a service provider that has the option of offering free trials to a new market while catering to an existing market of price and delay sensitive customers. Then, we pose the following question: Under what conditions is offering free trials beneficial and what is the extent of this benefit? This question is relevant because with free trials the firm essentially gives away part of its capacity to potentially generate more revenue in the future. Hence, it is not apriori clear if offering free trials is beneficial. To answer these questions, we pursue an asymptotic analysis that allows us to glean novel insights as to when and how a large-scale service firm should offer free trials: When the firm offers free trials with no control, we find, somewhat surprisingly, that offering free trials is beneficial only for small free trial market sizes with correspondingly small relative benefits. When capacity can be adjusted, we derive tractable expressions that can accurately determine when offering free trials is beneficial and the extent of this benefit. Consequently, we establish that offering free trials is always beneficial when capacity can be adjusted below a cost threshold. Thus, we conclude that service firms should opt to simultaneously increase capacity as free trials are offered to unlock the expected benefits, and otherwise not offer free trials at all.

Job assignment in large-scale service systems with affinity relations

We consider load balancing in service systems with affinity relations between jobs and servers. Specifically, an arriving job can be assigned to a fast, primary server from a particular selection associated with this job or to a secondary server to be processed at a slower rate. Such job–server affinity relations can model network topologies based on geographical proximity, or data locality in cloud scenarios. We introduce load balancing schemes that assign jobs to primary servers if available, and otherwise to secondary servers. A novel coupling construction is developed to obtain stability conditions and performance bounds. We also conduct a fluid limit analysis for symmetric model instances, which reveals a delicate interplay between the model parameters and load balancing performance.

Mean-field Macro Computation in Large-scale Cloud Service Systems with Resource Management and Job Scheduling

Service computing is an emerging and distributed computing mode in cloud service systems, and has become an interesting research direction for both academia and industry. Note that the cloud service systems always display new characteristics, such as stochasticity, large scale, loose coupling, concurrency, non-homogeneity and heterogeneity, thus their load balancing investigation has been more interesting, difficult and challenging until now. By using resource management and job scheduling, this paper proposes an integrated, real-time and dynamic control mechanism for large-scale cloud service systems and their load balancing through combining supermarket models with not only work stealing models but also scheduling of public reserved resource. To this end, this paper provides a novel stochastic model with weak interactions by means of nonlinear Markov processes. To overcome theoretical difficulties growing out of the state explosion in high-dimensional stochastic systems, this paper applies the mean-field theory to develop a macro computational technique in terms of an infinite-dimensional system of mean-field equations. Furthermore, this paper proves the asymptotic independence of the large-scale cloud service system, and show how to compute the fixed point by virtue of an infinite-dimensional system of nonlinear equations. Based on the fixed point, this paper provides effective numerical computation for performance analysis of this system under a high approximate precision. Therefore, we hope that the methodology and results given in this paper can be applicable to the study of more general large-scale cloud service systems.

New perspectives on the Erlang-A queue

AbstractThe nonstationary Erlang-A queue is a fundamental queueing model that is used to describe the dynamic behavior of large-scale multiserver service systems that may experience customer abandonments, such as call centers, hospitals, and urban mobility systems. In this paper we develop novel approximations to all of its transient and steady state moments, the moment generating function, and the cumulant generating function. We also provide precise bounds for the difference of our approximations and the true model. More importantly, we show that our approximations have explicit stochastic representations as shifted Poisson random variables. Moreover, we are also able to show that our approximations and bounds also hold for nonstationary Erlang-B and Erlang-C queueing models under certain stability conditions.

Functional regression-based monitoring of quality of service in hospital emergency departments

This article focuses on building a statistical monitoring scheme for service systems that experience time-varying arrivals of customers and have time-varying service rates. There is lack of research in the systematic statistical monitoring of large-scale service systems, which is critical for maintaining a high quality of service. Motivated by the emergency department at a major academic medical center, this article intends to fill this research gap and provide a practical statistical monitoring scheme capable of detecting changes in service using readily available time stamp data. The proposed method is focused on building a functional regression model based on customer arrival and departure time instances from an in-control system. The model finds the expected departure intensity function for an observed arrival intensity on any given day of operation. The mean squared difference between the expected departure intensity function and the observed departure intensity functions is used to generate an alarm indicating a significant change in service. This methodology is validated using simulation and real data case studies. The proposed method can identify patterns of inefficiency or delay in service that are hard to detect using traditional statistical monitoring algorithms. The method offers a practical approach for monitoring service systems and determining when staffing levels need to be re-optimized.

A Comprehensive Security Framework for Publish/Subscribe-Based IoT Services Communication

The publish/subscribe paradigm provides a loosely-coupled and scalable communication model for the large-scale IoT service systems, such as supervisory control and data acquisition (SCADA). Data confidentiality and service privacy are two crucial security issues for the publish/subscribe model deployed in different domains. The existing access control schemes for such model cannot address both the issues at the same time. In this paper, we propose a comprehensive access control framework (CACF) to bridge this gap. The design principle of the proposed framework is twofold: (a) a bi-directional policy matching scheme for protecting the privacy of IoT services; and (b) a fully homomorphic encryption scheme for encrypting published events to protect data confidentiality. We analyze the correctness and security of the CACF, moreover, we prototype CACF based on Apache ActiveMQ, an open source message broker, and evaluate its performance. The experimental results indicate that our security system meets the latency requirements for very high-quality SCADA services.

Determination of Cooking Degree of Boiled and Steam Cooked Potatoes (Solanum tuberosum L.) from a Physico-Chemical and Sensorial Perspective

Potatoes served in large-scale food service systems are often of poor quality. The current recommendation is to cook the potatoes until the core has reached a temperature of 96°C, but this recommendation is questionable. In this study, three potato varieties have been cooked by using conventional boiling (CB) to temperatures of 94°C and 96°C and steam cooking (SC) to temperatures of 92°C, 94°C and 96°C. The samples were thereafter analyzed by both texture and sensorial analyses. The results were evaluated with Partial Linear Squares Regression (PLSR). The perceived Softness and Cooking Degree from the sensorial analysis agreed with the hardness measured by the texture analyzer. The general impression of the samples showed that an undercooked tuber was preferable to an overcooked tuber, and that a cooking temperature of 96°C mostly resulted in overcooked tubers. The general impression depended on several parameters. Therefore, no optimal cooking temperature could be established.

Operational Perils and Benefits of Free Trials in Large Scale Service Systems

Operational Perils and Benefits of Free Trials in Large Scale Service Systems

Creating Work Breaks from Available Idleness

We develop new rules for assigning available service representatives to customers in customer contact centers and other large-scale service systems in order to create effective work breaks for the service representatives from naturally available idleness. These are unplanned breaks occurring randomly over time. We consider both announced breaks as well as unannounced breaks. Our goal is to make the mean and variance of the interval between successive breaks suitably small. Given a target break duration, we propose assigning idle servers based on the elapsed time since their last break. We show that our proposed server-assignment rules are optimal for the many-server heavy-traffic (MSHT) fluid model. Extensive simulation experiments support the proposed server-assignment rules in practical cases and confirm the MSHT approximation formulas when the number of servers is very large. The online appendix is available at https://doi.org/10.1287/msom.2017.0682 .

Dynamic rate Erlang-A queues

The multi-server queue with non-homogeneous Poisson arrivals and customer abandonment is a fundamental dynamic rate queueing model for large-scale service systems such as call centers and hospitals. Scaling the arrival rates and number of servers arises naturally when a manager updates a staffing schedule in response to a forecast of increased customer demand. Mathematically, this type of scaling ultimately gives us the fluid and diffusion limits as found in Mandelbaum et al. (Queueing Syst 30(1):149–201, 1998) for Markovian service networks. These asymptotics were inspired by the Halfin and Whitt (Oper Res 29(3):567–588, 1981) scaling for multi-server queues. In this paper, we provide a review and an in-depth analysis of the Erlang-A queueing model. We prove new results about cumulant moments of the Erlang-A queue, the transient behavior of the Erlang-A limit cycle, new fluid limits for the delay time of a virtual customer, and optimal static staffing policies for healthcare systems. We combine tools from queueing theory, ordinary differential equations, complex analysis, cumulant moments, orthogonal polynomials, and dynamic optimization to obtain new insights about this fundamental queueing model.

Behavior Reconstruction Models for Large-scale Network Service Systems

In large-scale network service systems, the phenomenon of instantaneous gathering of a large number of users can cause system abnormality, whenever the load imposed by the user behaviors does not match the system load. This paper proposes a behavior reconstruction model for large-scale network service systems integrated with Petri net reconstruction methodology, for the purpose of achieving load balancing in the system under increasing number of users. Based on the features of the user interaction behavior sequence, the behavioral load balancing model defines a user behavior membership function. Then, a random fuzzy Petri net with delay is presented to control the user behavior reconstruction. Experiments conducted by considering various changes in the number of user behaviors and their distribution in unit time demonstrate that the proposed methodology can effectively trigger the reconstructed model to balance the system load when the system load exceeds the defined warning point.

Observational Learning in Large-Scale Congested Service Systems

We study the impact of observational learning in large scale congested service systems with servers having heterogenous quality levels and customers that are heterogonously informed about the server quality. Providing congestion information to all customers allows them to avoid congested servers, but, also implies that less informed customers learn about the quality from observing the choices of other customers. Due to an exponentially growing state space in the number of servers, identifying Bayesian equilibria is intractable with a large, discrete number of servers. In this paper, we develop a tractable model with a continuum of servers. We find that the impact of observational learning on the customers' choice behavior may lead to severe imbalance of server load in the system, such that a decentralized system significantly under-performs in terms of the social welfare, compared with a centralized system. The decentralized system performs well only when (a) either the congestion costs are high and there are sufficient informed customers, or (b) when the congestion costs are medium or low and the aggregate capacity of high-quality servers matches the aggregate demand of informed customers. We also find situations in which making more customers informed about service quality leads to a decrease in social welfare. Our paper highlights the tension between observational learning and social welfare maximization and thus observational learning in large-scale service systems might require intervention of the platform manager.

Differentiation service of level in multiservice call centers with adaptation servers

In this article study large-scale service systems with several classes of customers. The following question of the necessary number of servers and compliance with the requirements of customers to reduce the cost of personnel in conditions of limiting the quality of service at the class level is considered. The characteristics of the planning and staffing scheme that are asymptotically optimal under constrained conditions are made, since the load on the system increases to infinity. Asymptotic modes are considered in accordance with the quality and efficiency of management, the quality of management and the effectiveness of management within a single class of service.

Staffing large-scale service systems with distributional uncertainty

This paper analyzes a staffing level problem for large-scale single-station queueing systems. The system manager operates an Erlang-C queueing system with a quality-of-service constraint on the probability that a customer is queued. However, in this model, the arrival rate is uncertain in the sense that even the arrival-rate distribution is not completely known to the manager. Rather, the manager has an estimate of the support of the arrival-rate distribution and the mean. The goal is to determine the number of servers needed to satisfy the quality-of-service constraint. Two cases are explored. First, the constraint is enforced on an overall delay probability, given the probability that different feasible arrival-rate distributions are selected. In the second case, the constraint has to be satisfied by every possible distribution. For both problems, asymptotically optimal solutions are developed based on Halfin---Whitt type scalings.

Large-scale heterogeneous service systems with general packing constraints

Abstract A service system with multiple types of customers, arriving according to Poisson processes, is considered. The system is heterogeneous in that the servers can also be of multiple types. Each customer has an independent, exponentially distributed service time, with the mean determined by its type. Multiple customers (possibly of different types) can be placed for service into one server, subject to `packing' constraints, which depend on the server type. Service times of different customers are independent, even if served simultaneously by the same server. The large-scale asymptotic regime is considered such that the customer arrival rates grow to ∞. We consider two variants of the model. For the infinite-server model, we prove asymptotic optimality of the greedy random (GRAND) algorithm in the sense of minimizing the weighted (by type) number of occupied servers in steady state. (This version of GRAND generalizes that introduced by Stolyar and Zhong (2015) for homogeneous systems, with all servers of the same type.) We then introduce a natural extension of the GRAND algorithm for finite-server systems with blocking. Assuming subcritical system load, we prove existence, uniqueness, and local stability of the large-scale system equilibrium point such that no blocking occurs. This result strongly suggests a conjecture that the steady-state blocking probability under the algorithm vanishes in the large-scale limit.

Large-scale Service Systems with Packing Constraints and Heterogeneous Servers

A service system with multiple types of arriving customers and multiple types of servers is considered. Several customers (possibly of different types) can be placed for concurrent service into same server, subject to "packing" constraints, which depend on the server type. Service times of different customers are independent, even if served simultaneously by the same server. The largescale asymptotic regime is considered such that the customer arrival rates grow to infinity. We consider two variants of the model. For the infinite-server model, we prove asymptotic optimality of the Greedy Random (GRAND) algorithm in the sense of minimizing the weighted (by type) number of occupied servers in steadystate. (This version of GRAND generalizes that introduced in [1] for the homogeneous systems, with all servers of same type.) We then introduce a natural extension of GRAND algorithm for finite-server systems with blocking. Assuming subcritical system load, we prove existence, uniqueness, and local stability of the large-scale system equilibrium point such that no blocking occurs. This result strongly suggests a conjecture that the steady-state blocking probability under the algorithm vanishes in the large-scale limit.