Request Dispatcher Research Articles

Dual Optimization of Revenue and Expense in Geo-Distributed Data Centers Using Smart Grid

Exorbitant energy expenses can supersede data center profits. Electricity prices often vary across the geographic regions, caused by gaps in the supply-demand, time of use, and production cost factors. Geo-distributed cloud data centers facilitated by a smart grid and enabled by cloud computing can potentially utilize the spatiotemporal diversity of energy prices to reduce operational expenditure and maximize profit. In this paper, we solve the data center profit by formulating it as a constrained multi-objective optimization problem. The proposed solution utilizes an evolutionary algorithm-based higher-level heuristic that optimizes data center revenue and expense objectives simultaneously. The proposed technique provides system managers with trade-off solutions suited to varied operational scenarios. Ours is a multi-step approach, utilizing the optimization scheme to obtain Pareto optimal solutions for the request dispatch and resource allocation problem. When broadly evaluated against a comparative resource optimization scheme, our technique increases revenue while lowering expense and collectively yields a higher profit. It exhibits such performance over a broad range of price changes regardless of the data center's size and utilization level. The extensive simulation results ascertain the effectiveness of the proposed approach across a myriad of system parameters.

EdgeMatrix: A Resource-Redefined Scheduling Framework for SLA-Guaranteed Multi-Tier Edge-Cloud Computing Systems

With the development of networking technology, the computing system has evolved towards the multi-tier paradigm gradually. However, challenges, such as multi-resource heterogeneity of devices, resource competition of services, and networked system dynamics, make it difficult to guarantee service-level agreement (SLA) for the applications. In this paper, we propose a multi-tier edge-cloud computing framework, EdgeMatrix, to maximize the throughput of the system while guaranteeing different SLA priorities. First, in order to reduce the impact of physical resource heterogeneity, EdgeMatrix introduces the Networked Multi-agent Actor-Critic (NMAC) algorithm to re-define physical resources with the same quality of service as logically isolated resource units and combinations, i.e., cells and channels. In addition, a multi-task mechanism is designed in EdgeMatrix to solve the problem of Joint Service Orchestration and Request Dispatch (JSORD) for matching the requests and services, which can significantly reduce the optimization runtime. For integrating above two algorithms, EdgeMatrix is designed with two time-scales, i.e., coordinating services and resources at the larger time-scale, and dispatching requests at the smaller time-scale. Realistic trace-based experiments proves that the overall throughput of EdgeMatrix is 36.7% better than that of the closest baseline, while the SLA priorities are guaranteed still.

A Joint Model Provisioning and Request Dispatch Solution for Low-Latency Inference Services on Edge.

With the advancement of machine learning, a growing number of mobile users rely on machine learning inference for making time-sensitive and safety-critical decisions. Therefore, the demand for high-quality and low-latency inference services at the network edge has become the key to modern intelligent society. This paper proposes a novel solution that jointly provisions machine learning models and dispatches inference requests to reduce inference latency on edge nodes. Existing solutions either direct inference requests to the nearest edge node to save network latency or balance edge nodes’ workload by reducing queuing and computing time. The proposed solution provisions each edge node with the optimal number and type of inference instances under a holistic consideration of networking, computing, and memory resources. Mobile users can thus be directed to utilize inference services on the edge nodes that offer minimal serving latency. The proposed solution has been implemented using TensorFlow Serving and Kubernetes on an edge cluster. Through simulation and testbed experiments under various system settings, the evaluation results showed that the joint strategy could consistently achieve lower latency than simply searching for the best edge node to serve inference requests.

Centaur

Due to the high density storage demand coming from applications from different domains, 3D NAND flash is becoming a promising candidate to replace 2D NAND flash as the dominant non-volatile memory. However, denser 3D NAND presents various performance and reliability issues, which can be addressed by the 3D NAND specific full-sequence program (FSP) operation. The FSP programs multiple pages simultaneously to mitigate the performance degradation caused by the long latency 3D NAND baseline program operations. However, the FSP-enabled 3D NAND-based SSDs introduce lifetime degradation due to the larger write granularities accessed by the FSP. To address the lifetime issue, in this paper, we propose and experimentally evaluate Centaur, a heterogeneous 2D/3D NAND heterogeneous SSD, as a solution. Centaur has three main components: a lifetime-aware inter-NAND request dispatcher, a lifetime-aware inter-NAND work stealer, and a data migration strategy from 2D NAND to 3D NAND. We used twelve SSD workloads to compare Centaur against a state-of-the-art 3D NAND-based SSD with the same capacity. Our experimental results indicate that the SSD lifetime and performance are improved by 3.7x and 1.11x, respectively, when using our 2D/3D heterogeneous SSD.

Centaur

Due to the high density storage demand coming from applications from different domains, 3D NAND flash is becoming a promising candidate to replace 2D NAND flash as the dominant non-volatile memory. However, denser 3D NAND presents various performance and reliability issues, which can be addressed by the 3D NAND specific full-sequence program (FSP) operation. The FSP programs multiple pages simultaneously to mitigate the performance degradation caused by the long latency 3D NAND baseline program operations. However, the FSP-enabled 3D NAND-based SSDs introduce lifetime degradation due to the larger write granularities accessed by the FSP. To address the lifetime issue, in this paper, we propose and experimentally evaluate Centaur, a heterogeneous 2D/3D NAND heterogeneous SSD, as a solution. Centaur has three main components: a lifetime-aware inter-NAND request dispatcher, a lifetime-aware inter-NAND work stealer, and a data migration strategy from 2D NAND to 3D NAND. We used twelve SSD workloads to compare Centaur against a state-of-the-art 3D NAND-based SSD with the same capacity. Our experimental results indicate that the SSD lifetime and performance are improved by 3.7x and 1.11x, respectively, when using our 2D/3D heterogeneous SSD.

Hierarchical resource allocation and consolidation framework in a multi‐core server cluster using a Markov decision process model

This paper investigates a service level agreements (SLAs)-based resource allocation problem in a server cluster. The objective is to maximise the total profit, which is the total revenue minus the operational cost of the server cluster. The total revenue depends on the average request response time, whereas the operating cost depends on the total energy consumption of the server cluster. A joint optimisation framework is proposed, comprised of request dispatching, dynamic voltage and frequency scaling (DVFS) for individual cores of the servers, as well as server- and core-level consolidations. Each DVFS-enabled core in the server cluster is modelled by using a continuous-time Markov decision process (CTMDP). A near-optimal solution comprised of a central manager and distributed local agents is presented. Each local agent employs linear programming-based CTMDP solving method to solve the DVFS problem for the corresponding core. On the other hand, the central manager solves the request dispatch problem and finds the optimal number of ON cores and servers, thereby achieving a desirable tradeoff between service response time and power consumption. To reduce the computational overhead, a two-tier hierarchical solution is utilized. Experimental results demonstrate the outstanding performance of the proposed algorithm over the baseline algorithms.

A Stackelberg Game-Based Optimization Framework of the Smart Grid With Distributed PV Power Generations and Data Centers

The emergence of cloud computing has established a trend toward building massive, energy-hungry, and geographically distributed data centers. Due to their enormous energy consumption, data centers are expected to have a major impact on the electric power grid by significantly increasing the load at locations where they are built. Dynamic energy pricing policies in the recently proposed smart power grid technology can incentivize the cloud controller to shift the computation load toward data centers in regions with cheaper electricity or with excessive electricity generated by renewable energy sources, e.g., photovoltaic (PV) and wind power. On the other hand, distributed data centers in the cloud also provide opportunities to help the power grid with distributed renewable energy sources to improve robustness and load balancing. To shed some light into these opportunities, this paper considers an interaction system of the smart power grid with distributed PV power generation and the cloud computing system, jointly accounting for the service request dispatch and routing problem in the cloud with the power flow analysis in power grid. The Stackelberg (sequential) game formulation is provided for the interaction system under two different dynamic pricing scenarios: 1) real-time power-dependent pricing; and 2) time-ahead pricing. The two players in the Stackelberg games are the power grid controller that sets the pricing signal and the cloud controller that performs resource allocation among data centers. The objective of the power grid controller is to maximize its own profit and perform load balancing among power buses, i.e., minimizing the power flow from one power bus to the others, whereas the objective of the cloud computing controller is to maximize its own profit with respect to the location-dependent pricing signal. Based on the backward induction method, this paper derives the near-optimal or suboptimal strategies of the two players in Stackelberg game using convex optimization and simulated annealing techniques.

Restful Implementation of Catalogue Service for Geospatial Data Provenance

Abstract. Provenance, also known as lineage, is important in understanding the derivation history of data products. Geospatial data provenance helps data consumers to evaluate the quality and reliability of geospatial data. In a service-oriented environment, where data are often consumed or produced by distributed services, provenance could be managed by following the same service-oriented paradigm. The Open Geospatial Consortium (OGC) Catalogue Service for the Web (CSW) is used for the registration and query of geospatial data provenance by extending ebXML Registry Information Model (ebRIM). Recent advance of the REpresentational State Transfer (REST) paradigm has shown great promise for the easy integration of distributed resources. RESTful Web Service aims to provide a standard way for Web clients to communicate with servers based on REST principles. The existing approach for provenance catalogue service could be improved by adopting the RESTful design. This paper presents the design and implementation of a catalogue service for geospatial data provenance following RESTful architecture style. A middleware named REST Converter is added on the top of the legacy catalogue service to support a RESTful style interface. The REST Converter is composed of a resource request dispatcher and six resource handlers. A prototype service is developed to demonstrate the applicability of the approach.

Coordination Algorithm Design of the Distributed Storage Proxy Servers for Streaming Media

For streaming media proxy server systems which are geographically far apart, distributed streaming media proxy server clusters can be a good solution to reduce upward bandwidth consumption, shorten access distance, and improve service capacity; the key is to solve the coordination problem among the proxy servers. The decision problem of the user request dispatch actually belongs to NP-complete. This article proposes an online coordination algorithm for the distributed storage streaming media proxy servers, and discusses the relative deviation of the approximate solution from the optimal solution, to serve as a reference to the specific applications of clustered streaming media proxy servers.

Coordination Algorithm Design of the Load Sharing Proxy Servers for Streaming Media

With the rapid development of streaming media applications, clustered streaming media proxy servers have become the mainstream of high-end streaming media proxy. To increase the IO throughput capacity of streaming media proxy servers, the most effective method is middle-ware technology in which several servers are bridged together to form a proxy cache server cluster, and the key is the coordination of individual proxy servers. The decision problem of the user request dispatch actually belongs to NP-complete. This article proposes an online coordination algorithm for the load sharing streaming media proxy servers, and discusses the relative deviation of the online algorithm from the optimal solution, to serve as a reference to the specific applications of clustered streaming media proxy servers.

“SOS 112 what has occurred?” Managing openings in children's emergency calls

This article examines the initial exchanges in calls to the Swedish emergency services, focusing on callers’ responses to the standardised opening phrase SOS one one two, what has occurred?. Comparisons across three age groups – children, teenagers, and adults – revealed significant differences in caller behaviour. Whereas teenagers and adults offered reports of the incident, child callers were more prone to request dispatch of specific assistance units. This pattern was only observable when children were accompanied by an adult relative, which leads us to propose that child callers may be operating under prior adult instruction concerning how to request help. The second part of the analysis examines the local organisation of participants’ actions, showing how turn-design and sequencing manifest the local concerns of the two parties. The analysis thus combines quantitative and qualitative methods to explore the ways through which the parties jointly produce an early sense of emergency incidents. These results are discussed in terms of children’s agency and competence as informants granted to them by emergency operators, and how such competence ascriptions run against commonsense conceptualisations of children as less-than-full-fledged members of society.

Adaptive admission control algorithm in a QoS-aware Web system

Internet traffic tends to show significant growth of demand at certain times of the day, or in response to special events. The consequence of these traffic peaks is that Web systems that are responding to user demands are congested due to their inability to serve a large volume of requests. The case for admission control in these situations is even stronger when Quality of Service (QoS) is considered as a primary objective in the Web system. In this work, we address two issues: on one hand, we consider and compare five throughput predictors to be used in a Web system in order to track its performance and, on the other hand, we propose a QoS-aware admission control and load balancing algorithm that prevents the Web system from sudden overload. The admission control algorithm is based on a resource allocation scheme that includes a throughput predictor. In order to obtain a low overhead, the monitoring of traffic arriving at the Web system is performed following an adaptive time slot scheduling based on the burstiness factor that we defined in previous work. Results show the benefits of our adaptive time slot scheduling compared to a fixed time scheduling. A discussion of the results of the five throughput predictors and the admission control algorithm is provided. We also compare the performance of our algorithm with Intelligent Queue-based Request Dispatcher (IQRD). The algorithm is designed to be included in a Web system composed by a set of Web servers distributed locally, which can also form part of a wider geographically distributed load balancing architecture.

A content-based load balancing algorithm with admission control for cluster web servers

With the growing demands for web-based applications, cluster web servers emerged as a reliable and leading resource in internet infrastructure. Managing performance of the cluster web servers under heavy load conditions is a critical task specifically when it comes to the advent of dynamic contents and database-driven applications. In this paper we propose a new load balancing algorithm namely IQRD (Intelligent Queue-based Request Dispatcher) for web-switches of the cluster web servers which operates at layer-7. The IQRD aims to achieve better load balancing with the help of request classification, performance isolation and dynamic remaining capacity estimation mechanisms. For this, a queuing model was employed for each class of requests in each node of the cluster to provide an estimation regarding the node remaining capacity. This value is used as a load descriptor (index) in the load balancing algorithm and also used by the admission control mechanism. The implementation results with synthetic and realistic workloads confirm that IQRD effectively balances loads among servers in the cluster and achieves better response time and throughput compared to other load balancing algorithms. However, the IQRD algorithm offered more processing overheads both in the web-switch and the web servers, but presented a better load balancing among web servers, even when the request rates were beyond the cluster capacity.

Efficient support for content‐aware request distribution and persistent connection in Web clusters

AbstractTo support Web clusters with efficient dispatching mechanisms and policies, we propose a light‐weight TCP connection transfer mechanism, TCP Rebuilding, and use it to develop a content‐aware request dispatching platform, LVS‐CAD, in which the request dispatcher can extract and analyze the content in requests and then dispatch each request by its content or type of service requested. To efficiently support HTTP/1.1 persistent connection in Web clusters, request scheduling should be performed per request rather than per connection. Consequently, multiple TCP Rebuilding, as an extension to normal TCP Rebuilding, is proposed and implemented. On this platform, we also devise fast TCP module handshaking to process the handshaking between clients and the request dispatcher in the IP layer instead of in the TCP layer for faster response times. Furthermore, we also propose content‐aware request distribution policies that consider cache locality and various types of costs for dispatching requests in this platform, which makes the resourceutilization of Web servers more effective. Experimental results of a practical implementation on Linux show that the proposed system, mechanisms, and policies can effectively improve the performance of Web clusters. Copyright © 2007 John Wiley & Sons, Ltd.

A performance study of robust load sharing strategies for distributed heterogeneous Web server systems

Replication of information across multiple servers is becoming a common approach to support popular Web sites. A distributed architecture with some mechanisms to assign client requests to Web servers is more scalable than any centralized or mirrored architecture. In this paper, we consider distributed systems in which the Authoritative Domain Name Server (ADNS) of the Web site takes the request dispatcher role by mapping the URL hostname into the IP address of a visible node, that is, a Web server or a Web cluster interface. This architecture can support local and geographical distribution of the Web servers. However, the ADNS controls only a very small fraction of the requests reaching the Web site because the address mapping is not requested for each client access. Indeed, to reduce Internet traffic, address resolution is cached at various name servers for a time-to-live (TTL) period. This opens an entirely new set of problems that traditional centralized schedulers of parallel/distributed systems do not have to face. The heterogeneity assumption on Web node capacity, which is much more likely in practice, increases the order of complexity of the request assignment problem and severely affects the applicability and performance of the existing load sharing algorithms. We propose new assignment strategies, namely adaptive TTL schemes, which tailor the TTL value for each address mapping instead of using a fixed value for all mapping requests. The adaptive TTL schemes are able to address both the nonuniformity of client requests and the heterogeneous capacity of Web server nodes. Extensive simulations show that the proposed algorithms are very effective in avoiding node overload, even for high levels of heterogeneity and limited ADNS control.