Service Request Scheduling Research Articles

QoS-aware edge AI placement and scheduling with multiple implementations in FaaS-based edge computing

Resource constraints on the computing continuum require that we make smart decisions for serving AI-based services at the network edge. AI-based services typically have multiple implementations (e.g., image classification implementations include SqueezeNet, DenseNet, and others) with varying trade-offs (e.g., latency and accuracy). The question then is how should AI-based services be placed across Function-as-a-Service (FaaS) based edge computing systems in order to maximize total Quality-of-Service (QoS). To address this question, we propose a problem that jointly aims to solve (i) edge AI service placement and (ii) request scheduling. These are done across two time-scales (one for placement and one for scheduling). We first cast the problem as an integer linear program. We then decompose the problem into separate placement and scheduling subproblems and prove that both are NP-hard. We then propose a novel placement algorithm that places services while considering device-to-device communication across edge clouds to offload requests to one another. Our results show that the proposed placement algorithm is able to outperform a state-of-the-art placement algorithm for AI-based services, and other baseline heuristics, with regard to maximizing total QoS. Additionally, we present a federated learning-based framework, FLIES, to predict the future incoming service requests and their QoS requirements. Our results also show that our FLIES algorithm is able to outperform a standard decentralized learning baseline for predicting incoming requests and show comparable predictive performance when compared to centralized training.

EdgeOptimizer: A programmable containerized scheduler of time-critical tasks in Kubernetes-based edge-cloud clusters

Edge computing has garnered significant attention in recent years, leading to the evolution of more delay-sensitive applications towards a three-tier architecture with edge-cloud collaboration. Concurrently, technologies associated with containerization have been maturing. Notably, K8s(Kubernetes) emerges as a prominent solution for the management of extensive, dynamically evolving, and intricate container clusters. However, optimizing performance in a K8s-based architecture requires careful consideration, as intelligent algorithms cannot be easily rehearsed or retracted, and a poorly functioning algorithm can result in significant damage. To enhance the success rate of time-critical task execution in real production environments, it is crucial to provide an algorithm optimizer for service orchestration and request dispatching, especially when dealing with different services. Traditionally, building a K8s-based experimental system during the early stages of experiments has been time-consuming and involved significant programming efforts. In this paper, we introduce EdgeOptimizer, a decoupled and modularized optimizer for scheduling in multiple clusters. EdgeOptimizer also serves as an online testbed for algorithm verification in K8s-based systems, offering detailed configuration options to facilitate cluster management. By collecting system information and employing an interface-oriented system architecture, EdgeOptimizer enables users to quickly develop, deploy, and switch between various algorithms, significantly reducing the upfront costs associated with setting up an experimental environment. We evaluated the performance of EdgeOptimizer based on overall overload and demonstrated its scalability and effectiveness in verifying the efficacy of service orchestration and request dispatching algorithms for time-critical tasks. Our findings illustrate the value of EdgeOptimizer in improving the overall success rate of executing time-critical tasks, thus highlighting its potential in real-world scenarios.

Request Scheduling Combined With Load Balancing in Mobile-Edge Computing

Edge servers can cache some delay-sensitive and resource-intensive applications to reduce the delay of user tasks. However, due to the limited resources of edge servers, they cannot cache all services and process all user requests like remote clouds. In order to take advantage of the low latency, we need to make full use of the limited resources of the edge servers and reasonably allocate user requests to edge servers for processing. Meanwhile, in order to maintain the efficient and long-term operation of the server cluster, we should also consider the load balancing of the cluster. How to balance the server load while making users have the best experience is an urgent problem to be solved. Solving the problem faces the challenge of the interaction between service placement and request scheduling, the tradeoff between communication and computation, and the consideration of response time and edge server load. We propose the service placement algorithm based on user visits and our request scheduling algorithm based on the simulated annealing algorithm. We verify the superiority of our algorithms in response time and server load balancing. The experimental results based on the real data sets prove our algorithm can cope with the actual situation and quickly converge to a favorable value.

Integrating deep reinforcement learning with pointer networks for service request scheduling in edge computing

With the increasing popularity of edge computing, service providers are more likely to deploy services at the edge of the network to reduce the latency of service requests. However, the resources offered by edge servers are extremely limited compared to those in the cloud. Therefore, a challenging issue in edge computing is how to sufficiently utilize service resources at the edge to satisfy as many service requests as possible. Existing service request scheduling methods mainly use a single optimization objective, e.g., resource utilization or running time. In this paper, the issue of service request scheduling with multiple requests is modeled as a sequential problem, where multiple optimization objectives, including resource utilization, running time, and waiting time, are involved. A reinforcement learning model with pointer networks is proposed to construct scheduling policies. Experiments conducted on three representative real-world datasets show that our proposed approach outperforms several state-of-the-art methods on the three metrics.

Dynamic service placement and request scheduling for edge networks

Multi-access computing (MEC) represents an effective technology to satisfy low-latency demands by capacitating the execution of data-intensive and latency-sensitive tasks at the edge of mobile networks. While recent studies have investigated cost-awareness regarding where to execute service tasks and how to schedule user requests to edge servers, most have primarily leveraged deep reinforcement learning or concentrated on less-complicated assumptions. In this paper, we design an online algorithm for joint service placement and request scheduling in MEC networks, subjected to multi-dimensional constraints, aiming to minimize the operational cost. However, the problem is non-trivial since it involves time-correlated service placement cost and future information. Our proposed online algorithm employs the online learning technique and rounding method to address these challenges, consisting of the following basic modules: (1) a regularization approach to decouple the offline cost-minimization problem into multiple convex sub-problems, each to be efficiently solved in each time slot; (2) a rounding method to transform an optimal fractional solution of the convex sub-problem into an integer solution of the original cost-minimization problem, with provable the performance guarantee. Our analytical results and simulations verify the effectiveness of the proposed online algorithm.

Service Placement and Request Scheduling for Data-Intensive Applications in Edge Clouds

Mobile edge computing provides the opportunity for wireless users to exploit the power of cloud computing without a large communication delay. To serve data-intensive applications (e.g., video analytics, machine learning tasks) from the edge, we need, in addition to computation resources, storage resources for storing server code and data as well as network bandwidth for receiving user-provided data. Moreover, due to time-varying demands, the code and data placement needs to be adjusted over time, which raises concerns of system stability and operation cost. In this paper, we address these issues by proposing a two-time-scale framework that jointly optimizes service (code and data) placement and request scheduling, while considering storage, communication, computation, and budget constraints. First, by analyzing the hardness of various cases, we completely characterize the complexity of our problem. Next, we develop a polynomial-time service placement algorithm by formulating our problem as a set function optimization, which attains a constant-factor approximation under certain conditions. Furthermore, we develop a polynomial-time request scheduling algorithm by computing the maximum flow in a carefully constructed auxiliary graph, which satisfies hard resource constraints and is provably optimal in the special case where requests have homogeneous resource demands. Extensive synthetic and trace-driven simulations show that the proposed algorithms achieve 90% of the optimal performance.

Joint Chain-Based Service Provisioning and Request Scheduling for Blockchain-Powered Edge Computing

Blockchain-powered edge computing (BEC) is a promising extension to strengthen the security and the trustworthiness among collaborative edge clouds for delivering computation-intensive and delay-sensitive services in the environments of IoT and 5G. A fundamental challenge is how to respond to the maximum number of IoT requests at the network edge instead of the remote cloud. Although some work has been done to consider service provisioning and request scheduling in collaborative edge clouds, they assume that a single service is used to respond to each request. This assumption, however, is not practical to meet the demand of emerging IoT applications. In reality, the request needs to call a set of services with a chain-based structure. To tackle this challenge, in this article, we first propose a chain-based service request model for emerging IoT applications and further study the joint service provisioning and request scheduling problem for chain-based service requests at the network edge. We characterize this problem as an integer linear programming (ILP) model and prove the NP-hardness of this joint optimization problem. Furthermore, we prove that the related problem is of approximate submodularity with an approximation ratio guarantee. Finally, a novel two-stage optimization (TSO) scheme is proposed, and the results of extensive experiments show the efficiency and the effectiveness of the TSO scheme.

Scheduling Internet of Things requests to minimize latency in hybrid Fog–Cloud​ computing

Delivering services for Internet of Things (IoT) applications that demand real-time and predictable latency is a challenge. Several IoT applications require stringent latency requirements due to the interaction between the IoT devices and the physical environment through sensing and actuation. The limited capabilities of IoT devices require applications to be integrated in Cloud and Fog computing paradigms. Fog computing significantly improves on the service latency as it brings resources closer to the edge. The characteristics of both Fog and Cloud computing will enable the integration and interoperation of a large number of IoT devices and services in different domains.This work models the scheduling of IoT service requests as an optimization problem using integer programming in order to minimize the overall service request latency. The scheduling problem by nature is NP-hard, and hence, exact optimization solutions are inadequate for large size problems. This work introduces a customized implementation of the genetic algorithm (GA) as a heuristic approach to schedule the IoT requests to achieve the objective of minimizing the overall latency. The GA is tested in a simulation environment that considers the dynamic nature of the environment. The performance of the GA is evaluated and compared to the performance of waited-fair queuing (WFQ), priority-strict queuing (PSQ), and round robin (RR) techniques. The results show that the overall latency for the proposed approach is 21.9% to 46.6% better than the other algorithms. The proposed approach also showed significant improvement in meeting the requests deadlines by up to 31%.

Utility-Centric Service Provisioning in Multi-Access Edge Computing

Recently, multi-access edge computing (MEC) is a promising paradigm to offer resource-intensive and latency-sensitive services for IoT devices by pushing computing functionalities away from the core cloud to the edge of networks. Most existing research has focused on effectively improving the use of computing resources for computation offloading while neglecting non-trivial amounts of data, which need to be pre-stored to enable service execution (e.g., virtual/augmented reality, video analytics, etc.). In this paper, we, therefore, investigate service provisioning in MEC consisting of two sub-problems: (i) service placement determining services to be placed in each MEC node under its storage capacity constraint, and (ii) request scheduling determining where to schedule each request considering network delay and computation limitation of each MEC node. The main objective is proposed to ensure the quality of experience (QoE) of users, which is also yet to be studied extensively. A utility function modeling user perception of service latency is used to evaluate QoE. We formulate the problem of service provisioning in MEC as an Integer Nonlinear Programming (INLP), aiming at maximizing the total utility of all users. We then propose a Nested-Genetic Algorithm (Nested-GA) consisting of two genetic algorithms, each of whom solves a sub-problem regarding service placement or request scheduling decisions. Finally, simulation results demonstrate that our proposal outperforms conventional methods in terms of the total utility and achieves close-to-optimal solutions.

Deadline-Constrained Connection Request Scheduling in Mobile Relay-Assisted LTE Networks

Connection and service request scheduling affects the performance of a mobile relay node (MRN) in a relay-assisted long-term evolution network in terms of crucial quality-of-service (QoS) metrics such as the blocking probability and the system's response time. In this paper, a novel Deadline-constrained Connection and service Request Scheduling (DCRS) scheme is proposed with the objective of improving an MRN's performance in terms of the above-mentioned metrics. In this paper, a stochastic queueing model is formulated for the purpose of capturing the MRN's behavioral dynamics and evaluating its performance as it operates under DCRS. Simulations are conducted in order to verify the model's results’ validity and accuracy. Moreover, the MRN performance achieved under DCRS is compared to that achieved under the typically adopted first-come-first-service (FCFS) scheme. Results indicate that DCRS remarkably outperforms FCFS.

Sustainable Service Allocation Using a Metaheuristic Technique in a Fog Server for Industrial Applications

Reducing energy consumption in the fog computing environment is both a research and an operational challenge for the current research community and industry. There are several industries such as finance industry or healthcare industry that require a rich resource platform to process big data along with edge computing in fog architecture. As a result, sustainable computing in a fog server plays a key role in fog computing hierarchy. The energy consumption in fog servers depends on the allocation techniques of services (user requests) to a set of virtual machines (VMs). This service request allocation in a fog computing environment is a nondeterministic polynomial-time hard problem. In this paper, the scheduling of service requests to VMs is presented as a bi-objective minimization problem, where a tradeoff is maintained between the energy consumption and makespan. Specifically, this paper proposes a metaheuristic-based service allocation framework using three metaheuristic techniques, such as particle swarm optimization (PSO), binary PSO, and bat algorithm. These proposed techniques allow us to deal with the heterogeneity of resources in the fog computing environment. This paper has validated the performance of these metaheuristic-based service allocation algorithms by conducting a set of rigorous evaluations.

Service deployment and scheduling for improving performance of composite cloud services

Cloud computing and services have greatly changed the way how people develop and use software, and also raised a lot of new research issues. In this paper, we investigate two such important issues, service deployment and service request scheduling, for composite cloud services in dynamic cloud environments. We present load-aware service deployment approaches for dynamic workload and a service request scheduling method based on task ranking mechanisms. The proposed approaches can effectively improve the execution performance of composite cloud services in dynamic cloud environments. A series of simulation experiments have been conducted to evaluate the proposed approaches. The experimental results show that our approaches outperform previous methods significantly in terms of average service response time.

Service Request Scheduling based on Quantification Principle using Conjoint Analysis and Z-score in Cloud

Service request scheduling has a major impact on the performance of the service processing design in a large-scale distributed computing environment like cloud systems. It is desirable to have a service request scheduling principle that evenly distributes the workload among the servers, according to their capacities. The capacities of the servers are termed high or low relative to one another. Therefore, there is a need to quantify the server capacity to overcome this subjective assessment. Subsequently, a method to split and distribute the service requests based on this quantified server capacity is also needed. The novelty of this research paper is to address these requirements by devising a service request scheduling principle for a heterogeneous distributed system using appropriate statistical methods, namely Conjoint analysis and Z-score. Suitable experiments were conducted and the experimental results show considerable improvement in the performance of the designed service request scheduling principle compared to a few other existing principles. Areas of further improvement have also been identified and presented.

Dynamic Service Request Scheduling for Mobile Edge Computing Systems

Nowadays, mobile services (applications) running on terminal devices are becoming more and more computation‐intensive. Offloading the service requests from terminal devices to cloud computing can be a good solution, but it would put a high burden on the network. Edge computing is an emerging technology to solve this problem, which places servers at the edge of the network. Dynamic scheduling of offloaded service requests in mobile edge computing systems is a key issue. It faces challenges due to the dynamic nature and uncertainty of service request patterns. In this article, we propose a Dynamic Service Request Scheduling (DSRS) algorithm, which makes request scheduling decisions to optimize scheduling cost while providing performance guarantees. The DSRS algorithm can be implemented in an online and distributed way. We present mathematical analysis which shows that the DSRS algorithm can achieve arbitrary tradeoff between scheduling cost and performance. Experiments are also carried out to show the effectiveness of the DSRS algorithm.

A Mobile Battery Swapping Service for Electric Vehicles Based on a Battery Swapping Van

This paper presents a novel approach for providing a mobile battery swapping service for electric vehicles (EVs) that is provided by a mobile battery swapping van. This battery swapping van can carry many fully charged batteries and drive up to an EV to swap a battery within a few minutes. First, a reasonable EV battery swapping architecture based on a battery swapping van is established in this paper. The function and role of each participant and the relationships between each participant are determined, especially their changes compared with the battery charging service. Second, the battery swapping service is described, including the service request priority and service request queuing model. To provide the battery swapping service efficiently and effectively, the battery swapping service request scheduling is analyzed well, and a minimum waiting time based on priority and satisfaction scheduling strategy (MWT-PS) is proposed. Finally, the battery swapping service is simulated, and the performance of MWT-PS is evaluated in simulation scenarios. The simulation results show that this novel approach can be used as a reference for a future system that provides reasonable and satisfying battery swapping service for EVs.

Cost-Aware Cloud Service Request Scheduling for SaaS Providers

As cloud computing becomes widely deployed, more and more cloud services are offered to end users in a pay-as-you-go manner. Today’s increasing number of end user-oriented cloud services are generally operated by Software as a Service (SaaS) providers using rental virtual resources from third-party infrastructure vendors. As far as SaaS providers are concerned, how to process the dynamic user service requests more cost-effectively without any SLA violation is an intractable problem. To deal with this challenge, we first establish a cloud service request model with SLA constraints, and then present a cost-aware service request scheduling approach based on genetic algorithm. According to the personalized features of user requests and the current system load, our approach can not only lease and reuse virtual resources on demand to achieve optimal scheduling of dynamic cloud service requests in reasonable time, but can also minimize the rental cost of the overall infrastructure for maximizing SaaS providers’ profits while meeting SLA constraints. The comparison of simulation experiments indicates that our proposed approach outperforms other revenue-aware algorithms in terms of virtual resource utilization, rate of return on investment and operation profit and provides a cost-effective solution for service request scheduling in cloud computing environments.

Profit-driven scheduling for cloud services with data access awareness

Resource sharing between multiple tenants is a key rationale behind the cost effectiveness in the cloud. While this resource sharing greatly helps service providers improve resource utilization and increase profit, it impacts on the service quality (e.g., the performance of consumer applications). In this paper, we address the reconciliation of these conflicting objectives by scheduling service requests with the dynamic creation of service instances. Specifically, our scheduling algorithms attempt to maximize profit within the satisfactory level of service quality specified by the service consumer. Our contributions include (1) the development of a pricing model using processor-sharing for clouds (i.e., queuing delay is embedded in processing time), (2) the application of this pricing model to composite services with dependency consideration, (3) the development of two sets of service request scheduling algorithms, and (4) the development of a prioritization policy for data service aiming to maximize the profit of data service.

Adaptive quality of service management for enterprise services

In the past, enterprise resource planning systems were designed as monolithic software systems running on centralized mainframes. Today, these systems are (re-)designed as a repository of enterprise services that are distributed throughout the available computing infrastructure. These service oriented architectures (SOAs) require advanced automatic and adaptive management concepts in order to achieve a high quality of service level in terms of, for example, availability, responsiveness, and throughput. The adaptive management has to allocate service instances to computing resources, adapt the resource allocation to unforeseen load fluctuations, and intelligently schedule individual requests to guarantee negotiated service level agreements (SLAs). Our AutoGlobe platform provides such a comprehensive adaptive service management comprising —static service-to-server allocation based on automatically detected service utilization patterns, —adaptive service management based on a fuzzy controller that remedies exceptional situations by automatically initiating, for example, service migration, service replication (scale-out), and —adaptive scheduling of individual service requests that prioritizes requests depending on the current degree of service level conformance. All three complementary control components are described in detail, and their effectiveness is analyzed by means of realistic business application scenarios.