Microservice Instance Research Articles

Microservice instances selection and load balancing in fog computing using deep reinforcement learning approach

Fog-native computing is an emerging paradigm that makes it possible to build flexible and scalable Internet of Things (IoT) applications using microservice architecture at the network edge. With this paradigm, IoT applications are decomposed into multiple fine-grained microservices, strategically deployed on various fog nodes to support a wide range of IoT scenarios, such as smart cities and smart farming. Nonetheless, the performance of these IoT applications is affected by their limited effectiveness in processing offloaded IoT requests originating from multiple IoT devices. Specifically, the requested IoT services are composed of multiple dependent microservice instances collectively referred to as a service plan (SP). Each SP comprises a series of tasks designed to be executed in a predefined order, with the objective of meeting heterogeneous Quality of Service (QoS) requirements (e.g., low service delays). Different from the cloud, selecting the appropriate service plan for each IoT request can be a challenging task in dynamic fog environments due to the dependency and decentralization of microservice instances, along with the instability of network conditions and service requests (i.e., change quickly over time). To deal with this challenge, we study the microservice instances selection problem for IoT applications deployed on fog platforms and propose a learning-based approach that employs Deep Reinforcement Learning (DRL) to compute the optimal service plans. The latter optimizes the delay of application requests while effectively balancing the load among microservice instances. In our selection process, we carefully address the plan-dependency to efficiently select valid service plans for every request by introducing two distinct approaches; an action masking approach and an adaptive action mapping approach. Additionally, we propose an improved experience replay to address delayed action effects and enhance our model training efficiency. A series of experiments were conducted to assess the performance of our Microservice Instances Selection Policy (MISP) approach. The results demonstrate that our model reduces the average failure rate by up to 65% and improves load balance by up to 45% on average when compared to the baseline algorithms.

Fully Decentralized Horizontal Autoscaling for Burst of Load in Fog Computing

With the increasing number of Web of Things devices, the network and processing delays in the cloud have also increased. As a solution, fog computing has emerged, placing computational resources closer to the user to lower the communication overhead and congestion in the cloud. In fog computing systems, microservices are deployed as containers, which require an orchestration tool like Kubernetes to support service discovery, placement, and recovery. A key challenge in the orchestration of microservices is automatically scaling the microservices in case of an unpredictable burst of load. In cloud computing, a centralized autoscaler can monitor the deployed microservice instances and make scaling actions based on the monitored metric values. However, monitoring an increasing number of microservices in fog computing can cause excessive network overhead and thereby delay the time to scaling action. We propose DESA, a fully DEcentralized Self-adaptive Autoscaler through which microservice instances make their own scaling decisions, cloning or terminating themselves through self-monitoring. We evaluate DESA in a simulated fog computing environment with different numbers of fog nodes. Furthermore, we conduct a case study with the 1998 World Cup website access log, examining DESA’s performance in a realistic scenario. The results show that DESA successfully reduces the scaling reaction time in large-scale fog computing systems compared to the centralized approach. Moreover, DESA resulted in a similar maximum number of instances and lower average CPU utilization during bursts of load.

Benchmarking scalability of stream processing frameworks deployed as microservices in the cloud

Context:The combination of distributed stream processing with microservice architectures is an emerging pattern for building data-intensive software systems. In such systems, stream processing frameworks such as Apache Flink, Apache Kafka Streams, Apache Samza, Hazelcast Jet, or the Apache Beam SDK are used inside microservices to continuously process massive amounts of data in a distributed fashion. While all of these frameworks promote scalability as a core feature, there is only little empirical research evaluating and comparing their scalability. Objective:The goal of this study to obtain evidence about the scalability of state-of-the-art stream processing framework in different execution environments and regarding different scalability dimensions. Method:We benchmark five modern stream processing frameworks regarding their scalability using a systematic method. We conduct over 740 h of experiments on Kubernetes clusters in the Google cloud and in a private cloud, where we deploy up to 110 simultaneously running microservice instances, which process up to one million messages per second. Results:All benchmarked frameworks exhibit approximately linear scalability as long as sufficient cloud resources are provisioned. However, the frameworks show considerable differences in the rate at which resources have to be added to cope with increasing load. There is no clear superior framework, but the ranking of the frameworks depends on the use case. Using Apache Beam as an abstraction layer still comes at the cost of significantly higher resource requirements regardless of the use case. We observe our results regardless of scaling load on a microservice, scaling the computational work performed inside the microservice, and the selected cloud environment. Moreover, vertical scaling can be a complementary measure to achieve scalability of stream processing frameworks. Conclusion:While scalable microservices can be designed with all evaluated frameworks, the choice of a framework and its deployment has a considerable impact on the cost of operating it.

Boosting the visibility of services in microservice architecture

Monolithic software architectures are no longer sufficient for the highly complex software-intensive systems, which modern society depends on. Service Oriented Architecture (SOA) surpassed monolithic architecture due to its reusability, platform independency, ease of maintenance, and scalability. Recent SOA implementations made use of cloud-native architectural approaches such as microservice architecture, which has resulted in a new challenge: the discovery difficulties of services. One way to dynamically discover and route traffic to service instances is to use a service discovery tool to locate the Internet Protocol (IP) address and port number of a microservice. In the event that replicated microservice instances are found to provide the same function, it is crucial to select the right microservice that provides the best overall experience for the end-user. Parameters including success rate, efficiency, delay time, and response time play a vital role in establishing a microservice’s Quality of Service (QoS). These assessments can be performed by means of a live health-check service, or, alternatively, by making a prediction of the current state of affairs with the application of machine learning-based approaches. In this research, we evaluate the performance of several classification algorithms for estimating the quality of microservices using the QWS dataset containing traffic data of 2505 microservices. Our research also analyzed the boosting algorithms, namely Gradient Boost, XGBoost, LightGBM, and CatBoost to improve the overall performance. We utilized parameter optimization techniques, namely Grid Search, Random Search, Bayes Search, Halvin Grid Search, and Halvin Random Search to fine-tune the hyperparameters of our classifier models. Experimental results demonstrated that the CatBoost algorithm achieved the highest level of accuracy (90.42%) in predicting microservice quality.

Cost-Efficient Fault-Tolerant Workflow Scheduling for Deadline-Constrained Microservice-Based Applications in Clouds

Microservices are becoming increasingly popular in the construction of cloud applications. On the basis of containers, microservice instances can be implemented with high scalability and maintainability. Due to the need of ensuring various quality of service (QoS) requirements and the two-layer resource structure of containers and virtual machines (VMs), microservice workflow scheduling in clouds is a challenging problem to address. This paper proposes a heuristic algorithm GSMS to minimize execution cost of a microservice-based workflow application while satisfying deadline and reliability constraints. GSMS adopts a greedy fault-tolerant scheduling strategy for replicas of each task to select appropriate resources that meet the sub-deadline and minimize the cost until the sub-reliability is guaranteed. Furthermore, a resource adjustment strategy is incorporated into GSMS to further improve resource utilization. By conducting extensive experiments with several realistic workflow applications, in comparison with existing algorithms, the effectiveness and efficiency of GSMS in achieving lower execution cost and meeting deadline and reliability requirements are validated.

ProScale: Proactive Autoscaling for Microservice With Time-Varying Workload at the Edge

Deploying microservice instances on the edge device close to end users can provide on-site processing thus reducing request response time. Each microservice has multiple instances that can process requests in parallel. To achieve high processing efficiency, the number of these instances is scaled according to the workload, which is also known as autoscaling. Previous studies of microservice autoscaling in the edge computing environment lack in-depth consideration of time-varying workload, they assume that the workload of each microservice always depends on that of its upstream. However, through an analysis of Alibaba's microservice trace with hundreds of millions of records, we find that the assumption is impractical thus hurting autoscaling effectiveness. To solve this problem, we propose ProScale, a prediction-driven proactive autoscaling framework for microservices at the edge. ProScale proactively forecasts the workload for each individual microservice per timeslot. Then it utilizes an efficient online algorithm to leverage the predicting results to determine the instance number for each microservice jointly with making placement decisions. For each microservice instance deployed on the edge device, ProScale handles burst requests using a designed offloading strategy. In addition, ProScale can also balance the load for multiple instances of each microservice. Extensive trace-driven experiments show that ProScale has great scalability. It can reduce average response time by 96.7% and resource usage by 96.5% compared with existing strategies and designed baselines.

Kubernetes-Oriented Microservice Placement With Dynamic Resource Allocation

Microservices and Kubernetes are widely used in the development and operations of cloud-native applications. By providing automated placement and scaling, Kubernetes has become the main tool for managing microservices. However, existing work and Kubernetes fail to consider the dynamic competition and availability of microservices as well as the problem of shared dependency libraries among multiple microservice instances. To this end, this paper proposes an integer nonlinear microservice placement model for Kubernetes with the goal of cost minimization. Specifically, we calculate the number of instances based on microservice availability and construct a model in which the total resource demand of multiple microservice instances exceeds the appropriate proportion of node resources when dynamic resource competition exists and the size of the shared dependency library is less than the node storage capacity. Finally, this paper solves the microservice placement model using an improved genetic algorithm. The experimental results demonstrate that higher throughput is obtained with the same costs and that the same throughput is obtained with lower costs.

Service Routing in Multi-Tier Edge Computing: A Matching Game Approach

Although microservice envisioned as a promising approach for edge applications which can improves the development efficiency and the deployment productivity, it also leads to the operational complexity during service runtime. This leads to the emergence of service mesh, a decided infrastructure layer over microservices for reliable service-to-service communication. However, when migrating the service mesh to the multi-tier edge computing, the corresponding extensions on the top of service mesh is needed to overcome the challenges brought by the multi-tier edge servers, shared microservices and diverse QoS requirements. Therefore, in this paper, we investigate the problem of service routing to achieve the efficient microservice-based service provision in multi-tier edge computing. The objective is to route user requests to the optimal microservice instances with low service delay and resource cost. To this end, we first formulate the service routing problem as an integer non-linear optimization problem, which is NP-hard. Then we map it to a many-to-one matching problem by employing the matching game theory and propose a dependency-aware deferred acceptance algorithm with dynamic quota. The experimental results based on a real-world dataset demonstrate that our proposed algorithm can significantly outperform existing representative algorithms in terms of service delay and resource cost.

Towards cost-effective and robust AI microservice deployment in edge computing environments

As a newly emerged promising computing paradigm, Multi-access Edge Computing (MEC) is capable of energizing massive Internet-of-Things (IoT) devices around us and novel mobile applications, especially the computing-intensive and latency-sensitive ones. Meanwhile, featured by the rapid development of cloud-native technologies in recent years, delivering Artificial-Intelligence (AI) capabilities in a microservice way in the MEC environments comes true nowadays. However, currently MEC systems are still restricted by the limited computing resources and highly dynamic network topology, which leads to high service deployment/maintenance cost. Therefore, how to cost-effectively and robustly deploy edge AI microservices in failure-prone MEC environments has become a hot issue. In this study, we consider an edge AI microservice that can be implemented by composing multiple Deep Neural Networks (DNN) models, in this way, features of different DNN models are aggregated and the deployment cost can be further reduced while fulfilling the Quality-of-Service (QoS) constraint. We propose a Three-Dimension-Dynamic-Programming-based algorithm (TDDP) to yield cost-effective multi-DNN orchestration and load allocation plans. For the robust deployment of the yield orchestration plan, we also develop a robust microservice instance placement algorithm (TLLB) by considering the three levels of load balance including applications, servers, and DNN models. Experiments based on real-world edge environments have demonstrated that the proposed orchestration and placement methods can achieve lower deployment costs and less QoS loss when faced with edge node failures than traditional approaches.

Modelling and analysing the reliability for microservice‐based cloud application based on predicate Petri net

AbstractMicroservice design is a new paradigm of cloud application development. Different from monolithic design, microservice enjoys merits of fine‐grained and loosely coupled services, and it is becoming more and more popular. The application developed with microservice has a good advantage in independent development and flexible deployment, especially for complex distributed systems. However, there is a big gap between the reliability requirements and microservice‐based cloud applications. This article proposes a reliability model of microservice‐based cloud application by using predicate Petri net. First, a microservice reliability requirement is given, some basic concepts of predicate Petri net are defined with syntax and semantics. Second, a microservice reliability strategy is proposed, which uses microservice instances and circuit breaker to improve the reliability of the system. Based on the constructed microservice reliability model, the correctness of predicate Petri net modelling and the effectiveness of the strategies are proven theoretically. Finally, an example is given to illustrate the establishment and analysis process of the model, and several groups of experiments are carried out to verify the effectiveness and feasibility of the method. Experimental results show that the proposed microservice reliability strategy is effective.

A Kubernetes controller for managing the availability of elastic microservice based stateful applications

The architectural style of microservices has been gaining popularity in recent years. In this architectural style, small and loosely coupled modules are deployed and scaled independently to compose cloud-native applications. Carrier-grade service providers are migrating their legacy applications to a microservice based architecture running on Kubernetes which is an open source platform for orchestrating containerized microservice based applications. However, in this migration, service availability remains a concern. Service availability is measured as the percentage of time the service is provisioned. High Availability (HA) is achieved when the service is available at least 99.999% of the time. In this paper, we identify possible architectures for deploying stateful microservice based applications with Kubernetes and evaluate Kubernetes from the perspective of availability it provides for its managed applications. The results of our experiments show that the repair actions of Kubernetes cannot satisfy HA requirements, and in some cases cannot guarantee service recovery. Therefore, we propose an HA State Controller which integrates with Kubernetes and allows for application state replication and automatic service redirection to the healthy microservice instances by enabling service recovery in addition to the repair actions of Kubernetes. Based on experiments we evaluate our solution and compare the different architectures from the perspective of availability and scaling overhead. The results of our investigations show that our solution can improve the recovery time of stateful microservice based applications by 50%.

Vessel to shore data movement through the Internet of Floating Things: A microservice platform at the edge

SummaryThe rise of the Internet of Things has generated high expectations about the improvement in people's lifestyles. In the last decade, we saw several examples of instrumented cities where different types of data were gathered, processed, and made available to inspire the next generation of scientists and engineers. In this framework, sensors and actuators became leading actors of technologically pervasive urban environments. However, in coastal areas, marine data crowdsourcing is difficult to apply due to the challenging operational conditions, extremely unstable network connectivity, and security issues in data movement. To fill this gap, we present a novel version of our DYNAMO transfer protocol (DTP), a platform‐independent data mover framework where data collected on board of vessels are stored locally and then moved from the edge to the cloud when the operating conditions are favorable. We evaluate the performance of DTP in a controlled environment with a private cloud by measuring the time it takes for the clouds ide to process and store a fixed amount of data while varying the number of microservice instances. We show that the time decreases exponentially when the number of microservice instances goes from 1 to 16 and it remains constant above that number.

Multi-objective microservice deployment optimization via a knowledge-driven evolutionary algorithm

For the deployment and startup of microservice instances in different resource centres, we propose an optimization problem model based on the evolutionary multi-objective theory. The objective functions of the model consider the computation and storage resource utilization rate, load balancing rate, and actual microservice usage rate in resource service centres. The constraints of the model are the completeness of service, total amount of storage resources, and total number of microservices. In this study, a knowledge-driven evolutionary algorithm (named MGR-NSGA-III) is proposed to solve the problem model and seek the optimal deployment and startup strategy of microservice instances in different resource centres. The proposed model and solution have been evaluated via real data experiments. The results show that our approach is better than the traditional microservice instance deployment and startup strategy. The average computation rate, storage idle rate, and actual microservice idle rate were 13.21%, 5.2%, and 16.67% lower than those in NSGA-III, respectively. After 50, 100, and 150 evolutionary generations in serval operations, the population members in NGR-NSGA-III dominated the population members in NSGA-III 6,270, 3,581, and 7,978 times in average, respectively, which means that NGR-NSGA-III can converge to the optimal solution much quicker than NSGA-III.

Optimal Application Deployment in Resource Constrained Distributed Edges

The dramatically increasing of mobile applications make it convenient for users to complete complex tasks on their mobile devices. However, the latency brought by unstable wireless networks and the computation failures caused by constrained resources limit the development of mobile computing. A popular approach to solve this problem is to establish a mobile service provisioning system based on a mobile edge computing (MEC) paradigm. In the MEC paradigm, plenty of machines are placed at the edge of the network so that the performance of applications can be optimized by using the involved microservice instances deployed on them. In this paper, we explore the deployment problem of microserivce-based applications in the MEC environment and propose an approach to help to optimize the cost of application deployment with the constraints of resources and the requirement of performance. We conduct a series of experiments to evaluate the performance of our approach. The result shows that our approach can improve the average response time of mobile services.

QoS-Constrained Service Selection for Networked Microservices

In the microservice system, microservices are generally deployed as many microservice instances connected by a network system, and there are several composite services (CSs) in the system, each of which consists of several tasks that need to be executed on instances and has some Quality of Service (QoS) constraints to represent the user requirement for QoS. At runtime, it is necessary to select the microservice instance for each task under constraints and form an execution path of the CS. However, traditional service selection methods focus on the QoS optimization of a single CS, while ignoring sharing and competition of instances. To address this problem, this paper presents a Microservice Service Selection algorithm (MSS) based on list scheduling. Firstly, a workflow model is employed to describe the CS and to analyze the processing speed of instances, the transferring speed of the network and the degree of task concurrency, to calculate the sub-deadline of each task. Then, according to the sub-deadline and other information, the scheduling urgency of each task is calculated and updated in real-time. Finally, two service selection strategies based on the sub-deadline and urgency are proposed to complete the microservice instances selection and constitute the service path. Experiments have been carried out using the standard workflow examples of real-world applications. The results show that the proposed selection strategies can effectively improve the performance of service selection in network systems.

Joint optimization of service request routing and instance placement in the microservice system

Microservice architecture is a promising architectural style. It decomposes monolithic software into a set of loosely coupled containerized microservices and associates them into multiple microservice chains to serve service requests. The new architecture creates flexibility for service provisioning but also introduces increased energy consumption and low service performance. Efficient resource allocation is critical. Unfortunately, existing solutions are designed at a coarse level for virtual machine (VM)-based clouds and not optimized for such chain-oriented service provisioning. In this paper, we study the resource allocation optimization problem for service request routing and microservice instance placement, so as to jointly reduce both resource usage and chains’ end-to-end response time for saving energy and guaranteeing Quality of Service (QoS). We design detailed workload models for microservices and chains and formulate the optimization problem as a bi-criteria optimization problem. To address it, a three-stage scheme is proposed to search and optimize the trade-off decisions, route service requests into instances and deploy instances to servers in a balanced manner. Through numerical evaluations, we show that while assuring the same QoS, our scheme performs significantly better than and faster than benchmarking algorithms on reducing energy consumption and balancing load.

DNS/DANE Collision-Based Distributed and Dynamic Authentication for Microservices in IoT †.

IoT devices provide real-time data to a rich ecosystem of services and applications. The volume of data and the involved subscribe/notify signaling will likely become a challenge also for access and core networks. To alleviate the core of the network, other technologies like fog computing can be used. On the security side, designers of IoT low-cost devices and applications often reuse old versions of development frameworks and software components that contain vulnerabilities. Many server applications today are designed using microservice architectures where components are easier to update. Thus, IoT can benefit from deploying microservices in the fog as it offers the required flexibility for the main players of ubiquitous computing: nomadic users. In such deployments, IoT devices need the dynamic instantiation of microservices. IoT microservices require certificates so they can be accessed securely. Thus, every microservice instance may require a newly-created domain name and a certificate. The DNS-based Authentication of Named Entities (DANE) extension to Domain Name System Security Extensions (DNSSEC) allows linking a certificate to a given domain name. Thus, the combination of DNSSEC and DANE provides microservices’ clients with secure information regarding the domain name, IP address, and server certificate of a given microservice. However, IoT microservices may be short-lived since devices can move from one local fog to another, forcing DNSSEC servers to sign zones whenever new changes occur. Considering DNSSEC and DANE were designed to cope with static services, coping with IoT dynamic microservice instantiation can throttle the scalability in the fog. To overcome this limitation, this article proposes a solution that modifies the DNSSEC/DANE signature mechanism using chameleon signatures and defining a new soft delegation scheme. Chameleon signatures are signatures computed over a chameleon hash, which have a property: a secret trapdoor function can be used to compute collisions to the hash. Since the hash is maintained, the signature does not have to be computed again. In the soft delegation schema, DNS servers obtain a trapdoor that allows performing changes in a constrained zone without affecting normal DNS operation. In this way, a server can receive this soft delegation and modify the DNS zone to cope with frequent changes such as microservice dynamic instantiation. Changes in the soft delegated zone are much faster and do not require the intervention of the DNS primary servers of the zone.

Instance expansion algorithm for micro‐service with prediction

Instance expansion of micro-service consumes time for constructing new instances, it cannot satisfy the requirement of low latency service, such as scientific calculation workflows. In order to reduce the time for expanding instances, an instance expansion algorithm for micro-service with prediction is proposed in this letter, which sets correntropy as cost function and uses maximum correntropy criteria to filter the burst service requirements to improve the accuracy of prediction, and use stochastic gradient descent algorithm to train the data set to predict the required micro-service instances in the next time. The performance of the proposed algorithm is analysed in a real experiment telecom office with the compared ones using A/B test, and the experimental results show that the proposed algorithm has 70% less time of instance expansion than the compared ones, and the accuracy of the proposed algorithm is 20% more than the compared one which uses least mean square as the cost function.