Serverless Platforms Research Articles

Workflow aware analytical model to predict performance and cost of serverless execution

SummaryServerless computing has emerged as a powerful deployment model based on the Function‐as‐a‐Service (FaaS) paradigm, where applications are orchestrated through a set of independent functions. The function orchestration within an application can be represented through a serverless workflow, which defines the overall execution plan of the application. To ensure the quality of service for serverless computing platforms, it is essential to develop performance and cost models that can predict the service quality that can be obtained from deploying and executing applications in the cloud platform. While several analytical models have been developed for various cloud deployment frameworks in recent years, there has been a lack of performance and cost analysis models for serverless computing platforms. The existing performance and cost monitoring tools available in serverless frameworks face several challenges, such as complexity, lack of transparency, and incomplete monitoring data. In this paper, we fill the gap by proposing an efficient workflow‐based analytical model that can estimate the end‐to‐end response time and cost of the serverless execution plan. The proposed model can handle complex structures like loop, cycles, self‐loop, and parallel substructures that exist in serverless workflows. Additionally, we propose a heuristic optimization algorithm to identify the optimal resource configuration to achieve the optimal response time under a given budget constraint. We evaluated the effectiveness of the proposed model by considering seven serverless applications in both AWS Lambda and Microsoft Azure platforms. We compared the accuracy of the proposed model with the real values of response time and cost obtained in AWS Lambda and Microsoft Azure serverless platforms. The proposed performance and cost model in the AWS Lambda platform has been observed to have an average accuracy of 99.2% and 98.7% respectively. In the Microsoft Azure platform, the average accuracy of the performance and cost model has been observed to be 98.6% and 98.2% respectively.

Event-Driven Serverless Pipelines for Video Coding and Quality Metrics

Nowadays, the majority of Internet traffic is multimedia content. Video streaming services are in high demand by end users and use HTTP Adaptive Streaming (HAS) as transmission protocol. HAS splits the video into non-overlapping chunks and each video chunk can be encoded independently using different representations. Therefore, these encode tasks can be parallelized and Cloud computing can be used for this. However, in the most extended solutions, the infrastructure must be configured and provisioned in advance. Recently, serverless platforms have made posible to deploy functions that can scale from zero to a configurable maximum. This work presents and analyses the behavior of event-driven serverless functions to encode video chunks and to compute, optionally, the quality of the encoded videos. These functions have been implemented using an adapted version of embedded Tomcat to deal with CloudEvents. We have deployed these event-driven serverless pipelines for video coding and quality metrics on an on-premises serverless platform based on Knative on one master node and eight worker nodes. We have tested the scalability and resource consumption of the proposed solution using two video codecs: x264 and AV1, varying the maximum number of replicas and the resources allocated to them (fat and slim function replicas). We have encoded different 4K videos to generate multiple representations per function call and we show how it is possible to create pipelines of serverless media functions. The results of the different tests carried out show the good performance of the serverless functions proposed. The system scales the replicas and distributes the jobs evenly across all the replicas. The overall encoding time is reduced by 18% using slim replicas but fat replicas are more adequate in live video streaming as the encoding time per chunk is reduced. Finally, the results of the pipeline test show an appropriate distribution and chaining among the available replicas of each function type.

Securing Serverless Computing: Challenges, Solutions, and Opportunities

Serverless computing is a new cloud service model that reduces both cloud providers' and consumers' costs through agile development, operation, and charging mechanisms. It has been widely applied since its emergence. Nevertheless, some characteristics of serverless computing, such as fragmented application boundaries, have raised new security challenges. Considerable literature has been committed to addressing these challenges. Commercial and open-source serverless platforms implement many security measures to enhance serverless environments. This paper presents the first survey of serverless security that considers both the literature and industrial security measures. We summarize the primary security challenges, analyze corresponding solutions from the literature and industry, and identify potential research opportunities. Then, we conduct a gap analysis of the academic and industrial solutions, as well as commercial and open-source serverless platforms' security capabilities. Finally, we present a complete picture of current serverless security research.

Executing Microservice Applications on Serverless, Correctly

While serverless platforms substantially simplify the provisioning, configuration, and management of cloud applications, implementing correct services on top of these platforms can present significant challenges to programmers. For example, serverless infrastructures introduce a host of failure modes that are not present in traditional deployments. Individual serverless instances can fail while others continue to make progress, correct but slow instances can be killed by the cloud provider as part of resource management, and providers will often respond to such failures by re-executing requests. For functions with side-effects, these scenarios can create behaviors that are not observable in serverful deployments. In this paper, we propose mu2sls, a framework for implementing microservice applications on serverless using standard Python code with two extra primitives: transactions and asynchronous calls. Our framework orchestrates user-written services to address several challenges, such as failures and re-executions, and provides formal guarantees that the generated serverless implementations are correct. To that end, we present a novel service specification abstraction and formalization of serverless implementations that facilitate reasoning about the correctness of a given application’s serverless implementation. This formalization forms the basis of the mu2sls prototype, which we then use to develop a few real-world microservice applications and show that the performance of the generated serverless implementations achieves significant scalability (3-5× the throughput of a sequential implementation) while providing correctness guarantees in the context of faults, re-execution, and concurrency.

FuncPipe: A Pipelined Serverless Framework for Fast and Cost-Efficient Training of Deep Learning Models

Training deep learning (DL) models in the cloud has become a norm. With the emergence of serverless computing and its benefits of true pay-as-you-go pricing and scalability, systems researchers have recently started to provide support for serverless-based training. However, the ability to train DL models on serverless platforms is hindered by the resource limitations of today's serverless infrastructure and DL models' explosive requirement for memory and bandwidth. This paper describes FuncPipe, a novel pipelined training framework specifically designed for serverless platforms that enable fast and low-cost training of DL models. FuncPipe is designed with the key insight that model partitioning can be leveraged to bridge both memory and bandwidth gaps between the capacity of serverless functions and the requirement of DL training. Conceptually simple, we have to answer several design questions, including how to partition the model, configure each serverless function, and exploit each function's uplink/downlink bandwidth. In particular, we tailor a micro-batch scheduling policy for the serverless environment, which serves as the basis for the subsequent optimization. Our Mixed-Integer Quadratic Programming formulation automatically and simultaneously configures serverless resources and partitions models to fit within the resource constraints. Lastly, we improve the bandwidth efficiency of storage-based synchronization with a novel pipelined scatter-reduce algorithm. We implement FuncPipe on two popular cloud serverless platforms and show that it achieves 7%-77% cost savings and 1.3X-2.2X speedup compared to state-of-the-art serverless-based frameworks.

CharmSeeker: Automated Pipeline Configuration for Serverless Video Processing

Video processing plays an essential role in a wide range of cloud-based applications. It typically involves multiple pipelined stages, which well fits the latest fine-grained serverless computing paradigm if properly configured to match the cost and delay constraints of video. Existing configuration tools, however, are primarily developed for traditional virtual machine clusters with general workloads. This paper presents CharmSeeker, an automated configuration tuning tool for serverless video processing pipelines. We first carefully examine the key steps and the performance bottlenecks for video processing over modern serverless platforms. Then, we identify the configuration space for processing pipelines and leverage a carefully designed Sequential Bayesian Optimization search scheme to identify promising configurations. We further address the practical challenges toward integrating our solution into real-world systems and develop a prototype with AWS Lambda. Evaluation results show that CharmSeeker can find out the optimal or near-optimal configurations that improve the relative processing time up to 408.77%. It is also more robust and scalable to various video processing pipelines compared with state-of-the-art solutions.

Performance Evaluation of Open-Source Serverless Platforms for Kubernetes

Serverless computing has grown massively in popularity over the last few years, and has provided developers with a way to deploy function-sized code units without having to take care of the actual servers or deal with logging, monitoring, and scaling of their code. High-performance computing (HPC) clusters can profit from improved serverless resource sharing capabilities compared to reservation-based systems such as Slurm. However, before running self-hosted serverless platforms in HPC becomes a viable option, serverless platforms must be able to deliver a decent level of performance. Other researchers have already pointed out that there is a distinct lack of studies in the area of comparative benchmarks on serverless platforms, especially for open-source self-hosted platforms. This study takes a step towards filling this gap by systematically benchmarking two promising self-hosted Kubernetes-based serverless platforms in comparison. While the resulting benchmarks signal potential, they demonstrate that many opportunities for performance improvements in serverless computing are being left on the table.

Threat modelling for serverless architectures identifying and mitigating risks in Function-As-A-Service (FAAS)

One significant method for developing software is Function as a Service (FaaS), which entails making little, tailored functions to deal with particular jobs. Coders put less emphasis on creating full apps and more on creating these functions that are called when certain events or requests come in. If you want to learn about and use Function as a Service, this article is for you. Customers using the serverless approach do not need to reserve hardware resources, which is a departure from the conventional cloud computing service model. Billing is dependent on real resource consumption, and code execution is event-driven (via HTTP requests, cron jobs, etc.). In exchange, the provider is liable for assigning resources and tasks. Serverless is most often seen as a public cloud service, although there are solutions being worked on and supported by strong players in the industry that will enable private cloud serverless platforms to be built. "Function as a Service" (FaaS), the initial serverless offering, has serious flaws that might cancel out any advantages for providers and customers alike, particularly when it comes to providers' capacity to multiplex resources and customers' ability to save money. Providers and tenants alike could save a ton of money and energy if these problems were solved. In order to prevent serverless from becoming the default cloud computing model, this chapter will provide a thorough overview of its limits and highlight state-of-the-art research to address these issues.

Benchmarking Serverless Computing

With the increasing adoption of serverless computing, there is a need for a benchmark. The aim of this paper is to present such a benchmark based on performance and usability testing to better understand serverless services as well as help practitioners to select between two major clouds, namely, Amazon and Azure. Jmeter tool and system usability scale are used to conduct performance and usability testing, respectively. In addition, a replication package is provided to increase the validity and reliability of the results. The main findings revealed that the serverless platforms are different in their architecture. Even though both of them support the same serverless concept, they differ considerably in structure, development, and creations of services. Overall, both the cloud vendors under study provide the same core capabilities one would expect but, there are some differences too. In particular, usability could be improved to extend the market and capture more customers.

Optimizing inference serving on serverless platforms

Serverless computing is gaining popularity for machine learning (ML) serving workload due to its autonomous resource scaling, easy to use and pay-per-use cost model. Existing serverless platforms work well for image-based ML inference, where requests are homogeneous in service demands. That said, recent advances in natural language processing could not fully benefit from existing serverless platforms as their requests are intrinsically heterogeneous. Batching requests for processing can significantly increase ML serving efficiency while reducing monetary cost, thanks to the pay-per-use pricing model adopted by serverless platforms. Yet, batching heterogeneous ML requests leads to additional computation overhead as small requests need to be "padded" to the same size as large requests within the same batch. Reaching effective batching decisions (i.e., which requests should be batched together and why) is non-trivial: the padding overhead coupled with the serverless auto-scaling forms a complex optimization problem. To address this, we develop Multi-Buffer Serving (MBS), a framework that optimizes the batching of heterogeneous ML inference serving requests to minimize their monetary cost while meeting their service level objectives (SLOs). The core of MBS is a performance and cost estimator driven by analytical models supercharged by a Bayesian optimizer. MBS is prototyped and evaluated on AWS using bursty workloads. Experimental results show that MBS preserves SLOs while outperforming the state-of-the-art by up to 8 x in terms of cost savings while minimizing the padding overhead by up to 37 x with 3 x less number of serverless function invocations.

Harnessing the Potential of Function-Reuse in Multimedia Cloud Systems

Cloud-based computing systems can get oversubscribed due to the budget constraints of their users or limitations in certain resource types. The oversubscription can, in turn, degrade the users perceived Quality of Service (QoS). The approach we investigate to mitigate both the oversubscription and the incurred cost is based on smart reusing of the computation needed to process the service requests (i.e., tasks). We propose a reusing paradigm for the tasks that are waiting for execution. This paradigm can be particularly impactful in serverless platforms where multiple users can request similar services simultaneously. Our motivation is a multimedia streaming engine that processes the media segments in an on-demand manner. We propose a mechanism to identify various types of “mergeable” tasks and aggregate them to improve the QoS and mitigate the incurred cost. We develop novel approaches to determine when and how to perform task aggregation such that the QoS of other tasks is not affected. Evaluation results show that the proposed mechanism can improve the QoS by significantly reducing the percentage of tasks missing their deadlines and reduce the overall time (and subsequently the incurred cost) of utilizing cloud services by more than 9 percent.

SERVERLESS COMPUTATIONS RESOURCE SCHEDULING BASED ON DATA DEPENDENCY

The serverless computing model is becoming quite widespread. This model allows developers to create flexible and fault tolerant applications with an attractive billing model. The increasing complexity of serverless functions has led to the necessity to use serverless workflows – serverless functions invoking other serverless functions. However, such concept imposes certain requirements on the serverless functions that make distributed computations. The overhead of transferring data between serverless functions can significantly increase the execution time of a program using this approach. One way to reduce overhead is to improve serverless scheduling techniques. This paper discusses an approach to scheduling serverless computations based on data dependency analysis. We propose to divide the problem of planning of the computation of a composite serverless function into three stages. For each stage we provide a description by a mathematical model. We carried out a review of algorithms used to schedule resources by compilers and in parallel computing in multiprocessor systems to determine the best algorithm to implement in a prototype scheduler. For each algorithm, it was specified how it could be used for resource scheduling in serverless platforms. We provide a description of the developed prototype based on the Fission serverless platform. The prototype implements the critical path heuristic. It is shown that the improvements can significantly reduce the execution time up to two times for some types of serverless functions.

Serverless: Next Generation of Cloud Computing

Serverless computing has emerged as a promising new paradigm for application and service deployment. The process of delivering code into production can be made easier with serverless computing. Serverless code can be utilized alongside standard deployment strategies such as microservices and monoliths. Alternatively, programs can be developed to be completely serverless, meaning they don't require any supplied servers. A serverless computing service concept is also called Function as a Service (FaaS), which allows developers to deploy specific functions to the cloud. FaaS has proven to be effective for a variety of computational activities, although using it for web apps can be difficult due to excessive response times on occasion. Nevertheless, serverless computing for web applications is gaining popularity. It is a testament to the maturity and widespread usage of cloud technologies, as it symbolizes the development of cloud programming paradigms, abstractions, and platforms. In this research, we investigated existing serverless platforms from industry, academia, and open-source initiatives, identifying key criteria.

Metaphorical Analysis of Serverless Computing Platforms And Related Challenges

In today’s global scenario, cloud technologies are leading the world form the front by facilitating the end users by providing the flawless infrastructure, platforms and services to every community. The current trending model of cloud technologies is Serverless model, which is also known as Function as a service (FaaS).This Serverless model has tremendous demand by various organizations as well as individual stakeholders because serverless model provides no provisioning and monitoring of required resources by any organization or individual.it also supports a wide range of programming languages, that make serverless model to be liked by everyone. However, the serverless approach to computing is not completely new. Presently there are many serverless service providers like AWS Lambda, Microsoft azure, Google Cloud Functions and IBM Open Whisk. These serverless platforms permits the individuals to run their code in provided environments. When Individuals trigger the code to run in serverless environment, the status of code execution does not known to the individual because these code runs in an isolated environment of the serverless platforms, which is the biggest challenge of these serverless model. In this proposed Paper, we have included the demand of Serverless model along with the comparison of available serverless platforms. This proposed paper also includes various challenges of Triggering of code in serverless environment in more detailed way.

Analysis of problems of storage and processing of data in serverless technologies

The object of research is the problems of storing and processing data in serverless technologies. The research carried out is based on a logical approach to storage, data processing and transmission processes. The main hypothesis of the study is that when moving from a monolithic architecture to microservice architecture, and then from a microservice architecture to a serverless architecture, the process of storing and processing data requires modifications and new approaches to solving classical problems of working with data. The problem of interacting with data is an integral part of the work of almost all computer systems, as they lay the basis for the goal of creating such systems. Serverless computing has already taken root in cloud computing. Improving its work is now one of the most popular tasks in the research field. In this article, let’s review one of the global problems – integrating serverless computing system with a database. As well as currently existing partial or complete solutions. Progress in this area can give impetus to the development of serverless technologies that supplant more outdated software development approaches. The result of these studies brings a certain understanding at what stage of development the above question is now. It also describes the advantages and disadvantages of the new systems. It is considered what innovations have been brought by the global giants in the development of serverless platforms, and what solutions are applied to open source platforms. This issue has not been fully resolved and requires developments and improvements, and therefore remains an excellent direction for development and new research.

λDNN: Achieving Predictable Distributed DNN Training With Serverless Architectures

Serverless computing is becoming a promising paradigm for Distributed Deep Neural Network (DDNN) training in the cloud, as it allows users to decompose complex model training into a number of <i>functions</i> without managing virtual machines or servers. Though provided with a simpler resource interface (i.e., function number and memory size), inadequate function resource provisioning (either under-provisioning or over-provisioning) easily leads to <i>unpredictable</i> DDNN training performance in serverless platforms. Our empirical studies on AWS Lambda indicate that, such <i>unpredictable performance</i> of serverless DDNN training is mainly caused by the resource bottleneck of Parameter Servers (PS) and small local batch size. In this article, we design and implement <i><inline-formula><tex-math notation="LaTeX">$\lambda$</tex-math><alternatives><mml:math><mml:mi>&#x03BB;</mml:mi></mml:math><inline-graphic xlink:href="xu-ieq1-3054656.gif"/></alternatives></inline-formula>DNN</i>, a cost-efficient function resource provisioning framework to provide predictable performance for serverless DDNN training workloads, while saving the budget of provisioned functions. Leveraging the PS network bandwidth and function CPU utilization, we build a <i>lightweight</i> analytical DDNN training performance model to enable our design of <i><inline-formula><tex-math notation="LaTeX">$\lambda$</tex-math><alternatives><mml:math><mml:mi>&#x03BB;</mml:mi></mml:math><inline-graphic xlink:href="xu-ieq2-3054656.gif"/></alternatives></inline-formula>DNN</i> resource provisioning strategy, so as to guarantee DDNN training performance with serverless functions. Extensive prototype experiments on AWS Lambda and complementary trace-driven simulations demonstrate that, <i><inline-formula><tex-math notation="LaTeX">$\lambda$</tex-math><alternatives><mml:math><mml:mi>&#x03BB;</mml:mi></mml:math><inline-graphic xlink:href="xu-ieq3-3054656.gif"/></alternatives></inline-formula>DNN</i> can deliver predictable DDNN training performance and save the monetary cost of function resources by up to 66.7 percent, compared with the state-of-the-art resource provisioning strategies, yet with an acceptable runtime overhead.

Operating Latency Sensitive Applications on Public Serverless Edge Cloud Platforms

Cloud native programming and serverless architectures provide a novel way of software development and operation. A new generation of applications can be realized with features never seen before while the burden on developers and operators will be reduced significantly. However, latency sensitive applications, such as various distributed IoT services, generally do not fit in well with the new concepts and today's platforms. In this article, we adapt the cloud native approach and related operating techniques for latency sensitive IoT applications operated on public serverless platforms. We argue that solely adding cloud resources to the edge is not enough and other mechanisms and operation layers are required to achieve the desired level of quality. Our contribution is threefold. First, we propose a novel system on top of a public serverless edge cloud platform, which can dynamically optimize and deploy the microservice-based software layout based on live performance measurements. We add two control loops and the corresponding mechanisms which are responsible for the online reoptimization at different timescales. The first one addresses the steady-state operation, while the second one provides fast latency control by directly reconfiguring the serverless runtime environments. Second, we apply our general concepts to one of today's most widely used and versatile public cloud platforms, namely, Amazon's AWS, and its edge extension for IoT applications, called Greengrass. Third, we characterize the main operation phases and evaluate the overall performance of the system. We analyze the performance characteristics of the two control loops and investigate different implementation options.

Performance Modeling of Serverless Computing Platforms

Analytical performance models have been leveraged extensively to analyze and improve the performance and cost of various cloud computing services. However, in the case of serverless computing, which is projected to be the dominant form of cloud computing in the future, we have not seen analytical performance models to help with the analysis and optimization of such platforms. In this work, we propose an analytical performance model that captures the unique details of serverless computing platforms. The model can be leveraged to improve the quality of service and resource utilization and reduce the operational cost of serverless platforms. Also, the proposed performance model provides a framework that enables serverless platforms to become <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">workload-aware</i> and operate differently for different workloads to provide a better trade-off between the cost and performance depending on the user's preferences. The current serverless offerings require the user to have extensive knowledge of the internals of the platform to perform efficient deployments. Using the proposed analytical model, the provider can simplify the deployment process by calculating the performance metrics for users even before physical deployments. We validate the applicability and accuracy of the proposed model by extensive experimentation on AWS Lambda. We show that the proposed model can calculate essential performance metrics such as average response time, probability of cold start, and the average number of function instances in the steady-state. Also, we show how the performance model can be used to tune the serverless platform for each workload, which will result in better performance or lower cost without scarifying the other. The presented model assumes no non-realistic restrictions, so that it offers a high degree of fidelity while maintaining tractability at large scale.

Optimized container scheduling for data-intensive serverless edge computing

Operating data-intensive applications on edge systems is challenging, due to the extreme workload and device heterogeneity, as well as the geographic dispersion of compute and storage infrastructure. Serverless computing has emerged as a compelling model to manage the complexity of such systems, by decoupling the underlying infrastructure and scaling mechanisms from applications. Although serverless platforms have reached a high level of maturity, we have found several limiting factors that inhibit their use in an edge setting. This paper presents a container scheduling system that enables such platforms to make efficient use of edge infrastructures. Our scheduler makes heuristic trade-offs between data and computation movement, and considers workload-specific compute requirements such as GPU acceleration. Furthermore, we present a method to automatically fine-tune the weights of scheduling constraints to optimize high-level operational objectives such as minimizing task execution time, uplink usage, or cloud execution cost. We implement a prototype that targets the container orchestration system Kubernetes, and deploy it on an edge testbed we have built. We evaluate our system with trace-driven simulations in different infrastructure scenarios, using traces generated from running representative workloads on our testbed. Our results show that (a) our scheduler significantly improves the quality of task placement compared to the state-of-the-art scheduler of Kubernetes, and (b) our method for fine-tuning scheduling parameters helps significantly in meeting operational goals.

Benchmarking Serverless Computing Platforms

We propose a benchmarking test suite to evaluate performance of cloud serverless platforms and an open source software tool to automate the test process. Additionally, we used this setup to compare the commercial offers of Amazon, Google, Microsoft, and IBM. The work builds on ideas and experiments reported in the literature that, nevertheless, did not offer a “standard” set of coherent and comprehensive tests, capable of analyzing diverse serverless platforms, across the same set of features. To that end, we defined seven tests that cover scalability (latency and throughput), the effect of allocated memory, the performance for CPU-bound cases, the impact of payload size, the influence of the programming language, resource management (e.g., reuse of containers), and overall platform overhead. We created a software tool to deploy the test code and collect metrics in a manner agnostic to the serverless platforms under study. At a time when serverless computing popularity is rising, this benchmarking suite and software test tool enable developers to take informed decisions on the suitability and performance of each provider’s serverless offer. Additionally, they also help to identify attributes in need for improvement in the existing platforms.