Serverless Platforms Research Articles

Enabling scalable and adaptive machine learning training via serverless computing on public cloud

In today’s production machine learning (ML) systems, models are continuously trained, improved, and deployed. ML design and training are becoming a continuous workflow of various tasks that have dynamic resource demands. Serverless computing is an emerging cloud paradigm that provides transparent resource management and scaling for users and has the potential to revolutionize the routine of ML design and training. However, hosting modern ML workflows on existing serverless platforms has non-trivial challenges due to their intrinsic design limitations such as stateless nature, limited communication support across function instances, and limited function execution duration. These limitations result in a lack of an overarching view and adaptation mechanism for training dynamics, and an amplification of existing problems in ML workflows.To address the above challenges, we propose SMLT, an automated, scalable and adaptive serverless framework on public cloud to enable efficient and user-centric ML design and training. SMLT employs an automated and adaptive scheduling mechanism to dynamically optimize the deployment and resource scaling for ML tasks during training. SMLT further enables user-centric ML workflow execution by supporting user-specified training deadline and budget limit. In addition, by providing an end-to-end design, SMLT solves the intrinsic problems in public cloud serverless platforms such as the communication overhead, limited function execution duration, need for repeated initialization, and also provides explicit fault tolerance for ML training. SMLT is open-sourced and compatible with all major ML frameworks. Our experimental evaluation with large, sophisticated modern ML models demonstrates that SMLT outperforms the state-of-the-art VM-based systems and existing public cloud serverless ML training frameworks in both training speed (up to 8×) and monetary cost (up to 3×).

Serverless Computing in the Edge-Cloud Continuum : Challenges, Opportunities, and a Novel Framework

This article explores the integration of serverless computing across the edge-cloud continuum, addressing the growing demand for low-latency, data-intensive applications in the era of the Internet of Things (IoT). We present EdgeServe, a novel framework that extends the serverless paradigm to encompass edge devices and cloud resources, overcoming the limitations of traditional cloud-centric approaches. Through comprehensive simulations and real-world case studies, including a smart city traffic management system, an industrial IoT predictive maintenance application, and a mobile augmented reality gaming platform, we evaluate the performance, scalability, and cost-effectiveness of our proposed framework. Our results demonstrate significant improvements in application response times, with latency reductions of up to 82% in time-critical functions, and an average cost reduction of 43% compared to cloud-only serverless deployments. We also observe a 28% decrease in overall energy consumption in certain scenarios. The article addresses key challenges in resource management, data consistency, adaptive function placement, and security in distributed environments. Our findings have important implications for application architects and cloud service providers, paving the way for a new generation of edge-native serverless platforms. This research contributes to the growing body of knowledge on distributed systems and offers insights into the future evolution of serverless computing in heterogeneous computing environments.

Toward security quantification of serverless computing

Serverless computing is one of the recent compelling paradigms in cloud computing. Serverless computing can quickly run user applications and services regardless of the underlying server architecture. Despite the availability of several commercial and open-source serverless platforms, there are still some open issues and challenges to address. One of the key concerns in serverless computing platforms is security. Therefore, in this paper, we present a multi-layer abstract model of serverless computing for an security investigation. We conduct a quantitative analysis of security risks for each layer. We observe that the Attack Tree and Attack-Defense Tree methodologies are viable approaches in this regard. Consequently, we make use of the Attack Tree and the Attack-Defense Tree to quantify the security risks and countermeasures of serverless computing. We also propose a novel measure called the Relative Risk Matrix (RRM) to quantify the probability of attack success. Stakeholders including application developers, researchers, and cloud providers can potentially apply these findings and implications to better understand and further enhance the security of serverless computing.

Application of Proximal Policy Optimization for Resource Orchestration in Serverless Edge Computing

Serverless computing is a new cloud computing model suitable for providing services in both large cloud and edge clusters. In edge clusters, the autoscaling functions play a key role on serverless platforms as the dynamic scaling of function instances can lead to reduced latency and efficient resource usage, both typical requirements of edge-hosted services. However, a badly configured scaling function can introduce unexpected latency due to so-called “cold start” events or service request losses. In this work, we focus on the optimization of resource-based autoscaling on OpenFaaS, the most-adopted open-source Kubernetes-based serverless platform, leveraging real-world serverless traffic traces. We resort to the reinforcement learning algorithm named Proximal Policy Optimization to dynamically configure the value of the Kubernetes Horizontal Pod Autoscaler, trained on real traffic. This was accomplished via a state space model able to take into account resource consumption, performance values, and time of day. In addition, the reward function definition promotes Service-Level Agreement (SLA) compliance. We evaluate the proposed agent, comparing its performance in terms of average latency, CPU usage, memory usage, and loss percentage with respect to the baseline system. The experimental results show the benefits provided by the proposed agent, obtaining a service time within the SLA while limiting resource consumption and service loss.

Comparative Security and Compliance Analysis of Serverless Computing Platforms: AWS Lambda, Azure Functions, and Google Cloud Functions

Serverless computing has revolutionized cloud services by abstracting infrastructure management, enabling developers to focus on application logic. This research examines the security and compliance features of three major serverless platforms: AWS Lambda, Azure Functions, and Google Cloud Functions. By evaluating authentication mechanisms, data encryption practices, vulnerability management, and compliance certifications, we aim to provide a comparative analysis that informs businesses and developers on the most secure and compliant platform for their needs. Keywords: Serverless Computing, Security, Compliance, AWS Lambda, Azure Functions, Google Cloud Functions, Cloud Security Alliance, Cloud Controls Matrix

Word frequency statistics based on MapReduce on serverless platforms

This paper investigates the application of serverless computing in conjunction with the MapReduce framework, particularly in machine learning (ML) tasks. The MapReduce programming model has been widely used to process large-scale datasets by simplifying parallel and distributed data processing. This study explores how the combination of these two technologies can provide more efficient and cost-effective ML solutions. Through a detailed analysis of serverless environments and the MapReduce framework, this paper shows how the combination can advance the fields of cloud computing and machine learning. The experimental part includes the implementation of Map-Reduce model on a serverless platform, exploring the impact of different parameter settings on performance and improving efficiency by optimizing the data processing flow. In addition, the paper analyzes the use of memory and CPU resources and derives the relationship between dataset size, memory consumption and processor configuration and execution time. Through these experiments and analyses, this paper provides an empirical basis and theoretical support for the optimization of cloud computing frameworks.

Transactions and Serverless are Made for Each Other

Database-backed applications are an exciting new frontier for serverless computation. By tightly integrating application execution and data management, a transactional serverless platform enables many new features not possible in either existing serverless platforms or server-based deployments.

Self-Provisioning Infrastructures for the Next Generation Serverless Computing

Serverless computing has ushered in a transformative paradigm, with a promise to alleviate developers from the intricacies of infrastructure management. However, current serverless platforms predominantly offer only serverless compute capabilities. As a consequence, the application developers are once again tasked to explicitly provision and manage the backend services (BaaS), such as object stores or API gateways, the infrastructure, and the configuration models. This violates the main promise of serverless computing and erases much of the practical benefits of the serverless paradigm. It also introduces the challenges of managing the application execution environment, which includes maintaining provisioning and deployment scripts, configuring and managing access permissions, and scaling the services during runtime. To address these challenges, in this paper we introduce a novel paradigm for the next generation of serverless computing, called self-provisioning infrastructure. The self-provisioning infrastructure is an infrastructure that is capable to automatically and autonomously (with zero-configuration and zero-touch) provision serverless functions, their infrastructure, and their supporting BaaS services. To achieve this vision, we introduce novel design principles, models, and mechanisms that are formalized via novel programming, function, and system models. With this novel paradigm, we intend to fortify the core design principles of serverless computing but also extend them to the entire application execution environment. By doing so, we aim to enable the next-generation serverless computing in the Edge-Cloud continuum.

Incorporating Serverless Computing into P2P Networks for ML Training: In-Database Tasks and Their Scalability Implications (Student Abstract)

Distributed ML addresses challenges from increasing data and model complexities. Peer to peer (P2P) networks in distributed ML offer scalability and fault tolerance. However, they also encounter challenges related to resource consumption, and communication overhead as the number of participating peers grows. This research introduces a novel architecture that combines serverless computing with P2P networks for distributed training. Serverless computing enhances this model with parallel processing and cost effective scalability, suitable for resource-intensive tasks. Preliminary results show that peers can offload expensive computational tasks to serverless platforms. However, their inherent statelessness necessitates strong communication methods, suggesting a pivotal role for databases. To this end, we have enhanced an in memory database to support ML training tasks.

The Promise of Serverless Computing within Peer-to-Peer Architectures for Distributed ML Training

My thesis focuses on the integration of serverless computing with Peer to Peer (P2P) architectures in distributed Machine Learning (ML). This research aims to harness the decentralized, resilient nature of P2P systems, combined with the scalability and automation of serverless platforms. We explore using databases not just for communication but also for in-database model updates and gradient averaging, addressing the challenges of statelessness in serverless environments.

Privacy-Preserving Serverless Edge Learning With Decentralized Small-Scale Mobile Data

In next-generation (i.e., 6G) networking systems, the data-driven approach will play an essential role, being an efficient tool for networking system management and bringing popular user applications. With those unprecedented and novel usages, existing frameworks fail to consider the complex nature of the next-generation networking system and consequently fail to be applied to future communication systems directly. Moreover, existing frameworks also fail to support popular privacy-preserving learning strategies efficiently by presenting special designs to respond to the resource-demanding nature of the aforementioned strategies. To fill this gap, this paper extends conventional serverless platforms with serverless edge learning architectures, providing a mature and efficient distributed training framework by fully exploiting limited wireless communication and edge computation resources in the considered networking system with the following three features. Firstly, this framework dynamically orchestrates resources among heterogeneous physical units to efficiently fulfill privacy-preserving learning objectives. The design jointly considers learning task requests and underlying infrastructure heterogeneity, including last-mile transmissions, computation abilities of edge and cloud computing centers, and loading status of infrastructure. Secondly, the proposed framework can easily work with data-driven approaches to improve network management efficiency, realizing AI for network promise of next-generation networking systems to provide efficient network automation. Lastly, to significantly reduce distributed training overheads, small-scale data training is proposed by integrating with a general, simple data classifier. This low-load enhancement can seamlessly work with various distributed deep models in the proposed framework to improve communications and computation efficiencies during the training phase. Based on the above innovations, open challenges, and future research directions encourage the research community to develop efficient privacy-preserving learning techniques.

MapReduce based on serverless platforms

This study intends to investigate the application of the MapReduce (MR) framework based on serverless computing in big data processing. By combining the MapReduce model with serverless computing, efficient data processing is achieved. In this framework, the phases of Map task execution, reduce task execution, etc. are accomplished through stateless serverless functions, and data storage is realized with the help of cloud storage platforms (e.g., OSS). In this paper, the author introduces the basic theory of MR, the basic theory of serverless computing, describes the framework implementation process, and discusses the role of OSS in distributed computing. The outcomes of the trial indicate the average execution time of the framework for a WordCount task on 100 pieces of TOEFL English reading data is 6.81 seconds. The discussion analyzes the frameworks advantages (high elasticity, resource utilization) and disadvantages (cold start latency, unsuitable for long time tasks). Future research directions encompass performance optimization, long-time task processing, state management, etc. In summary, this study provides valuable insights for the practical application of serverless computing in big data processing.

Functions as a service for distributed deep neural network inference over the cloud‐to‐things continuum

AbstractThe use of serverless computing has been gaining popularity in recent years as an alternative to traditional Cloud computing. We explore the usability and potential development benefits of three popular open‐source serverless platforms in the context of IoT: OpenFaaS, Fission, and OpenWhisk. To address this we discuss our experience developing a serverless and low‐latency Distributed Deep Neural Network (DDNN) application. Our findings indicate that these serverless platforms require significant resources to operate and are not ideal for constrained devices. In addition, we archived a 55% improvement compared to Kafka‐ML's performance under load, a framework without dynamic scaling support, demonstrating the potential of serverless computing for low‐latency applications.

Rescheduling serverless workloads across the cloud-to-edge continuum

Serverless computing was a breakthrough in Cloud computing due to its high elasticity capabilities and fine-grained pay-per-use model offered by the main public Cloud providers. Meanwhile, open-source serverless platforms supporting the FaaS (Function as a Service) model allow users to take advantage of many of their benefits while operating on the on-premises platforms of organizations. This opens the possibility to deploy and exploit them on the different layers of the cloud-to-edge continuum, either on IoT (Internet of Things) devices located at the Edge (i.e. next to data acquisition devices), in on-premises clusters closer to the data sources (i.e. Fog computing) or directly on the Cloud.This paper presents two strategies to mitigate the overload that disparate data ingestion rates may cause in low-powered devices at the Edge or Fog layers. To this end, it is proposed to delegate and reschedule serverless jobs between the different layers of the cloud-to-edge continuum using an open-source platform for event-driven file processing. To demonstrate the performance of these strategies, a use case for fire detection is proposed that includes processing in the Fog via minified Kubernetes clusters located near the Edge, in the private Cloud via on-premises elastic clusters and, finally, in the public Cloud by using the AWS (Amazon Web Services) Lambda FaaS service. The results indicate that these strategies can mitigate overloads in use cases involving processing across the cloud-to-edge continuum by coordinating several layers of computing resources.

Implementing Serverless Architecture: Discuss the practical aspects and challenges

Function as a Service (FaaS) has garnered significant attention for its approach to deploying computations to serverlessbackends across various cloud environments. It simplifies the complexity of provisioning and managing resources for applications by leveraging cloud providers' capabilities, offering users an illusion of perpetual resource availability. Among these providers, the AWS serverless platform stands out, offering a novel paradigm for cloud application development, abstracting away concerns about underlying hardware infrastructure while ensuring scalability, security, and cost-effectiveness.However, due to the absence of standardized benchmarks, serverless functions often rely on ad-hoc solutions for building cost-efficient and scalable applications. The development of the SeBS framework has addressed this gap, enabling comprehensive testing, evaluation, and performance analysis across different cloud providers. While previous research has explored serverless platforms among various providers, little attention has been given to AWS Lambda service's performance within the ARM64 architecture and its comparison with traditional x86 architectures.

A Seer knows best: Auto-tuned object storage shuffling for serverless analytics

Serverless platforms offer high resource elasticity and pay-as-you-go billing, making them a compelling choice for data analytics. To craft a “pure” serverless solution, the common practice is to transfer intermediate data between serverless functions via serverless object storage (IBM COS; AWS S3). However, prior works have led to inconclusive results about the performance of object storage systems, since they have left large margin for optimization. To verify that object storage has been underrated, we devise a novel shuffle manager for serverless data analytics called Seer. Specifically, Seer dynamically chooses between two shuffle algorithms to maximize performance. The algorithm choice is made online based on some predictive models, and very importantly, without end users having to specify intermediate shuffle data sizes at the time of the job submission. We integrate Seer with PyWren-IBM [31], a well-known serverless analytics framework, and evaluate it against both serverful (e.g., Spark) and serverless systems (e.g., Google BigQuery, Caerus [46] and SONIC [22]). Our results certify that our new shuffle manager can deliver performance improvements over them.

FaaSLight : General Application-level Cold-start Latency Optimization for Function-as-a-Service in Serverless Computing

Serverless computing is a popular cloud computing paradigm that frees developers from server management. Function-as-a-Service (FaaS) is the most popular implementation of serverless computing, representing applications as event-driven and stateless functions. However, existing studies report that functions of FaaS applications severely suffer from cold-start latency. In this article, we propose an approach, namely, FaaSLight , to accelerating the cold start for FaaS applications through application-level optimization. We first conduct a measurement study to investigate the possible root cause of the cold-start problem of FaaS. The result shows that application code loading latency is a significant overhead. Therefore, loading only indispensable code from FaaS applications can be an adequate solution. Based on this insight, we identify code related to application functionalities by constructing the function-level call graph and separate other code (i.e., optional code) from FaaS applications. The separated optional code can be loaded on demand to avoid the inaccurate identification of indispensable code causing application failure. In particular, a key principle guiding the design of FaaSLight is inherently general, i.e., platform - and language-agnostic . In practice, FaaSLight can be effectively applied to FaaS applications developed in different programming languages (Python and JavaScript), and can be seamlessly deployed on popular serverless platforms such as AWS Lambda and Google Cloud Functions, without having to modify the underlying OSes or hypervisors, nor introducing any additional manual engineering efforts to developers. The evaluation results on real-world FaaS applications show that FaaSLight can significantly reduce the code loading latency (up to 78.95%, 28.78% on average), thereby reducing the cold-start latency. As a result, the total response latency of functions can be decreased by up to 42.05% (19.21% on average). Compared with the state-of-the-art, FaaSLight achieves a 21.25× improvement in reducing the average total response latency.

Disaggregated GPU Acceleration for Serverless Applications

Serverless platforms have been attracting applications from traditional platforms because infrastructure management responsibilities are shifted from users to providers. Many applications well-suited to serverless environments could leverage GPU acceleration to enhance their performance. Unfortunately, current serverless platforms do not expose GPUs to serverless applications.

Navigating Performance-Efficiency Tradeoffs in Serverless Computing: Deduplication to the Rescue!

Navigating the performance and efficiency trade-offs is critical for serverless platforms, where the providers ideally want to give the illusion of warm function startups while maintaining low resource costs. Limited controls, provided via toggling sandboxes between warm and cold states and keepalives, force operators to sacrifice significant resources to achieve good performance.

A Fast Cold-Start Solution: Container Space Reuse Based on Resource Isolation

With the emergence of cloud-native computing, serverless computing has become a popular way to deploy intensive applications due to its scalability and flexibility, and it has been increasingly applied in the field of big data processing on service platforms. Currently, the development momentum of cloud-native computing is strong, and serverless computing has become more attractive to the growing number of Internet services. However, how to more effectively address the issues of container resource usage and service startup time for serverless computing remains a huge challenge when exploring its potential. Our research is based on the complete life cycle of serverless functions and improves the performance of serverless computing by changing the original method of exchanging space for time or time for space. We focus on how to shorten the cold-start time of serverless computing while maximizing the usage of container resources. The research innovation is the dynamic control of functions and container pools, which mainly includes three aspects. First, we create a container pool with the classification identification based on the usage rate of functions. Then, we use namespace technology to achieve container resource reuse in the security isolated state. Next, we adaptively match the correspondence between functions and reusable container resources through system resource monitoring. Finally, the test results prove that converting the remaining space resources of the container into a prewarm container for new functions can effectively reduce the resource waste caused by idle function-containers, and container resource reuse can further shorten the cold-start time while ensuring the safety and isolation of functions. Compared to other open-source serverless platforms, our solution can reduce the cold-start time of general function calls to less than 20 ms and improve the ability to alleviate cold starts by 90% without enabling container prewarming.