Stateless Functions Research Articles

Relational Programming with Foundational Models

Foundation models have vast potential to enable diverse AI applications. The powerful yet incomplete nature of these models has spurred a wide range of mechanisms to augment them with capabilities such as in-context learning, information retrieval, and code interpreting. We propose Vieira, a declarative framework that unifies these mechanisms in a general solution for programming with foundation models. Vieira follows a probabilistic relational paradigm and treats foundation models as stateless functions with relational inputs and outputs. It supports neuro-symbolic applications by enabling the seamless combination of such models with logic programs, as well as complex, multi-modal applications by streamlining the composition of diverse sub-models. We implement Vieira by extending the Scallop compiler with a foreign interface that supports foundation models as plugins. We implement plugins for 12 foundation models including GPT, CLIP, and SAM. We evaluate Vieira on 9 challenging tasks that span language, vision, and structured and vector databases. Our evaluation shows that programs in Vieira are concise, can incorporate modern foundation models, and have comparable or better accuracy than competitive baselines.

Serverless-like platform for container-based YARN clusters

Serverless computing is an emerging paradigm that has gained a lot of relevance in recent years, as it allows users to consume computing resources without worrying about the underlying infrastructure and pay only for what they actually use. Most current services that implement this paradigm typically rely on the Function-as-a-Service (FaaS) model, which works perfectly for simple applications based on stateless functions triggered by specific events. However, these services are not designed to run more complex applications with intricate interactions, usually presenting a significant degree of configuration difficulty and/or low ability to customise the execution environment. They also tend to be designed for short and simple workloads, with some services even limiting their maximum runtime to just a few minutes. In this paper, we present a platform based on Hadoop YARN oriented to the execution of Big Data workloads in a containerised and serverless way, so that the resources allocated to such containers are automatically and dynamically scaled according to their actual usage. An experimental evaluation has been carried out to compare our serverless-like platform with a standard YARN deployment when executing Big Data workloads concurrently. Our results have shown experimental evidence of enhancing both performance and overall resource efficiency, providing runtime reductions and resource usage improvements of up to 41% and 50%, respectively.

Serverless application composition leveraging function fusion: Theory and algorithms

Serverless computing is a novel cloud computing paradigm enabling flexible, cost-efficient and fine-granular development of cloud-native applications, although without any guarantees on scheduling or execution times. Thus, various high-level solutions have been proposed in recent years to find proper configurations of individual FaaS functions, while considering user-given QoS requirements. However, the ever-increasing complexity of invocation patterns among stateless functions, externalized management of intermediate states, and diverse public cloud resources pose new challenges to the composition of highly data-intensive serverless applications. In this paper, we fill this gap by proposing novel algorithms based on the emerging function fusion technique, along with the related cost/performance models of composite functions supporting implicit instance parallelization and internal state propagation. We prove the NP-completeness of the underlying latency-constrained tree partitioning problem, and design a bicriteria approximation scheme and a greedy heuristic to derive cost-efficient deployment configurations in polynomial time. With the help of extensive simulations using synthetic call graphs generated from public cloud traces, we demonstrate the applicability and superior runtime performance of our proposed methods compared to state-of-the-art solutions. In addition, we showcase that further cost-reduction of up to 3–6 % can be achieved compared to the optimal partitioning with the allowance of tolerable latency violations.

SD-SRF: An Intelligent Service Deployment Scheme for Serverless-operated Cloud-Edge Computing in 6G Networks

With the development of serverless computing, developers can implement and deploy business applications as a combination of stateless functions. Although originally proposed for cloud computing, serverless computing is gradually applied to cloud-edge systems for service deployment to provide users with high-quality, low-latency services. However, optimized service deployment in 6G networks is a very challenging issue because of the vast number of deployable devices in the network, and its permutations are highly exponential. In this paper, we propose optimized service deployment schemes for online and offline, respectively, to minimize the overall latency at a lower cost. (1) First, a SD-SRF algorithm based on the greedy algorithm is proposed to optimize the service deployment for a multi-layer edge network, which consists of two phases: SR and SF. (a) Services are deployed in the nearest ancestor devices in the routing tree of all such service requests with the least cost of deployment. (b) When the deployment cost of moving some service replicas to devices in the lower layer is less than the benefit, the service will fall. (2) However, the offline algorithm relies on the availability of prior information heavily, such as request arrival pattern and number, which is difficult to obtain. Therefore, this paper proposes a PPO-MSD algorithm for optimal deployment online, where a Markov decision process (MDP) is modeled. Extensive simulation results show that PPO-MSD outperforms existing algorithms in terms of overall delay and utility, and its performance is close to the optimal ones obtained by SD-SRF, with the SD-SRF and PPO-MSD algorithms reducing the delay on average by 32.40% and 9.91%.

FaaSLight : General Application-level Cold-start Latency Optimization for Function-as-a-Service in Serverless Computing

Serverless computing is a popular cloud computing paradigm that frees developers from server management. Function-as-a-Service (FaaS) is the most popular implementation of serverless computing, representing applications as event-driven and stateless functions. However, existing studies report that functions of FaaS applications severely suffer from cold-start latency. In this article, we propose an approach, namely, FaaSLight , to accelerating the cold start for FaaS applications through application-level optimization. We first conduct a measurement study to investigate the possible root cause of the cold-start problem of FaaS. The result shows that application code loading latency is a significant overhead. Therefore, loading only indispensable code from FaaS applications can be an adequate solution. Based on this insight, we identify code related to application functionalities by constructing the function-level call graph and separate other code (i.e., optional code) from FaaS applications. The separated optional code can be loaded on demand to avoid the inaccurate identification of indispensable code causing application failure. In particular, a key principle guiding the design of FaaSLight is inherently general, i.e., platform - and language-agnostic . In practice, FaaSLight can be effectively applied to FaaS applications developed in different programming languages (Python and JavaScript), and can be seamlessly deployed on popular serverless platforms such as AWS Lambda and Google Cloud Functions, without having to modify the underlying OSes or hypervisors, nor introducing any additional manual engineering efforts to developers. The evaluation results on real-world FaaS applications show that FaaSLight can significantly reduce the code loading latency (up to 78.95%, 28.78% on average), thereby reducing the cold-start latency. As a result, the total response latency of functions can be decreased by up to 42.05% (19.21% on average). Compared with the state-of-the-art, FaaSLight achieves a 21.25× improvement in reducing the average total response latency.

Performance Optimization of Serverless Computing for Latency-Guaranteed and Energy-Efficient Task Offloading in Energy-Harvesting Industrial IoT

Serverless architecture enables various intelligent applications to be run without managing infrastructure. In this architecture, the computing cost is generally proportional to the number of requested stateless functions and this number can affect the task completion time and, thus, it is prominent to decide an appropriate number of requested stateless functions. In this article, we propose a latency-guaranteed and energy-efficient task offloading (LETO) system where an Internet of Things (IoT) device decides the number of stateless functions requested to the cloud by considering the deadline on the task completion time and its energy level. To minimize the computing cost while guaranteeing sufficiently short task completion time and low energy outage probability, we formulate a constrained Markov decision process (CMDP) problem and convert the CMDP problem into an equivalent linear programming (LP) model. By solving the LP model, the optimal policy on the number of requested stateless functions can be achieved. Evaluation results illustrate that LETO can cut down the operating expenditure (OPEX) by up to 59% compared to a latency-guaranteed offloading scheme while keeping the task completion time and the energy outage probability below desirable levels.

DIFFUSE: A DIstributed and decentralized platForm enabling Function composition in Serverless Environments

Serverless computing is an emerging proposition in the cloud offering landscape that promotes a higher level of abstraction, further decoupling software operations from the underlying hardware. Often recognized as an economically driven computational approach, the serverless model relies on the execution of short-lived stateless functions, enabling a fine-grained accounting and control of resources. In this context, function composition represents an appealing feature, allowing the composition of two or more functions to create tailored processing pipelines, incentivizing modularity and reusability of functions, while paving the way to application-specific run-time optimizations. This work presents DIFFUSE: a DIstributed and decentralized platForm enabling Function composition in Serverless Environments. DIFFUSE embodies an innovative infrastructural support, enabling the efficient and transparent composition of functions by relying on pluggable middleware support, serving as a conveyor of messages among the platform components. Broadening the deployment spectrum of our proposal, we assess different middleware solutions, each presenting distinct delivery profiles, evidencing the tradeoffs that emerge.

Transactions across serverless functions leveraging stateful dataflows

Serverless computing is currently the fastest-growing cloud services segment. The most prominent serverless offering is Function-as-a-Service (FaaS), where users write functions and the cloud automates deployment, maintenance, and scalability. Although FaaS is a good fit for executing stateless functions, it does not adequately support stateful constructs like microservices and scalable, low-latency cloud applications. Recently, there have been multiple attempts to add first-class support for state in FaaS systems, such as Microsoft Orleans, Azure Durable Functions, or Beldi. These approaches execute business code inside stateless functions, handing over their state to an external database. In contrast, approaches such as Apache Flink’s StateFun follow a different design: a dataflow system such as Apache Flink handles all state management, messaging, and checkpointing by executing a stateful dataflow graph providing exactly-once state processing guarantees. This design relieves programmers from having to “pollute” their business logic with distributed systems error checking, management, and mitigation. Although programmers can easily develop applications without worrying about messaging and state management, executing transactions across stateful functions remains an open problem.In this paper, we introduce a programming model and implementation for transaction orchestration of stateful serverless functions. Our programming model supports serializable distributed transactions with two-phase commit, as well as eventually consistent workflows with Sagas. We design and implement our programming model on Apache Flink StateFun to leverage Flink’s exactly-once processing and state management guarantees. Our experiments show that the approach of building transactional systems on top of dataflow graphs can achieve very high throughput, but with latency overhead due to checkpointing mechanism guaranteeing the exactly-once processing. We compare our approach to Beldi that implements two-phase commit on AWS lambda functions backed by DynamoDB for state management, as well as an implementation of a system that makes use of CockroachDB as its backend.

PAPS: A Serverless Platform for Edge Computing Infrastructures

Edge computing infrastructures are often employed to run applications with low latency requirements. Users can exploits nodes that are close to their physical positions so that the delay of sending computations and data to the Cloud is mitigated. Since users frequently change their locations, and the resources available in the Edge are limited, the management of these infrastructures poses new, difficult challenges. This paper presents PAPS (Partitioning, Allocation, Placement, and Scaling), a framework for the efficient, automated and scalable management of large-scale Edge topologies. PAPS acts as a serveless platform for the Edge. Service providers can upload applications as compositions of lightweight and stateless functions along with latency constraints. At runtime, PAPS manages these applications by executing them in containers, it changes their placement in the Edge topology according to the geographical distribution of the workload, and efficiently allocates resources according to their needs. This paper also presents the architecture of a PAPS prototype built atop Kubernetes and OpenFaaS. The assessment shows both the feasibility of the approach and the ability of efficiently managing hundreds of serverless concurrent functions and of dealing with intense and unpredictable workload variations.

Improving the Formal Verification of Reachability Policies in Virtualized Networks

Network Function Virtualization (NFV) and Software Defined Networking (SDN) are new emerging paradigms that changed the rules of networking, shifting the focus on dynamicity and programmability. In this new scenario, a very important and challenging task is to detect anomalies in the data plane, especially with the aid of suitable automated software tools. In particular, this operation must be performed within quite strict times, due to the high dynamism introduced by virtualization. In this article, we propose a new network modeling approach that enhances the performance of formal verification of reachability policies, checked by solving a Satisfiability Modulo Theories (SMT) problem. This performance improvement is motivated by the definition of function models that do not work on single packets, but on packet classes. Nonetheless, the modeling approach is comprehensive not only of stateless functions, but also stateful functions such as NATs and firewalls. The implementation of the proposed approach achieves high scalability in complex networked systems consisting of several heterogeneous functions.

A Decentralized Framework for Serverless Edge Computing in the Internet of Things

Serverless computing is becoming widely adopted among cloud providers, thus making increasingly popular the Function-as-a-Service (FaaS) programming model, where the developers realize services by packaging sequences of stateless function calls. The current technologies are very well suited to data centers, but cannot provide equally good performance in decentralized environments, such as edge computing systems, which are expected to be typical for Internet of Things (IoT) applications. In this paper, we fill this gap by proposing a framework for efficient dispatching of stateless tasks to in-network executors so as to minimize the response times while exhibiting short- and long-term fairness, also leveraging information from a virtualized network infrastructure when available. Our solution is shown to be simple enough to be installed on devices with limited computational capabilities, such as IoT gateways, especially when using a hierarchical forwarding extension. We evaluate the proposed platform by means of extensive emulation experiments with a prototype implementation in realistic conditions. The results show that it is able to smoothly adapt to the mobility of clients and to the variations of their service request patterns, while coping promptly with network congestion.

Cloudburst

Function-as-a-Service (FaaS) platforms and "serverless" cloud computing are becoming increasingly popular due to ease-of-use and operational simplicity. Current FaaS offerings are targeted at stateless functions that do minimal I/O and communication. We argue that the benefits of serverless computing can be extended to a broader range of applications and algorithms while maintaining the key benefits of existing FaaS offerings. We present the design and implementation of Cloudburst, a stateful FaaS platform that provides familiar Python programming with low-latency mutable state and communication, while maintaining the autoscaling benefits of serverless computing. Cloudburst accomplishes this by leveraging Anna, an autoscaling key-value store, for state sharing and overlay routing combined with mutable caches co-located with function executors for data locality. Performant cache consistency emerges as a key challenge in this architecture. To this end, Cloudburst provides a combination of lattice-encapsulated state and new definitions and protocols for distributed session consistency. Empirical results on benchmarks and diverse applications show that Cloudburst makes stateful functions practical, reducing the state-management overheads of current FaaS platforms by orders of magnitude while also improving the state of the art in serverless consistency.