RPC Mechanism Research Articles

SplitRPC: A {Control + Data} Path Splitting RPC Stack for ML Inference Serving

The growing adoption of hardware accelerators driven by their intelligent compiler and runtime system counterparts has democratized ML services and precipitously reduced their execution times. This motivates us to shift our attention to efficiently serve these ML services under distributed settings and characterize the overheads imposed by the RPC mechanism ('RPC tax') when serving them on accelerators. The RPC implementations designed over the years implicitly assume the host CPU services the requests, and we focus on expanding such works towards accelerator-based services. While recent proposals calling for SmartNICs to take on this task are reasonable for simple kernels, serving complex ML models requires a more nuanced view to optimize both the data-path and the control/orchestration of these accelerators. We program today's commodity network interface cards (NICs) to split the control and data paths for effective transfer of control while efficiently transferring the payload to the accelerator. As opposed to unified approaches that bundle these paths together, limiting the flexibility in each of these paths, we design and implement SplitRPC - a control + data path optimizing RPC mechanism for ML inference serving. SplitRPC allows us to optimize the datapath to the accelerator while simultaneously allowing the CPU to maintain full orchestration capabilities. We implement SplitRPC on both commodity NICs and SmartNICs and demonstrate how GPU-based ML services running different compiler/runtime systems can benefit. For a variety of ML models served using different inference runtimes, we demonstrate that SplitRPC is effective in minimizing the RPC tax while providing significant gains in throughput and latency over existing kernel by-pass approaches, without requiring expensive SmartNIC devices.

Build real-time communication for hybrid dual-OS system

Abstract In automotive/avionic/robot systems, consolidating multiple domains on a powerful SoC is an important trend. From the perspectives of safety and security, each of these domains must execute independently with a self-governed Operating System (OS). Such systems are termed as hybrid multi-OS systems. Generally, in a commonly used hybrid dual-OS system, a real-time OS (RTOS) and a general-purpose OS (GPOS) are integrated to satisfy different requirements. Inter-OS communication is the key to such systems. Specifically, services are often requested by tasks in RTOS from GPOS, but it is challenging to satisfy the requirements on time efficiency and predictability in RTOS, due to the uncertainty in communication maintenance and the time-sharing policy of the GPOS. Current existing approaches can not provide predictable time guarantees to the circumstances. This paper aims to build a time predictable and secure RPC mechanism (i.e., the primary and critical communication unit in a hybrid multi-OS system). On the foundation of a detailed analysis of the time delays in each phase of a RPC from RTOS to GPOS, we point out the drawbacks in conventional RPC mechanisms and propose an effective model and some methods to guarantee the predictability in all the phases of RPC. Furthermore, a real-time RPC scheme (termed RTRG-RPC) is proposed and applied to our ready-built TrustZone-based consolidation system (i.e., TZDKS). RTRG-RPC achieves accurate time control through three mechanisms: SGI (Software Generate Interrupt) message transforming, interrupt handler RPC servicing, and priority-swapping. Evaluations show that RTRG-RPC can achieve real-time predictability and managed priority inversion. Moreover, the design of RTRG-RPC also provides lightweight and essential protection on the security, without violation of the efficiency.

An abstract layered model for Web‐inclusive distributed computing leading to enhancing GRIDSpace with Web 2.0

AbstractService‐oriented computing (SOC), Grid computing, and Web2.0 computing have increasingly received momentum in both research and industry. In this paper, we propose an abstract Web‐inclusive distributed computing model (WIDCOM) in order to analyse these three important distributed computing paradigms. We then advance the notion of software in existing Grid services as discussed in open Grid services architecture, which provides software middleware or wrappers for accessing hardware resources towards the notion that the resources provided can be hardware, software, or hybrid hardware/software. It also proposes an approach using the integration of Grid service, semantic Grid, and Web2.0 to overcome some of the limitations of the existing Web services architecture, which relies on having the Web version of the RPC mechanism and thus has difficulty in dealing with massive scale of user participation and communication across the Internet. The new approach produces a novel Grid architecture—GRIDSpace. Copyright © 2008 John Wiley & Sons, Ltd.

Ninf-G: A Reference Implementation of RPC-based Programming Middleware for Grid Computing

GridRPC, which is an RPC mechanism tailored for the Grid, is an attractive programming model for Grid computing. This paper reports on the design and implementation of a GridRPC programming system called Ninf-G. Ninf-G is a reference implementation of the GridRPC API which has been proposed for standardization at the Global Grid Forum. In this paper, we describe the design, implementations and typical usage of Ninf-G. A preliminary performance evaluation in both WAN and LAN environments is also reported. Implemented on top of the Globus Toolkit, Ninf-G provides a simple and easy programming interface based on standard Grid protocols and the API for Grid Computing. The overhead of remote procedure calls in Ninf-G is acceptable in both WAN and LAN environments.

Remote procedure calls and Java Remote Method Invocation

Remote procedure call systems have been around since around 1984 when they were first proposed (A.D. Birrell and B.J. Nelson, 1984). During the intervening 15 years, numerous evolutionary improvements have occurred in the basic RPC system, leading to improved systems-such as NCS (T.H. Dineen et al., 1987)-that offer programmers more functionality or greater simplicity. The Common Object Request Broker Architecture from the Object Management Group and Microsoft's Distributed Common Object Model are this evolutionary process's latest outgrowths. With the introduction of Java Developer's Kit release 1.1, a third alternative for creating distributed applications has emerged. The Java Remote Method Invocation system has many of the same features of other RPC systems, letting an object running in one Java virtual machine make a method call on an object running in another, perhaps on a different physical machine. On the surface, the RMI system is just another RPC mechanism, much like Corba and DCOM. But on closer inspection, RMI represents a very different evolutionary progression, one that results in a system that differs not just in detail but in the very set of assumptions made about the distributed systems in which it operates. These differences lead to differences in the programming model, capabilities, and the way the mechanisms interact with the code that implements and built the distributed systems.

Client-server based ray-tracer using ASTRA: an asynchronous RPC mechanism

A distributed ray tracer is implemented in a network of workstations using ASTRA, an asynchronous remote procedure call mechanism. The motivation of this work is to utilize the idle cycles available on workstations to speed up the execution of the ray tracing task. The application uses the client-server model with a central dispatcher as the client, sending ray tracing jobs to the servers which handle the actual ray tracing computations. A new job is given to a server that has completed its present job and successfully sent back the results. Performance tests are conducted on a group of homogeneous and heterogeneous workstations using different scene complexities and varying number of servers. Our results showed that for some of the databases, it is possible to achieve 64 times speedup with 85 servers. The paper discusses the ASTRA call primitives, the client and server parts of our distributed ray tracer program, and presents the results of the performance tests.

Communicative Action Notation with Shared Storage

A variation of the Communicative Action Notation is presented, in order to allow the sharing of storage among agents. Examples of use of the new notation are given by means of the description of semaphores, monitors and RPC mechanisms. The operational semantics of the new notation is also presented.