Middleware Stack Research Articles

More Is Less: Model Augmentation for Intermittent Deep Inference

Energy harvesting creates an emerging intermittent computing paradigm but poses new challenges for sophisticated applications such as intermittent deep neural network (DNN) inference. Although model compression has adapted DNNs to resource-constrained devices, under intermittent power, compressed models will still experience multiple power failures during a single inference. Footprint-based approaches enable hardware-accelerated intermittent DNN inference by tracking footprints, independent of model computations, to indicate accelerator progress across power cycles. However, we observe that the extra overhead required to preserve progress indicators can severely offset the computation progress accumulated by intermittent DNN inference. This work proposes the concept of model augmentation to adapt DNNs to intermittent devices. Our middleware stack, JAPARI, appends extra neural network components into a given DNN, to enable the accelerator to intrinsically integrate progress indicators into the inference process, without affecting model accuracy. Their specific positions allow progress indicator preservation to be piggybacked onto output feature preservation to amortize the extra overhead, and their assigned values ensure uniquely distinguishable progress indicators for correct inference recovery upon power resumption. Evaluations on a Texas Instruments device under various DNN models, capacitor sizes, and progress preservation granularities show that JAPARI can speed up intermittent DNN inference by 3× over the state of the art, for common convolutional neural architectures that require heavy acceleration.

WLCG Token Usage and Discovery

Since 2017, the Worldwide LHC Computing Grid (WLCG) has been working towards enabling token based authentication and authorisation throughout its entire middleware stack. Following the publication of the WLCG Common JSON Web Token (JWT) Schema v1.0 [1] in 2019, middleware developers have been able to enhance their services to consume and validate the JWT-based [2] OAuth2.0 [3] tokens and process the authorization information they convey. Complex scenarios, involving multiple delegation steps and command line flows, are a key challenge to be addressed in order for the system to be fully operational. This paper expands on the anticipated token based workflows, with a particular focus on local storage of tokens and their discovery by services. The authors include a walk-through of this token flow in the RUCIO managed data-transfer scenario, including delegation to FTS and authorised access to storage elements. Next steps are presented, including the current target of submitting production jobs authorised by Tokens within 2021.

Everything Leaves Footprints: Hardware Accelerated Intermittent Deep Inference

Current peripheral execution approaches for intermittently powered systems require full access to the internal hardware state for checkpointing or rely on application-level energy estimation for task partitioning to make correct forward progress. Both requirements present significant practical challenges for energy-harvesting, intelligent edge Internet-of-Things devices, which perform hardware-accelerated deep neural network (DNN) inference. Sophisticated compute peripherals may have an inaccessible internal state, and the complexity of DNN models makes it difficult for programmers to partition the application into suitably sized tasks that fit within an estimated energy budget. This article presents the concept of inference footprinting for intermittent DNN inference, where accelerator progress is accumulatively preserved across power cycles. Our middleware stack, HAWAII, tracks and restores inference footprints efficiently and transparently to make inference forward progress, without requiring access to the accelerator internal state and application-level energy estimation. Evaluations were carried out on a Texas Instruments device, under varied energy budgets and network workloads. Compared to a variety of task-based intermittent approaches, HAWAII improves the inference throughput by 5.7%-95.7%, particularly achieving higher performance on heavily accelerated DNNs.

Testing the limits of an LVS - GridFTP cluster as a replacement for BeSTMan

The Worldwide LHC Computing Grid (WLCG) is the largest grid computing infrastructure in the world pooling the resources of 170 computing centers (sites). One advantage of grid computing is that multiple copies of data can be distributed across different sites, allowing user access that is independent of geographic location or software. Each site is able to communicate using software stacks collectively referred to as “middleware”. One key middleware piece is the storage element (SE), which provides remote POSIX-like access to a site’s storage.The middleware stack managed by the Open Science Grid (OSG) used a storage resource manager (SRM) protocol implementation that, among other things, allowed sites to load-balance servers providing the Grid File Transfer Protocol (GridFTP) interface. OSG is eliminating the use of an SRM entirely and is transitioning to a solution based solely on GridFTP load-balanced at the network level with Linux Virtual Server (LVS). LVS is a core component of the Linux kernel, so this change increases both maintainability and reduces complexity of the site. In this document, we outline our methodologies and results from the large scale testing of an LVS+GridFTP cluster for data reads. Additionally, we discuss potential optimizations to the cluster to maximize total throughput.

Towards effective scheduling policies for many-task applications: Practice and experience based on HTCaaS

Summary In this paper, we conduct a comparative study of relatively simple yet effective scheduling policies for many-task applications where multiple users with varying numbers of tasks are actively sharing a common system infrastructure. We have implemented three different scheduling mechanisms that can address fairness and user response time respectively in a common middleware stack called HTCaaS, which is a pilot-job–based multi-level scheduling system running on top of production-level clusters. As a representative case of our many-task applications, we have leveraged the virtual screening application, which is a computational technique used in drug discovery process to select the most promising candidate drugs for in vitro testing from millions of chemical compounds, which typically requires a substantial amount of computing resources and efficient processing of docking simulations. Our comparative experimental results of different scheduling policies show how we can effectively support multiple users in a shared resource environment by balancing between user satisfaction and overall system performance, provide guidelines to improve system utilization, and address additional technical challenges to support various many-task applications.

Using Hadoop File System and MapReduce in a small/medium Grid site

Data storage and data access represent the key of CPU-intensive and data-intensive high performance Grid computing. Hadoop is an open-source data processing framework that includes fault-tolerant and scalable distributed data processing model and execution environment, named MapReduce, and distributed File System, named Hadoop distributed File System (HDFS).HDFS was deployed and tested within the Open Science Grid (OSG) middleware stack. Efforts have been taken to integrate HDFS with gLite middleware. We have tested the File System thoroughly in order to understand its scalability and fault-tolerance while dealing with small/medium site environment constraints. To benefit entirely from this File System, we made it working in conjunction with Hadoop Job scheduler to optimize the executions of the local physics analysis workflows. The performance of the analysis jobs which used such architecture seems to be promising, making it useful to follow up in the future.

Hiding the Complexity: Building a Distributed ATLAS Tier-2 with a Single Resource Interface using ARC Middleware

Since their inception, Grids for high energy physics have found management of data to be the most challenging aspect of operations. This problem has generally been tackled by the experiment's data management framework controlling in fine detail the distribution of data around the grid and the careful brokering of jobs to sites with co-located data. This approach, however, presents experiments with a difficult and complex system to manage as well as introducing a rigidity into the framework which is very far from the original conception of the grid.In this paper we describe how the ScotGrid distributed Tier-2, which has sites in Glasgow, Edinburgh and Durham, was presented to ATLAS as a single, unified resource using the ARC middleware stack. In this model the ScotGrid 'data store' is hosted at Glasgow and presented as a single ATLAS storage resource. As jobs are taken from the ATLAS PanDA framework, they are dispatched to the computing cluster with the fastest response time. An ARC compute element at each site then asynchronously stages the data from the data store into a local cache hosted at each site. The job is then launched in the batch system and accesses data locally.We discuss the merits of this system compared to other operational models and consider, from the point of view of the resource providers (sites), and from the resource consumers (experiments); and consider issues involved in transitions to this model.

Another look at the middleware for dependable distributed computing

AbstractKey concepts of reliable distributed computing developed during the 1980s and 1990s (e.g., transactions, replication) influenced the standards based middleware such as CORBA and Java EE. This middleware has evolved steadily over the years with message passing facilities to support construction of loosely coupled systems. However, the way networked computing is being used for business and social uses is undergoing rapid changes as new ways of constructing distributed execution environments from varieties of resources, ranging from computational, storage, network to application level services, provided by globally distributed service providers are emerging. In light of these developments, the paper examines what core concepts, components, and techniques that will be required in the next-generation middleware for dependable distributed computing. The paper puts forward the case for five topics for further research: better coordination facilities for loosely coupled systems, restructuring of the middleware stack for supporting multi-tenancy, replication in the large, negotiation, and enforcement of service agreements, and accountability.

On the usability of grid middleware and security mechanisms

Usability is an all too often neglected aspect of grid computing, although it is one of the principal factors militating against the widespread uptake of distributed computing. Many resource providers on a grid infrastructure deploy a standard middleware stack and expect users to invoke the default client tools for that middleware stack to access their resources. Unfortunately, many of these middleware client tools have been developed as an afterthought, and are widely considered difficult to use. Such tools typically require a user to interact with a machine, to stage data and launch jobs, and to use digital certificates. Our experience of working with grids over many years has led us to propose a new model of grid interaction, which we call the user-application interaction model. Similar considerations have also led us to develop environments that remove digital certificates from the user's experience, replacing them with familiar username and password authentication credentials. In this paper, we investigate the usability of this interaction model and its security system through a series of tests, which compare the usability of our systems with commonly deployed middleware tools using five usability metrics. Our middleware and security solutions are judged to be more usable than the systems in use by most of today's computationalgrids.

Coupled-Simulation e-Science Support in the NAREGI Grid

NAREGI's middleware stack provides comprehensive support for coupled applications in the nanoscience discipline. The authors describe several mechanisms that are application-area agnostic and can be applied to other disciplines, such as weather and climate simulations, multiresolution simulation in life sciences, and other complex systems.

A configuration complexity model and its application to a change management system

The complexity of configuring computing systems is a major impediment to the adoption of new information technology (IT) products and greatly increases the cost of IT services. This paper develops a model of configuration complexity and demonstrates its value for a change management system. The model represents systems as a set of nested containers with configuration controls. From this representation, we derive various metrics that indicate configuration complexity, including execution complexity, parameter complexity, and memory complexity. We apply this model to a J2EE-based enterprise application and its associated middleware stack to assess the complexity of the manual configuration process for this application. We then show how an automated change management system can greatly reduce configuration complexity.