High Throughput Computing System Research Articles

Hyperspectral in-memory computing with optical frequency combs and programmable optical memories

The rapid rise of machine learning drives demand for extensive matrix-vector multiplication operations, thereby challenging the capacities of traditional von Neumann computing systems. Researchers explore alternatives, such as in-memory computing architecture, to find energy-efficient solutions. In particular, there is renewed interest in optical computing systems, which could potentially handle matrix-vector multiplication in a more energy-efficient way. Despite promising initial results, developing high-throughput optical computing systems to rival electronic hardware remains a challenge. Here, we propose and demonstrate a hyperspectral in-memory computing architecture, which simultaneously utilizes space and frequency multiplexing, using optical frequency combs and programmable optical memories. Our carefully designed three-dimensional opto-electronic computing system offers remarkable parallelism, programmability, and scalability, overcoming typical limitations of optical computing. We have experimentally demonstrated highly parallel, single-shot multiply-accumulate operations with precision exceeding 4 bits in both matrix-vector and matrix-matrix multiplications, suggesting the system’s potential for a wide variety of deep learning and optimization tasks. Our approach presents a realistic pathway to scale beyond peta operations per second, a major stride towards high-throughput, energy-efficient optical computing.

Using Virtual Machine live migration in trace-driven energy-aware simulation of high-throughput computing systems

Virtual Machine (VM) live migration is one of the strategic approaches that can be employed to reduce energy consumption and increase the utilisation of a large computing infrastructure. In this paper, we implement the pre-copy live migration algorithm to provide a test environment for live job migration in a high throughput computing system (HTC). We present details of our extension into the HTC-Sim simulation framework. Furthermore, we demonstrate the performance and energy impact of our responsive live migration approach in high throughput computing (HTC) systems with various migration policies. The proposed policies focus on which computer to select for migration from an HTC resource pool. We evaluate our migration policies through the use of our HTC-Sim simulation framework. Moreover, we compare the results between the policies as well as the system where migration is not considered. We demonstrate that our responsive migration could save approximately 75% of the system wasted energy due to job evictions by user interruptions where migration is not employed as a fault-tolerance mechanism.

High-Throughput Cloud Computing with the Cloudscheduler VM Provisioning Service

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large-scale production particle physics workloads for many years using thousands of cores on three continents. A decade after its initial design and implementation, cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The updated cloudscheduler is more resilient and scalable, with expanded capabilities. We present an overview of the original design and then describe the new version of the distributed compute cloud system. We conclude with a review of the current status and future plans.

Cloudscheduler: a VM provisioning system for a distributed compute cloud

We describe a high-throughput computing system for running jobs on public and private computing clouds using the HTCondor job scheduler and the cloudscheduler VM provisioning service. The distributed cloud computing system is designed to simultaneously use dedicated and opportunistic cloud resources at local and remote locations. It has been used for large scale production particle physics workloads for many years using thousands of cores on three continents. Cloudscheduler has been modernized to take advantage of new software designs, improved operating system capabilities and support packages. The result is a more resilient and scalable system, with expanded capabilities.

Динамическое построение прогноза времени завершения вычислительного эксперимента в Desktop Grid

Desktop Grid is a powerful tool to perform high-throughput computing. Desktop Grid is a form of distributed high-throughput computing system, which uses idle time of non-dedicated geographically distributed computing nodes connected over low-speed network. It has significant differences from computing clusters and computational GRIDs, and needs special operation tackle. In this paper, we present a mechanism for dynamic forecasting of the completion time of a computational experiment in a Desktop Grid. We propose a statistical approach based on the linear regression model with the calculation of a confidence interval, taking into account the accumulation of statistical error and, if needed, changing the forecast. The developed approach is used to implement a forecasting algorithm and software module for a Desktop Grid. We present experimental results based on data from the RakeSearch volunteer computing project.

Wrangling distributed computing for high-throughput environmental science: An introduction to HTCondor.

Biologists and environmental scientists now routinely solve computational problems that were unimaginable a generation ago. Examples include processing geospatial data, analyzing -omics data, and running large-scale simulations. Conventional desktop computing cannot handle these tasks when they are large, and high-performance computing is not always available nor the most appropriate solution for all computationally intense problems. High-throughput computing (HTC) is one method for handling computationally intense research. In contrast to high-performance computing, which uses a single "supercomputer," HTC can distribute tasks over many computers (e.g., idle desktop computers, dedicated servers, or cloud-based resources). HTC facilities exist at many academic and government institutes and are relatively easy to create from commodity hardware. Additionally, consortia such as Open Science Grid facilitate HTC, and commercial entities sell cloud-based solutions for researchers who lack HTC at their institution. We provide an introduction to HTC for biologists and environmental scientists. Our examples from biology and the environmental sciences use HTCondor, an open source HTC system.

Mosaicking Copernicus Sentinel-1 Data at Global Scale

This paper presents a processing chain for handling big volume of remotely sensing data for generating wide extent mosaics. More specifically, the data under consideration are level-1 ground range detected Sentinel-1 products with dual polarisation (VV+VH or HH+HV). Two approaches for a) distribution discretization accompanied by false color composition and b) image rendering and mosaicking are proposed. While these two components are necessary constituents of the presented mosaicking workflow, they can operate independently of each other. The design of the processing chain satisfies three objectives: i) contrasting derivative products of the input Sentinel-1 imagery such as the Global Human Settlement Layer, ii) adapting on a high-throughput computing system for fast execution, and iii) allowing potential extensions to more complex applications such as the image classification. Fast processing, process automation, incremental adjustment and information distinction are the main advantages of the proposed method. Elaboration and focus on these features are carried out during the presentation of the results.

New advances in large‐scale distributed simulation and real‐time applications

New advances in large‐scale distributed simulation and real‐time applications

Lark: An effective approach for software-defined networking in high throughput computing clusters

High throughput computing (HTC) systems are widely adopted in scientific discovery and engineering research. They are responsible for scheduling submitted batch jobs to utilize the cluster resources. Current systems mostly focus on managing computing resources like CPU and memory; however, they lack flexible and fine-grained management mechanisms for network resources. This has increasingly been an urgent need as current batch systems may be distributed among dozens of sites around the globe like Open Science Grid. The Lark project was motivated by this need to re-examine how the HTC layer interacts with the network layer.In this paper, we present the system architecture of Lark and its implementation as a plugin of HTCondor which is a popular HTC software project. Lark achieves lightweight network virtualization at per-job granularity for HTCondor by utilizing Linux container and virtual Ethernet devices; this provides each batch job with a unique network address in a private network namespace. We extended HTCondor’s description language, ClassAds, so users can specify networking requirements in the job submission script. HTCondor can perform matchmaking to make sure user-specified network requirements and resource-specific policies are fulfilled. We also extended the job agent, condor_starter, so that it can manage and configure the job’s network environment. Given this important building block as the core, we implement bandwidth management functionality at both the host and network levels utilizing software-defined networking (SDN). In addition to HTCondor, Wide area network bandwidth management for GridFTP traffic is designed and implemented. Our experiments and evaluations show that Lark can effectively manage network resources simultaneously for both applications inside the cluster environment. By not resorting to heavyweight VMs, we keep startup overheads minimal compared to “regular” batch jobs. This mechanism provides the users with better predictability of their job execution and the administrators more policy flexibility in allocation of network resources.

Competition-based failure-aware scheduling for High-Throughput Computing systems on peer-to-peer networks

In a High-Throughput Computing (HTC) system, system failures and churning pose an important performance limitation. The time used by tasks running in a node that suddenly fails (or abandons the system) constitutes a waste of resources. These aborted tasks are usually reinserted into the system for automatic re-execution, causing additional overheads. This problem has been partially addressed via fault tolerant techniques such as checkpointing and replication. However, these solutions cause additional overheads. In this work, we present several failure-aware scheduling policies that aim to reduce the waste of resources by means of mechanisms to match the submitted tasks with the best node to run it, taking into consideration the (predicted) duration of the task and the (expected) survival time of the nodes. Experimentation through simulation, in the context of an HTC system built on top of a peer-to-peer network, confirms that our policies, compared to several state-of-the-art alternatives, result in a more effective distribution of workload whose consequence is a higher task throughput.

SU-FF-T-242: High-Throughput Computing in Condor Using a Nested Partitions Framework for IMRT Beam and Dose Optimization

Purpose: To utilize a High Throughput Computing (HTC) system to provide distributed computing on a network of computer workstations for a metaheuristic, Nested Partitions (NP), for beam angle selection (BAS) in IMRT. Method and Materials: An important feature of the NP approach is that its partition/sampling processes are naturally parallelized, allowing the integration of this powerful optimization methodology with high-performance distributed environments such as the Condor system. Condor is a specialized workload management system that can used on existing computer networks for compute-intensive jobs, allowing the submission of many jobs at the same time, and performing these jobs by locating and utilizing idle computers in the network. Our algorithm first solves an integer programming formulation of the BAS problem that uses aggregate mean organ-at-risk data. Based on this initial angle set, NP uses sampling on a partition of the global solution space to generate alternative angle sets that are distributed via Condor for evaluation via dose optimization. The corresponding dose optimization problems for the beam set samples are thus solved in parallel. NP repeats this procedure iteratively to obtain high quality angle sets for IMRT. Results: Compared to evaluating each beam angle set sequentially, parallel evaluation of the NP-selected beam angle sets 2457_2via Condor on 15 workstations yielded a factor of 14 speedup. Overall, BAS for a difficult clinical case in this setting required 15–28 minutes. Using a weighted sum of percentage violations of DVH thresholds for OARs as an overall measure of solution quality, a 78% improvement (head-and-neck case) and a 30% improvement (pelvic case) were achieved over equally-spaced beams. Conclusions: High-quality beam angle sets for IMRT can be efficiently obtained in an automated manner by utilizing an HTC system on an existing computer network to implement an NP-based metaheuristic. Conflict of Interest: N/A.

A Tool for Prioritizing DAGMan Jobs and its Evaluation

It is often difficult to perform efficiently a collection of jobs with complex job dependencies due to temporal unpredictability of the Grid. One way to mitigate the unpredictability is to schedule job execution in a manner that constantly maximizes the number of jobs that can be sent to workers. A recently developed scheduling theory provides a basis to meet that optimization goal. Intuitively, when the number of such jobs is always large, high parallelism can be maintained, even if the number of workers changes over time in an unpredictable manner. In this paper we present the design, implementation, and evaluation of a practical scheduling tool inspired by the theory. Given a DAGMan input file with interdependent jobs, the tool prioritizes the jobs. The resulting schedule significantly outperforms currently used schedules under a wide range of system parameters, as shown by simulation studies. For example, a scientific data analysis application, AIRSN, was executed at least 13% faster with 95% confidence. An implementation of the tool was integrated with the Condor high-throughput computing system.

Master–Worker: An Enabling Framework for Applications on the Computational Grid

The goal of this work is to create a tool that allows users to easily distribute large scientific computations on computational grids. Our tool MW relies on the simple master–worker paradigm. MW provides both a top Level interface to application software and a bottom Level interface to existing Grid computing toolkits. Both interfaces are briefly described. We conclude with a case study, where the necessary Grid services are provided by the Condor high-throughput computing system, and the MW-enabled application code is used to solve a combinatorial optimization problem of unprecedented complexity.

Matchmaking: An extensible framework for distributed resource management

Federated distributed systems present new challenges to resource management. Conventional resource managers are based on a relatively static resource model and a centralized allocator that assigns resources to customers. Distributed environments, particularly those built to support high-throughput computing (HTC), are often characterized by distributed management and distributed ownership. Distributed management introduces resource heterogeneity: Not only the set of available resources, but even the set of resource types is constantly changing. Distributed ownership introduces policy heterogeneity: Each resource may have its own idiosyncratic allocation policy. To address these problems, we designed and implemented the Matchmaking resource management framework. Customers and resources are all described by classified advertisements (classads) written in a simple but powerful formal language that describes their attributes and allocation policies. A Matchmaker server uses a policy-independent matching operation to discover pairings. It notifies the parties to the match, which use a separate, bilateral claiming protocol to confirm the allocation. The resulting framework is robust, scalable and flexible, and can evolve with changing resources. Matchmaking is the core of the Condor High Throughput Computing System developed at the University of Wisconsin – Madison. Condor is a production-quality system used by scientists and engineers at sites around the world. Condor derives much of its flexibility, robustness and efficiency from the matchmaking architecture. We describe the use of matchmaking in Condor, presenting several examples that illustrate its flexibility and expressiveness.