Cluster Of Workstations Research Articles

The NSWAP module for network swap

Swapping between nodes in a cluster of workstations to improve system performance has been discussed by several research papers in the recent past [2] [3]. Studies of workstation usage patterns show that most of the resources in a cluster of workstation are free most of the time even at peak workloads [1]. As a result, there will almost always be idle remote memory for an overloaded node to use as swap space. In addition, faster networks and slowly increasing disk speeds make swapping to remote memory an increasingly attractive option. We present a network swapping system called “nswap” implemented as a loadable kernel module for for the Linux kernel. nswap features a rapid response time even in the case of out of date information about remote hosts, a load balancing scheme through page migrations, and smooth fail-over to disk in case network swap space becomes scarce. The nswap module, conceptually divided into a pseudo-device driver client and a memory server, achieves these goal using a set of protocols described in the poster.

The black-box Niederreiter algorithm and its implementation over the binary field

The most time-consuming part of the Niederreiter algorithm for factoring univariate polynomials over finite fields is the computation of elements of the nullspace of a certain matrix. This paper describes the so-called Niederreiter algorithm, in which these elements are found by using a method developed by Wiedemann. The main advantages over an approach based on Gaussian elimination are that the matrix does not have to be stored in memory and that the computational complexity of this approach is lower. The black-box Niederreiter algorithm for factoring polynomials over the binary field was implemented in the C programming language, and benchmarks for factoring high-degree polynomials over this field are presented. These benchmarks include timings for both a sequential implementation and a parallel implementation running on a small cluster of workstations. In addition, the Wan algorithm, which was recently introduced, is described, and connections between (implementation aspects of) Wan's and Niederreiter's algorithm are given.

Parallel programming of gradient-based iterative image reconstruction schemes for optical tomography

Optical tomography (OT) is a fast developing novel imaging modality that uses near-infrared (NIR) light to obtain cross-sectional views of optical properties inside the human body. A major challenge remains the time-consuming, computational-intensive image reconstruction problem that converts NIR transmission measurements into cross-sectional images. To increase the speed of iterative image reconstruction schemes that are commonly applied for OT, we have developed and implemented several parallel algorithms on a cluster of workstations. Static process distribution as well as dynamic load balancing schemes suitable for heterogeneous clusters and varying machine performances are introduced and tested. The resulting algorithms are shown to accelerate the reconstruction process to various degrees, substantially reducing the computation times for clinically relevant problems.

Taking advantage of heterogeneity in disk arrays

Disk arrays, or RAIDs, have become the solution to increase the capacity and bandwidth of most storage system, but their usage has some limitations because all the disks in the array have to be equal. Nowadays, assuming a homogeneous set of disks to build an array is becoming a not very realistic assumption in many environments, especially in low-cost clusters of workstations. It is difficult to find a disk with the same characteristics as the ones in the array and replacing or adding new disks breaks the homogeneity. In this paper, we propose two block-distribution algorithms (one for RAID0 and an extension for RAID5) that can be used to build disk arrays from a heterogeneous set of disks. We also show that arrays using this algorithm are able to serve many more disk requests per second than when blocks are distributed assuming that all disks have the lowest common speed, which is the solution currently being used.

Distributed and shared memory parallelism with a smoothed particle hydrodynamics code

The Smoothed Particle Hydrodynamics (?) method, being Lagrangian in nature, provides advantages in modelling flows containing interfaces. The ? method is completely mesh-free, modelling the fluid as a collection of N particles which move with the fluid velocity. The continuum fluid properties at a particular location are interpolated as weighted sums of the properties of surrounding particles in a process known as kernel interpolation. In its simplest form the ? method requires that every particle is used in the updating of every other particle. This leads to a solution time that is proportional to N 2 . Methods using only nearby particles reduce this requirement significantly; however, the ? method is still expensive and would benefit from being implemented on a parallel computer. An example ? code has been parallelised using ?, OpenMP and ?, and its performance has been measured on an ?, Origin 2000 and a Beowulf workstation cluster.

Solving hub arc location problems on a cluster of workstations

Many modern transportation systems rely on a network of hub facilities to help concentrate flows of freight or passengers to exploit the economies of scale in transportation. The design of a hub network, including location of the hub facilities, is a key determinant of the cost and competitiveness of a transportation and logistics system. This paper reports on a parallel implementation of an algorithm for the hub arc location model to design such a network. Computational work was performed on a cluster of workstations with data for air passenger traffic in the United States and postal operations in Sydney, Australia.

Adaptive scheduling under memory constraints on non-dedicated computational farms

This paper presents scheduler extensions that enable better adaptation of parallel programs to the execution conditions of non-dedicated computational farms with limited memory resources. The purpose of the techniques is to prevent thrashing and co-schedule communicating threads, using two disjoint, yet cooperating extensions to the kernel scheduler. A thrashing prevention module enables memory-bound programs to adapt to memory shortage, via suspending their threads at selected points of execution. Thread suspension is used so that memory is not over-committed by parallel jobs—which are assumed to be running as guests on the nodes of the computational farm—at memory allocation points. In the event of thrashing, parallel jobs are the first to release memory and help local resident jobs make progress. Adaptation is implemented using a shared-memory interface in the /proc filesystem and upcalls from the kernel to the user space. On an orthogonal axis, co-scheduling is implemented in the kernel with a heuristic that boosts periodically the priority of communicating threads. Using experiments on a cluster of workstations, we show that when a guest parallel job competes with general-purpose interactive, I/O-intensive, or CPU and memory-intensive load on the nodes of the cluster, thrashing prevention reduces drastically the slowdown of the job at memory utilization levels of 20% or higher. The slowdown of parallel jobs is reduced by up to a factor of 7. Co-scheduling provides a limited performance improvement at memory utilization levels below 20%, but has no significant effect at higher memory utilization levels.

Efficient collective communication in distributed heterogeneous systems

With recent advances in high-speed networks, distributed heterogeneous computing has emerged as an attractive computational paradigm. Wide-area grid infrastructures will enable distributed applications—such as video conferencing and distributed interactive simulation—to seamlessly integrate collections of heterogeneous workstations, multiprocessors, and mobile nodes. The underlying network is typically a collection of several heterogeneous links, of different networking technologies. Such a heterogeneous network is also typical in local area workstation clusters, which are increasingly being used as alternatives to parallel computing systems. This paper introduces a framework for developing efficient collective communication schedules over such heterogeneous networks. We focus on application-level communication, between processes of a parallel program. Our framework consists of analytical models of the heterogeneous system, scheduling algorithms for the collective communication pattern, and performance evaluation mechanisms. We show that previous models, which considered node heterogeneity but ignored network heterogeneity, can lead to solutions which are worse than the optimal by an unbounded factor. We then introduce an enhanced communication model, and develop three heuristic algorithms for the broadcast and multicast patterns. The completion time of the schedule is chosen as the performance metric. The heuristic algorithms are fastest edge first (FEF), earliest completing edge first (ECEF), and ECEF with look-ahead. For small system sizes, we find the optimal solution using exhaustive search. Our simulation experiments indicate that the performance of our heuristic algorithms is close to optimal. For performance evaluation of larger systems, we have also developed a simple lower bound on the completion time. Our heuristic algorithms achieve significant performance improvements over previous approaches.

Performance of a distributed architecture for query processing on workstation clusters

A network of workstations (NOWs) is an attractive alternative to parallel database systems due to the cost advantage. In a typical database, client workstations (nodes) submit queries/transactions and receive responses from the database server. With even recent PC-based client nodes providing traditional workstation-class performance, performance improvements can be obtained by offloading some of the processing typically done on the traditional server node to these powerful client nodes. Parallel query processing takes advantage of the idle cycles on the client nodes to process the query. In this paper we introduce a distributed architecture for parallel query processing and study the performance of this architecture in various scenarios. We have implemented the distributed architecture using parallel virtual machine (PVM) on a Pentium-based NOW system. Our results show that the distributed architecture successfully achieves good speedups and scaleups. Furthermore, our results show that the distributed architecture handles both light and heavy workloads in the presence of a background (non-query) load.

Supporting dynamic parallel object arrays

AbstractWe present efficient support for generalized arrays of parallel data driven objects. Array elements are regular C++ objects, and are scattered across the parallel machine. An individual element is addressed by its ‘index’, which can be an arbitrary object rather than a simple integer. For example, an array index can be a series of numbers, supporting multidimensional sparse arrays; a bit vector, supporting collections of quadtree nodes; or a string. Methods can be invoked on any individual array element from any processor, and the elements can participate in reductions and broadcasts. Individual elements can be created or deleted dynamically at any time. Most importantly, the elements can migrate from processor to processor at any time. We discuss support for message delivery and collective operations in the face of such dynamic behavior. The migration capabilities of array elements have proven extremely useful, for example, in implementing flexible load balancing strategies and for exploiting workstation clusters adaptively. We present the design, an implementation, and performance results. Copyright © 2003 John Wiley & Sons, Ltd.

Parallel Monte Carlo algorithms for information retrieval

In any data mining applications, automated text and text and image retrieval of information is needed. This becomes essential with the growth of the Internet and digital libraries. Our approach is based on the latent semantic indexing (LSI) and the corresponding term-by-document matrix suggested by Berry and his co-authors. Instead of using deterministic methods to find the required number of first “ k” singular triplets, we propose a stochastic approach. First, we use Monte Carlo method to sample and to build much smaller size term-by-document matrix (e.g. we build k× k matrix) from where we then find the first “ k” triplets using standard deterministic methods. Second, we investigate how we can reduce the problem to finding the “ k”-largest eigenvalues using parallel Monte Carlo methods. We apply these methods to the initial matrix and also to the reduced one. The algorithms are running on a cluster of workstations under MPI and results of the experiments arising in textual retrieval of Web documents as well as comparison of the stochastic methods proposed are presented.

Experience teaching hands-on parallel computing at a small college

Parallel computing can be taught at a small college given a well-planned approach and utilizing easily obtained tools. Parallel computing is defined as a combination of parallel architecture and parallel programming and is one of the few hardware/software continuums. The course hardware is a cluster of workstations that can be adapted for use from most CS labs. Most of the software used in the course is based in cooperative learning, group activities which lend themselves to the experimental nature of the topic.

Starfish: Fault-Tolerant Dynamic MPI Programs on Clusters of Workstations

This paper reports on the architecture and design of Starfish, an environment for executing dynamic (and static) MPI-2 programs on a cluster of workstations. Starfish is unique in being efficient, fault-tolerant, highly available, and dynamic as a system internally, and in supporting fault-tolerance and dynamicity for its application programs as well. Starfish achieves these goals by combining group communication technology with checkpoint/restart, and uses a novel architecture that is both flexible and portable and keeps group communication outside the critical data path, for maximum performance.

A fast BNM (Best Neighborhood Matching): Algorithm and parallel processing for image restoration

AbstractBest Neighborhood Matching (BNM) algorithm is a good approach of error concealment in terms of restored image quality. However, this kind of error concealment algorithm is commonly computation‐intensive, which restricts their real applications on large‐scale image or video sequence restoration. In this article, we propose a fast method, named Jump and look around Best Neighborhood Matching (JBNM), which reduces computing time to one sixth of that by BNM, while the quality of the restored images remains almost the same. To further reduce processing time and meet large‐scale image restorations, a parallel JBNM working on a cluster of workstations is proposed. Several critical techniques, including reading policy, overlap stripe data distribution, and communication strategies, have been developed to obtain high performance. Both theoretical analysis and experiment results indicate that our parallel JBNM provides an efficient technique for image restoration applications. © 2003 Wiley Periodicals, Inc. Int J Imaging Syst Technol, 13, 189–200, 2003; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ima.10057

Parallel Optimization Scheme for Industrial Steam Cracking Process.

The solution of realistic simulation and optimization problems is computationally very intense. Along with the development of parallel computing environment, it is necessary to construct an effective parallel strategy in order to exploit supercomputing power. A parallel solution strategy was developed for the optimization of the whole periodic operation of the cracking coils. The most time-consuming computation step for the gradients is deserialized, an approximate linear speed-up is acquired and the optimization calculation is well accelerated. The optimization work was carried out successfully on a cluster of workstations and the properties of the coarse-grained parallel computation technique were analyzed. The results suggest that the execution time of the steam cracking process optimization can be reduced significantly by distributing the heavy simulation duties onto several computing nodes, which communicate via communication networks.

Database Allocation Strategies for Parallel BLAST Evaluation on Clusters

In this work we investigate the parallel evaluation of BLAST, the most popular tool for comparing biological sequences. Our goal is to study database distribution issues that, besides workload balancing, could improve the performance of a set of BLAST processes running in a workstation cluster. We consider different partitioning strategies within actual BLAST executions against a few relevant molecular databases. We have implemented multiple databases and input sequence configurations and show that there are many important parameters, such as the fragment generation method and sequence similarities, that must be taken into account in order to make full use of such parallel environment.

Investigation of time-dependency of intracranial brain shift and its relation to the extent of tumor removal using intra-operative MRI

The object of the paper is to investigate intra-operative brainshift and its relation to the extent of tumor removal. Repeated T1w 3D datasets were acquired at different time points intra-operatively (T0; T1; T2... Tx) using a vertical open 0.5T MR scanner in six patients with intracranial tumor. An offline analysis with initial linear registration, intensity adjustment and finally nonlinear registration of the first versus subsequent time points (T0/T1; T0/T2... T0/Tx) was performed, yielding a 3D displacement vector field that describes the brainshift. Brainshift was analysed qualitatively and quantitatively. A semi-automatic segmentation technique was used for calculation of the tumor size and the size of tumor remnants. Semi-automatic segmentation was reliable in all but two cases. Segmentation was difficult and unreliable in astrocytomas grade II. The shift basically followed gravity. The major shift reached levels up to 25 mm. Significant shift was observed at the first time point (T0). Intra-operative brainshift can be analysed qualitatively and also captured quantitatively. Neuronavigation that is based on pre-operatively acquired datasets is associated with a significant risk of surgical morbidity at a very early time point. Parallelisation on a workstation cluster may reduce computation time so that information about the displacement can facilitate updated navigation.

Comparing Parallel Functional Languages: Programming and Performance

This paper presents a practical evaluation and comparison of three state-of-the-art parallel functional languages. The evaluation is based on implementations of three typical symbolic computation programs, with performance measured on a Beowulf-class parallel architecture. We assess three mature parallel functional languages: PMLS, a system for implicitly parallel execution of ML programss GPH, a mainly implicit parallel extension of Haskells and Eden, a more explicit parallel extension of Haskell designed for both distributed and parallel execution. While all three languages employ a completely implicit approach to communication, each language takes a different approach to specifying and controlling parallelism, ranging from explicit identification of processes as language constructs (Eden) through annotation of potential parallelism (GPH) to automatic detection of parallel skeletons in sequential code (PMLS). We present detailed performance measurements of all three systems on a widely available parallel architecture: a Beowulf cluster of low-cost commodity workstations. We use three representative symbolic applications: a matrix multiplication algorithm, an exact linear system solver, and a simple ray-tracer. Our results show how moderate speedups can be achieved with little or no changes to the sequential code, and that parallel performance can be significantly improved even within our high-level model of parallel functional programming by controlling key aspects of the program such as load distribution and thread granularity.

Hot interconnects 10 - thinking beyond the internet

Microcomputers and their users rely on interconnects to communicate. At the Hot Interconnects 10 conference, which took place 21 to 23 August 2002 at Stanford University, designers and architects from universities and companies met to discuss new technologies for high-performance switches, routers, and packet processing engines. These technologies enable computers and users to seamlessly exchange data over the Internet. They also transport data between multi-processors, interconnect clusters of workstations, and implement the core switching in storage area networks.

Fair scheduling for input buffered switches

Input buffered switch architecture has become attractive for implementing high performance switches for workstation clusters. It is challenging to provide a scheduling technique that is both highly efficient and fair in resource allocation. In this paper, we first introduce an iterative Fair Scheduling (iFS) scheme for input buffered switches that supports fair bandwidth distribution among the flows and achieves asymptotically 100% throughput. We then apply the idea of fair scheduling to switches with multicasting capability and propose an mFS scheme which allocates bandwidth to various flows according to their reservations. We show that mFS produces throughput comparable to the existing schemes while distributing the bandwidth as per the given reservations. Extensive simulation results are presented to validate the effectiveness of our proposed schemes.