Heterogeneous Network Of Workstations Research Articles

Improving the scalability of hyperspectral imaging applications on heterogeneous platforms using adaptive run-time data compression

Latest generation remote sensing instruments (called hyperspectral imagers) are now able to generate hundreds of images, corresponding to different wavelength channels, for the same area on the surface of the Earth. In previous work, we have reported that the scalability of parallel processing algorithms dealing with these high-dimensional data volumes is affected by the amount of data to be exchanged through the communication network of the system. However, large messages are common in hyperspectral imaging applications since processing algorithms are pixel-based, and each pixel vector to be exchanged through the communication network is made up of hundreds of spectral values. Thus, decreasing the amount of data to be exchanged could improve the scalability and parallel performance. In this paper, we propose a new framework based on intelligent utilization of wavelet-based data compression techniques for improving the scalability of a standard hyperspectral image processing chain on heterogeneous networks of workstations. This type of parallel platform is quickly becoming a standard in hyperspectral image processing due to the distributed nature of collected hyperspectral data as well as its flexibility and low cost. Our experimental results indicate that adaptive lossy compression can lead to improvements in the scalability of the hyperspectral processing chain without sacrificing analysis accuracy, even at sub-pixel precision levels.

Evaluation of a distributed numerical simulation optimization approach applied to aquifer remediation

In this paper we evaluate a distributed approach which uses numerical simulation and optimization techniques to automatically find remediation solutions to a hypothetical contaminated aquifer. The repeated execution of the numerical simulation model of the aquifer through the optimization cycles tends to be computationally expensive. To overcome this drawback, the numerical simulations are executed in parallel using a network of heterogeneous workstations. Performance metrics for heterogeneous environments are not trivial; a new way of calculating speedup and efficiency for Bag-of-Tasks (BoT) applications is proposed. The performance of the parallel approach is evaluated.

Improving the Performance of Hyperspectral Image and Signal Processing Algorithms Using Parallel, Distributed and Specialized Hardware-Based Systems

Advances in sensor technology are revolutionizing the way remotely sensed data is collected, managed and analyzed. The incorporation of latest-generation sensors to airborne and satellite platforms is currently producing a nearly continual stream of high-dimensional data, and this explosion in the amount of collected information has rapidly created new processing challenges. For instance, hyperspectral signal processing is a new technique in remote sensing that generates hundreds of spectral bands at different wavelength channels for the same area on the surface of the Earth. Many current and future applications of remote sensing in Earth science, space science, and soon in exploration science will require (near) real-time processing capabilities. In recent years, several efforts have been directed towards the incorporation of high-performance computing (HPC) systems and architectures in remote sensing missions. With the aim of providing an overview of current and new trends in parallel and distributed systems for remote sensing applications, this paper explores three HPC-based paradigms for efficient implementation of the Pixel Purity Index (PPI) algorithm, available from the popular Kodak's Research Systems ENVI software package, as a representative case study for demonstration purposes. Several different parallel programming techniques are used to improve the performance of the PPI on a variety of parallel platforms, including a set of message passing interface (MPI)-based implementations on a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center in Maryland and on a variety of heterogeneous networks of workstations at University of Maryland; a Handel-C implementation of the algorithm on a Virtex-II field programmable gate array (FPGA); and a compute unified device architecture (CUDA)-based implementation on graphical processing units (GPUs) of NVidia. Combined, these parts deliver an excellent snapshot of the state-of-the-art in those areas, and offer a thoughtful perspective on the potential and emerging challenges of adapting HPC systems to remote sensing problems.

Enhancing the Schedulability of Real-Time Heterogeneous Networks of Workstations (NOWs)

This paper proposes a realtime duplication based algorithm (RT-DBA) for scheduling precedence-related periodic tasks with hard deadlines on networks of workstations (NOWs). We have utilized selective subtask duplication that enables some tasks to have earlier start times, which enables additional tasks (and, hence, task sets) to finish before their deadlines, thereby increasing the schedulability of a real-time application. We strongly believe that duplication can be used as a tool for obtaining a better quality of service (QoS) from the real-time heterogeneous system, and this is our major contribution. We have taken both the computation and the communication heterogeneities into account while modeling such a system. Both data and control dependencies between the tasks have also been considered. Our algorithm exhibits scalability, fully exploits the underlying parallelism, and is capable of scheduling an application, even if the available number of processors is less than the required number of processors. Based on extensive simulation studies, we observe that RT-DBA offers an enhanced success ratio as compared to other scheduling schemes when communication is a dominant factor.

Direct simulation of price particles for option pricing using Monte-Carlo

Absence of closed form solutions for many financial models has given rise to numerical and simulation techniques in the recent past. In the current study direct simulation, which is one of the popular approaches in aerospace engineering is used for studying applications in finance, especially option pricing.Monte Carlo (MC) is one of the popular simulation approaches for approximating the value of the quantity under question. However, the slow convergence rate, O(N−1/2) for N number of samples of the MC method has motivated research in Quasi Monte-Carlo (QMC) techniques. QMC methods use low discrepancy (LD) sequences that provide faster, more accurate results than MC methods. In this paper, we focus on the parallelization of the QMC method on a heterogeneous network of workstations (HNOWs) for option pricing. HNOWs are machines with different processing capabilities and have distinct execution time for the same task. It is, therefore, vital to allocate and schedule the tasks depending on the performance a...

Parallel Processing of Remotely Sensed Hyperspectral Images On Heterogeneous Networks of Workstations Using HeteroMPI

The development of efficient techniques for transforming massive volumes of remotely sensed hyperspectral data into scientific understanding is critical for space-based Earth science and planetary exploration. Although most available parallel processing strategies for information extraction and mining from hyperspectral imagery assume homogeneity in the underlying computing platform, heterogeneous networks of computers (HNOCs) have become a promising cost-effective solution, expected to play a major role in many on-going and planned remote sensing missions. In this paper, we develop a new morphological parallel algorithm for hyperspectral image classification using HeteroMPI, an extension of MPI for programming high-performance computations on HNOCs. The main idea of HeteroMPI is to automate and optimize the selection of a group of processes that executes a heterogeneous algorithm faster than any other possible group in a heterogeneous environment. In order to analyze the impact of many-to-one (gather) communication operations introduced by our proposed algorithm, we resort to a recently proposed collective communication model. The parallel algorithm is validated using two heterogeneous clusters at University College Dublin and a massively parallel Beowulf cluster at NASA's Goddard Space Flight Center.

An experimental comparison of parallel algorithms for hyperspectral analysis using heterogeneous and homogeneous networks of workstations

Imaging spectroscopy, also known as hyperspectral imaging, is a new technique that has gained tremendous popularity in many research areas, including satellite imaging and aerial reconnaissance. In particular, NASA is continuously gathering high-dimensional image data from the surface of the earth with hyperspectral sensors such as the Jet Propulsion Laboratory’s Airborne Visible-Infrared Imaging Spectrometer (AVIRIS) or the Hyperion hyperspectral imager aboard NASA’s Earth Observing-1 (EO-1) spacecraft. Despite the massive volume of scientific data commonly involved in hyperspectral imaging applications, very few parallel strategies for hyperspectral analysis are currently available, and most of them have been designed in the context of homogeneous computing platforms. However, heterogeneous networks of workstations represent a very promising cost-effective solution that is expected to play a major role in the design of high-performance computing platforms for many on-going and planned remote sensing missions. Our main goal in this paper is to understand parallel performance of hyperspectral imaging algorithms comprising the standard hyperspectral data processing chain (which includes pre-processing, selection of pure spectral components and linear spectral unmixing) in the context of fully heterogeneous computing platforms. For that purpose, we develop an exhaustive quantitative and comparative analysis of several available and new parallel hyperspectral imaging algorithms by comparing their efficiency on both a fully heterogeneous network of workstations and a massively parallel homogeneous cluster at NASA’s Goddard Space Flight Center in Maryland.

Parallel techniques for information extraction from hyperspectral imagery using heterogeneous networks of workstations

Recent advances in space and computer technologies are revolutionizing the way remotely sensed data is collected, managed and interpreted. In particular, NASA is continuously gathering very high-dimensional imagery data from the surface of the Earth with hyperspectral sensors such as the Jet Propulsion Laboratory's airborne visible-infrared imaging spectrometer (AVIRIS) or the Hyperion imager aboard Earth Observing-1 (EO-1) satellite platform. The development of efficient techniques for extracting scientific understanding from the massive amount of collected data is critical for space-based Earth science and planetary exploration. In particular, many hyperspectral imaging applications demand real time or near real-time performance. Examples include homeland security/defense, environmental modeling and assessment, wild-land fire tracking, biological threat detection, and monitoring of oil spills and other types of chemical contamination. Only a few parallel processing strategies for hyperspectral imagery are currently available, and most of them assume homogeneity in the underlying computing platform. In turn, heterogeneous networks of workstations (NOWs) have rapidly become a very promising computing solution which is expected to play a major role in the design of high-performance systems for many on-going and planned remote sensing missions. In order to address the need for cost-effective parallel solutions in this fast growing and emerging research area, this paper develops several highly innovative parallel algorithms for unsupervised information extraction and mining from hyperspectral image data sets, which have been specifically designed to be run in heterogeneous NOWs. The considered approaches fall into three highly representative categories: clustering, classification and spectral mixture analysis. Analytical and experimental results are presented in the context of realistic applications (based on hyperspectral data sets from the AVIRIS data repository) using several homogeneous and heterogeneous parallel computing facilities available at NASA's Goddard Space Flight Center and the University of Maryland.

On “Exploiting” Node-Heterogeneous Clusters Optimally

It is proved that “FIFO” worksharing protocols provide asymptotically optimal solutions to two problems related to sharing large collections of independent tasks in a heterogeneous network of workstations (HNOW) $\mathcal{N}$. In the $\mathsf{HNOW-Exploitation Problem}$, one seeks to accomplish as much work as possible on $\mathcal{N}$ during a prespecified fixed period of L time units. In the $\mathsf{HNOW-Rental Problem}$, one seeks to complete W units of work by “renting” $\mathcal{N}$ for as short a time as necessary. The worksharing protocols we study are crafted within an architectural model that characterizes $\mathcal {N}$ via parameters that measure $\mathcal{N}$ ’s workstations’ computational and communicational powers. All valid protocols are self-scheduling, in the sense that they determine completely both an amount of work to allocate to each of $\mathcal{N}$ ’s workstations and a schedule for all related interworkstation communications. The schedules provide either a value for W given L, or a value for L given W, hence solve both of the motivating problems. A protocol observes a FIFO regimen if it has $\mathcal{N}$ ’s workstations finish their assigned work, and return their results, in the same order in which they are supplied with their workloads. The proven optimality of FIFO protocols resides in the fact that they accomplish at least as much work as any other protocol during all sufficiently long worksharing episodes, and that they complete sufficiently large given collections of tasks at least as fast as any other protocol. Simulation experiments illustrate that the superiority of FIFO protocols is often observed during worksharing episodes of only a few minutes’ duration.

Broadcasting in Heterogeneous Networks

In this paper we study a well-known broadcasting heuristic for heterogeneous networks of workstations, called fastest node first. We show that this heuristic produces an optimal solution for minimizing the sum of all completion times and, in addition, produces a 1.5 approximation for the problem of minimizing the maximum completion time. We also develop a polynomial-time approximation scheme for this problem. We extend these results to show that the same bounds can be obtained for the multicast operation on such heterogeneous networks. In addition we show that the problem of minimizing the maximum completion time is NP-hard, which settles the complexity of this open problem.

Impact of platform heterogeneity on the design of parallel algorithms for morphological processing of high-dimensional image data

The main objective of this paper is to describe a realistic framework to understand parallel performance of high-dimensional image processing algorithms in the context of heterogeneous networks of workstations (NOWs). As a case study, this paper explores techniques for mapping hyperspectral image analysis techniques onto fully heterogeneous NOWs. Hyperspectral imaging is a new technique in remote sensing that has gained tremendous popularity in many research areas, including satellite imaging and aerial reconnaissance. The automation of techniques able to transform massive amounts of hyperspectral data into scientific understanding in valid response times is critical for space-based Earth science and planetary exploration. Using an evaluation strategy which is based on comparing the efficiency achieved by an heterogeneous algorithm on a fully heterogeneous NOW with that evidenced by its homogeneous version on a homogeneous NOW with the same aggregate performance as the heterogeneous one, we develop a detailed analysis of parallel algorithms that integrate the spatial and spectral information in the image data through mathematical morphology concepts. For comparative purposes, performance data for the tested algorithms on Thunderhead (a large-scale Beowulf cluster at NASA's Goddard Space Flight Center) are also provided. Our detailed investigation of the parallel properties of the proposed morphological algorithms provides several intriguing findings that may help image analysts in selection of parallel techniques and strategies for specific applications.

Efficient distributed SAT and SAT-based distributed Bounded Model Checking

SAT-based Bounded Model Checking (BMC), though a robust and scalable verification approach, still is computationally intensive, requiring large memory and time. Even with the recent development of improved SAT solvers, the memory limitation of a single server rather than time can become a bottleneck for doing deeper BMC search for large designs. Distributing computing requirements of BMC over a network of workstations can overcome the memory limitation of a single server, albeit at increased communication cost. In this paper, we present (a) a method for distributed SAT over a network of workstations using a Master/Client model where each Client workstation has an exclusive partition of the SAT problem and uses knowledge of partition topology to communicate with other Clients, (b) a method for distributing SAT-based BMC using the distributed SAT. For the sake of scalability, at no point in the BMC computation does a single workstation have all the information. We experimented on a network of heterogeneous workstations interconnected with a standard Ethernet LAN. To illustrate, on an industrial design with ∼13 K FFs and ∼0.5 million gates, the non-distributed BMC on a single workstation (with 4 GB memory) ran out of memory after reaching a depth of 120; on the other hand, our SAT-based distributed BMC over 5 similar workstations was able to go up to 323 steps with a communication overhead of only 30%.

Memory latency consideration for load sharing on heterogeneous network of workstations

Although most modern processors have hardware support for double precision or double-extended precision floating-point multiplication, this support is inadequate for many scientific computations. This paper presents the architecture of a quadruple precision floating-point multiplier that also supports two parallel double precision multiplications. Since hard-ware support for quadruple precision arithmetic is expensive, a new technique is presented that requires much less hard-ware than a fully parallel quadruple precision multiplier. With this architecture, quadruple precision multiplication has a latency of three cycles and two parallel double precision multiplications have latencies of only two cycles. The multiplier is pipelined so that two double precision multiplications can begin every cycle or a quadruple precision multiplication can begin every other cycle. The technique used for the dual-mode quadruple precision multiplier is also applied to the design of a dual-mode double precision floating-point multiplier that performs a double precision multiplication or two single precision multiplications in parallel. Synthesis results show that the dual-mode double precision multiplier requires 43% less area than a conventional double precision multiplier. The correctness of all the multipliers presented in this paper is tested and verified through extensive simulation.

Efficient Broadcast in Heterogeneous Networks of Workstations Using Two Sub-Networks

This paper presents efficient algorithms for broadcasting on heterogeneous networks of workstations (HNOW) by two partitioned sub-networks. In an HNOW, many multiple speed types of workstations can have different send and receive overheads. Previous research presented that routing by two sub-networks in a NOW can significantly increase system's performance according to Lin et al. (2001). Similarly, EBS and VBBS (Lin, 2003) designed by Chao and using the concept of fastest nodes first can be executed in O(nlog(n)) time, where n is the number of workstations. This paper proposes two schemes TWO-EBS and TWO-VBBS for broadcasting in an HNOW. These two schemes divide an HNOW into two sub-networks that are routed concurrently and combine EBS and VBBS to broadcast in an HNOW. Based on simulation results, TWO-VBBS outperforms all other schemes presented before.

Heuristic Contention-Free Broadcast in Heterogeneous Networks of Workstations with Multiple Send and Receive Speeds

This paper presents three efficient contention-free algorithms for broadcasting on heterogeneous networks of workstations (HNOW). In an HNOW, many multiple speed types of workstations that their send and receive overhead are distinct. Previous research has shown that finding an optimal routing scheme in an HNOW is not easy [R. Libesking-Hadas and J. Hartline, 2000, M. Banikazemi, et al., 1998], because properly arranging the scheduling tree in routing is difficult. Therefore, this investigation focuses mainly on enhancing the performance of an HNOW by properly arranging fastest nodes to the internal nodes of upper levels in the scheduling tree. The fastest node first is fundamental in designing an efficient algorithm. This paper presents three schemes named EBS, VBBS and VBBSWF. All of these three schemes can be implemented in O(nlog(n)) time, where n is the number of workstations. They are all contention-free when broadcasting in an HNOW. The post-order recursive doubling is used as a platform to design our algorithms. Based on the simulation result, the performance outperforms the broadcast with minimal steps [R. Libeskind-Hadas, et al., 1998] and the scheduling tree [A. Singhal, et al., 2001] generated by dynamic programming in an HNOW.

Efficient Multiple Multicast on Heterogeneous Network of Workstations

In recent years, network of workstations/PCs (so called NOW) are becoming appealing vehicles for cost-effective parallel computing. Due to the commodity nature of workstations and networking equipment, LAN environments are gradually becoming heterogeneous. The diverse sources of heterogeneity in NOW systems pose a challenge on the design of efficient communication algorithms for this class of systems. In this paper, we propose efficient algorithms for multiple multicast on heterogeneous NOW systems, focusing on heterogeneity in processing speeds of workstations/PCs. Multiple multicast is an important operation in many scientific and industrial applications. Multicast on heterogeneous systems has not been investigated until recently. Our work distinguishes itself from others in two aspects: (1) In contrast to the blocking communication model used in prior works, we model communication in a heterogeneous cluster more accurately by a non-blocking communication model, and design multicast algorithms that can fully take advantage of non-blocking communication. (2) While prior works focus on single multicast problem, we propose efficient algorithms for general, multiple multicast (in which single multicast is a special case) on heterogeneous NOW systems. To our knowledge, our work is the earliest effort that addresses multiple multicast for heterogeneous NOW systems. These algorithms are evaluated using a network simulator for heterogeneous NOW systems. Our experimental results on a system of up to 64 nodes show that some of the algorithms outperform others in many cases. The best algorithm achieves completion time that is within 2.5 times of the lower bound.

Optimization of combustor wall heat transfer and pollutant emissions for preliminary design using evolutionary techniques

This paper presents the concept and application of a design optimization toolbox, based upon evolutionary techniques, for the preliminary design of a gas turbine combustor. The toolbox has been designed to interface with existing analysis packages and to perform optimization in parallel over a heterogeneous network of workstations. The optimization capabilities of the toolbox are demonstrated for gas turbine combustor design by automatically attaining 22 performance targets in a combustor design while performing minimization of wall cooling flow and NO x, emissions.

Process recovery in heterogeneous systems

Heterogeneous computing environments, where computers may have different instruction set architectures, data representations, and operating systems, complicate checkpointing and recovery of processes. This paper describes an approach to recovery and an implementation, PREACHES, that provides portable checkpointing of single-process applications in heterogeneous systems using checkpoint propagation. The checkpoint propagation mechanism creates machine-dependent checkpoints for different architectures in the heterogeneous environment. A process is restored on a specific machine with the checkpoint that is appropriate for the architecture. An implementation of PREACHES has been evaluated on a heterogeneous network of workstations, including Sun, HP, and Pentium machines. The experimental results show that PREACHES achieves efficient checkpointing and rapid recovery.

PORTING REGULAR APPLICATIONS ON HETEROGENEOUS WORKSTATION NETWORKS: PERFORMANCE ANALYSIS AND MODELING

Heterogeneous networks of workstations and/or personal computers (NOW) are increasingly used as a powerful platform for the execution of parallel applications. When applications previously developed for traditional parallel machines (homogeneous and dedicated) are ported to NOWs, performance worsens owing in part to less efficient communications but more often to unbalancing. In this paper, we address the problem of the efficient porting to heterogeneous NOWs of data-parallel applications originally developed using the SPMD paradigm for homogeneous parallel systems with regular topology like ring. To achieve good performance, the computation time on the various machines composing the NOW must be as balanced as possible. This can be obtained in two ways: by using an heterogeneous data partition strategy with a single process per node, or by splitting homogeneously data among processes and assigning to each node a number of processes proportional to its computing power. The first method is however more difficult, since some modifications in the code are always needed, whereas the second approach requires very few changes. We carry out a simplified but reliable analysis, and propose a simple model able to simulate performance in the various situations. Two test cases, matrix multiplication and computation of long-range interactions, are considered, obtaining a good agreement between simulated and experimental results. Our analysis shows that an efficient porting of regular homogeneous data-parallel applications on heterogeneous NOWs is possible. Particularly, the approach based on multiple processes per node turns out to be a straightforward and effective way for achieving very satisfying performance in almost all situations, even dealing with highly heterogeneous systems.

Broadcast Scheduling Optimization for Heterogeneous Cluster Systems

Network of workstation (NOW) is a cost-effective alternative to massively parallel supercomputers. As commercially available off-the-shelf processors become cheaper and faster, it is now possible to build a PC or workstation cluster that provides high computing power within a limited budget. However, a cluster may consist of different types of processors, and this heterogeneity within a cluster complicates the design of efficient collective communication protocols. This paper shows that a simple heuristic called fastest-node-first (FNF) (1998, M. Banikazemi, V. Moorthy, and D. K. Panda, in “Proceedings of the International Parallel Processing Conference”) is very effective in reducing the broadcast time for heterogeneous cluster systems. Despite the fact that the FNF heuristic fails to give the optimal broadcast time for a general heterogeneous network of workstations, we prove that FNF always gives the optimal broadcast time in several special cases of clusters. Based on these special case results, we show that FNF is an approximation algorithm that guarantees a competitive ratio of 2. From these theoretical results we also derive techniques to speed up the branch-and-bound search for the optimal broadcast schedule in HNOW.