Heterogeneous Computer Networks Research Articles

An efficient concurrent implementation of a neural network algorithm

The focus of this study is how we can efficiently implement the neural network backpropagation algorithm on a network of computers (NOC) for concurrent execution. We assume a distributed system with heterogeneous computers and that the neural network is replicated on each computer. We propose an architecture model with efficient pattern allocation that takes into account the speed of processors and overlaps the communication with computation. The training pattern set is distributed among the heterogeneous processors with the mapping being fixed during the learning process. We provide a heuristic pattern allocation algorithm minimizing the execution time of backpropagation learning. The computations are overlapped with communications. Under the condition that each processor has to perform a task directly proportional to its speed, this allocation algorithm has polynomial-time complexity. We have implemented our model on a dedicated network of heterogeneous computers using Sejnowski’s NetTalk benchmark for testing. Copyright c © 2005 John Wiley & Sons, Ltd.

HeteroMPI: Towards a message-passing library for heterogeneous networks of computers

The paper presents Heterogeneous MPI (HeteroMPI), an extension of MPI for programming high-performance computations on heterogeneous networks of computers. It allows the application programmer to describe the performance model of the implemented algorithm in a generic form. This model allows the specification of all the main features of the underlying parallel algorithm, which have an impact on its execution performance. These features include the total number of parallel processes, the total volume of computations to be performed by each process, the total volume of data to be transferred between each pair of the processes, and how exactly the processes interact during the execution of the algorithm. Given a description of the performance model, HeteroMPI tries to create a group of processes that executes the algorithm faster than any other group. The principal extensions to MPI are presented. We demonstrate the features of the library by performing experiments with parallel simulation of the interaction of electric and magnetic fields and parallel matrix multiplication.

The System PARCS-JAVA for Parallel Computations on Computer Networks

The system PARCS-JAVA provides software tools for solution of problems on computer networks. It can be installed on heterogeneous computer networks and allows users of small computers to use parallel data processing.

An adaptive scheduling framework for heterogeneous computer networks

This paper presents an adaptive scheduling framework for large-scale computer networks facing significant environmental dynamics such as shifts in job mix and available resources. The presented framework detects changes in the job stream, and other events, and evolves scheduling policies at two distinct levels—operational and tactical—in response to the changes, making adaptations such as reassigning resources and adjusting sequencing heuristics. The paper also presents a simulation of a computer network facing dynamic job arrival rates, which demonstrates that even a simple implementation of the framework can lead to major gains in performance.

Parallel Processing of Remote Sensing Images in a High-Performance Heterogeneous Computer Network

A distributed architecture of the software package for the processing of remote sensing images is presented. Since the analysis and processing of remote sensing images require the mobilization of all presently available hardware and software, a new technology is proposed for solving this problem on the basis of high-performance parallel computer systems. The preliminary results of application of this technology are presented.

Adaptive parallel computing on heterogeneous networks with mpC

The paper presents a new advanced version of the mpC parallel language. The language was designed specially for programming high-performance parallel computations on heterogeneous networks of computers. The advanced version allows the programmer to define at runtime all the main features of the underlying parallel algorithm, which have an impact on the application execution performance, namely, the total number of participating parallel processes, the total volume of computations to be performed by each of the processes, the total volume of data to be transferred between each pair of the processes, and how exactly the processes interact during the execution of the algorithm. Such an abstraction of parallel algorithm is called a network type in mpC. Given a network type, the programmer can define a network object of this type and describe in details all the computations and communications to be performed on the network object. The mpC programming system uses the information extracted from the network-type definition together with information about the actual performance of the executing network to map the processes of the parallel program to this network in such a way that leads to its better execution time. In addition, the programmer can use a special operator, timeof, which predicts the total time of the algorithm execution on the underlying hardware without its real execution. That feature allows the programmer to write such a parallel program that can follow different parallel algorithms to solve the same problem, making choice at runtime depending on the particular executing network and its actual performance. The paper describes both the language model of parallel algorithm and the model of executing parallel environment used by the mpC programming system. It also discusses principles of the implementation of mapping mpC network objects to the computing network.

Data collection and restoration for heterogeneous process migration

AbstractThis study presents a practical solution for data collection and restoration to migrate a process written in high‐level stack‐based languages such as C and Fortran over a network of heterogeneous computers. We first introduce a logical data model, namely the Memory Space Representation (MSR) model, to recognize complex data structures in process address space. Then, novel methods are developed to incorporate the MSR model into a process, and to collect and restore data efficiently. We have implemented prototype software and performed experiments on different programs. Experimental and analytical results show that: (1) a user‐level process can be migrated across different computing platforms; (2) semantic information of data structures in the process's memory space can be correctly collected and restored; (3) costs of data collection and restoration depend on the complexity of the MSR graph in the memory space and the amount of data involved; and (4) the implantation of the MSR model into the process is not a decisive factor of incurring execution overheads. With appropriate program analysis, we can practically achieve low overhead. Copyright © 2002 John Wiley & Sons, Ltd.

Matching and scheduling algorithms for minimizing execution time and failure probability of applications in heterogeneous computing

In a heterogeneous distributed computing system, machine and network failures are inevitable and can have an adverse effect on applications executing on the system. To reduce the effect of failures on an application executing on a failure-prone system, matching and scheduling algorithms which minimize not only the execution time but also the probability of failure of the application must be devised. However, because of the conflicting requirements, it is not possible to minimize both of the objectives at the same time. Thus, the goal of this paper is to develop matching and scheduling algorithms which account for both the execution time and the reliability of the application. This goal is achieved by modifying an existing matching and scheduling algorithm. The reliability of resources is taken into account using an incremental cost function proposed in this paper and the new algorithm is referred to as the reliable dynamic level scheduling algorithm. The incremental cost function can be defined based on one of the three cost functions developed here. These cost functions are unique in the sense that they are not restricted to tree-based networks and a specific matching and scheduling algorithm. The simulation results confirm that the proposed incremental cost function can be incorporated into matching and scheduling algorithms to produce schedules where the effect of failures of machines and network resources on the execution of the application is reduced and the execution time of the application is minimized as well.

PORTING REGULAR APPLICATIONS ON HETEROGENEOUS WORKSTATION NETWORKS: PERFORMANCE ANALYSIS AND MODELING

Heterogeneous networks of workstations and/or personal computers (NOW) are increasingly used as a powerful platform for the execution of parallel applications. When applications previously developed for traditional parallel machines (homogeneous and dedicated) are ported to NOWs, performance worsens owing in part to less efficient communications but more often to unbalancing. In this paper, we address the problem of the efficient porting to heterogeneous NOWs of data-parallel applications originally developed using the SPMD paradigm for homogeneous parallel systems with regular topology like ring. To achieve good performance, the computation time on the various machines composing the NOW must be as balanced as possible. This can be obtained in two ways: by using an heterogeneous data partition strategy with a single process per node, or by splitting homogeneously data among processes and assigning to each node a number of processes proportional to its computing power. The first method is however more difficult, since some modifications in the code are always needed, whereas the second approach requires very few changes. We carry out a simplified but reliable analysis, and propose a simple model able to simulate performance in the various situations. Two test cases, matrix multiplication and computation of long-range interactions, are considered, obtaining a good agreement between simulated and experimental results. Our analysis shows that an efficient porting of regular homogeneous data-parallel applications on heterogeneous NOWs is possible. Particularly, the approach based on multiple processes per node turns out to be a straightforward and effective way for achieving very satisfying performance in almost all situations, even dealing with highly heterogeneous systems.

Discrete model for fragmentation with random stopping

In this work, we present the numerical results obtained from large scale parallel and distributed simulations of a model for two- and three-dimensional discrete fragmentation. Its main features are: (1) uniform and independent random distribution of the forces that generate the fracture; (2) deterministic criteria for the fracture process at each step of the fragmentation, based on these forces and a random stopping criteria. By large scale parallel and distributed simulations, implemented over a heterogeneous network of high performance computers, different behaviors were obtained for the fragment size distribution, which includes power law behavior with positive exponents for a wide range of the main parameter of the model: the stopping probability. Also, by a sensitive analysis we prove that the value of the main parameter of the model does not affect these results. The power law distribution is a non-trivial result which reproduces empirical results of some highly energetic fracture processes.

Online algorithms supporting emergence in distributed CAD systems

The rule-based approach from traditional AI and the conventional constraint satisfaction algorithms are not adequate to cope with the unpredictable events and interactive computations in distributed CAD environments. This paper claims that the problems faced by distributed CAD systems require solutions based on the concepts of emergence, reactivity, and online algorithms. The present paper presents Extended Constraints Graphs (ECGs) as online algorithms supporting emergence in a network of reactive agents. ECGs represent an effective solution for the nonlinear constraint problem in cyclic graphs, which are distributed over a heterogeneous computer network. The relationships between agents in an ECG are generic, distributed, recursive and nonlinear — a problem not solved in the literature. Also this paper presents a CORBA model integrated with a Geometry Bus to support distributed virtual prototype whose variables can be defined in terms of ECGs.

Miro: Middleware for Autonomous Mobile Robots

Implementing software for autonomous mobile robots is a non-trivial task, because such robots incorporate several sensor systems and actuators that must be controlled simultaneously by a heterogeneous ensemble of networked computers and microcontrollers. Additionally, the use of modern software engineering technologies like object-oriented and distributed programming and client/server architectures is essential in order to maintain program code effectively. In this paper, we present Miro, a new CORBA-based robot programming framework which allows a rapid development of reliable and safe software on heterogeneous computer networks and supports the mixed use of several programming languages.

Heterogeneous Distribution of Computations Solving Linear Algebra Problems on Networks of Heterogeneous Computers

This paper presents and analyzes two different strategies of heterogeneous distribution of computations solving dense linear algebra problems on heterogeneous networks of computers. The first strategy is based on heterogeneous distribution of processes over processors and homogeneous block cyclic distribution of data over the processes. The second is based on homogeneous distribution of processes over processors and heterogeneous block cyclic distribution of data over the processes. Both strategies were implemented in the mpC language—a dedicated parallel extension of ANSI C for efficient and portable programming of heterogeneous networks of computers. The first strategy was implemented using calls to ScaLAPACK; the second strategy was implemented with calls to LAPACK and BLAS. Cholesky factorization on a heterogeneous network of workstations is used to demonstrate that the heterogeneous distributions have an advantage over the traditional homogeneous distribution.

Algorithms, Models and Tools for High Performance Computing on Heterogeneous Networks

Grid computing is a very important research area with a great practical impact. By nature, the topics of Grid computing cover many types of systems. Speaking about metacomputing implies to have built a metacomputer, which can be defined as a collection of elements providing a variety of functionalities (networking, computation, graphics) that can be used to run multiple tasks with several computational requirements. The key point is that, to the individual user, it should look and act like a single computer. Of course, all these capabilities can be a achieved by a set of high-performance machines interconnected by a fast dedicated local-area network. But a metacomputer may also be made of geographically distributed computational and information resources spread over the Internet, since emerging high-performance networks promise to enable a wide range of emerging application concepts such as remote computing, distributed supercomputing, tele-immersion, smart instruments, and data mining. On such a metacomputing environment, all well known standard services such as authentication, resource location, resource allocation, configuration, communication, file access, fault detection, and executable management… are becoming harder to deploy, to maintain or simply to access. Thus, applications in this kind of environment may achieve performance by exploiting the adequacy between the problem to solve and the different platforms or paradigms available. The challenge for the computer science community is to provide a solid, integrated middleware foundation on which to build wide-area applications. A metacomputing system must support the illusion of a single machine by transparently scheduling application components on processors, managing data migration, caching, transfer and masking of data-format differences between systems; detecting and managing faults and ensuring that users' data and physical resources are protected. Unlike dedicated parallel computer systems, local networks are inherently heterogeneous. They consist of diverse computers of different performances interconnected via mixed network equipment providing communication links of different speeds and bandwidths. Traditional parallel algorithms and tools are aimed at homogeneous multiprocessors and cannot be efficiently used for parallel computing on heterogeneous clusters. New ideas, dedicated algorithms and tools are needed to efficiently use this most common but most irregular parallel architecture. The first and second International Workshops on Metacomputing Systems and Applications (MSA'2000 and MSA'2001) as well as the first International Workshop on Algorithms and Tools for Parallel Computing on Heterogeneous Clusters (HeteroPar'2001) were recently held in conjunction with ICPP '2000 and PDPTA'2001 respectively. Building on the success of these workshops, this special issue of Parallel and Distributed Computing Practice features papers that explore algorithms, programming languages, systems, tools and theoretical models aimed at high performance computing on heterogeneous networks of computers. Guest Editors Frederic Desprez, LIP ENS Lyon, France Eric Fleury, CITI INSA-Lyon, France Alexey Kalinov, Institute for System Programming, Moscow, Russia Alexey Lastovetsky, Dept of Computer Science, University College Dublin, Ireland

Decentralized Network Management Using Distributed Artificial Intelligence

Centralized approaches to Network Management have demonstrated a clear inadequacy for efficient management of large and heterogeneous computer networks. Considerable research is being carried out o...

Applications of CORBA in the ATLAS prototype DAQ

This paper presents the experience of using the Common Object Request Broker Architecture (CORBA) in the ATLAS prototype DAQ project. Many communication links in the DAQ system have been designed and implemented using the CORBA standard. A public domain package, called Inter-Language Unification (ILU) has been used to implement CORBA based communications between DAQ components in a local area network (LAN) of heterogeneous computers. The CORBA Naming Service provides the principal mechanism through which most clients of an ORE-based system locate objects that they intend to use. In our project, conventions are employed that meaningfully partition the name space of the Naming Service according to divisions in the DAQ system itself. The Inter Process Communication (IPC) package, implemented in C++ on the top of CORBA/ILU, incorporates this facility and hides the details of the naming schema is described. The development procedure and environment for remote database access using IPC is described. Various end-user interfaces have been implemented using the Java language that communicate with C++ servers via CORBA/ILU. To support such interfaces, a second implementation of IPC in Java has been developed. The design and implementation of such connections are described. An alternative CORBA implementation, ORBacus, has been evaluated and compared with ILU.

A parallel language and its programming system for heterogeneous networks

The paper presents a new parallel language, mpC, designed specially for programming high-performance computations on heterogeneous networks of computers, as well as its supportive programming environment. The main idea underlying mpC is that an mpC application explicitly defines an abstract network and distributes data, computations and communications over the network. The mpC programming environment uses at run time this information as well as information about any real executing network in order to map the application to the real network in such a way that ensures efficient execution of the application on this real network. Experience of using mpC for solving both regular and irregular real-life problems on networks of heterogeneous computers is also presented.

SIMA: A Java Tool for Constructing Image Processing Applications on a Heterogeneous Network

In this work we present SIMA (System for distributed IMage Applications), a tool for contructing distributed applications for the analysis of biomedical images. Such applications involve a set of operations that go from improvement of images to production of results that allow users to diagnose. Using SIMA it is possible to exploit, in an integral way, the available resources in a heterogeneous computer network for image processing . This tool has been developed in JAVA and no previous knowledge in distributed programming is required for its proper use it. A graphic interface allows the definition of applications for image processing. A distributed application is constructed by composition of operations, starting from a set of default basic operations (segmentation, filters, calculation of areas).

Distributed short-circuit analysis in heterogeneous computer networks

Probabilistic criteria allows for less conservative substation designs, but requires a more careful observation of short-circuit levels. In this paper we are particularly interested on the possibility of using this technique for monitoring short-circuit levels as a normal function of a modern control centre. The paper demonstrates the feasibility of determining probability density curves by running a number of simulations in a heterogeneous computing network used as a parallel virtual machine. The techniques described in the paper can be applied to a range of loosely coupled problems such as contingency analysis, multi-case transient stability studies, reliability evaluation, and probabilistic short-circuit analysis. This paper specifically deals with the distributed implementation of short-circuit analysis. Results include performance indices such as efficiencies, speedups, total computing times, and communication times. Tests have been performed with three real-life systems.

Synchronisation and Load Balancing for Parallel Hierarchical Radiosity of Complex Scenes on a Heterogeneous Computer Network

In this paper we propose a SPMD parallel hierarchical radiosity algorithm relying on a novel partitioning method which may apply to any kind of architectural scene. This algorithm is based on MPI (Message Passing Interface), a communication library which allows the use of either a heterogeneous set of concurrent computers or a parallel computer or both. The database is stored on a common directory and accessed by all the processors (through NFS in case of a network of computers). As the objective is to handle complex scenes such as building interiors, to cope with the problem of memory size, only a subset of the database resides in memory of each processor. This subset is determined with the help of a partitioning into 3D cells, clustering and visibility calculations. A graph expressing visibility between the resulting clusters is determined, partitioned (with a new method based on classification of K‐means type) and distributed amongst all the processors. Each processor is responsible for gathering energy (using the Gauss‐Seidel method) only for its subset of clusters. In order to reduce the disk transfers due to downloading these subsets of clusters, we use an ordering strategy based on the traveling salesman algorithm. Dynamic load balancing relies on a task stealing approach while termination is detected by configuring the processors into a ring and moving a token around this ring. The parallel iterative resolution is of group iterative type. Its mathematical convergence is proven in the appendix.