Mesh-connected Multicomputers Research Articles

An Efficient All Shapes Busy List Processor Allocation Algorithm for 3D Mesh Multicomputers

Contiguous processor allocation is useful for security and accounting reasons. This is due to the allocated jobs are separated from one another, where each sub-mesh of processors is allocated to an exclusive job request, and the allocated sub-mesh has the same size and shape of the requested job. The size and shape constraint leads to high processor fragmentation. Most recent contiguous allocation strategies suggested for 3D mesh-connected multiconputers try all possible orientations of an allocation request when allocation fails for the requested orientation, which reduces processor fragmentation and hence improves system performance. However, none of them considers all shapes of the request in the process of allocation. To generalize this restricted rotation, we propose, in this paper, a new contiguous allocation strategy for 3D mesh-connected multicomputers, referred to as All Shapes Busy List (ASBL for short), which takes into consideration all possible contiguous request shapes when attempting allocation for a job request. ASBL depends on the list of allocated sub-meshes, in the method suggested in (Bani-Mohammad et al., 2006), for selecting an allocated sub-mesh. The performance of the proposed ASBL allocation strategy has been evaluated considering several important scheduling strategies under a variety of system loads based on different job size distributions. The simulation results have shown that the ASBL allocation strategy improves system performance in terms of parameters such as the average turnaround time of jobs and system utilization under all scheduling strategies considered.

Corner-Boundary Processor Allocation for 3D Mesh-Connected Multicomputers

This research paper presents a new contiguous allocation strategy for 3D mesh-connected multicomputers. The proposed strategy maintains a list of maximal free sub-meshes and gives priority to allocating corner and boundary free sub-meshes. The goal of corner and boundary allocation is to decrease the number of leftover free sub-meshes and increase their sizes, which is expected to reduce processor fragmentation and improve overall system performance. The proposed strategy, which is referred to as Turning Corner-Boundary Free List (TCBFL) strategy, is compared, using extensive simulation experiments, to several existing allocation strategies for 3D meshes. These are the First-Fit (FF), Turning First-Fit Free List (TFFFL), and Turning Busy List (TBL) allocation strategies. The simulation results show that TCBFL produces average turnaround times and mean system utilization values that are superior to those of previous strategies.

The Effect of Real Workloads and Synthetic Workloads on the Performance of Job Scheduling for Non-Contiguous Allocation in 2D Mesh Multicomputers

The performance of non-contiguous allocation has been traditionally carried out by means of simulations based on synthetic workloads, and also it can be significantly affected by the job scheduling strategy used for determining the order in which jobs are selected for execution. To validate the performance of the non-contiguous allocation algorithms, there has been a need to evaluate the algorithm's performance based on a real workload trace. In this paper, the performance of the well-known Greedy Available Busy List (GABL) non-contiguous allocation strategy for 2D mesh-connected multicomputers is revisited considering several important job scheduling strategies based on a real workload trace, and the results are compared to those obtained from using a synthetic workload. The scheduling strategies used are the First-Come-First-Served (FCFS), Out-of-Order (OO), and Window-Based job scheduling strategies. These strategies have been selected because they are common and they have been used in related works (Ababneh & Bani-Mohammad, 2011). Extensive simulation results based on synthetic and real workload models indicate that the Window-Based job scheduling strategy can improve both overall system performance and fairness (i.e., maximum job waiting delays) by adopting a large job scheduling window. Moreover, the relative performance merits of the scheduling strategies when a real workload trace is used are in general compatible with those obtained when a synthetic workload is used.

A Distributed Parallel Pipelined Hardware-Level Barrier Synchronization Method for Mesh-Connected Multicomputers

In the article, the authors present a new distributed hardware-level method for barrier synchronization of parallel programs in a mesh-connected multicomputer, which is based on the usage of a virtual multislice pipeline coordinating environment timed by clock pulse waves.

Review of Methods of Distributed Barrier Synchronization of Parallel Processes in Matrix VLSI Systems

The problem of barrier synchronization between parallel processes in VLSI-based mesh-connected multicomputers is under consideration. This paper introduces and reviews description of hardware synchronization methods (HSM) in modern microprocessing systems.

The effect of communication on the performance of allocation request shape changes in 2D mesh-connected multicomputers

Contiguous allocation in multicomputers is useful for security and accounting reasons. In mesh-connected systems, each job is allocated an exclusive submesh of processors, and the allocated submesh has the same shape and size as that requested by the job. Because of this size and shape constraint, contiguous allocation typically suffers from high processor fragmentation. Processor fragmentation can be reduced by considering contiguous allocation to all possible request shapes, while maintaining contiguity. However, request shape modification can influence communication overhead. In this paper, we study the effect of request shape modification on communication overhead and performance in 2D mesh-connected multicomputers, where several common communication patterns are simulated in detail. The simulation results based on both synthetic and real workload models show that permitting all request shape changes can improve system performance substantially. For example, it improves the performance of first-fit allocation by up to about 60% in terms of average job turnaround time.

On the performance of non-contiguous allocation for common communication patterns in 2D mesh-connected multicomputers

The communication pattern used by applications can have a major influence on the performance of non-contiguous processor allocation in multicomputers. In this paper, the performance of well-known non-contiguous allocation strategies for 2D mesh multicomputers is re-visited considering several important communication patterns. These are the Near Neighbour, Ring, Divide and Conquer Binomial Tree (DQBT), Fast Fourier Transform (FFT), and Random communication patterns. The allocation strategies investigated are the Greedy Available Busy List (GABL), Multiple Buddy Strategy (MBS), Adaptive Non-contiguous Allocation (ANCA), and Paging(0). They are compared using detailed flit-level simulations. The results show that GABL is overall superior to the remaining non-contiguous allocation strategies. It produces superior average turnaround times and mean system utilization.

Performance evaluation of noncontiguous allocation algorithms for 2D mesh interconnection networks

Several noncontiguous allocation strategies have been proposed for 2D mesh-connected multicomputers. These allocation strategies differ in their ability to detect free submeshes and in the degree of contiguity that exists among the submeshes they allocate to the same job. The previous Adaptive Noncontiguous Allocation (ANCA) policy was evaluated based on a proposed formula that estimates the job execution time when the job is allocated noncontiguous submeshes. Using this formula, simulation results had shown that ANCA could outperform the preceding Multiple Buddy Strategy (MBS). However, the execution times of jobs under noncontiguous allocation depend on message sizes, the number of messages sent, message contention and distances messages traverse. In this paper, we evaluate ANCA for different communication patterns using an event-driven simulator operating at the flit level, which allows for a more realistic evaluation that takes into account the shape of allocation and contention among messages. Moreover, we compare the performance of ANCA with that of other noncontiguous allocation strategies, MBS, Greedy Available Busy List (GABL), and Paging(0). The results show that ANCA is inferior to the remaining strategies, and that GABL has the best performance results, expressed in terms of the average turnaround time and mean system utilization performance parameters.

A new window-based job scheduling scheme for 2D mesh multicomputers

Allocating submeshes to jobs in mesh-connected multicomputers in a FCFS fashion can lead to poor system performance (e.g., long job waiting delays) because the job at the head of the waiting queue can prevent the allocation of free submeshes to other waiting jobs with smaller submesh requirements. However, serving jobs aggressively out-of-order can lead to excessive waiting delays for jobs with large allocation requests. In this paper, we propose a scheduling scheme that uses a window of consecutive jobs from which it selects jobs for allocation and execution. This window starts with the current oldest waiting job and corresponds to the lookahead of the scheduler. The performance of the proposed window-based scheme has been compared to that of FCFS and other previous job scheduling schemes. Extensive simulation results based on synthetic workloads and real workload traces indicate that the new scheduling strategy exhibits good performance when the scheduling window size is large. In particular, it is substantially superior to FCFS in terms of system utilization, average job turnaround times, and maximum waiting delays under medium to heavy system loads. Also, it is superior to aggressive out-of-order scheduling in terms of maximum job waiting delays. Window-based job scheduling can improve both overall system performance and fairness (i.e., maximum job waiting delays) by adopting large lookahead job scheduling windows.

An adaptive job scheduling scheme for mesh-connected multicomputers

Allocating submeshes to jobs in mesh-connected multicomputers in an FCFS fashion leads to poor system performance because a large job at the head of the waiting queue can prevent the allocation of free submeshes to other smaller waiting jobs. However, serving jobs aggressively out-of-order can lead to excessive waiting delays for large jobs located at the head of the waiting queue. In this paper, we show that the ability of the job scheduling algorithm to bypass the head of the waiting queue should increase with the load, and we propose a scheduling scheme that can bypass the waiting queue head in a load-dependent adaptive fashion. Also, giving priority to large jobs because they are more difficult to accommodate is investigated. The performance of the proposed scheme has been compared to that of FCFS, aggressive out-of-order scheduling, and other previous job scheduling schemes. Extensive simulation results based on synthetic workloads and real workload traces indicate that our scheduling strategy is a good strategy when both average and maximum job waiting delays are considered. In particular, it is substantially superior to FCFS in terms of mean turnaround times, and to aggressive out-of-order scheduling in terms of maximum waiting delays.

On submesh allocation for 2D mesh multicomputers using the free-list approach: Global placement schemes

Two global placement schemes for contiguous processor allocation in two-dimensional mesh-connected multicomputers are proposed in this paper. The first scheme gives preference to allocating a free peripheral submesh that has the largest number of mesh-boundary processors. This peripheral placement has for goal producing large leftover free submeshes, which can improve system performance. Another characteristic of this scheme is that it reduces the search space by halting the search process when a large-enough multicomputer corner submesh is found. The second proposed scheme considers allocation in the corners of all large-enough free submeshes and allocates a submesh that has the maximum number of allocated neighbors and multicomputer peripheral nodes. Using extensive simulations, we evaluated the proposed schemes and compared them with previous promising schemes. The simulation results show that the peripheral placement scheme produces the best average turnaround times, and its measured allocation and de-allocation times are smaller than those of the previous schemes. The second proposed scheme ranks overall second in terms of turnaround times, however it is last in terms of efficiency.

Availability-based noncontiguous processor allocation policies for 2D mesh-connected multicomputers

Various contiguous and noncontiguous processor allocation policies have been proposed for mesh-connected multicomputers. Contiguous allocation suffers from high external processor fragmentation because it requires that the processors allocated to a parallel job be contiguous and have the same topology as the multicomputer. The goal of lifting the contiguity condition in noncontiguous allocation is reducing processor fragmentation. However, this can increase the communication overhead because the distances traversed by messages can be longer, and messages from different jobs can interfere with each other by competing for communication resources. The extra communication overhead depends on how the allocation request is partitioned and mapped to free processors. In this paper, we investigate a new class of noncontiguous allocation schemes for two-dimensional mesh-connected multicomputers. These schemes are different from previous ones in that request partitioning is based on the submeshes available for allocation. The available submeshes selected for allocation to a job are such that a high degree of contiguity among their processors is achieved. The proposed policies are compared to previous noncontiguous policies using detailed simulations, where several common communication patterns are considered. The results show that the proposed policies can reduce the communication overhead and improve performance substantially.

A new processor allocation strategy with a high degree of contiguity in mesh-connected multicomputers

Relaxing the contiguity condition in non-contiguous allocation can reduce processor fragmentation and increase processor utilization. However, communication overhead could increase due to the potential increase in message distances. The communication overhead depends on how the allocation request is partitioned and allocated to free sub-meshes. In this paper, a new non-contiguous processor allocation strategy, referred to as Greedy-Available-Busy-List (GABL for short), is suggested for the mesh network, and is compared against the existing non-contiguous and contiguous allocation strategies. To demonstrate the performance gains achieved by our proposed strategy, we have conducted simulation runs under the assumption of wormhole routing technique. The results have revealed that the new strategy can reduce communication overhead and considerably improve performance in terms of the job turnaround time, system utilization, and jobs finish time.

An efficient free-list submesh allocation scheme for two-dimensional mesh-connected multicomputers

This paper presents an efficient free-list submesh allocation scheme for two-dimensional mesh-connected systems. The scheme maintains an unordered list of free submeshes. For allocation, it selects the first free submesh that has at least the same size as the request, and when the selected submesh is larger than the request the part actually allocated is one that has the largest number of busy neighbors and mesh boundary processors. Whereas the best previously proposed schemes have time complexities that are either quadratic or cubic in the number of free or allocated submeshes, the time complexity of the proposed scheme is linear in the number of free submeshes. To evaluate the effectiveness of the scheme, its system performance in terms of parameters such as the average job turnaround time was compared to that of promising previously proposed schemes. Simulation results show that the scheme performs at least as well as these schemes, yet it has the lowest time complexity.

On Constructing the Minimum Orthogonal Convex Polygon for the Fault-Tolerant Routing in 2-D Faulty Meshes

The rectangular faulty block model is the most commonly used fault model for designing fault-tolerant, and deadlock-free routing algorithms in mesh-connected multicomputers. The convexity of a rectangle facilitates simple, efficient ways to route messages around fault regions using relatively few or no virtual channels to avoid deadlock. However, such a faulty block may include many nonfaulty nodes which are disabled, i.e., they are not involved in the routing process. Therefore, it is important to define a fault region that is convex, and at the same time, to include a minimum number of nonfaulty nodes. In this paper, we propose an optimal solution that can quickly construct a set of minimum faulty polygons, called orthogonal convex polygons, from a given set of faulty blocks in a 2-D mesh (or 2-D torus). The formation of orthogonal convex polygons is implemented using either a centralized, or distributed solution. Both solutions are based on the formation of faulty components, each of which consists of adjacent faulty nodes only, followed by the addition of a minimum number of nonfaulty nodes to make each component a convex polygon. Extensive simulation has been done to determine the number of nonfaulty nodes included in the polygon, and the result obtained is compared with the best existing known result. Results show that the proposed approach can not only find a set of minimum faulty polygons, but also does so quickly in terms of the number of rounds in the distributed solution.

A quad-tree based algorithm for processor allocation in 2D mesh-connected multicomputers

Mesh-connected multicomputers are highly scalable and efficient class of computer systems which are used for the execution of supercomputing applications. In such systems, an application is divided into a set of smaller independent tasks each of which requires separate subsystems to execute it. Hence, there is a need of partitioning the system dynamically and allocating these partitions to the incoming tasks. At the same time, partitions must be deallocated whenever they become available after the task completion. In this paper, a quad-tree based processor-allocation algorithm called MQTR is proposed. It is a modification of an existing algorithm called QTR. MQTR reduces the system fragmentation by maintaining the largest possible free subsystem after allocating a subsystem to the incoming task. Thus a system can accommodate more number of tasks at a time, thereby improving the system performance. A generalized simulation framework is also developed on which both QTR and MQTR algorithms are implemented. Simulation results show that MQTR algorithm improves the system performance in terms of system utilization and system fragmentation as compared to QTR.

Comments on "A fast and efficient processor allocation scheme for mesh-connected multicomputers"

In a recent paper by B.S. Yoo and C.R. Das (2002), the so-called stack-based allocation (SBA) algorithm is claimed to be, at worst, O(B/sup 2/) expensive. In this paper, we present an exception for which the time complexity of SBA is at least O(B/sup 3/). Furthermore, we point out the discrepancy in the complexity analysis.

A Distributed Formation of Smallest Faulty Orthogonal Convex Polygons in 2-D Meshes

The rectangular faulty block model is the most commonly used fault model for designing fault-tolerant and deadlock-free routing algorithms in mesh-connected multicomputers. The convexity of a rectangle facilitates simple and efficient ways to route messages around fault regions using relatively few virtual channels to avoid deadlock. However, such a faulty block may include many nonfaulty nodes which are disabled, i.e., they are not involved in the routing process. Therefore, it is important to define a fault region that is convex, and at the same time, to include a minimum number of nonfaulty nodes. In this paper, we propose a simple and efficient distributed algorithm that can quickly construct a set of special convex polygons, called orthogonal convex polygons, from a given set of rectangular faulty blocks in a 2-D mesh (or 2-D torus). The formation of orthogonal convex polygons is done through a labeling scheme based on iterative message exchanges among neighboring nodes. For a given faulty block, after some nonfaulty nodes have been removed, the block is split into a set of orthogonal convex polygons, each of which is the smallest orthogonal convex polygon that contains all the faults it covers. Moreover, we show that the number of nonfaulty nodes covered in these polygons is no more than the one in the smallest orthogonal convex polygon that includes all the faulty nodes in the original faulty block. Finally, we discuss one open problem and present extensions to multidimensional meshes.

A fast and efficient processor allocation scheme for mesh-connected multicomputers

Efficient processor allocation is crucial for obtaining high performance in space-shared parallel computers. A good processor allocation algorithm should find available processors for incoming jobs, if they exist, with minimum overhead. In this paper, we propose such a fast and efficient processor allocation scheme for mesh-connected multicomputers. By using simple coordinate calculation and spatial subtraction, the proposed scheme reduces the search space drastically and, hence, can locate a free submesh very quickly. The algorithm is implemented efficiently using a stack and therefore is called the stack-based allocation (SBA) algorithm. Extensive simulation reveals that our scheme incurs much less allocation overhead than all of the existing allocation algorithms, while delivering competitive performance.

Efficient processor management schemes for mesh-connected multicomputers

This paper investigates various processor management techniques for improving the performance of mesh-connected multicomputers. Unlike almost all prior work where the focus was on improving the submesh recognition ability of the processor allocation algorithms, this research examines other alternatives to improve system performance beyond what is achievable with usually assumed first come first served (FCFS) scheduling and any allocation. First, we use the smallest job first (SJF) policy to improve the spatial parallelism in a mesh. Next, we introduce a generic processor management scheme called multitasking and multiprogramming (M 2). Then, an M 2 policy for mesh-connected multicomputers called virtual mesh (VM) is proposed and analyzed. The proposed VM scheme allows multiprogramming of jobs on several VMs. Finally, a novel approach called limit allocation is used for job allocation. With this scheme, a job (submesh) size is reduced if the job cannot be allocated. The objective here is to reduce the job waiting time and hence improve the overall performance. While all of the three approaches are viable alternatives to reduce the average job response time under various workloads, the VM and the limit allocation techniques are especially attractive for providing some additional features. The VM scheme brings in the concept of time-sharing execution for better efficiency and limit allocation shows how job size restriction can be beneficial for performance and fault-tolerance in a mesh topology. Moreover, the limit allocation scheme using even the simplest allocation policy can outperform any other approach.