Advanced Computer Architectures Research Articles

Big Data Analytics Performance for Large Out-of-Core Matrix Solvers on Advanced Hybrid Architectures

Abstract This paper examines the performance of advanced computer architectures for large Out-Of-Core matrices to assess the optimal Big Data system configurations., The performance evaluation is based on a large dense Lower-Upper Matrix Decomposition (LUD) employing a highly tuned, I/O managed, slab based LUD software package developed by the Lockheed Martin Corporation. We present extensive benchmark studies conducted with this package on UMBC's Bluegrit and Bluewave clusters, and NASA-GFSC's Discover cluster systems. Our results show the speedup for a single node achieved by Phi Co-Processors relative to the host CPU SandyBridge processor is about a 1.5X improvement, which is an even smaller relative performance gain compared with the studies by F. Masci where he obtains a 2-2.5x performance. Surprisingly, the Westmere with the Tesla GPU scales comparably with the Sandy Bridge and the Phi Co-Processor up to 12 processes and then fails to continue to scale. The performances across 20 CPU nodes of SandyBridge obtains a uniform speedup of 0.5X over Westmere for problem sizes of 10K, 20K and 40K unknowns. With an Infiniband DDR, the performance of Nehalem processors is comparable to Westmere without the interconnect.

Ab initio nonadiabatic dynamics of multichromophore complexes: a scalable graphical-processing-unit-accelerated exciton framework.

Conspectus Although advances in computer hardware and algorithms tuned for novel computer architectures are leading to significant increases in the size and time scale for molecular simulations, it remains true that new methods and algorithms will be needed to address some of the problems in complex chemical systems, such as electrochemistry, excitation energy transport, proton transport, and condensed phase reactivity. Ideally, these new methods would exploit the strengths of emerging architectures. Fragment based approaches for electronic structure theory decompose the problem of solving the electronic Schrodinger equation into a series of much smaller problems. Because each of these smaller problems is largely independent, this strategy is particularly well-suited to parallel architectures. It appears that the most significant advances in computer architectures will be toward increased parallelism, and therefore fragment-based approaches are an ideal match to these trends. When the computational effort involved scales with the third (or higher) power of the molecular size, there is a large benefit to fragment-based approaches even on serial architectures. This is the case for many of the well-known methods for solving the electronic structure theory problem, especially when wave function-based approaches including electron correlation are considered. A major issue in fragment-based approaches is determining or improving their accuracy. Since the Achilles' heel of any such method lies in the approximations used to stitch the smaller problems back together (i.e., in the treatment of the cross-fragment interactions), it can often be important to ensure that the size of the smaller problems is "large enough." Thus, there are two frontiers that need to be extended in order to enable molecular simulations for large systems and long times: the strongly coupled problem of medium sized molecules (100-500 atoms) and the more weakly coupled problem of decomposing ("fragmenting") a molecular system and then stitching it back together. In this Account, we address both of these problems, the first by using graphical processing units (GPUs) and electronic structure algorithms tuned for these architectures and the second by using an exciton model as a framework in which to stitch together the solutions of the smaller problems. The multitiered parallel framework outlined here is aimed at nonadiabatic dynamics simulations on large supramolecular multichromophoric complexes in full atomistic detail. In this framework, the lowest tier of parallelism involves GPU-accelerated electronic structure theory calculations, for which we summarize recent progress in parallelizing the computation and use of electron repulsion integrals (ERIs), which are the major computational bottleneck in both density functional theory (DFT) and time-dependent density functional theory (TDDFT). The topmost tier of parallelism relies on a distributed memory framework, in which we build an exciton model that couples chromophoric units. Combining these multiple levels of parallelism allows access to ground and excited state dynamics for large multichromophoric assemblies. The parallel excitonic framework is in good agreement with much more computationally demanding TDDFT calculations of the full assembly.

Advances in tracking and trigger concepts

Increasing beam intensities and input data rates require to rethink the traditional approaches in trigger concepts. At the same time the advanced many-core computer architectures providing new dimensions in programming require to rework the standard methods or to develop new methods of track reconstruction in order to efficiently use parallelism of the computer hardware. As a results a new tendency appears to replace the standard (usually implemented in FPGA) hardware triggers by clusters of computers running software reconstruction and selection algorithms. In addition that makes possible unification of the offline and on-line data processing and analysis in one software package running on a heterogeneous computer farm.

Unsupervised image classification over supercomputers Kraken, Keeneland and Beacon

The iterative self-organizing data analysis technique algorithm (ISODATA) was implemented over supercomputers Kraken, Keeneland and Beacon to explore scalable and high-performance solutions for image processing and analytics using emerging advanced computer architectures. When 10 classes are extracted from one 18-GB image tile, the calculation can be reduced from several hours to no more than 90 seconds when 100 CPU, GPU or MIC processors are utilized. High-performance scalability tests were further implemented over Kraken using 10,800 processors to extract various number of classes from 12 image tiles totalling 216 gigabytes. As the first geospatial computations over GPU clusters (Keeneland) and MIC clusters (Beacon), the success of this research illustrates a solid foundation for exploring the potential of scalable and high-performance geospatial computation for the next generation cyber-enabled image analytics.

Explaining Parallel Architecture Design

Natalie Enright Jerger reviews a book (Parallel Computer Organization and Design, by Michel Dubois, Murali Annavaram, and Per Senström) that explains advanced computer architecture topics from the perspective of parallel computer organization.

Variant-Frequency Semantics for Green Futures

This paper describes an operational semantics for futures, with the primary target on energy efficiency. The work in progress is built around an insight that different threads can coordinate by running at different "paces," so that the time for synchronization and the resulting wasteful energy consumption can be reduced. We exploit several inherent characteristics of futures to determine how the paces of involving threads can be coordinated. The semantics is inspired by recent advances in computer architectures, where the frequencies of CPU cores can be adjusted dynamically. The work is a first-step toward a direction where variant frequencies are directly modeled as an essential semantic feature in concurrent programming languages.

Facilitating analysis of Monte Carlo dense matrix inversion algorithm scaling behaviour through simulation

With the latest developments in the area of advanced computer architectures, we are already seeing large-scale machines at petascale level and are faced with the exascale computing challenge. All these require scalability at system, algorithmic and mathematical model levels. In particular, efficient scalable algorithms are required to bridge the performance gap. Being able to predict application demeanour, performance and scalability of currently used software on new supercomputers of different architectures, varying sizes, and utilising distinct ways of intercommunication, can be of great benefit for researchers as well as application developers. This paper is concerned with scaling characteristics of Monte Carlo based algorithms for matrix inversion. The algorithmic behaviour on both, a shared memory and a large-scale cluster system will be predicted with the help of an extreme-scale high-performance computing (HPC) simulator.

Playable Serious Games for Studying and Programming Computational Stem and Informatics Applications of Distributed and Parallel Computer Architectures

Carefully engineered playable games can serve as vehicles for students and practitioners to learn and explore the programming of advanced computer architectures to execute applications, such as high performance computing (HPC) and complex, inter-networked, distributed systems. The article presents families of playable games that are grounded in the concurrent, parallel, and distributed manipulations and processing of sequences. Some examples of such manipulations are games that involve pattern matching, recognition, alignment, transformation, and evolution of sequences.

Johns Hopkins on the chip: microsystems and cognitive machines for sustainable, affordable, personalised medicine and healthcare

Semiconductor technology is contributing to the advancement of biotechnology, medicine and healthcare delivery in ways that it was never envisioned - from chip micro-arrays, to scientific grade CMOS imagers and ion sensing arrays to implantable prosthesis. This exponential growth of sensory microsystems has led to an exponential growth of data. Cognitive machines, i.e. advanced computer architectures and algorithms, are carefully co-designed to extract knowledge from such health data making rational decisions and recommendations for therapies. Nano, micro and macro robotics driven by sophisticated algorithms interface to the human body at different levels and scales, from nano-scale molecules to micron-scale cells to networks and all the way to the scale of organisms. The present era is one where semiconductor technology and the -chip- is the foundation of sustainable and affordable personalised medicine and healthcare delivery.

Special Section: 2010 Copper Mountain Conference

The biennial Copper Mountain Conference on Iterative Methods was held April 4–9, 2010. This meeting included more than 140 presentations covering many scientific computing areas, such as uncertainty quantification, optimization, Markov chains, saddle-point systems, inverse problems, direct factorizations, Krylov methods, algebraic multigrid, software frameworks, and advanced computer architectures. A partial list of the represented application domains included acoustic scattering, material science, circuit simulation, neutron transport, image processing, magnetohydrodynamics, and porous media flow. Howard Elman (University of Maryland) and Ray Tuminaro (Sandia National Laboratories) served as co-chairs. Management services were provided by Annette Anthony of Front Range Scientific Computations, Inc. The Conference is gratefully for financial support from the Department of Energy, IBM, Lawrence Livermore National Laboratory, Los Alamos National Laboratory, Sandia National Laboratories, and the National Science Foundation. The Copper Mountain Conference is organized in cooperation with the Society for Industrial and Applied Mathematics (SIAM). Submissions to this special section were open to the public. This was advertised in advance on web sites of the SIAM Journal on Scientific Computing (SISC) and the Copper Mountain Conference. The collection of papers in this section all center on iterative methods. However, many diverse topics are covered spanning contributions with a theoretical orientation to those that are more applied or driven by a specific application area. Additionally, several papers are highly influenced by current computer architecture trends. Some of the submissions also appeared in the student paper competition, a unique event at the Copper Mountain conferences that has proven to be a successful forum for presenting the latest results in the field. We have had increasingly strong participation from talented recent Ph.D. holders and current Ph.D. students in this competition. I wish to thank the guest editorial board, the SIAM/SISC staff, and the referees for their hard work to help meet deadlines. Special acknowledgment is due to Mitch Chernoff (SIAM Publications Manager) and Heather Blythe (SIAM Senior Publications Coordinator) for their efforts on this special section. We hope that this special section will continue to foster new developments in this important area of research. The diversity of topics clearly demonstrates the significant impact of iterative methods across many scientific computing areas and the need for continued research.

Monte Carlo scalable algorithms for Computational Finance

With the latest developments in the area of advanced computer architectures, we are already seeing large scale machines at petascale level and we are faced with the exascale computing challenge. All these require scalability at system, algorithmic and mathematical model level. In particular, e_cient scalable algorithms are required to bridge the performance gap. In this paper, examples of various approaches of designing scalable algorithms for such advanced architectures will be given. We will briefly present our approach to Monte Carlo scalable algorithms for Linear Algebra and explain how these approaches are extended to the field of Computational Finance. Implementation examples will be presented using Linear Algebra Problems and problems from Computational Finance. Furthermore, the corresponding properties of these algorithms will be outlined and discussed.

CIM-EARTH: Framework and Case Study

Abstract General equilibrium models have been used for decades to obtain insights into the economic implications of policies and decisions. Despite successes, however, these economic models have substantive limitations. Many of these limitations are due to computational and methodological constraints that can be overcome by leveraging recent advances in computer architecture, numerical methods, and economics research. Motivated by these considerations, we are developing a new modeling framework: the Community Integrated Model of Economic and Resource Trajectories for Humankind (CIM-EARTH). In this paper, we describe the key features of the CIM-EARTH framework and initial implementation, detail the model instance we use for studying the impacts of a carbon tax on international trade and the sensitivity of these impacts to assumptions on the rate of change in energy efficiency and labor productivity, and present results on the extent to which carbon leakage limits global reductions in emissions for some policy scenarios.

Parallel multivariate slice sampling

Slice sampling provides an easily implemented method for constructing a Markov chain Monte Carlo (MCMC) algorithm. However, slice sampling has two major drawbacks: (i) it requires repeated evaluation of likelihoods for each update, which can make it impractical when evaluations are expensive or as the number of evaluations grows (geometrically) with the dimension of the slice sampler, and (ii) since it can be challenging to construct multivariate updates, the updates are typically univariate, which often results in slow mixing samplers. We propose an approach to multivariate slice sampling that naturally lends itself to a parallel implementation. Our approach takes advantage of recent advances in computer architectures, for instance, the newest generation of graphics cards can execute roughly 30,000 threads simultaneously. We demonstrate that it is possible to construct a multivariate slice sampler that has good mixing properties and is efficient in terms of computing time. The contributions of this article are therefore twofold. We study approaches for constructing a multivariate slice sampler, and we show how parallel computing can be useful for making MCMC algorithms computationally efficient. We study various implementations of our algorithm in the context of real and simulated data.

On Developing New Models, with Paging as a Case Study

As computer science has progressed, numerous models and measures have been developed over the years. Among the most commonly used in theoretical computer science are the RAM model, the I/O model, worst case analysis, space (memory) usage, average case analysis, amortized analysis, adaptive analysis and the competitive ratio. New models are added to this list every few years to re ect varying constraints imposed by novel application or advances in computer architectures. Examples of alternative models are the transdichotomous RAM or word-RAM, the data stream model, the MapReduce model, the cache oblivious model and the smoothed analysis model. New models and measures, when successful expand our understanding of computation and open new avenues of inquiry. As it is to be expected relatively few models and paradigms are introduced every year, and even less are eventually proven successful. In this paper we discuss rst certain shortcomings of the online competitive analysis model particularly as it concerns paging, discuss existing solutions in the literature as well as present recent progress in developing models and measures that better re ect actual practice for the case of paging. From there we proceed to a more general discussion on how to measure and evaluate new models within theoretical computer science and how to contrast them, when appropriate, to existing models. Lastly, we highlight certain \natural" choices and assumptions of the standard worst-case model which are often unstated and rarely explicitly justied. We contrast these choices to those made in the formalization of probability theory.

How to simulate Galaxy Evolution: strengths and weaknesses of numerical techniques

Galaxy formation and evolution is one of the most challenging tasks in astrophysics. There is strongly increasing and substantial progress in this field not only due to the increase of telescope size and developments of new techniques providing observations allover the electromagnetic spectrum, but also the development of advanced computer architectures and of sophisticated numerical codes allow to tackle this problem from both sides and to link both research approaches to a fruitful progress in astrophysics. This review will not focus on the application of a single numerical method and the description of its result, but is instead aiming at providing an overview of different numerical tools on the basis of particle and grid schemes and to referring to important publications for the intersted reader.

Automated Event Detection and Activity Monitoring in Long Molecular Dynamics Simulations.

Events of scientific interest in molecular dynamics (MD) simulations, including conformational changes, folding transitions, and translocations of ligands and reaction products, often correspond to high-level structural rearrangements that alter contacts between molecules or among different parts of a molecule. Due to advances in computer architecture and software, MD trajectories representing such structure-changing events have become easier to generate, but the length of these trajectories poses a challenge to scientific interpretation and analysis. In this paper, we present automated methods for the detection of potentially important structure-changing events in long MD trajectories. In contrast with traditional tools for the analysis of such trajectories, our methods provide a detailed report of broken and formed contacts that aids in the identification of specific time-dependent side-chain interactions. Our approach employs a coarse-grained representation of amino acid side chains, a contact metric based on higher order generalizations of Delaunay tetrahedralization, techniques for detecting significant shifts in the resulting contact time series, and a new kernel-based measure of contact alteration activity. The analysis methods we describe are incorporated in a newly developed package, called TimeScapes, which is freely available and compatible with trajectories generated by a variety of popular MD programs. Tests based on actual microsecond time scale simulations demonstrate that the package can be used to efficiently detect and characterize important conformational changes in realistic protein systems.

Automated Event Detection and Activity Monitoring in Long Time-Scale Molecular Dynamics

Molecular dynamics trajectories of biological systems contain many events of great scientific interest, including conformational transitions, folding processes, and translocations of ligands and reaction products. In proteins these events often correspond to high-level tertiary or quaternary structure rearrangements, which alter the contacts between amino acid residues. Due to advances in computer architecture and software, molecular dynamics trajectories representing such structure-changing events have become easy to generate, but their length complicates scientific interpretation. The goal of this work is to simplify an important part of the analysis workflow (and to complement traditional visual inspection) by automating the mining of long trajectories. We present several new algorithms and implementation techniques that enable the detection of significant structure-changing events in a molecular dynamics trajectory. These algorithms include a coarse graining of side chain contacts, a contact metric based on higher-order generalizations of the Delaunay tetrahedralization, and median filters for detecting significant shifts in the ensemble mean of the resulting time series. We have also developed numerical techniques for suppressing trivial re-crossing events and a new kernel-based estimator of the contact alteration activity. These methods will be disseminated in a newly developed package, “TimeScapes,” which is compatible with molecular dynamics trajectories generated from any of a variety of popular simulation programs. Tests on microsecond time scale simulations suggest that the implementation is efficient and requires very little parameterization. The analysis provides a detailed listing of broken and formed contacts, and reliably detects allosteric and folding transitions, as well as stable intermediates, in the protein dynamics.

The new landscape of parallel computer architecture

The past few years has seen a sea change in computer architecture that will impact every facet of our society as every electronic device from cell phone to supercomputer will need to confront parallelism of unprecedented scale. Whereas the conventional multicore approach (2, 4, and even 8 cores) adopted by the computing industry will eventually hit a performance plateau, the highest performance per watt and per chip area is achieved using manycore technology (hundreds or even thousands of cores). However, fully unleashing the potential of the manycore approach to ensure future advances in sustained computational performance will require fundamental advances in computer architecture and programming models that are nothing short of reinventing computing. In this paper we examine the reasons behind the movement to exponentially increasing parallelism, and its ramifications for system design, applications and programming models.

Absolutely positively on time: what would it take? [embedded computing systems

Despite considerable progress in software and hardware techniques, many recent computing advances do more harm than good when embedded computing systems absolutely must meet tight timing constraints. For example, while synchronous digital logic delivers precise timing determinacy, advances in computer architecture and software have made it difficult or impossible to estimate or predict software's execution time. Moreover, networking techniques introduce variability and stochastic behavior, while operating systems rely on best-effort techniques. Worse, programming language semantics do not handle time well, so developers can only specify timing requirements indirectly. Thus, achieving precise timeliness in a networked embedded system - an absolutely essential goal - requires sweeping changes. For embedded computing to realize its full potential, we must reinvent computer science. Resource limitations have influenced embedded software's evolution. Embedded software differs from other software in more fundamental ways.

Advanced computer architecture and parallel processing

1. Introduction to Advanced Computer Architecture and Parallel Processing.1.1 Four Decades of Computing.1.2 Flynn's Taxonomy of Computer Architecture.1.3 SIMD Architecture.1.4 MIMD Architecture.1.5 Interconnection Networks.1.6 Chapter Summary.Problems.References.2. Multiprocessors Interconnection Networks.2.1 Interconnection Networks Taxonomy.2.2 Bus-Based Dynamic Interconnection Networks.2.3 Switch-Based Interconnection Networks.2.4 Static Interconnection Networks.2.5 Analysis and Performance Metrics.2.6 Chapter Summary.Problems.References.3. Performance Analysis of Multiprocessor Architecture.3.1 Computational Models.3.2 An Argument for Parallel Architectures.3.3 Interconnection Networks Performance Issues.3.4 Scalability of Parallel Architectures.3.5 Benchmark Performance.3.6 Chapter Summary.Problems.References.4. Shared Memory Architecture.4.1 Classification of Shared Memory Systems.4.2 Bus-Based Symmetric Multiprocessors.4.3 Basic Cache Coherency Methods.4.4 Snooping Protocols.4.5 Directory Based Protocols.4.6 Shared Memory Programming.4.7 Chapter Summary.Problems.References.5. Message Passing Architecture.5.1 Introduction to Message Passing.5.2 Routing in Message Passing Networks.5.3 Switching Mechanisms in Message Passing.5.4 Message Passing Programming Models.5.5 Processor Support for Message Passing.5.6 Example Message Passing Architectures.5.7 Message Passing Versus Shared Memory Architectures.5.8 Chapter Summary.Problems.References.6. Abstract Models.6.1 The PRAM Model and Its Variations.6.2 Simulating Multiple Accesses on an EREW PRAM.6.3 Analysis of Parallel Algorithms.6.4 Computing Sum and All Sums.6.5 Matrix Multiplication.6.6 Sorting.6.7 Message Passing Model.6.8 Leader Election Problem.6.9 Leader Election in Synchronous Rings.6.10 Chapter Summary.Problems.References.7. Network Computing.7.1 Computer Networks Basics.7.2 Client/Server Systems.7.3 Clusters.7.4 Interconnection Networks.7.5 Cluster Examples.7.6 Grid Computing.7.7 Chapter Summary.Problems.References.8. Parallel Programming in the Parallel Virtual Machine.8.1 PVM Environment and Application Structure.8.2 Task Creation.8.3 Task Groups.8.4 Communication Among Tasks.8.5 Task Synchronization.8.6 Reduction Operations.8.7 Work Assignment.8.8 Chapter Summary.Problems.References.9. Message Passing Interface (MPI).9.1 Communicators.9.2 Virtual Topologies.9.3 Task Communication.9.4 Synchronization.9.5 Collective Operations.9.6 Task Creation.9.7 One-Sided Communication.9.8 Chapter Summary.Problems.References.10 Scheduling and Task Allocation.10.1 The Scheduling Problem.10.2 Scheduling DAGs without Considering Communication.10.3 Communication Models.10.4 Scheduling DAGs with Communication.10.5 The NP-Completeness of the Scheduling Problem.10.6 Heuristic Algorithms.10.7 Task Allocation.10.8 Scheduling in Heterogeneous Environments.Problems.References.