Supercomputing Resources Research Articles

Optimization of analytic density functionals by parallel genetic algorithm

We report the construction and implementation of a parallel genetic algorithm (PGA) for use in optimization of analytic density-functional theory. Our new algorithm demonstrates comparable performance to our previous simplex optimizer when benchmarked against the extended G2 set, and the considerable scatter between different optimizations shows that robust methods are required. This new PGA will allow us to further expand our parameter space while efficiently utilizing cluster and supercomputing resources as the problems grow larger.

Computational challenges of large-scale, long-time, first-principles molecular dynamics

Plane wave density functional calculations have traditionally been able to use the largest available supercomputing resources. We analyze the scalability of modern projector-augmented wave implementations to identify the challenges in performing molecular dynamics calculations of large systems containing many thousands of electrons. Benchmark calculations on the Cray XT4 demonstrate that global linear-algebra operations are the primary reason for limited parallel scalability. Plane-wave related operations can be made sufficiently scalable. Improving parallel linear-algebra performance is an essential step to reaching longer timescales in future large-scale molecular dynamics calculations.

9.1.2 Overcoming Engineering Challenges of Providing an Effective User Interface to a Large Scale Distributed Synthetic Environment on the US Teragrid: A Systems Engineering Success Story

AbstractOver recent years large scale distributed synthetic environment enterprises have been evolving in a diverse range of scientific and engineering fields. These computer modelling and simulation systems are increasing in scale and dimension in order to allow scientists and engineers to explore the attributes and emergent properties of a given system design. Within the field of computational science the grid has been developed to facilitate very large scale collaborative simulation enterprises. The grid is similar to DIS/HLA in that it supports interconnectivity but differs in the sense that it supports intercommunication of large super computing resources. An important factor in the rapid adoption of the grid has been its role in enabling access to significant supercomputing resources not usually available at a single institution. However, the major challenge for the grid has been the lack of an effective and ubiquitous interface to the huge computational resource (which can comprise over 6000 CPUs distributed across the globe) at any time and from any location. This paper describes a unique user interface built on systems engineering principles and practices to solve the problem of delivering real‐time interaction (from lightweight computing devices such as PDAs to high end computing platforms) with simulations delivering high resolution 3D images. The application of our work is likely to have far reaching benefits for many sectors including: aerospace, medical informatics, engineering design, distributed simulation and modelling.

GRID-enabled Solution of Groundwater Inverse Problems on the TeraGrid Network

Grid-computing environments are becoming increasingly popular for scientific computing due to the significant increase in capacity that they represent when compared to a single computational resource (e.g., a single cluster), their ubiquitous availability and advances in grid middleware components. Several such specialized grid environments are now available for users in the commercial and research sectors. One such effort for research is the TeraGrid, consisting of a collection of geographically distributed heterogeneous supercomputer resources including data storage resources. Parallel implementations for these environments are inherently multilevel and obtaining efficient mapping of work to processors can be eXtremely challenging. This paper eXtends an eXisting MPI application to the grid via the use of grid-enabled MPI libraries. The application uses a simulation-optimization framework involving coarse-grained parallelism in the optimizer and fine-grained parallelism in the finite-element-based simulator. Using parallelism at both these levels is essential for problems involving computationally intensive simulation steps. A hierarchical grid architecture consisting of a collection of supercomputers is ideally suited for these types of problems as a good application-to-architecture mapping can be obtained with a proper implementation. This paper presents the performance results of our implementation on the TeraGrid network consisting of three geographically distributed supercomputers.

Recovering transient data

It has become a national priority to build and use PetaFlop supercomputers. The dependability of such large systems has been recognized as a key issue that can impact their usability. Even with smaller, existing machines, failures are the norm rather than an exception. Research has shown that storage systems are the primary source of faults leading to supercomputer unavailability. In this paper, we envision two mechanisms, namely on-demand data reconstruction and eager data offloading , to address the availability of job input/output data. These two techniques aim to allow parallel jobs and post-job processing tools to continue execution despite storage system failures in supercomputers. Fundamental to both approaches is the definition and acquisition of recovery-related parallel file system metadata, which is then coupled with transparent remote data accesses. Our approach attempts to maximize the utilization of precious supercomputer resources by improving the accessibility of transient job data. Further, the proposed methods are best-effort in nature and complement existing file system recovery schemes, which are designed for persistent data. Several of our previous studies help in demonstrating the feasibility of the proposed approaches.

Robust Machine Learning Applied to Astronomical Data Sets. I. Star‐Galaxy Classification of the Sloan Digital Sky Survey DR3 Using Decision Trees

We provide classifications for all 143 million nonrepeat photometric objects in the Third Data Release of the SDSS using decision trees trained on 477,068 objects with SDSS spectroscopic data. We demonstrate that these star/galaxy classifications are expected to be reliable for approximately 22 million objects with r ≲ 20. The general machine learning environment Data-to-Knowledge and supercomputing resources enabled extensive investigation of the decision tree parameter space. This work presents the first public release of objects classified in this way for an entire SDSS data release. The objects are classified as either galaxy, star, or nsng (neither star nor galaxy), with an associated probability for each class. To demonstrate how to effectively make use of these classifications, we perform several important tests. First, we detail selection criteria within the probability space defined by the three classes to extract samples of stars and galaxies to a given completeness and efficiency. Second, we investigate the efficacy of the classifications and the effect of extrapolating from the spectroscopic regime by performing blind tests on objects in the SDSS, 2dFGRS, and 2QZ surveys. Given the photometric limits of our spectroscopic training data, we effectively begin to extrapolate past our star-galaxy training set at r ~ 18. By comparing the number counts of our training sample with the classified sources, however, we find that our efficiencies appear to remain robust to r ~ 20. As a result, we expect our classifications to be accurate for 900,000 galaxies and 6.7 million stars and remain robust via extrapolation for a total of 8.0 million galaxies and 13.9 million stars.

Integrative Biology — the challenges of developing a collaborative research environment for heart and cancer modelling

The Integrative Biology project is building a Grid infrastructure which will enable biomedical scientists to develop accurate multi-scale computational models of the heart and of cancer tumours. This infrastructure will remove many of the constraints limiting research progress today by providing access to leading-edge supercomputing and data management resources and by improving facilities to support collaboration between scientists distributed around the world. This paper describes how the project is addressing some specific technical challenges in developing such a collaborative research environment.

Hub-based Simulation and Graphics Hardware Accelerated Visualization for Nanotechnology Applications

The Network for Computational Nanotechnology (NCN) has developed a science gateway at nanoHUB.org for nanotechnology education and research. Remote users can browse through online seminars and courses, and launch sophisticated nanotechnology simulation tools, all within their web browser. Simulations are supported by a middleware that can route complex jobs to grid supercomputing resources. But what is truly unique about the middleware is the way that it uses hardware accelerated graphics to support both problem setup and result visualization. This paper describes the design and integration of a remote visualization framework into the nanoHUB for interactive visual analytics of nanotechnology simulations. Our services flexibly handle a variety of nanoscience simulations, render them utilizing graphics hardware acceleration in a scalable manner, and deliver them seamlessly through the middleware to the user. Rendering is done only on-demand, as needed, so each graphics hardware unit can simultaneously support many user sessions. Additionally, a novel node distribution scheme further improves our system's scalability. Our approach is not only efficient but also cost-effective. Only a half-dozen render nodes are anticipated to support hundreds of active tool sessions on the nanoHUB. Moreover, this architecture and visual analytics environment provides capabilities that can serve many areas of scientific simulation and analysis beyond nanotechnology with its ability to interactively analyze and visualize multivariate scalar and vector fields.

The John von Neumann Institute for Computing (NIC): A survey of its supercomputer facilities and its Europe-wide computational science activities

The John von Neumann Institute for Computing (NIC) at the Research Centre Juelich, Germany, is one of the leading supercomputing centres in Europe. Founded as a national centre in the mid-eighties it now provides more and more resources to European scientists. This happens within EU-funded projects (I3HP, DEISA) or Europe-wide scientific collaborations. Beyond these activities NIC started an initiative towards the new EU member states in summer 2004. Outstanding research groups are offered to exploit the supercomputers at NIC to accelerate their investigations on leading-edge technology. The article gives an overview of the organisational structure of NIC, its current supercomputer systems, and its user support. Transnational Access (TA) within I3HP is described as well as access by the initiative for new EU member states. The volume of these offers and the procedure of how to apply for supercomputer resources is introduced in detail.

Using Scientific Visualization to Represent Soil Hydrology Dynamics

Understanding the relationships between soil, landscape, and hydrology is important for making sustainable land management decisions. In this study, scientific visualization was explored as a means to visually represent the complex spatial and temporal variations in the hydrologic status of soils. Soil hydrology data was collected at seven stations along a 125-meter summit to depression transect in south-central Minnesota from 1995 to 1998. Three-dimensional (X, Y, and time) and 4-dimensional (X, Y, Z, and time) animations of water table fluctuations were constructed using an off-the-shelf product and supercomputing resources. When constructing and evaluating the animations, specific attention was given to educational value, user functionality, portability, and ease of animation development. The animations developed in this study can be used in a variety of educational settings (soil/hydrology courses, wetland delineation training, septic system suitability assessment, etc.) and will make it easier for users of all educational backgrounds to more effectively understand complex hydrologic processes.

Computational network federations: a middleware architecture for network-based computing

Computational Network Federations (CNFs) enable an arbitrary set of heterogeneous hosts which are connected via any type of network to form dynamic virtual distributed systems that cooperate to execute an application, or serve as generalized application service platforms to end users. CNFs motivate a view of the Internet as a vast unified host: a repository of information, application services, and an omnipresent supercomputing resource regardless of the type of access device or access methodology. CNFs provide a powerful way of virtualizing generalized enterprise networks (or even the Internet), and an economic and resilient model for deploying enterprise applications, (such as CRM) and peer-2-peer services (e.g., chatrooms). This paper describes a middleware architecture that enables network-based computing, communications, and services through a unified, access, and platform-independent approach. CNFs borrow from the capabilities of grid computing and aim toward intelligent computational service networks that are ubiquitous, secure, and adaptive to user and access-method idiosyncrasies. CNFs encompass a set of abstractions and interfaces that provide: 1) a unified service-oriented view of the network to the user; 2) a homogeneous host abstraction to applications; and 3) a shared-memory abstraction to software developers. This paper outlines the architecture of CNFs and describes in more detail i-DVM, a distributed multithreaded meta-OS that forms the core of a CNF and implements the virtual machine abstraction and location transparency.

A grid-enabled lightweight computational steering client: a .NET PDA implementation

The grid has been developed to support large-scale computer simulations in a diverse range of scientific and engineering fields. Consequently, the increasing availability of powerful distributed computing resources is changing how scientists undertake large-scale modelling/simulation. Instead of being limited to local computing resources, scientists are now able to make use of supercomputing facilities around the world. These grid resources comprise specialized distributed three-dimensional visualization environments through to massive computational systems. The scientist usually accesses these resources from reasonably high-end desktop computers. Even though most modern desktop computers are provided with reasonably powerful three-dimensional graphical hardware, not all scientific applications require high-end three-dimensional visualization because the data of interest is essentially numerical or two-dimensional graphical data. For these applications, a much simpler two-dimensional graphical displays can be used. Since large jobs can take many hours to complete the scientist needs access to a technology that will allow them to still monitor and control their job while away from their desks. This paper describes an effective method of monitoring and controlling a set of chained computer simulations by means of a lightweight steering client based on a small personal digital assistant (PDA). The concept of using a PDA to steer a series of computational jobs across a supercomputing resource may seem strange at first but when scientists realize they can use these devices to connect to their computation wherever there is a wireless network (or cellular phone network) the concept becomes very compelling. Apart from providing a much needed easy-to-use interface, the PDA-based steering client has the benefit of freeing the scientist from the desktop. It is during this monitoring stage that the hand-held PDA client is of particular value as it gives the application scientist greater freedom to leave his or her desk but still communicate with their simulation, with the proviso that they remain within the range of a wireless network.

QCDgrid: A Grid Resource for Quantum Chromodynamics

Quantum Chromodynamics (QCD) is an application area that requires access to large supercomputing resources and generates large amounts of raw data. The UK's national lattice QCD collaboration UKQCD currently stores and requires access to around five Tbytes of data, a figure that is growing dramatically as the collaboration's purpose built supercomputing system, QCDOC [P.A. Boyle, D. Chen, N.H. Christ, M. Clark, S.D. Cohen, C. Cristian, Z. Dong, A. Gara, B. Joo, C. Jung, C. Kim, L. Levkova, X. Liao, G. Liu, R.D. Mawhinney, S. Ohta, K. Petrov, T. Wettig and A. Yamaguchi, “Hardware and software status of QCDOC, arXiv: hep-lat/0309096”, Nuclear Physics. B, Proceedings Supplement, Vol. 838, pp. 129–130, 2004. See: http://www.ph.ed.ac.uk/ukqcd/community/qcdoc/; P.A. Boyle, D. Chen, N.H. Christ, M.A. Clark, S.D. Cohen, C. Cristian, Z. Dong, A. Gara, B. Joo, C. Jung, C. Kim, L.A. Levkova, X. Liao, R.D. Mawhinney, S. Ohta, K. Petrov, T. Wettig and A. Yamaguchi, “Overview of the QCDSP and QCDOC computers”, IBM Journal of Research and Development, Vol. 49, No. 2/3, p. 351, 2005] came into full production service towards the end of 2004. This data is stored on QCDgrid, a data Grid currently composed of seven storage elements at five separate UK sites.

Large-scale grid-enabled lattice Boltzmann simulations of complex fluid flow in porous media and under shear.

Well-designed lattice Boltzmann codes exploit the essentially embarrassingly parallel features of the algorithm and so can be run with considerable efficiency on modern supercomputers. Such scalable codes permit us to simulate the behaviour of increasingly large quantities of complex condensed matter systems. In the present paper, we present some preliminary results on the large-scale three-dimensional lattice Boltzmann simulation of binary immiscible fluid flows through a porous medium, derived from digitized X-ray micro-tomographic data of Bentheimer sandstone, and from the study of the same fluids under shear. Simulations on such scales can benefit considerably from the use of computational steering, and we describe our implementation of steering within the lattice Boltzmann code, called LB3D, making use of the RealityGrid steering library. Our large-scale simulations benefit from the new concept of capability computing, designed to prioritize the execution of big jobs on major supercomputing resources. The advent of persistent computational grids promises to provide an optimal environment in which to deploy these mesoscale simulation methods, which can exploit the distributed nature of computer, visualization and storage resources to reach scientific results rapidly; we discuss our work on the grid-enablement of lattice Boltzmann methods in this context.

Poisson–Boltzmann Methods for Biomolecular Electrostatics

Publisher Summary This chapter presents Poisson-Boltzmann (PB) methods for biomolecular electrostatics. The understanding of electrostatic properties is a basic aspect of the investigation of biomolecular processes. Structures of proteins and other biopolymers are being determined at an increasing rate through structural genomics and other effort. The nonlinear equations obtained from the full PB equation require more specialized techniques, such as Newton methods, to determine the solution to the discretized algebraic equation. The increasing size and number of biomolecular structures have necessitated the development of new parallel finite difference and finite element methods for solving the PB equation. These parallel techniques allow users to leverage supercomputing resources to determine the electrostatic properties of single-structure calculations on large biological systems consisting of millions of atoms. In addition, advances in improving the efficiency of PB force calculations have paved the way for a new era of biomolecular dynamics simulation methods using detailed implicit solvent models. It is anticipated that PB methods will continue to play an important role in computational biology as the study of biological systems grow from macromolecular to cellular scales.

Role of parallel and distributed computing in beam physics

In this paper the role of distributed and parallel computing in beam physics is discussed. The natural parallel and distributed structures of beam physics problems allow the use of parallel and distributed computer systems. But the usual approaches based on traditional numerical methods demand using the resources of supercomputers. This leads to the impossibility of using such multiprocessing systems as computational clusters. In this paper some examples of beam physics problems are discussed from the computational point of view using clustered systems.

A fast high-order solver for problems of scattering by heterogeneous bodies

A new high-order integral algorithm for the solution of scattering problems by heterogeneous bodies is presented. Here, a scatterer is described by a (continuously or discontinuously) varying refractive index n(x) within a two-dimensional (2D) bounded region; solutions of the associated Helmholtz equation under given incident fields are then obtained by high-order inversion of the Lippmann-Schwinger integral equation. The algorithm runs in O(Nlog(N)) operations where N is the number of discretization points. A wide variety of numerical examples provided include applications to highly singular geometries, high-contrast configurations, as well as acoustically/electrically large problems for which supercomputing resources have been used recently. Our method provides highly accurate solutions for such problems on small desktop computers in CPU times of the order of seconds.

Ligature: Component Architecture for High Performance Applications

The increasing feasibility of developing applications spanning nationwide supercomputing resources makes possible the creation of simulations composed of multiple interdisciplinary components and capable of modeling natural and social phenomena of national importance with unprecedented speed and accuracy. However, the potential offered by hardware technology often fails to be fully realized due to the lack of software environments supporting such efforts. Furthermore, the complexity of combining within one application components with different performance characteristics often prevents such applications from achieving required performance levels. The Ligature project at Los Alamos National Laboratory addresses the issue of designing a software infrastructure enabling fast and efficient development of multicomponent applications and that of providing performance guidance to the programmer using this infrastructure. Ligature allows the programmer to define component interfaces specifying how heterogeneous, distributed components can interact within a larger system and provides a reusable infrastructure capable of connecting these components. These interfaces, as well as information about component performance, are accessible through a database. Within this framework, we are trying to understand how information about the performance of individual components and information about performance of the framework can be combined to develop a performance-aware multicomponent application.

A mobile agent based push methodology for global parallel computing

The 1990s are seeing the explosive growth of the Internet and Web-based information sharing and dissemination systems. The Internet is also showing a potential of forming of a supercomputing resource out of networked computers. Parallel computing on the Internet often works in a machine-centric “pull” execution model. That is, a coordinator machine maintains a pool of tasks and distributes the tasks to other participants on demands. This paper proposes a novel mobile agent based “push” methodology from the perspective of applications. In the method, users declare their computation-bound jobs as autonomous agents. The computational agents will roam on the Internet to find servers to run. Since the agents can be programmed to satisfy their goals, even if they move and lose contact with their creators, they can survive intermittent or unreliable network connection. During their lifetime, the agents can also move themselves autonomously from one machine to another for load balancing, enhancing data locality, and tolerating faults. We present an agent-oriented programming and resource brokerage infrastructure, TRAVELER, in support of global parallel computing. The TRAVELER provides a mechanism for clients to wrap their parallel applications as mobile agents. The agents are dispatched to a resource broker. The broker forms a parallel virtual machine atop available servers to execute the agents. TRAVELER relies on an integrated distributed shared array (DSA) runtime system to support inter-agent communication and synchronization on clusters of servers. We demonstrate the feasibility of the TRAVELER in parallel sorting and LU factorization problems.

The Earth's deep interior: advances in theory and experiment

The Earth extends some 6400 km to the centre where the conditions of pressure ( P ) and temperature ( T ) reach over three million times atmospheric pressure and ca .6000°C. We stand on thin brittle crustal plates moving through geological time over a continuously deforming mantle of slowly convecting hot rock. The mantle extends about halfway through the Earth to a liquid outer core and a solid inner core. Although the mantle and core make up 99% of the Earth by volume and mass, we are only able to sample mantle material directly to a few hundred kilometres, from inclusions in diamonds that are brought up to the surface by volcanic intrusions; the remaining 90% of the Earth is effectively inaccessible. The most direct knowledge we have of the Earth9s deep interior comes from the seismic waves generated from earthquakes. A knowledge of material properties coupled with these seismic waves tell us that the mantle is made up of complex silicates and that the core is predominantly made of solid and liquid iron with some alloying elements. However, the detailed structure of the Earth9s deep interior is poorly constrained. Major advances toward the understanding of the composition, structure and dynamics of the Earth9s deep interior are to be gained only by a combination of experimental and theoretical techniques. It is already clear that many of the large–scale geological processes responsible for the conditions at the surface are driven from the Earth9s core. However, there are many questions yet to be answered about the exact nature of the core and mantle, and the interaction between them. For example, we have yet to fully define the major– and minor–element chemistry of the mantle, the convective regime of the mantle, the alloying elements in the core, the nature of the core–mantle boundary and the dynamical processes in the outer core governing the geodynamo. Advances in high– P / T experimental techniques over the last two decades allow laboratory simulation of the physical conditions from the surface of the Earth to the core, shedding light on the physics and chemistry of the Earth9s deep interior. High P and T can be maintained for significant periods (minutes to days) in multi–anvil and diamond–anvil presses. Shock experiments produce high T and P in the megabar range for tiny durations (milliseconds), but, in doing so, they shed light on the physics of the solid inner core. The current development of in situ high–pressure research such as P– and S–wave interferometry, electrical conductivity and synchrotron–based X–ray techniques will, over the coming decades, allow significant improvements in our understanding of processes in the deep Earth. Even so, with increasing depth, it becomes increasingly difficult to mimic the extreme conditions of P and T precisely. An alternative to laboratory experiments is the use of computer simulations, which allow us to test which models best match the seismic evidence and experimental data. In particular, with increasingly powerful supercomputer resources, emphasis is now being placed on the use of ab initio quantum–mechanical calculations to simulate materials at the conditions of pressure and temperature to be found in the Earth9s deep interior. This approach allows us to predict the properties of candidate mantle silicates with remarkable accuracy when compared with seismic data and the results of laboratory experiments. With these simulation techniques, we are also trying to solve many problems that are out of the reach of experimentation involving simultaneously high P and T , such as the nature of iron and iron alloys under the extreme conditions of the core where iron is squeezed to about half its normal volume, and we will soon be able provide constraints on the temperature profile of the Earth, which, at core depths, is known only to within a few thousand degrees! It is therefore the challenge of the next few years among deep–Earth scientists to develop accurate measurements and models of the properties of the high–pressure silicates and iron alloys at deep–Earth conditions. With an interdisciplinary approach involving theory, experiment and seismology we will be able to determine the nature, evolution and influence of the Earth9s deep interior.