High Performance Fortran Research Articles

RoseNNa: A performant, portable library for neural network inference with application to computational fluid dynamics

The rise of neural network-based machine learning ushered in high-level libraries, including TensorFlow and PyTorch, to support their functionality. Computational fluid dynamics (CFD) researchers have benefited from this trend and produced powerful neural networks that promise shorter simulation times. For example, multilayer perceptrons (MLPs) and Long Short Term Memory (LSTM) recurrent-based (RNN) architectures can represent sub-grid physical effects, like turbulence. Implementing neural networks in CFD solvers is challenging because the programming languages used for machine learning and CFD are mostly non-overlapping, We present the roseNNa library, which bridges the gap between neural network inference and CFD. RoseNNa is a non-invasive, lightweight (1000 lines), and performant tool for neural network inference, with focus on the smaller networks used to augment PDE solvers, like those of CFD, which are typically written in C/C++ or Fortran. RoseNNa accomplishes this by automatically converting trained models from typical neural network training packages into a high-performance Fortran library with C and Fortran APIs. This reduces the effort needed to access trained neural networks and maintains performance in the PDE solvers that CFD researchers build and rely upon. Results show that RoseNNa reliably outperforms PyTorch (Python) and libtorch (C++) on MLPs and LSTM RNNs with less than 100 hidden layers and 100 neurons per layer, even after removing the overhead cost of API calls. Speedups range from a factor of about 10 and 2 faster than these established libraries for the smaller and larger ends of the neural network size ranges tested. Program summaryProgram Title: RoseNNaCPC Library link to program files:https://doi.org/10.17632/srbrfx8k74.1Developer's repository link:https://github.com/comp-physics/roseNNaLicensing provisions: MITProgramming language: Fortran90 and PythonNature of problem: Neural network applications in computational fluid dynamics exhibit high promise. However, they are often not deployed on HPC systems due to limited deep-learning support in Fortran or C/C++. Previous attempts to solve this issue rely on significant manual intervention. This leaves researchers with no practical solution for conveniently porting their trained models for use in performant HPC codebases.Solution method: RoseNNa converts trained neural networks from popular machine learning frameworks to Fortran/C code for inference, requiring minimal manual intervention. Metaprogramming enables automatic code creation, and ONNX ensures flexible machine learning library support.

A high-performance Fortran code to calculate spin- and parity-dependent nuclear level densities

A high-performance Fortran code is developed to calculate the spin- and parity-dependent shell model nuclear level densities. The algorithm is based on the extension of methods of statistical spectroscopy and implies exact calculation of the first and second Hamiltonian moments for different configurations at fixed spin and parity. The proton–neutron formalism is used. We have applied the method for calculating the level densities for a set of nuclei in the sd-, pf-, and pf+g9/2- model spaces. Examples of the calculations for 28Si (in the sd-model space) and 64Ge (in the pf+g9/2-model space) are presented. To illustrate the power of the method we estimate the ground state energy of 64Ge in the larger model space pf+g9/2, which is not accessible to direct shell model diagonalization due to the prohibitively large dimension, by comparing with the nuclear level densities at low excitation energy calculated in the smaller model space pf. Program summaryProgram title: MMCatalogue identifier: AENM_v1_0Program summary URL:http://cpc.cs.qub.ac.uk/summaries/AENM_v1_0.htmlProgram obtainable from: CPC Program Library, Queen’s University, Belfast, N. IrelandLicensing provisions: Standard CPC licence, http://cpc.cs.qub.ac.uk/licence/licence.htmlNo. of lines in distributed program, including test data, etc.: 193181No. of bytes in distributed program, including test data, etc.: 1298585Distribution format: tar.gzProgramming language: Fortran 90, MPI.Computer: Any architecture with a Fortran 90 compiler and MPI.Operating system: Linux.RAM: Proportional to the system size, in our examples, up to 75MbClassification: 17.15.External routines: MPICH2 (http://www.mcs.anl.gov/research/projects/mpich2/)Nature of problem:Calculating of the spin- and parity-dependent nuclear level density.Solution method:The algorithm implies exact calculation of the first and second Hamiltonian moments for different configurations at fixed spin and parity. The code is parallelized using the Message Passing Interface and a master-slaves dynamical load-balancing approach.Restrictions:The program uses two-body interaction in a restricted single-level basis. For example, GXPF1A in the pf-valence space.Running time:Depends on the system size and the number of processors used (from 1 min to several hours).

The rise and fall of high performance Fortran

HPF pioneered a high-level approach to parallel programming but failed to win over a broad user community.

Automated refactorings for high performance Fortran programmes

Refactoring is a software engineering technique aimed at improving the design of software applications, without changing their external behaviour. Several refactorings have been proposed for object-oriented languages, but there are few related works focusing on procedural programming. Fortran is a procedural language heavily used in high performance computing, which is not fully explored considering refactoring support. In this paper, we describe a set of automated refactorings for Fortran based on the Photran plug-in, which is integrated with the Eclipse integrated development environment (IDE). We present a set of experiments to evaluate the impact of the proposed refactorings in third-party Fortran applications. The results show that the proposed refactorings improve the design of existing applications without compromising their performance.

Gas-kinetic numerical study of complex flow problems covering various flow regimes

The Boltzmann simplified velocity distribution function equation, as adapted to various flow regimes, is described on the basis of the Boltzmann–Shakhov model from the kinetic theory of gases in this study. The discrete velocity ordinate method of gas-kinetic theory is studied and applied to simulate complex multi-scale flows. On the basis of using the uncoupling technique on molecular movements and collisions in the DSMC method, the gas-kinetic finite difference scheme is constructed by extending and applying the unsteady time-splitting method from computational fluid dynamics, which directly solves the discrete velocity distribution functions. The Gauss-type discrete velocity numerical quadrature technique for flows with different Mach numbers is developed to evaluate the macroscopic flow parameters in the physical space. As a result, the gas-kinetic numerical algorithm is established for studying the three-dimensional complex flows with high Mach numbers from rarefied transition to continuum regimes. On the basis of the parallel characteristics of the respective independent discrete velocity points in the discretized velocity space, a parallel strategy suitable for the gas-kinetic numerical method is investigated and, then, the HPF (High Performance Fortran) parallel programming software is developed for simulating gas dynamical problems covering the full spectrum of flow regimes. To illustrate the feasibility of the present gas-kinetic numerical method and simulate gas transport phenomena covering various flow regimes, the gas flows around three-dimensional spheres and spacecraft-like shapes with different Knudsen numbers and Mach numbers are investigated to validate the accuracy of the numerical methods through HPF parallel computing. The computational results determine the flow fields in high resolution and agree well with the theoretical and experimental data. This computing, in practice, has confirmed that the present gas-kinetic algorithm probably provides a promising approach for resolving hypersonic aerothermodynamic problems with the complete spectrum of flow regimes from the gas-kinetic point of view for solving the mesoscopic Boltzmann model equation.

A Two-Level Scheduling Strategy for optimising communications of data parallel programs in clusters

Irregular data distribution and redistribution have been attracting attention recently since it can distribute different size of data segments to heterogeneous processors. High Performance Fortran Version 2 (HPF2) provides GEN_BLOCK distribution format which facilitates generalised block distributions. In this paper, we present a two-level scheduling method to minimise the communication cost in such operations. The proposed technique isolates local messages from inter-processor messages and schedules both kinds of messages in separated steps to efficiently avoid synchronisation delay. The proposed technique which has been implemented with the traditional scheduling approach yields improved schedules for heterogeneous processors environment.

Gas-kinetic numerical studies of three-dimensional complex flows on spacecraft re-entry

The gas-kinetic numerical algorithm solving the Boltzmann model equation is extended and developed to study the three-dimensional hypersonic flows of spacecraft re-entry into the atmosphere in perfect gas. In this study, the simplified velocity distribution function equation for various flow regimes is presented on the basis of the kinetic Boltzmann–Shakhov model. The discrete velocity ordinate technique and numerical quadrature methods, such as the Gauss quadrature formulas with the weight function 2/ π 1/2exp(− V 2) and the Gauss–Legendre numerical quadrature rule, are studied to resolve the barrier in simulating complex flows from low Mach numbers to hypersonic problems. Specially, the gas-kinetic finite-difference scheme is constructed for the computation of three-dimensional flow problems, which directly captures the time evolution of the molecular velocity distribution function. The gas-kinetic boundary conditions and numerical procedures are studied and implemented by directly acting on the velocity distribution function. The HPF (high performance fortran) parallel implementation technique for the gas-kinetic numerical method is developed and applied to study the hypersonic flows around three-dimensional complex bodies. The main purpose of the current research is to provide a way to extend the gas-kinetic numerical algorithm to the flow computation of three-dimensional complex hypersonic problems with high Mach numbers. To verify the current method and simulate gas transport phenomena covering various flow regimes, the three-dimensional hypersonic flows around sphere and spacecraft shape with different Knudsen numbers and Mach numbers are studied by HPF parallel computing. Excellent results have been obtained for all examples computed.

From FORTRAN 77 to Locality‐Aware High Productivity Languages for Peta‐Scale Computing

When the first specification of the FORTRAN language was released in 1956, the goal was to provide an "automatic programming system" that would enhance the economy of programming by replacing assembly language with a notation closer to the domain of scientific programming. A key issue in this context, explicitly recognized by the authors of the language, was the requirement to produce efficient object programs that could compete with their hand‐coded counterparts. More than 50 years later, a similar situation exists with respect to finding the right programming paradigm for high performance computing systems. FORTRAN, as the traditional language for scientific programming, has played a major role in the quest for high‐productivity programming languages that satisfy very strict performance constraints. This paper focuses on high‐level support for locality awareness, one of the most important requirements in this context. The discussion centers on the High Performance Fortran (HPF) family of languages, and their influence on current language developments for peta‐scale computing. HPF is a data‐parallel language that was designed to provide the user with a high‐level interface for programming scientific applications, while delegating to the compiler the task of generating an explicitly parallel message‐passing program. We outline developments that led to HPF, explain its major features, identify a set of weaknesses, and discuss subsequent languages that address these problems. The final part of the paper deals with Chapel, a modern object‐oriented language developed in the High Productivity Computing Systems (HPCS) program sponsored by DARPA. A salient property of Chapel is its general framework for the support of user‐defined distributions, which is related in many ways to ideas first described in Vienna Fortran. This framework is general enough to allow a concise specification of sparse data distributions. The paper concludes with an outlook to future research in this area.

Three-dimensional particle-in-cell simulation of scrape-off-layer plasmas

Three-dimensional particle-in-cell simulation code for the investigation of physics in scrape-off-layer plasmas is developed. The code is parallelled by using High Performance Fortran (HPF) and good performance is attained in the computation at the Plasma Simulator at the National Institute for Fusion Science (NIFS) (NECSX-7/160M5). The code represents resonable physics related to sheath and presheath formation as well as the formation of the self-consistent potential structure related to blob transport across a magnetic field.

Analysis of two-level data mapping in an HPF compiler for distributed-memory machines

Application programs in scientific and technological fields have grown increasingly large and complex. Thus, it is becoming more difficult to parallelize these programs by hand using message-passing libraries. To reduce this difficulty, we are researching the compilation technology for an industry-standard data-parallel language: High Performance Fortran (HPF). This paper proposes new data-mapping analysis applicable to HPF data-remapping directives more general than those previous research has coped with. This analysis allows programmers to develop highly productive HPF programs. We implemented the analysis in our HPF compiler and evaluated using some benchmark programs. As a result, both programs with general HPF directives and with restricted ones were transformed to similar codes and these codes ran in similar processing speed on a Hitachi SR2201 supercomputer.

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

Development of the Efficient Electromagnetic Particle Simulation Code with High Performance Fortran on a Vector-Parallel Supercomputer

A three-dimensional, relativistic, electromagnetic particle simulation code is developed using the “Exact Charge Conservation Scheme”on a vector-parallel supercomputer. This scheme is a method for calculating current densities. Applying this method, correction of electric fields is not needed. In this paper, some techniques to optimize the above scheme for a vector-parallel supercomputer are shown. The method for vectorization and parallelization in shared memory and in distributed memories are discussed. The code is written in Fortran90 and High Performance Fortran (HPF). Examination of this code is also made.

Efficient parallel implementations of near Delaunay triangulation with High Performance Fortran

Unstructured mesh generation exposes highly irregular computation patterns, which imposes a challenge in implementing triangulation algorithms on parallel machines. This paper reports on an efficie...

A Divide-and-Conquer Algorithm for Irregular Redistribution in Parallelizing Compilers

In order to achieve higher load balancing, it is necessary to solve irregular block redistribution problems, which are different from regular block-cyclic redistribution. High Performance Fortran version 2 (HPF-2) provides irregular distribution functionalities, such as GENlBLOCK and INDIRECT. This paper is devoted to develop an efficient algorithm that attempts to obtain near optimal scheduling while satisfying the conditions of minimal message size of total steps and the minimal number of steps for irregular array redistribution. The algorithm intends to decrease the computation costs by dividing the whole block into sub-blocks and solving the sub-problems accordingly, and then merging them together to get final results. Simulation results show that our algorithm has comparable performance with a relocation algorithm developed previously (H. Yook and M. Park. Proceedings of the IASTED International Conference Parallel and Distributed Computing and Systems, Nov. 3–6, MIT, Boston, USA, 1999).

Efficient parallel implementations of near Delaunay triangulation with High Performance Fortran

AbstractUnstructured mesh generation exposes highly irregular computation patterns, which imposes a challenge in implementing triangulation algorithms on parallel machines. This paper reports on an efficient parallel implementation of near Delaunay triangulation with High Performance Fortran (HPF). Our algorithm exploits embarrassing parallelism by performing sub‐block triangulation and boundary merge independently at the same time. The sub‐block triangulation is a divide & conquer Delaunay algorithm known for its sequential efficiency, and the boundary triangulation is an incremental construction algorithm with low overhead. Compared with prior work, our parallelization method is both simple and efficient. In the paper, we also describe a solution to the collinear points problem that usually arises in large data sets. Our experiences with the HPF implementation show that with careful control of the data distribution, we are able to parallelize the program using HPF's standard directives and extrinsic procedures. Experimental results on several parallel platforms, including an IBM SP2 and a DEC Alpha farm, show that a parallel efficiency of 42–86% can be achieved for an eight‐node distributed memory system. We also compare efficiency of the HPF implementation with that of a similarly hand‐coded MPI implementation. Copyright © 2004 John Wiley & Sons, Ltd.

An efficient algorithm for communication set generation of data parallel programs with block-cyclic distribution

Data parallel programming languages, such as High Performance Fortran, are widely regarded as a promising means for writing portable programs for distributed-memory machines. In this paper, we present a new algorithm for computing the communication sets in array section movements with block-cyclic (cyclic (k) in HPF) distribution. Our framework can handle multi-level alignments, multi-dimensional arrays, array intrinsic functions, affine indices and axis exchanges in the array subscript. Instead of employing the linear diophantine equation solver, a new algorithm which does not rely on the linear diophantine equation solver to calculate communication sets is proposed. We use formal proof and experimental results to show that it is more efficient than previous solutions to the same problem. Another important contribution of this paper is that we prove that the compiler is able to compute efficiently the communication sets of block-cyclic distribution as long as the block sizes of the arrays are set to be identical or the lowest common multiple (LCM) of block sizes is not a huge integer. We demonstrate it by thorough complexity analyses and extensive experimental results.

An efficient algorithm for communication set generation of data parallel programs with block-cyclic distribution

Data parallel programming languages, such as High Performance Fortran, are widely regarded as a promising means for writing portable programs for distributed-memory machines. In this paper, we present a new algorithm for computing the communication sets in array section movements with block-cyclic (cyclic ( k) in HPF) distribution. Our framework can handle multi-level alignments, multi-dimensional arrays, array intrinsic functions, affine indices and axis exchanges in the array subscript. Instead of employing the linear diophantine equation solver, a new algorithm which does not rely on the linear diophantine equation solver to calculate communication sets is proposed. We use formal proof and experimental results to show that it is more efficient than previous solutions to the same problem. Another important contribution of this paper is that we prove that the compiler is able to compute efficiently the communication sets of block-cyclic distribution as long as the block sizes of the arrays are set to be identical or the lowest common multiple (LCM) of block sizes is not a huge integer. We demonstrate it by thorough complexity analyses and extensive experimental results.

Compiling data‐parallel programs for clusters of SMPs

AbstractClusters of shared‐memory multiprocessors (SMPs) have become the most promising parallel computing platforms for scientific computing. However, SMP clusters significantly increase the complexity of user application development when using the low‐level application programming interfaces MPI and OpenMP, forcing users to deal with both distributed‐memory and shared‐memory parallelization details. In this paper we present extensions of High Performance Fortran (HPF) for SMP clusters which enable the compiler to adopt a hybrid parallelization strategy, efficiently combining distributed‐memory with shared‐memory parallelism. By means of a small set of new language features, the hierarchical structure of SMP clusters may be specified. This information is utilized by the compiler to derive inter‐node data mappings for controlling distributed‐memory parallelization across the nodes of a cluster and intra‐node data mappings for extracting shared‐memory parallelism within nodes. Additional mechanisms are proposed for specifying inter‐ and intra‐node data mappings explicitly, for controlling specific shared‐memory parallelization issues and for integrating OpenMP routines in HPF applications. The proposed features have been realized within the ADAPTOR and VFC compilers. The parallelization strategy for clusters of SMPs adopted by these compilers is discussed as well as a hybrid‐parallel execution model based on a combination of MPI and OpenMP. Experimental results indicate the effectiveness of the proposed features. Copyright © 2004 John Wiley & Sons, Ltd.

Generalized multipartitioning of multi-dimensional arrays for parallelizing line-sweep computations

Multipartitioning is a strategy for decomposing multi-dimensional arrays into tiles and mapping the resulting tiles onto a collection of processors. This class of partitionings enables efficient parallelization of “line-sweep” computations that solve one-dimensional recurrences along each dimension of a multi-dimensional array. Multipartitionings yield balanced parallelism for line sweeps by assigning each processor the same number of data tiles to compute at each step of a sweep along any array dimension. Also, they induce only coarse-grain communication. This paper considers the problem of computing generalized multipartitionings, which decompose d-dimensional arrays, d⩾2, onto an arbitrary number of processors. We describe an algorithm that computes an optimal multipartitioning onto all of the processors for this general case. We use a cost model to select the dimensionality of the best partitioning and the number of cuts to make along each array dimension; then, we show how to construct a mapping that assigns the resulting data tiles to each of the processors. The assignment of tiles to processors induced by this class of multipartitionings corresponds to an instance of a latin hyper-rectangle, a natural extension of latin squares, which have been widely studied in mathematics and statistics. Finally, we describe how we extended the Rice dHPF compiler for High Performance Fortran to generate code that employs our strategy for generalized multipartitioning and show that the compiler's generated code for the NAS SP computational fluid dynamics benchmark achieves scalable high performance.

Using High Performance Fortran to parallelise a multi-layer atmospheric transport model

An atmospheric transport model, FRAME (Fine Resolution AMmonia Exchange), has been used to model the spatial pattern of ammonia concentrations and deposition over the British Isles. The model uses a multi-layer approach with diffusion through 33 layers to describe vertical concentration profiles in the atmosphere explicitly. Together with the necessary description of atmospheric reactions with sulphur and oxidised nitrogen, this imposes a major computational requirement, with the model having a run-time of 8.5 days on a mid-range workstation. Improvement in the model run-time was sought by developing a parallel implementation coded in a data-parallel approach using High Performance Fortran. Running the code on a Cray T3 E with 128 processors provided a speedup by a factor of 120. The code’s portability, scalability and performance on different parallel platforms are discussed.