Large-scale Scientific Computing Research Articles

Machine Learning-Based Kernel Selector for SpMV Optimization in Graph Analysis

Sparse Matrix and Vector multiplication (SpMV) is one of the core algorithms in various large-scale scientific computing and real-world applications. With the rapid development of AI and big data, the input vector in SpMV becomes sparse in many application fields. Especially in some graph analysis calculations, the sparsity of the input vector will change with the running of the program, and the non-zero element distribution of the adjacency matrix of some graph data has the power law property, leading to serious load imbalance, which requires additional optimization means. Therefore, the optimal SpMV kernel may be different, and a single SpMV kernel can no longer meet the acceleration requirements. In this paper, we propose a decision tree-based adaptive SpMV framework, named DTSpMV, that can automatically select appropriate SpMV kernels according to different input data in iterations of graph computation. Based on the analysis of computing patterns, bit-array compression algorithms, and serial and parallel algorithms, we encapsulate nine SpMV kernels within the framework. We explore machine learning-based kernel selectors in terms of both accuracy and runtime overhead. Experimental results on NVIDIA Tesla T4 GPU show that our adaptive framework achieves the arithmetic average performance improvement of \(152\% \) compared to the SpMV kernel in cuSPARSE.

A Survey of Accelerating Parallel Sparse Linear Algebra

Sparse linear algebra includes the fundamental and important operations in various large-scale scientific computing and real-world applications. There exists performance bottleneck for sparse linear algebra since it mainly contains the memory-bound computations with low arithmetic intensity. How to improve its performance has increasingly become a focus of research efforts. Using parallel computing techniques to accelerate sparse linear algebra is currently the most popular method, while facing various challenges, e.g., large-scale data brings difficulties in storage, and the sparsity of data leads to irregular memory accesses and parallel load imbalance. Therefore, this article provides a comprehensive overview on acceleration of sparse linear algebra operations using parallel computing platforms, where we focus on four main classifications: sparse matrix-vector multiplication (SpMV), sparse matrix-sparse vector multiplication (SpMSpV), sparse general matrix-matrix multiplication (SpGEMM), and sparse tensor algebra. The takeaways from this article include the following: understanding the challenges of accelerating linear sparse algebra on various hardware platforms; understanding how structured data sparsity can improve storage efficiency; understanding how to optimize parallel load balance; understanding how to improve the efficiency of memory accesses; understanding how do the adaptive frameworks automatically select the optimal algorithms; and understanding recent design trends for acceleration of parallel sparse linear algebra.

Guest Editorial Reproducibility Initiative at the SC Conference Series: A Preface to the Special Section

Welcome to our special section on reproducibility in large-scale computational science. Reproducibility is one of the foundations of science, ensuring not only the transparency and rigor of the methods utilized for discovery but also that the research involved can be built upon. In order to enable research and to support open science, it is crucial that the computational work performed on high performance computing (HPC) systems is also reproducible.

Numerical modeling of the propagation process of landslide surge using physics-informed deep learning

The landslide surge is a common secondary disaster of reservoir bank landslides, which can cause more serious damage than the landslide itself in many cases. With the development of large-scale scientific and engineering computing, many new techniques have been applied to the study of hydrodynamic problems to make up for the shortcomings of traditional methods. In this paper, we use the physics-informed neural network (PINN) to simulate the propagation process of surges caused by landslides. We study different characteristics of landslide surges by changing water depth and particle density. We find that: (1) the landslide surge propagation process simulation method based on the physics-informed neural network has good applicability, and the stages of landslide surge propagation can be well presented; (2) the depth of water influences the landslide surge propagation as the amplitude of the surge increases with deeper water; (3) the particle density of water influences the landslide surge propagation as the fluctuation of the surge is more obvious with larger particle density. Our study is helpful to understand the propagation process of landslide surges more clearly and provides new ideas for the follow-up study of this kind of complex fluid–structure interaction problem.

Demonstrating the viability of Lagrangian in situ reduction on supercomputers

Performing exploratory analysis and visualization of large-scale time-varying computational science applications is challenging due to inaccuracies that arise from under-resolved data. In recent years, Lagrangian representations of the vector field computed using in situ processing are being increasingly researched and have emerged as a potential solution to enable exploration. However, prior works have offered limited estimates of the encumbrance on the simulation code as they consider “theoretical” in situ environments. Further, the effectiveness of this approach varies based on the nature of the vector field, benefitting from an in-depth investigation for each application area. With this study, an extended version of Sane et al. (2021), we contribute an evaluation of Lagrangian analysis viability and efficacy for simulation codes executing at scale on a supercomputer. We investigated previously unexplored cosmology and seismology applications as well as conducted a performance benchmarking study by using a hydrodynamics mini-application targeting exascale computing. To inform encumbrance, we integrated in situ infrastructure with simulation codes, and evaluated Lagrangian in situ reduction in representative homogeneous and heterogeneous HPC environments. To inform post hoc accuracy, we conducted a statistical analysis across a range of spatiotemporal configurations as well as a qualitative evaluation. Additionally, our study contributes cost estimates for distributed-memory post hoc reconstruction. In all, we demonstrate viability for each application — data reduction to less than 1% of the total data via Lagrangian representations, while maintaining accurate reconstruction and requiring under 10% of total execution time in over 90% of our experiments.

Transparency and Reproducibility Practice in Large-Scale Computational Science: A Preface to the Special Section

With this special section we bring you a practice and experience effort in transparency and reproducibility for large-scale computational science. A unique section, it consists of a research work plus six critques, each by a student team that reproduced the work. The original research work has been expanded in its science and also in its contribution to open science with a discussion of the student effort. Our letter contemplates implications as well.

Inference of dynamic spatial GRN models with multi-GPU evolutionary computation.

Reverse engineering mechanistic gene regulatory network (GRN) models with a specific dynamic spatial behavior is an inverse problem without analytical solutions in general. Instead, heuristic machine learning algorithms have been proposed to infer the structure and parameters of a system of equations able to recapitulate a given gene expression pattern. However, these algorithms are computationally intensive as they need to simulate millions of candidate models, which limits their applicability and requires high computational resources. Graphics processing unit (GPU) computing is an affordable alternative for accelerating large-scale scientific computation, yet no method is currently available to exploit GPU technology for the reverse engineering of mechanistic GRNs from spatial phenotypes. Here we present an efficient methodology to parallelize evolutionary algorithms using GPU computing for the inference of mechanistic GRNs that can develop a given gene expression pattern in a multicellular tissue area or cell culture. The proposed approach is based on multi-CPU threads running the lightweight crossover, mutation and selection operators and launching GPU kernels asynchronously. Kernels can run in parallel in a single or multiple GPUs and each kernel simulates and scores the error of a model using the thread parallelism of the GPU. We tested this methodology for the inference of spatiotemporal mechanistic gene regulatory networks (GRNs)-including topology and parameters-that can develop a given 2D gene expression pattern. The results show a 700-fold speedup with respect to a single CPU implementation. This approach can streamline the extraction of knowledge from biological and medical datasets and accelerate the automatic design of GRNs for synthetic biology applications.

An efficient pattern-based approach for workflow supporting large-scale science: The DagOnStar experience

Workflow engines are commonly used to orchestrate large-scale scientific computations such as, but not limited to weather, climate, natural disasters, food safety, and territorial management. However, to implement, manage, and execute real-world scientific applications in the form of workflows on multiple infrastructures (servers, clusters, cloud) remains a challenge. In this paper, we present DagOnStar (Directed Acyclic Graph OnAnything), a lightweight Python library implementing a workflow paradigm based on parallel patterns that can be executed on any combination of local machines, on-premise high performance computing clusters, containers, and cloud-based virtual infrastructures. DagOnStar is designed to minimize data movement to reduce the application storage footprint. A case study based on a real-world application is explored to illustrate the use of this novel workflow engine: a containerized weather data collection application deployed on multiple infrastructures. An experimental comparison with other state-of-the-art workflow engines shows that DagOnStar can run workflows on multiple types of infrastructure with an improvement of 50.19% in run time when using a parallel pattern with eight task-level workers.

Large-Scale Scientific Computing in the Fight Against COVID-19

U.S. computing leaders, including the National Science Foundation, have partnered with universities, government agencies, and the private sector to accelerate research into responses to COVID-19—providing an unprecedented collection of resources that include some of the fastest computers in the world. This article expands on last month's Leadership Computing article by continuing to showcase the range of contributions that the national cyberinfrastructure is making global efforts to stop the pandemic. This article touches on research efforts to learn how SARS-CoV-2 spreads among different populations, the biology and structure of the virus, and its mechanisms of infection, and to develop effective vaccines for prevention and antiviral therapies for treatment. Even though we are still early in the process of developing an effective therapeutic response, the rapid mobilization of the national research cyberinfrastructure is a timely reminder of the strategic importance of robust, ongoing investments in large-scale scientific computing.

Memory-Efficient and Skew-Tolerant MapReduce Over MPI for Supercomputing Systems

Data analytics has become an integral part of large-scale scientific computing. Among various data analytics frameworks, MapReduce has gained the most traction. Although some efforts have been made to enable efficient MapReduce for supercomputing systems, they are often limited to fairly homogeneous workloads where equal partitioning of input data across tasks results in essentially equal output or temporary data generated on each task. For workloads that are more skewed, however, current implementations can result in imbalance in memory usage and, consequently, can cause a slowdown in execution time and a loss in data scalability. To tackle this problem, we enhance a previously published memory-conscious MapReduce over MPI framework called Mimir. Our enhancements to Mimir include combiner and dynamic repartition optimizations to minimize and balance memory usage and to achieve close to optimal balance of the memory usage across processes and to reduce the execution time by up to 12 times. Experimental results show that Mimir can scale to at least 3072 processes on the Tianhe-2 supercomputer on skewed datasets.

Toward Social Media Content Recommendation Integrated with Data Science and Machine Learning Approach for E-Learners

Electronic Learning (e-learning) has made a great success and recently been estimated as a billion-dollar industry. The users of e-learning acquire knowledge of diversified content available in an application using innovative means. There is much e-learning software available—for example, LMS (Learning Management System) and Moodle. The functionalities of this software were reviewed and we recognized that learners have particular problems in getting relevant recommendations. For example, there might be essential discussions about a particular topic on social networks, such as Twitter, but that discussion is not linked up and recommended to the learners for getting the latest updates on technology-updated news related to their learning context. This has been set as the focus of the current project based on symmetry between user project specification. The developed project recommends relevant symmetric articles to e-learners from the social network of Twitter and the academic platform of DBLP. For recommendations, a Reinforcement learning model with optimization is employed, which utilizes the learners’ local context, learners’ profile available in the e-learning system, and the learners’ historical views. The recommendations by the system are relevant tweets, popular relevant Twitter users, and research papers from DBLP. For matching the local context, profile, and history with the tweet text, we recognized that terms in the e-learning system need to be expanded to cover a wide range of concepts. However, this diversification should not include such terms which are irrelevant. To expand terms of the local context, profile and history, the software used the dataset of Grow-bag, which builds concept graphs of large-scale Computer Science topics based on the co-occurrence scores of Computer Science terms. This application demonstrated the need and success of e-learning software that is linked with social media and sends recommendations for the content being learned by the e-Learners in the e-learning environment. However, the current application only focuses on the Computer Science domain. There is a need for generalizing such applications to other domains in the future.

Issues on the use of a modified Bunch and Kaufman decomposition for large scale Newton\u2019s equation

In this work, we deal with Truncated Newton methods for solving large scale (possibly nonconvex) unconstrained optimization problems. In particular, we consider the use of a modified Bunch and Kaufman factorization for solving the Newton equation, at each (outer) iteration of the method. The Bunch and Kaufman factorization of a tridiagonal matrix is an effective and stable matrix decomposition, which is well exploited in the widely adopted SYMMBK (Bunch and Kaufman in Math Comput 31:163–179, 1977; Chandra in Conjugate gradient methods for partial differential equations, vol 129, 1978; Conn et al. in Trust-region methods. MPS-SIAM series on optimization, Society for Industrial Mathematics, Philadelphia, 2000; HSL, A collection of Fortran codes for large scale scientific computation, http://www.hsl.rl.ac.uk/; Marcia in Appl Numer Math 58:449–458, 2008) routine. It can be used to provide conjugate directions, both in the case of 1times 1 and 2times 2 pivoting steps. The main drawback is that the resulting solution of Newton’s equation might not be gradient–related, in the case the objective function is nonconvex. Here we first focus on some theoretical properties, in order to ensure that at each iteration of the Truncated Newton method, the search direction obtained by using an adapted Bunch and Kaufman factorization is gradient–related. This allows to perform a standard Armijo-type linesearch procedure, using a bounded descent direction. Furthermore, the results of an extended numerical experience using large scale CUTEst problems is reported, showing the reliability and the efficiency of the proposed approach, both on convex and nonconvex problems.

A Fortran-Keras Deep Learning Bridge for Scientific Computing

Implementing artificial neural networks is commonly achieved via high-level programming languages such as Python and easy-to-use deep learning libraries such as Keras. These software libraries come preloaded with a variety of network architectures, provide autodifferentiation, and support GPUs for fast and efficient computation. As a result, a deep learning practitioner will favor training a neural network model in Python, where these tools are readily available. However, many large-scale scientific computation projects are written in Fortran, making it difficult to integrate with modern deep learning methods. To alleviate this problem, we introduce a software library, the Fortran-Keras Bridge (FKB). This two-way bridge connects environments where deep learning resources are plentiful with those where they are scarce. The paper describes several unique features offered by FKB, such as customizable layers, loss functions, and network ensembles. The paper concludes with a case study that applies FKB to address open questions about the robustness of an experimental approach to global climate simulation, in which subgrid physics are outsourced to deep neural network emulators. In this context, FKB enables a hyperparameter search of one hundred plus candidate models of subgrid cloud and radiation physics, initially implemented in Keras, to be transferred and used in Fortran. Such a process allows the model’s emergent behavior to be assessed, i.e., when fit imperfections are coupled to explicit planetary-scale fluid dynamics. The results reveal a previously unrecognized strong relationship between offline validation error and online performance, in which the choice of the optimizer proves unexpectedly critical. This in turn reveals many new neural network architectures that produce considerable improvements in climate model stability including some with reduced error, for an especially challenging training dataset.

New challenges for distributed computing at the CMS experiment

The Large Hadron Collider (LHC) experiments soon step into the next period of run-3 data taking with an increased data rate and high pileup requiring an excellent working computing infrastructure. In the future High-Luminosity LHC (HL-LHC) data-taking period, the compute, storage and network facilities have to be further extended by large factors and flexible and sophisticated computing models are essential. New techniques of modern state-of-the-art methods in physics analysis and data science, Deep Learning and Big Data tools, are crucial to handle high-dimensional and more complex problems. Beside flexible cloud computing technologies the usage of High Performance Computing (HPC) at the LHC experiments are explored. In this presentation, I will discuss the LHC run-3 and future HL-LHC runs computing technologies and the utilization of modern physics analysis and data science methods for the increasing and complex demands of large-scale scientific computing.

Post-processing and visualization of large-scale DEM simulation data with the open-source VELaSSCo platform

Regardless of its origin, in the near future the challenge will not be how to generate data, but rather how to manage big and highly distributed data to make it more easily handled and more accessible by users on their personal devices. VELaSSCo (Visualization for Extremely Large-Scale Scientific Computing) is a platform developed to provide new visual analysis methods for large-scale simulations serving the petabyte era. The platform adopts Big Data tools/architectures to enable in-situ processing for analytics of engineering and scientific data and hardware-accelerated interactive visualization. In large-scale simulations, the domain is partitioned across several thousand nodes, and the data (mesh and results) are stored on those nodes in a distributed manner. The VELaSSCo platform accesses this distributed information, processes the raw data, and returns the results to the users for local visualization by their specific visualization clients and tools. The global goal of VELaSSCo is to provide Big Data tools for the engineering and scientific community, in order to better manipulate simulations with billions of distributed records. The ability to easily handle large amounts of data will also enable larger, higher resolution simulations, which will allow the scientific and engineering communities to garner new knowledge from simulations previously considered too large to handle. This paper shows, by means of selected Discrete Element Method (DEM) simulation use cases, that the VELaSSCo platform facilitates distributed post-processing and visualization of large engineering datasets.

Translation of array-based loops to distributed data-parallel programs

Large volumes of data generated by scientific experiments and simulations come in the form of arrays, while programs that analyze these data are frequently expressed in terms of array operations in an imperative, loop-based language. But, as datasets grow larger, new frameworks in distributed Big Data analytics have become essential tools to large-scale scientific computing. Scientists, who are typically comfortable with numerical analysis tools but are not familiar with the intricacies of Big Data analytics, must now learn to convert their loop-based programs to distributed data-parallel programs. We present a novel framework for translating programs expressed as array-based loops to distributed data parallel programs that is more general and efficient than related work. We report on a prototype implementation on top of Spark and evaluate the performance of our system relative to hand-written programs.

Scheduling strategy for computational-intensive data flow in generalized cluster environments

A classical problem in the field of distributed computation and parallel processing concerns reasonable allocation of resources among computational-intensive data flows. We introduce the generalized cluster in this paper for processing large-scale scientific computations and to further explore a consecutive cooperation game-based dynamic scheduling strategy. We construct an abstract generalized cluster environment and summarize the types of data flows. We then convert the multi-objective scheduling problem into a multi-objective expectation function-based continuous cooperation game model and discuss its strategy and a solution for its kernel. We also propose a dynamic scheduling mechanism to address the instability of generalized clusters to ensure a reasonable, real-time adjusting allocation scheme by monitoring and compensating appropriately. Finally, we apply our method to a real world application to demonstrate our successful scheduling strategy that achieves superior results for overall cost, cost-performance index, and mean run time when compared with other methods.

Service Degradability Supported by Forecasting System in Optical Data Center Networks

A large-scale distributed scientific computing develops rapidly, making the interconnection of data centers (DCs) become the focus in both academic and industrial communities. Since the fiber transmission has the characteristics of high capacity, low power consumption, and strong reliability, establishing the optical data center network (ODCN)—where DCs are interconnected via optical backbone—is an inevitable trend. In ODCNs, the oversize inter-DC traffic demand suffers from the denial-of-service caused by network overload. Therefore, we propose a provident service degradation (service degradability) framework based on the traffic-demand forecasting system. This forecasting system helps us to precisely know when service degradability becomes necessary, together with the exact location of overload. As a result, service degradability can be adopted at appropriate times and locations. Specifically, we perform bandwidth degradability to overcome the overload of optical backbone, while utilizing computing-resource degradability if there is a DC overload. We simulate and analyze the performance of our prediction system and service degradability algorithm. The simulation results show that the system's prediction result well matches actual traffic demand, and the average absolute relative error is merely 0.0507; and compared with benchmarks, our algorithm has the better performance on maximizing service provider profit via mitigating network congestion.

Budget and Deadline Aware e-Science Workflow Scheduling in Clouds

Basic science is becoming ever more computationally intensive, increasing the need for large-scale compute and storage resources, be they within a High Performance Computer cluster, or more recently within the cloud. In most cases, large scale scientific computation is represented as a workflow for scheduling and runtime provisioning. Such scheduling become an even more challenging problem on cloud systems due to the dynamic nature of the cloud, in particular, the elasticity, the pricing models (both static and dynamic), the non-homogeneous resource types, the vast array of services, and virtualization. This mapping of workflow tasks on to a set of provisioned instances is an example of the general scheduling problem and is NP-complete. In addition, we also need to ensure that certain runtime constraints are met - the most typical being the cost of the computation and the time which that computation requires to complete. In this article, we introduce a new heuristic scheduling algorithm, Budget Deadline Aware Scheduling (BDAS), that addresses eScience workflow scheduling under budget and deadline constraints in Infrastructure as a Service (IaaS) clouds. The novelty of our work is satisfying both budget and deadline constraints while introducing a tunable cost-time trade off over heterogeneous instances. In addition, we study the stability and robustness of our algorithm by performing sensitivity analysis. The results demonstrate that overall BDAS finds a viable schedule for more than 40000 test cases accomplishing both defined constraints: budget and deadline. Moreover, our algorithm achieves a 17.0-23.8 percent higher success rate when compared to state of the art algorithms.

Performance evaluation of parallel re-computing algorithm in different data distribution modes

With the rapid increase of spatial data resolution, the huge volume of datasets makes geo-computation more time-consuming especially when operating some complex algorithms, i.e. viewshed analysis and drainage network extraction in digital terrain analysis. Parallel computing is regarded as an efficient solution by utilizing more computing resources. Among them, the stable and credible services play an irreplaceable role in parallel computing, especially when an error occurs in the large-scale scientific computing. In this paper, a master/slave approach to implement the parallel re-computing is proposed based on redundancy mechanism. Once some errors in application layer are detected, the original data block with computation errors is further partitioned into several sub-blocks which are re-computed by the surviving processes concurrently to improve the efficiency of failure recovery. The multi-thread strategy in the main process is responsible for the distribution of data blocks, detecting errors and star...