Hybrid MPI Research Articles

Surface geometry inversion of transient electromagnetic data

We investigate an emerging method called surface geometry inversion (SGI) for the inversion of transient electromagnetic (TEM) data. Conventional minimum-structure inversion methods parameterize the earth model with many mesh cells within which the physical properties are constant, and construct a physical property model that is usually smoothly varying as well as fits the observations. With these smooth models, it is difficult to extract the interface between different geologic units, and it can be especially difficult to target drillholes for thin, plate-like targets, which are frequently encountered in mineral exploration. Our SGI parameterizes the model in terms of the coordinates of the nodes (vertices) used to connect together the surfaces that define the geologic interfaces. The algorithm then inverts for the locations of these nodes, which directly provides geometric information about the target. This can be more useful than a fuzzy image of conductivity, especially for an exploration project. A genetic algorithm (GA) is used to solve the nonlinear, overdetermined optimization problem. We use a finite-element solver to solve the TEM forward-modeling problem of each candidate model in the GA population. Because forward modeling is independent for each model, we implement a hybrid message passing interface (MPI) + OpenMP parallel method to improve computational efficiency. We investigate a new parameterization method specifically designed for thin, plate-like structures, which is more efficient and can effectively avoid self-intersection. We first illustrate the effectiveness of our SGI algorithm on a synthetic block model before testing the new parameterization method on a synthetic thin plate model. Finally, we apply our SGI to a real data set collected for the exploration of thin graphitic faults in a uranium exploration project in Canada. The constructed model from our SGI corresponds well with the drilling data.

TARS: A parallel tetrahedral discontinuous finite element code for the solution of the discrete ordinates neutron transport equation

In this study, we introduce TARS, a neutron transport code based on the discrete-ordinates discontinuous finite element (SN-DFEM) method, which tackles existing challenges in the field by providing accurate and efficient solutions to the neutron transport equation. The use of DFEM enables higher-order spatial discretization, improving numerical stability, while parallelizing transport sweeping reduces computational time and resource requirements. TARS' performance was evaluated using the Takeda benchmark problems. The results showed that TARS generates highly accurate solutions, with eigenvalue errors no more than 90 pcm compared to MC solutions. The relative differences of region-averaged fluxes were below 1% in most regions, except those with severe flux gradients. Parallel calculation efficiency analysis revealed a PCE of over 70% even at 1024 CPU cores. However, hybrid MPI and OpenMP parallel programming did not outperform pure MPI programming, suggesting that the latter might be more suitable for TARS.

An Alternate GPU-Accelerated Algorithm for Very Large Sparse LU Factorization

The LU factorization of very large sparse matrices requires a significant amount of computing resources, including memory and broadband communication. A hybrid MPI + OpenMP + CUDA algorithm named SuperLU3D can efficiently compute the LU factorization with GPU acceleration. However, this algorithm faces difficulties when dealing with very large sparse matrices with limited GPU resources. Factorizing very large matrices involves a vast amount of nonblocking communication between processes, often leading to a break in SuperLU3D calculation due to the overflow of cluster communication buffers. In this paper, we present an improved GPU-accelerated algorithm named SuperLU3D_Alternate for the LU factorization of very large sparse matrices with fewer GPU resources. The basic idea is “divide and conquer”, which means dividing a very large matrix into multiple submatrices, performing LU factorization on each submatrix, and then assembling the factorized results of all submatrices into two complete matrices L and U. In detail, according to the number of available GPUs, a very large matrix is first divided into multiple submatrices using the elimination tree. Then, the LU factorization of each submatrix is alternately computed with limited GPU resources, and its intermediate LU factors from GPUs are saved to the host memory or hard disk. Finally, after finishing the LU factorization of all submatrices, these factorized submatrices are assembled into a complete lower triangular matrix L and a complete upper triangular matrix U, respectively. The SuperLU3D_Alternate algorithm is suitable for hybrid CPU/GPU cluster systems, especially for a subset of nodes without GPUs. To accommodate different hardware resources in various clusters, we designed the algorithm to run in the following three cases: sufficient memory for GPU nodes, insufficient memory for GPU nodes, and insufficient memory for the entire cluster. The results from LU factorization test on different matrices in various cases show that the larger the matrix is, the more efficient this algorithm is under the same GPU memory consumption. In our numerical experiments, SuperLU3D_Alternate achieves speeds of up to 8× that of SuperLU3D (CPU only) and 2.5× that of SuperLU3D (CPU + GPU) on the hybrid cluster with six Tesla V100S GPUs. Furthermore, when the matrix is too big to be handled by SuperLU3D, SuperLU3D_Alternate can still utilize the cluster’s host memory or hard disk to solve it. By reducing the amount of data exchange to prevent exceeding the buffer’s limit of the cluster MPI nonblocking communication, our algorithm enhances the stability of the program.

Hybrid MPI and CUDA paralleled finite volume unstructured CFD simulations on a multi-GPU system

Porting unstructured Computational Fluid Dynamics (CFD) analysis of compressible flow to Graphics Processing Units (GPUs) confronts two difficulties. Firstly, non-coalescing access to the GPU’s global memory is induced by indirect data access leading to performance loss. Secondly, data exchange among multi-GPU is complex due to data communication between processes and transfer between host and device, which degrades scalability. For increasing data locality on unstructured finite volume GPU simulations for compressible flow, we perform some optimizations, including cell and face renumbering, data dependence resolving, nested loops split, and loop mode adjustment. Then, a hybrid MPI-CUDA parallel framework with packing and unpacking exchange data on GPU is established for multi-GPU computing. Finally, after optimizations, the performance of the whole application on a GPU is increased by around 50%. Simulations of ONERA M6 cases on a single GPU (Nvidia Tesla V100) can achieve an average of 13.4 speedup compared to those on 28 CPU cores (Intel Xeon Gold 6132). On the baseline of 2 GPUs, strong scaling results show a parallel efficiency of 42% on 200 GPUs, while weak scaling tests give a parallel efficiency of 82.4% up to 200 GPUs.

Parallelization of Array Method with Hybrid Programming: OpenMP and MPI

For parallelization of applications with high processing times and large amounts of storage in High Performance Computing (HPC) systems, shared memory programming and distributed memory programming have been used; a parallel application is represented by Parallel Task Graphs (PTGs) using Directed Acyclic Graphs (DAGs). For the execution of PTGs in HPC systems, a scheduler is executed in two phases: scheduling and allocation; the execution of the scheduler is considered an NP-complete combinatorial problem and requires large amounts of storage and long processing times. Array Method (AM) is a scheduler to execute the task schedule in a set of clusters; this method was programmed sequentially, analyzed and tested using real and synthetic application workloads in previous work. Analyzing the proposed designs of this method in this research work, the parallelization of the method is extended using hybrid OpenMP and MPI programming in a server farm and using a set of geographically distributed clusters; at the same time, a novel method for searching free resources in clusters using Lévy random walks is proposed. Synthetic and real workloads have been experimented with to evaluate the performance of the new parallel schedule and compare it to the sequential schedule. The metrics of makespan, waiting time, quality of assignments and search for free resources were evaluated; the results obtained and described in the experiments section show a better performance with the new version of the parallel algorithm compared to the sequential version. By using the parallel approach with hybrid programming applied to the extraction of characteristics of the PTGs, applied to the search for geographically distributed resources with Lévy random walks and applied to the metaheuristic used, the results of the metrics are improved. The makespan is decreased even when the loads increase, the times of the tasks in the waiting queue are decreased, the quality of assignments in the clusters is improved by causing the tasks with their subtasks to be assigned in the same clusters or in cluster neighbors and, finally, the searches for free resources are executed in different geographically distributed clusters, not sequentially.

HSMA: An O(N) electrostatics package implemented in LAMMPS

We implement two recently developed fast Coulomb solvers, HSMA3D (Zhao et al. (2018) [49]) and HSMA2D (Liang et al. (2020) [50]), into a new user package HSMA for the molecular dynamics simulation engine LAMMPS. The HSMA package is designed for efficient and accurate modeling of electrostatic interactions in 3D and 2D periodic systems with dielectric effects at O(N) cost. The implementation is hybrid MPI and OpenMP parallelized and compatible with existing LAMMPS functionalities. The vectorization technique following AVX512 instructions is adopted for acceleration. To establish the validity of our implementation, we have presented extensive comparisons to the widely used particle-particle particle-mesh (PPPM) algorithm in LAMMPS and other dielectric solvers. With the proper choice of algorithm parameters and parallelization setup, the package enables calculations of electrostatic interactions that is efficient especially for systems with dielectric mismatch. Program summaryProgram Title: HSMACPC Library link to program files:https://doi.org/10.17632/z39zyh573s.1Developer's repository link:https://github.com/LiangJiuyang/HSMA-Harmonic-surface-mapping-algorithm-in-LAMMPSLicensing provisions: GPLv3Programming language: C++Nature of problem: Evaluation of long-range electrostatic interactions for charged system with fully periodic condition or confined by planar dielectric interfaces.Solution method: We implement the Harmonic Surface Mapping algorithm (HSMA), which combines the image-charge method with the harmonic surface mapping and converts the contribution of infinite images into a finite number of surface charges on an auxiliary sphere, into simulation package LAMMPS. The HSMA package works for both fully periodic systems and partially periodic systems with planar dielectric interfaces, achieving truly linear O(N) complexity by employing fast multipole method. Our package can be applied to general all-atom simulations and a broad range of charged complex fluids under dielectric confinement.Additional comments including restrictions and unusual features: Hybrid MPI plus OpenMP parallelization is used in HSMA package for high-performance computing. We provide vector optimization by using the AVX512 instructions and Intel MKL library for further acceleration. The Intel Parallel Studio is required for these techniques.

Parallel algorithms for solving diffusion-convection problems on a multiprocessor computer system using hybrid MPI / OpenMP technology

The paper deals with the mathematical models, algorithms and software for mathematical modeling of coastal systems’ water pollution spreading dynamics under various unfavorable phenomena of natural and artificial genesis, developed for high-performance cluster systems. Methods for partitioning the computational domain for solving diffusion-convection problems have been developed, which allow for efficient parallelization of a computationally complex modeling problem, taking into account the architecture of the multiprocessor system used. The developed mathematical models are based on high-precision models of hydrophysics and hydrobiology and take into account the peculiarities of water systems in the south of the Rostov region, as well as factors of hydrobiological dynamics such as microturbulent diffusion and advective transport in various directions, mechanisms of primary and secondary pollution of coastal systems, taking into account currents. The paper presents algorithms for solving a simulated problem based on MPI parallelization technology, as well as based on mixed MPI + OpenMP technology. Numerical experiments have been carried out and the two technologies efficiency comparison has been made in the conditions of computing cluster used.

Hybrid MPI and OpenMP parallel implementation of large-scale linear-response time-dependent density functional theory with plane-wave basis set

High performance computing is a powerful tool to accelerate the Kohn–Sham density functional theory calculations on modern heterogeneous supercomputers. Here, we describe a massively parallel implementation of large-scale linear-response time-dependent density functional theory (LR-TDDFT) to calculate the excitation energies and wave functions of solids with plane-wave basis set. We adopt a two-level parallelization strategy that combines the message passing interface with open multi-processing parallel programming to deal with the matrix operations and data communications of constructing and diagonalizing the LR-TDDFT Hamiltonian matrix. Numerical results illustrate that the LR-TDDFT calculations can scale up to 24 576 processing cores on modern heterogeneous supercomputers to study the excited state properties of bulky silicon systems containing thousands of atoms (4,096 atoms). We demonstrate that the LR-TDDFT calculations can be used to investigate the photoinduced charge separation of water molecule adsorption on rutile TiO2(110) surface from an excitonic perspective.

Perturbo: A software package for ab initio electron–phonon interactions, charge transport and ultrafast dynamics

Perturbo is a software package for first-principles calculations of charge transport and ultrafast carrier dynamics in materials. The current version focuses on electron–phonon interactions and can compute phonon-limited transport properties such as the conductivity, carrier mobility and Seebeck coefficient. It can also simulate the ultrafast nonequilibrium electron dynamics in the presence of electron–phonon scattering. Perturbo uses results from density functional theory and density functional perturbation theory calculations as input, and employs Wannier interpolation to reduce the computational cost. It supports norm-conserving and ultrasoft pseudopotentials, spin–orbit coupling, and polar electron–phonon interactions for bulk and 2D materials. Hybrid MPI plus OpenMP parallelization is implemented to enable efficient calculations on large systems (up to at least 50 atoms) using high-performance computing. Taken together, Perturbo provides efficient and broadly applicable ab initio tools to investigate electron–phonon interactions and carrier dynamics quantitatively in metals, semiconductors, insulators, and 2D materials. Program summaryProgram Title:PerturboCPC Library link to program files:https://doi.org/10.17632/34m2p6v79t.1Developer’s repository link:https://perturbo-code.github.ioLicensing provisions: GNU General Public Licence 3.0Programming language:Fortran, PythonExternal routines/libraries:LAPACK, HDF5, MPI, OpenMP, FFTW, Quantum-ESPRESSO, Wannier90Nature of problem: Computing transport properties from first-principles in materials, including the electrical conductivity, carrier mobility and Seebeck coefficient; Simulating ultrafast nonequilibrium electron dynamics, such as the relaxation of excited carriers via interactions with phonons.Solution method: We implement the first-principles Boltzmann transport equation, which employs materials properties such as the electronic structure, lattice dynamics, and electron–phonon collision terms computed with density functional theory and density functional perturbation theory. The Boltzmann transport equation is solved numerically to compute charge transport and simulate ultrafast carrier dynamics. Wannier interpolation is employed to reduce the computational cost.Additional comments: Hybrid MPI plus OpenMP parallelization is implemented to run large calculations and take advantage of high-performance computing. Most results are output to HDF5 file format, which is portable and convenient for post-processing using high-level languages such as Python and Julia.

Hybrid MPI and CUDA Parallelization for CFD Applications on Multi-GPU HPC Clusters

Graphics processing units (GPUs) have a strong floating-point capability and a high memory bandwidth in data parallelism and have been widely used in high-performance computing (HPC). Compute unified device architecture (CUDA) is used as a parallel computing platform and programming model for the GPU to reduce the complexity of programming. The programmable GPUs are becoming popular in computational fluid dynamics (CFD) applications. In this work, we propose a hybrid parallel algorithm of the message passing interface and CUDA for CFD applications on multi-GPU HPC clusters. The AUSM + UP upwind scheme and the three-step Runge–Kutta method are used for spatial discretization and time discretization, respectively. The turbulent solution is solved by the K−ω SST two-equation model. The CPU only manages the execution of the GPU and communication, and the GPU is responsible for data processing. Parallel execution and memory access optimizations are used to optimize the GPU-based CFD codes. We propose a nonblocking communication method to fully overlap GPU computing, CPU_CPU communication, and CPU_GPU data transfer by creating two CUDA streams. Furthermore, the one-dimensional domain decomposition method is used to balance the workload among GPUs. Finally, we evaluate the hybrid parallel algorithm with the compressible turbulent flow over a flat plate. The performance of a single GPU implementation and the scalability of multi-GPU clusters are discussed. Performance measurements show that multi-GPU parallelization can achieve a speedup of more than 36 times with respect to CPU-based parallel computing, and the parallel algorithm has good scalability.

Effects of total pressures and equivalence ratios on kerosene/air rotating detonation engines using a paralleling CE/SE method

In this paper, the kerosene/air rotating detonation engines(RDE) are numerically investigated, and the emphasis is laid on the effects of total pressures and equivalence ratios on the operation characteristics of RDE including the initiation, instabilities, and propulsive performance. A hybrid MPI + OpenMP parallel computing model is applied and it is proved to be able to obtain a more effective parallel performance on high performance computing(HPC) systems. A series of cases with the total pressure of 1 MPa, 1.5 MPa, 2 MPa, and the equivalence ratio of 0.9, 1, 1.4 are simulated. On one hand, the total pressure shows a significant impact on the instabilities of rotating detonation waves. The instability phenomenon is observed in cases with low total pressure (1 MPa) and weakened with the increase of the total pressure. The total pressure has a small impact on the detonation wave velocity and the specific impulse. On the other hand, the equivalence ratio shows a negligible influence on the instabilities, while it affects the ignition process and accounts for the detonation velocity deficit. It is more difficult to initiate rotating detonation waves directly in the lean fuel operation condition. Little difference was observed in the thrust with different equivalence ratios of 0.9, 1, and 1.4. The highest specific impulse was obtained in the lean fuel cases, which is around 2700 s. The findings could provide insights into the understanding of the operation characteristics of kerosene/air RDE.

Reproducibility of parallel preconditioned conjugate gradient in hybrid programming environments

The Preconditioned Conjugate Gradient method is often employed for the solution of linear systems of equations arising in numerical simulations of physical phenomena. While being widely used, the solver is also known for its lack of accuracy while computing the residual. In this article, we propose two algorithmic solutions that originate from the ExBLAS project to enhance the accuracy of the solver as well as to ensure its reproducibility in a hybrid MPI + OpenMP tasks programming environment. One is based on ExBLAS and preserves every bit of information until the final rounding, while the other relies upon floating-point expansions and, hence, expands the intermediate precision. Instead of converting the entire solver into its ExBLAS-related implementation, we identify those parts that violate reproducibility/non-associativity, secure them, and combine this with the sequential executions. These algorithmic strategies are reinforced with programmability suggestions to assure deterministic executions. Finally, we verify these approaches on two modern HPC systems: both versions deliver reproducible number of iterations, residuals, direct errors, and vector-solutions for the overhead of less than 37.7% on 768 cores.

Three-Dimensional Reverse Time Migration of Ground-Penetrating Radar Signals

Three-dimensional (3D) ground-penetrating radar (GPR) systems and 3D seismic imaging techniques have been developing fast and evolving rapidly in the last decade. Ray-based migration methods have been successfully applied to processing 3D GPR signals based on the similarity between electromagnetic and seismic waves. However, reverse time migration (RTM) of 3D GPR signals has not been well studied in the past. In this paper, we present a 3D RTM based on Maxwell’s equations for 3D GPR surveys. Migration recovers the true subsurface structure from a distorted and unfocused time profile and suppresses common electromagnetic clutter and noise. RTM based on Maxwell’s equations can consider conductivity directly and compensate for the attenuation within a high-conductivity zone. Compared with 2D RTM, 3D RTM back-propagates both in-line and cross-line signals simultaneously and can include complex 3D permittivity and conductivity models. We have integrated a parallel finite-difference time-domain (FDTD) algorithm based on a hybrid MPI and OpenMP scheme to reduce the computational cost of 3D problems. The 3D RTM experiments on an anomaly of “EM” shape and a realistic sand dune model demonstrate the effective recovery of 3D subsurface structures.

Improvement of workload balancing using parallel loop self-scheduling on Intel Xeon Phi

In recent years, Intel promotes its new product Xeon Phi coprocessor, which is similar to the x86 architecture coprocessor. It has about 60 cores and can be regarded as a single computing node, with the computing power that cannot be ignored. This work aims to improve the workload balance by parallel loop self-scheduling scheme performed on Xeon Phi-based computer cluster. The proposed concept is implemented by hybrid MPI and OpenMP parallel programming in C language. Since parallel loop self-scheduling composes of static and dynamic allocation, weighting algorithm is adopted in the static part, while the well-known loop self-scheduling is adopted in dynamic part. The loop block is partitioned according to the weighting of MIC and HOST nodes. Accordingly, Xeon Phi with many-core is adopted to implement parallel loop self-scheduling. Finally, we test the performance in the experiments by four applicable problems: matrix multiplication, sparse matrix multiplication, Mandelbrot set and circuit meet. The experimental results indicate how to do the weight allocation and which scheduling method can achieve the best performance.

A conservative scheme for Vlasov Poisson Landau modeling collisional plasmas

We have developed a deterministic conservative solver for the inhomogeneous Fokker–Planck–Landau equation coupled with the Poisson equation, which is a classical mean-field primary model for collisional plasmas. Two subproblems, i.e. the Vlasov–Poisson problem and homogeneous Landau problem, are obtained through time-splitting methods, and treated separately by the Runge–Kutta Discontinuous Galerkin method and a conservative spectral method, respectively. To ensure conservation when projecting between the two different computing grids, a special conservation routine is designed to link the solutions of these two subproblems. This conservation routine accurately enforces conservation of moments in Fourier space. The entire numerical scheme is implemented with parallelization with hybrid MPI and OpenMP. Numerical experiments are provided to study linear and nonlinear Landau Damping problems and two-stream flow problem as well.

Embedded multicore computing and applications

Embedded multicore computing and applications

An approach of performance comparisons with OpenMP and CUDA parallel programming on multicore systems

SummaryIn the past, the tenacious semiconductor problems of operating temperature and power consumption limited the performance growth for single‐core microprocessors. Microprocessor vendors hence adopt the multicore chip organizations with parallel processing because the new technology promises faster and lower power needed. In a short time, this trend floods first the development of CPU, then also the other peripherals like GPU. Modern GPUs are very efficient in manipulating computer graphics, and their highly parallel structure makes them even more effective than general‐purpose CPUs for a range of graphical complex algorithms. However, technology of multicore processor brought revolution and unavoidable collision to the programming personnel. Multicore processor has high performance; however, parallel processing brings not only the opportunity but also a challenge. The issue of efficiency and the way how programmer or compiler parallelizes the software explicitly are the keys that enhance the performance on multicore chip. In this paper, we propose a parallel programming approach using hybrid CUDA, OpenMP, and MPI programming. There would be two verificational experiments presented in the paper. In the first, we would verify the availability and correctness of the auto‐parallel tools, and discuss the performance issues on CPU, GPU, and embedded system. In the second, we would verify how the hybrid programming could surely improve performance. Copyright © 2016 John Wiley & Sons, Ltd.

BLESS 2: accurate, memory-efficient and fast error correction method.

The most important features of error correction tools for sequencing data are accuracy, memory efficiency and fast runtime. The previous version of BLESS was highly memory-efficient and accurate, but it was too slow to handle reads from large genomes. We have developed a new version of BLESS to improve runtime and accuracy while maintaining a small memory usage. The new version, called BLESS 2, has an error correction algorithm that is more accurate than BLESS, and the algorithm has been parallelized using hybrid MPI and OpenMP programming. BLESS 2 was compared with five top-performing tools, and it was found to be the fastest when it was executed on two computing nodes using MPI, with each node containing twelve cores. Also, BLESS 2 showed at least 11% higher gain while retaining the memory efficiency of the previous version for large genomes. Freely available at https://sourceforge.net/projects/bless-ec dchen@illinois.edu Supplementary data are available at Bioinformatics online.

TOD-Tree: Task-Overlapped Direct Send Tree Image Compositing for Hybrid MPI Parallelism and GPUs.

Modern supercomputers have thousands of nodes, each with CPUs and/or GPUs capable of several teraflops. However, the network connecting these nodes is relatively slow, on the order of gigabits per second. For time-critical workloads such as interactive visualization, the bottleneck is no longer computation but communication. In this paper, we present an image compositing algorithm that works on both CPU-only and GPU-accelerated supercomputers and focuses on communication avoidance and overlapping communication with computation at the expense of evenly balancing the workload. The algorithm has three stages: a parallel direct send stage, followed by a tree compositing stage and a gather stage. We compare our algorithm with radix-k and binary-swap from the IceT library in a hybrid OpenMP/MPI setting on the Stampede and Edison supercomputers, show strong scaling results and explain how we generally achieve better performance than these two algorithms. We developed a GPU-based image compositing algorithm where we use CUDA kernels for computation and GPU Direct RDMA for inter-node GPU communication. We tested the algorithm on the Piz Daint GPU-accelerated supercomputer and show that we achieve performance on par with CPUs. Last, we introduce a workflow in which both rendering and compositing are done on the GPU.

Distributed multi-scale muscle simulation in a hybrid MPI–CUDA computational environment

We present Mexie, an extensible and scalable software solution for distributed multi-scale muscle simulations in a hybrid MPI–CUDA environment. Since muscle contraction relies on the integration of physical and biochemical properties across multiple length and time scales, these models are highly processor and memory intensive. Existing parallelization efforts for accelerating multi-scale muscle simulations imply the usage of expensive large-scale computational resources, which produces overwhelming costs for the everyday practical application of such models. In order to improve the computational speed within a reasonable budget, we introduce the concept of distributed calculations of multi-scale muscle models in a mixed CPU–GPU environment. The concept is applied to a two-scale muscle model, in which a finite element macro model is coupled with the microscopic Huxley kinetics model. Finite element calculations of a continuum macroscopic model take place strictly on the CPU, while numerical solutions of the partial differential equations of Huxley’s cross-bridge kinetics are calculated on both CPUs and GPUs. We present a modular architecture of the solution, along with an internal organization and a specific load balancer that is aware of memory boundaries in such a heterogeneous environment. Solution was verified on both benchmark and real-world examples, showing high utilization of involved processing units, ensuring high scalability. Speed-up results show a boost of two orders of magnitude over any previously reported distributed multi-scale muscle models. This major improvement in computational feasibility of multi-scale muscle models paves the way for new discoveries in the field of muscle modeling and future clinical applications.