Finite Element Tearing And Interconnecting Research Articles

Advanced numerical techniques for predicting frequency response functions

Numerical methods have been widely used to predict Frequency Response Functions (FRF) of structures. The FRF can be used to characterize structural dynamics and support sound and vibration control. Generally, frequency sweep is conducted in the FRF calculations to identify damping and resonant frequencies of structures under excitations. However, as numerical models are getting complex and larger, the demands for computational resources (e.g., CPU time, memory, disk space) are greatly increased. Lately, high performance computing (HPC) at the DoD HPC center has been used with advanced numerical techniques to reduce the overall computational time. Those advanced numerical techniques include Krylov subspace and Galerkin Projection (KGP) and Pade Approximation. Significantly CPU time reduction has been demonstrated for relatively smaller FE models. This paper will further discuss adaptive KGP by automating the frequency sweep process via calculated relative residues, then combine with Finite Element Tearing and Interconnecting (FETI) to solve for FRF of a large FEM model in the order of 60M DOF. A flat plate with viscoelastic layer will serve as an example to demonstrate accuracy and efficiency of AKGP with FETI. The benefits of using HPC with advanced numerical techniques to solve for large FEM models are clearly displayed.

A coupled SBFETI-BDEs method for solving 2D static and dynamic contact problem with friction

In this study, we propose a coupled approach called the SBFETI-BDEs method for solving the 2D elastic frictional contact problem. The contact system is decomposed into multiple subdomains, similar to the finite-element tearing and interconnecting (FETI) algorithm. Each subdomain can be regarded as a scaled boundary finite element (SBFE), where only the boundaries are discretized. To avoid computing the global contact flexibility matrix using the force method, we consider the generalized inverse of the stiffness matrix for each subdomain as a subcontact-flexibility matrix. We uniformly formulate the contact equations, displacement continuity equations, and the equilibrium equations of force for the floating subdomains, as a B-differentiable equations (BDEs) system, which we solve using the B-differentiable damped Newton-Raphson method that possesses a global convergence property. The proposed method offers increased computational efficiency without compromising accuracy when compared to the original BDEs algorithm without domain decomposition. We present the dynamic scheme of the SBFETI-BDEs method within the framework of the HHT time-integration scheme, where the equilibrium equations of force for the floating subdomains disappear due to their nonsingular equivalent stiffness matrices. Numerical examples are provided to demonstrate the convergence, accuracy, efficiency, and capability in solving engineering problems of the proposed method.

A coupled FETI-BDNM for solving 3D elastic frictional contact problem

Based on the B-differentiable Newton method (BDNM) and finite element tearing and interconnecting (FETI) algorithm, a coupled approach named FETI-BDNM is developed to solve the 3D elastic contact problem with friction. In this method, the contact bodies are decomposed into a number of subdomains. The interfaces between the two adjacent substructures will be skillfully considered as the special contact surfaces that are always in the stick state. By resorting to the generalized inverse of stiffness matrices of the subdomains, the relationships between the relative displacements and forces on contact/interface, i.e., contact flexibility matrices will be computed efficiently. The contact equations, displacement continuity equations and equilibrium equations for each subdomain are expressed in the form of B-differentiable equations (BDEs) uniformly, which can be solved by the damped Newton-Raphson method with 1D Armijo type linear search. The proposed method makes it possible to solve large-scale contact problems without compromising the solution accuracy compared to the original BDNM, and the good convergence of BDEs and parallel scalability of the FETI algorithm will be preserved. Numerical examples validate the accuracy, efficiency, robustness and effects of the number of subdomains on the computational time of the proposed method.

A Wave Equation-Based Hybridizable Discontinuous Galerkin–Robin Transmission Condition Algorithm for Electromagnetic Problems Analyzing

In this work, a wave-equation-based discontinuous Galerkin (DG) method hybridized with the Robin transmission condition (DG-RTC) is developed to solve the frequency-domain electromagnetic (EM) problems. The proposed DG method directly discretizes the vector electric field wave equation in each subdomain, and subsequently, a term named numerical flux is introduced at the subdomain interfaces to connect the solutions between neighboring subdomains. However, the numerical flux depends not only on the electric field <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {E}$ </tex-math></inline-formula> but also on the magnetic field <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {H}$ </tex-math></inline-formula> residing over the interface. Thereby, another equation is essential for solving <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {E}$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {H}$ </tex-math></inline-formula> simultaneously. Realizing that <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {H}$ </tex-math></inline-formula> only situates at the interface of adjacent subdomains, it is thus desired that the auxiliary equation also merely involves <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {H}$ </tex-math></inline-formula> at the interfaces. To reach this aim, at the subdomain interfaces, a Robin transmission condition (RTC) based on the tangential continuity of EM fields across the interface is introduced as the second equation. With this proposed DG-RTC algorithm, only <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {E}$ </tex-math></inline-formula> is a volume variable, while <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\mathbf {H}$ </tex-math></inline-formula> is a two-dimensional (2-D) one, and the degrees of freedom (DoFs) are thus greatly reduced compared with the traditional DG method. On the other hand, in the frequency domain, the established subdomain matrix equations are implicitly coupled with each other, resulting in a global matrix system. Directly solving it is not yet cheap. To alleviate the computational cost, a finite-element tearing and interconnecting (FETI)-like approach resorts. With the block-diagonal preconditioner and a restriction operator, the global matrix is decomposed into a surface matrix equation and several local matrix equations pertinent to each subdomain. In this way, a direct solver can be applied to solve these small matrix equations efficiently. To validate the proposed DG method in solving various EM problems, several representative examples are presented.

A Hybridizable Discontinuous Galerkin Time-Domain Method With Robin Transmission Condition for Transient Thermal Analysis of 3-D Integrated Circuits

In this article, a discontinuous Galerkin time-domain (DGTD) method hybridized with Robin transmission condition (RTC-DGTD) is proposed for the transient thermal analysis of 3-D integrated circuits (ICs). As a kind of domain decomposition method (DDM), the DGTD method employs a term named numerical flux for the information exchange of solutions between neighboring subdomains. The numerical flux is a function of both temperature <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> and heat flux q, and thus for the heat equation, apart from <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> , the variable q has to be solved as well. To reach this, the traditional DGTD method decomposes the second-order partial-differential thermal equation into two first-order ones, which results that both <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> and q need to be solved in the whole computational domain, thereby the computational cost is extremely expensive. In this work, to get this trouble-treated, the proposed DGTD method is directly applied to discretize the second-order heat equation. An RTC is established at the interfaces of subdomains, which serves as the second governing equation and meanwhile bridges another connection of solutions in adjacent subdomains. In this way, the new variable q is now only present at the interfaces of subdomains. Consequently, the total number of unknowns is significantly reduced compared with that in the traditional DGTD method. To further speed up the time-marching scheme, an adaptive time-stepping method in terms of the proportional–integral–derivative (PID) control algorithm is employed. Besides, by introducing a restriction operator, the globally coupled matrix equation is torn into a number of small matrix equations pertinent to each subdomain, which is later solved via a finite-element tearing and interconnecting (FETI)-like method. To validate and verify the accuracy and efficiency of the proposed algorithm, several representative examples are studied and compared with COMSOL simulations.

DC IR-Drop Analysis of Power Distribution Networks by a Robin Transmission Condition-Enhanced Discontinuous Galerkin Method

In this work, a novel Robin transmission condition (RTC)-enhanced discontinuous Galerkin (DG) method is proposed for the dc IR-drop analysis of power distribution networks with Joule heating effects included. Unlike the conventional DG method, the proposed DG method straightforwardly applied to discretize the second-order spatial partial differential governing equations for the electrostatic potential <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$\Phi $ </tex-math></inline-formula> and the steady-state temperature <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$T$ </tex-math></inline-formula> . The numerical flux in DG used to facilitate the information exchange among neighboring subdomains introduces two additional variables: the current density <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$J$ </tex-math></inline-formula> for the electrical potential equation and the thermal flux <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$q$ </tex-math></inline-formula> for the thermal equation. To solve them, at the interface of neighboring subdomains, an RTC is presented as the second equation to establish another connection for solutions in neighboring subdomains. With this strategy, the number of degrees of freedom (DoFs) involved in the proposed RTC-DG method is dramatically reduced compared with the traditional DG method. The finalized matrix system is solved in an finite-element tearing and interconnecting (FETI)-like procedure, namely the unknowns are obtained in a subdomain-by-subdomain scheme. Finally, the accuracy and the efficiency of the proposed RTC-DG method are validated by several representative examples.

Nonlinearly Preconditioned FETI Solver for Substructured Formulations of Nonlinear Problems

We consider the finite element approximation of the solution to elliptic partial differential equations such as the ones encountered in (quasi)-static mechanics, in transient mechanics with implicit time integration, or in thermal diffusion. We propose a new nonlinear version of preconditioning, dedicated to nonlinear substructured and condensed formulations with dual approach, i.e., nonlinear analogues to the Finite Element Tearing and Interconnecting (FETI) solver. By increasing the importance of local nonlinear operations, this new technique reduces communications between processors throughout the parallel solving process. Moreover, the tangent systems produced at each step still have the exact shape of classically preconditioned linear FETI problems, which makes the tractability of the implementation barely modified. The efficiency of this new preconditioner is illustrated on two academic test cases, namely a water diffusion problem and a nonlinear thermal behavior.

Schur complement spectral bounds for large hybrid FETI-DP clusters and huge three-dimensional scalar problems

Abstract Bounds on the spectrum of Schur complements of subdomain stiffness matrices with respect to the interior variables are key ingredients of the convergence analysis of FETI (finite element tearing and interconnecting) based domain decomposition methods. Here we give bounds on the regular condition number of Schur complements of ‘floating’ clusters arising from the discretization of 3D Laplacian on a cube decomposed into cube subdomains. The results show that the condition number of the cluster defined on a fixed domain decomposed into m × m × m cube subdomains connected by face and optionally edge averages increases proportionally to m. The estimates support scalability of unpreconditioned H-FETI-DP (hybrid FETI dual-primal) method. Though the research is most important for the solution of variational inequalities, the results of numerical experiments indicate that unpreconditioned H-FETI-DP with large clusters can be useful also for the solution of huge linear problems.

Efficient transient thermal simulation with Laguerre-based finite-element method and domain decomposition

In this article, a Laguerre-based finite-element method (LBFEM) combined with domain decomposition is proposed for large-scale transient thermal simulation. The unconditionally stable marching-on-in-order scheme is obtained using the weighted Laguerre polynomials and Galerkin testing procedure to eliminate the time variable of the temperature field. An effective data filtering method is utilized to reduce the number of basis functions and improve the efficiency of simulation. To handle the large-scale problems, a domain decomposition scheme named finite element tearing and interconnecting (FETI) is applied. The entire computational domain can be divided into a series of small-scale subdomains to reduce the computational resources. Numerical results are presented to demonstrate the accuracy and efficiency of the proposed method.

Bounds on the spectra of Schur complements of large H‐TFETI‐DP clusters for 2D Laplacian

AbstractBounds on the spectrum of Schur complements of subdomain stiffness matrices of the discretized Laplacian with respect to interior variables are important in the convergence analysis of finite element tearing and interconnecting (FETI)‐based domain decomposition methods. Here, we are interested in bounds on the regular condition number of Schur complements of “floating” clusters, that is, of matrices comprising the Schur complements of subdomains with prescribed zero Neumann conditions that are joined on the primal level by edge averages. Using some known results, angles of subspaces, and known bounds on the spectrum of Schur complements associated with square domains, we give bounds on the regular condition number of the Schur complement of some “floating” clusters arising from the discretization and decomposition of 2D Laplacian on domains comprising square subdomains. The results show that the condition number of the cluster defined on a fixed domain decomposed into m × m square subdomains joined by edge averages increases proportionally to m. The estimates are compared with numerical values and used in the analysis of H‐FETI‐DP methods. Though the research has been motivated by an effort to extend the scope of scalability of FETI‐based solvers to variational inequalities, the experiments indicate that H‐TFETI‐DP with large clusters can be useful for the solution of huge linear elliptic problems discretized by sufficiently regular grids.

Fast and Novel Computational Methods for Multi-scale and Multi-physics: FETI and POD-ROM

In this paper, the finite element tearing and interconnecting (FETI) method and proper orthogonal decomposition-based reduced order model (POD-ROM) are suggested as fast and novel computational approaches for multi-scale and multi-physics analysis. FETI is one of non-overlapping domain decomposition methods, facilitating the parallel computation. POD-ROM reduces the number of degrees of freedom (DoFs) to several representative POD modes which completely describe the behavior of the full order model. The computational time for the analyses using large number of DoFs can be decreased by using the previously mentioned methods. The present FETI-local and POD-ROM are found to reduce computational time significantly in each example.

Improvement of DDM Preprocessing for the Simulation of the GRAVES Radar Densified Sparse Array for Space Surveillance and Tracking

In a recent work (A. Jouade and A. Barka, “Massively parallel implementation of FETI-2LM methods for the simulation of the sparse receiving array evolution of the GRAVES radar system for space surveillance and tracking, IEEE Access , vol. 7, pp. 128968–128979, 2019), we have discussed our implementation of the finite-element tearing and interconnecting (FETI) method and the domain decomposition method (DDM) for the full-wave simulation of the large GRAVES sparse array (60 m diameter disk). Owing to the nonregular distribution of the antennas of the sparse array, such simulations in reasonable times are not accessible with existing FETI methods optimized for repetitive geometries, which do not benefit from an effective parallelization technique. A particularity of the preprocessing step of our DDM is that the complete sparse array mesh is not built upstream of the resolution of the electromagnetic problem. Each subdomain in our decomposition is equipped with its own local mesh, belonging to a limited set of unit-cell meshes, generated separately. Initially, a total of 13 692 subdomains were chosen among three types of subdomains, considering those corresponding to antennas (200), air subdomains (800), and ground plane subdomains (12 592). This simulation has required 13 692 Intel Xeon Broadwell E5-2680v4 processing cores equipped with 4 GB of memory, each handling one subdomain. A new strategy is shown, which requires only 2408 cores for the same simulation, leading to an 82.41% reduction in the required number of cores, as well as a 30% reduction in the simulation time.

Comparison of selected FETI coarse space projector implementation strategies

This paper deals with scalability improvements of the FETI (Finite Element Tearing and Interconnecting) domain decomposition method solving elliptic PDEs. The main bottleneck of FETI is the solution of a coarse problem that is part of the projector onto the natural coarse space. This paper introduces and compares two strategies for the FETI coarse problem solution. The first one is a classical solution with either direct (factorization + forward/backward substitutions) or iterative solvers (conjugate gradient and deflated conjugate gradient methods). The second one is the assembly of an explicit inverse using a direct solver with the coarse problem solution realised by dense matrix-vector products. MPI subcommunicators are employed to increase arithmetic intensity and, crucially, to decrease the communication cost. PERMON library for quadratic programming implementing the Total FETI variant of FETI was used for the numerical experiments.

An Iteration-Free Domain Decomposition Method for the Fast Finite Element Analysis of Electromagnetic Problems

An iteration-free finite element domain decomposition method (DDM) is proposed for the simulation of 3-D electromagnetic problems. The finite element tearing and interconnecting (FETI) with Robin-type transmission condition is introduced to construct a domain decomposition framework. Based on hierarchical ( $\mathcal {H}$ -) matrix algorithm, the numerical Green’s function of each subdomain can be represented by a data-sparse matrix form. Subsequently, the original 3-D problem is reduced to a 2-D problem on the skeleton. Derived from the finite element discretization of pseudo-differential operator, the skeleton matrix can also be efficiently approximated with an $\mathcal {H}$ -matrix by assembling all the numerical Green’s functions of subdomains. Based on $\mathcal {H}$ -LU factorization algorithm, the computational complexity and memory requirement for the direct solution of the skeleton equation can be reduced to be logarithmic linear. The resulting direct FETI (D-FETI) method completely depends on direct method without any iterative process. Hence, it does not suffer from the issue of convergence rate, and avoids redundant computations for multiple right-hand side problems. Numerical examples demonstrate the effectiveness of the proposed method.

Parallel computation for three-dimensional shell analysis of curved configuration based on domain decomposition method

In this paper, a parallel computational algorithm is developed based on finite element tearing and interconnecting (FETI) method, specifically, localized Lagrange multipliers. The proposed FETI method decomposes large-size structures into non-overlapping subdomains via localized Lagrange multipliers. To consider the curved configuration of large-size structures, a facet shell element created by combining an optimal triangle membrane and discrete Kirchhoff triangle bending plate (OPT-DKT) is suggested and used by introducing rotational operators. Moreover, practical performance of the present OPT-DKT facet shell element is evaluated through static and dynamic analysis. Finally, parallel computation is implemented for the proposed approach using message passing interface (MPI).

An efficient parallel algorithm for flexible multibody systems based on domain decomposition method

Recently, a large number of lightweight flexible structures are successfully used in the field of aerospace industry, intelligent robot and 3D printing. Because of the increasing flexibility of elastic components, some large scale mechanical systems will perform typical rigid-flexible coupled dynamic characteristic. Previous studies have indicated that the traditional modeling methods of the flexible multibody system based on the assumptions of small rotations and small deformation, such as the floating frame of reference method, can not lead to correct results for the dynamic response of those large deformation structures. Under the framework of Isogeometric analysis, the basis functions of the NURBS are employed to discretize the displacement field of elastic components with large deformation. Based on the large deformation theory of the continuum mechanics, the geometrical nonlinearity of planar structures undergoing the overall motion and large deformation is accurately taken into account. In order to improve the computational efficiency, some efficient formulations to calculate the elastic force and the tangent stiffness matrix are proposed via the invariant matrix method. Furthermore, based on the finite element tearing and interconnecting (FETI) method, an efficient parallel algorithm is presented to deal with the dynamic equations of motion for flexible multibody systems. First, the governing partial differential equations of multibody systems are transformed into a set of nonlinear algebraic equations after spatial and time discretization. Then, the preconditioned conjugate gradient method is exploited to solve the linearization equation in parallel. Compared with the existing parallel direct methods for flexible multibody systems, the proposed method can improve the computational efficiency significantly. Finally, several numerical examples are given to validate the effectiveness of the proposed parallel algorithm, including the complexity, the speed-up ratio and the scalability.

Elastic Image Registration Based on Domain Decomposition with Mesh Adaptation

Medical images are increasingly used within healthcare for diagnosis, planning treatment, and monitoring disease progression. The images acquired at different times, with different imaging modalities, from different subjects etc. often provide an additional clinical information that is not revealed in the separate images. The spatial relation between the images has to be found and this process is called image registration. In our contribution, we use elastic registration which assumes that the images are two different observations of an elastic body which is discretized by the finite element method. We are especially interested in the problems where the requirements on the registration prevent the application of standard FFT based solvers to the solution of auxiliary linear problems, which is the case when the part of the two observations can be related by a rigid body motion. Because the medical images usually contain a large area of background and a small area of changes, a regular discretization results in waste of computational resources due to the fine refinement of the space outside the region of interest (especially in 3D). To avoid this, we use coarser grid with local refinement that takes into account specific features of the images and their differences. The related elasticity problems are solved by TFETI, which is a variant of the Finite Element Tearing and Interconnecting (FETI) domain decomposition method for massively parallel numerical solution of elliptic Partial Differential Equations (PDE) with optimal complexity.

Multiphysics Simulation of 3-D ICs With Integrated Microchannel Cooling

A multiphysics-based co-simulation technique is developed based on the finite element method for the characterization of 3-D high-frequency integrated circuits (ICs) with integrated microchannel cooling. The multiphysics in the co-simulation includes full-wave electromagnetic, fluid, and transient conjugate heat transfer analyses. For incompressible and fully developed flows, the governing equations of fluid flow are decoupled from temperature, and the velocity field is obtained analytically. The full-wave electromagnetic and the transient conjugate heat transfer analyses are coupled through temperature-dependent material properties. The efficiency of the co-simulation is enhanced through an adaptive time-stepping scheme, a domain decomposition scheme called the finite element tearing and interconnecting (FETI), and FETI-enabled parallel computing. The capability and the efficiency of the proposed co-simulation in analyzing practical and large-scale 3-D ICs are demonstrated through numerical examples. The effectiveness of heat removal through integrated microchannel cooling and the thermal impact on high-frequency characteristics of 3-D IC designs are also demonstrated.

Implementation of the efficient communication layer for the highly parallel total FETI and hybrid total FETI solvers

This paper describes the implementation, performance, and scalability of our communication layer developed for Total FETI (TFETI) and Hybrid Total FETI (HTFETI) solvers. HTFETI is based on our variant of the Finite Element Tearing and Interconnecting (FETI) type domain decomposition method. In this approach a small number of neighboring subdomains is aggregated into clusters, which results in a smaller coarse problem. To solve the original problem TFETI method is applied twice: to the clusters and then to the subdomains in each cluster. The current implementation of the solver is focused on the performance optimization of the main CG iteration loop, including: implementation of communication hiding and avoiding techniques for global communications; optimization of the nearest neighbor communication - multiplication with a global gluing matrix; and optimization of the parallel CG algorithm to iterate over local Lagrange multipliers only. The performance is demonstrated on a linear elasticity 3D cube and real world benchmarks.

Hybrid parallelization of the total FETI solver

This paper describes our new hybrid parallelization of the Finite Element Tearing and Interconnecting (FETI) method for the multi-socket and multi-core computer cluster. This is an essential step in our development of the Hybrid FETI solver were small number of neighboring subdomains is aggregated into clusters and each cluster is processed by a single compute node.In our previous work we have implemented FETI solver using MPI parallelization into our ESPRESO solver. The proposed hybrid implementation provides better utilization of resources of modern HPC machines using advanced shared memory runtime systems such as Cilk++ runtime. Cilk++ is an alternative to OpenMP which is used by ESPRESO for shared memory parallelization.We have compared the performance of the hybrid parallelization to MPI-only parallelization. The results show that we have reduced both solver runtime and memory utilization. This allows a solver to use a larger number of smaller sub-domains and in order to solve larger problems using a limited number of compute nodes. This feature is essential for users with smaller computer clusters.In addition, we have evaluated this approach with large-scale benchmarks of size up to 1.3 billion of unknowns to show that the hybrid parallelization also reduces runtime of the FETI solver for these types of problems.