Large-scale Density Functional Theory Calculations Research Articles

Massively parallel implementation of iterative eigensolvers in large-scale plane-wave density functional theory

The Kohn-sham density functional theory (DFT) is a powerful method to describe the electronic structures of molecules and solids in condensed matter physics, computational chemistry and materials science. However, large and accurate DFT calculations within plane waves process a cubic-scaling computational complexity, which is usually limited by expensive computation and communication costs. The rapid development of high performance computing (HPC) on leadership supercomputers brings new opportunities for developing plane-wave DFT calculations for large-scale systems. Here, we implement parallel iterative eigensolvers in large-scale plane-wave DFT calculations, including Davidson, locally optimal block preconditioned conjugate gradient (LOBPCG), projected preconditioned conjugate gradient (PPCG) and the Chebyshev subspace iteration (CheFSI) algorithms, and analyze the performance of these algorithms in massively parallel plane-wave computing tasks. We adopt a two-level parallelization strategy that combines the message passing interface (MPI) with open multi-processing (OpenMP) parallel programming to handle data exchange and matrix operations in the construction and diagonalization of large-scale Hamiltonian matrix within plane waves. Numerical results illustrate that these iterative eigensolvers can scale up to 42,592 processing cores with high peak performance of 30% on leadship supercomputers to study the electronic structures of bulk silicon systems containing 10,648 atoms. Program summaryProgram Title: Plane wave density functional theory (PWDFT)CPC Library link to program files:https://doi.org/10.17632/c8v2mx5vn4.1Developer's repository link:https://bitbucket.org/berkeleylab/scalesLicensing provisions: BSD 3-clauseProgramming language: C++Nature of problem: PWDFT is used for electronic structure calculations based on Kohn-Sham density functional theory. The key challenge to address is a constrained energy minimization problem, which can also be formulated as a nonlinear eigenvalue problem. MPI/OpenMP-based approaches are employed to provide multi-core acceleration for the study of the chemical and material properties of larger-scale molecules and solids.Solution method: PWDFT implements self-consistent field (SCF) iterations and direct constrained minimization algorithms with various acceleration strategies. It is written in C++ and offers parallel acceleration based on MPI/OpenMP.

Theoretical search for characteristic atoms in supported gold nanoparticles: a large-scale DFT study.

The size and site dependences of atomic and electronic structures in isolated and supported gold nanoparticles have been investigated using large-scale density functional theory (DFT) calculations using multi-site support functions. The effects of the substrate on nanoparticles with diameters of 2 nm and several different shapes have been examined. First, isolated gold nanoparticles with diameters of 0.6 nm (13 atoms) to 4.5 nm (2057 atoms), which have comparable sizes to nanoparticles used in experiments, were considered. To analyse huge amounts of data obtained from large-scale DFT calculations, we performed principal component analysis (PCA), which helps systematically and efficiently clarify the electronic structures of large nanoparticles. The PCA results reveal the site dependence of the electronic structures. Notably, the atoms in the surface and subsurface have different electronic structures to those located in the inner layers, especially at the vertexes of the particles. The convergence of local electronic structures with respect to the particle size has also been demonstrated. For supported nanoparticles, PCA helps indicate which atoms are affected, and how much, by the substrate. The correlation between the PCA results and site dependence of reaction activity is also discussed herein.

Large-scale DFT calculations of multi-component glass systems (SiO2)0.70(Al2O3)0.13(XO)0.17 (X = Mg, Ca, Sr, Ba) : Accuracy of classical force fields

Although molecular dynamics (MD) simulation is a powerful tool for investigating the atomic-scale structures of complex materials, several challenges limit their reliable and accurate application to multi-component glass systems. The available force fields (FFs) that can treat many elements in a multi-component glass are limited, and even if such a FF exists, its accuracy is suspicious due to the large variety and complexity of chemical environments in these materials. First-principles calculations based on the density functional theory (DFT) are reliable, but prohibitively expensive with conventional methods. In this study, we use large-scale DFT techniques and demonstrate that it is possible to perform efficient and accurate DFT calculations of multi-component glass systems, such as (SiO2)0.70(Al2O3)0.13 (XO)0.17 (X= Mg, Ca, Sr, Ba), containing about 1000–5000 atoms. From the results of large-scale DFT calculations, we evaluate the accuracy of some classical FFs, and show that the accuracy for non-bridging oxygen atoms is very low especially when the SiO distance is short. Large differences in the distribution of SiOSi angles observed in the FF-MD and DFT-MD simulations and the unique electronic structure in the case of X= Mg are also discussed.

Isomeric Excitation Energy for ^{99}In^{m} from Mass Spectrometry Reveals Constant Trend Next to Doubly Magic ^{100}Sn.

The excitation energy of the 1/2^{-} isomer in ^{99}In at N=50 is measured to be 671(37)keV and the mass uncertainty of the 9/2^{+} ground state is significantly reduced using the ISOLTRAP mass spectrometer at ISOLDE/CERN. The measurements exploit a major improvement in the resolution of the multireflection time-of-flight mass spectrometer. The results reveal an intriguing constancy of the 1/2^{-} isomer excitation energies in neutron-deficient indium that persists down to the N=50 shell closure, even when all neutrons are removed from the valence shell. This trend is used to test large-scale shell model, abinitio, and density functional theory calculations. The models have difficulties describing both the isomer excitation energies and ground-state electromagnetic moments along the indium chain.

Finite Difference Interpolation for Reduction of Grid-Related Errors in Real-Space Pseudopotential Density Functional Theory.

The real-space pseudopotential approach is a well-known method for large-scale density functional theory (DFT) calculations. One of its main limitations, however, is the introduction of errors associated with the positioning of the underlying real-space grid, a phenomenon usually known as the "egg-box" effect. The effect can be controlled by using a finer grid, but this raises the cost of the calculations or even undermines their feasibility altogether. Therefore, there is ongoing interest in the reduction of the effect per a given real-space grid. Here, we present a finite difference interpolation of electron orbitals as a means of exploiting the high resolution of the pseudopotential to reduce egg-box effects systematically. We implement the method in PARSEC, a finite difference real-space pseudopotential DFT code, and demonstrate error mitigation and improved convergence at a low additional computational cost.

Electron/Hole Mobilities of Periodic DNA and Nucleobase Structures from Large-Scale DFT Calculations.

Electron/hole transfer mechanisms in DNA and polynucleotide structures continue to garner considerable interest as emerging charge-transport systems and molecular electronics. To shed mechanistic insight into these electronic properties, we carried out large-scale density functional theory (DFT) calculations (up to 650 atoms) to systematically analyze the structural and electron/hole transport properties of fully periodic single- and double-stranded DNA. We examined the performance of various exchange-correlation functionals (LDA, BLYP, B3LYP, and B3LYP-D) and found that single-stranded thymine (T) and cytosine (C) are predominantly hole conductors, whereas single-stranded adenine (A) and guanine (G) are better electron conductors. For double-stranded DNA structures, the periodic A-T and G-C electronic band structures undergo a significant renormalization, which causes hole transport to only occur on the A and G nucleobases. Our calculations (1) provide new benchmarks for periodic nucleobase structures using dispersion-corrected hybrid functionals with large basis sets and (2) highlight the importance of dispersion effects for obtaining accurate geometries and electron/hole mobilities in these extended systems.

Gaussian-Approximated Poisson Preconditioner for Iterative Diagonalization in Real-Space Density Functional Theory

Various real-space methods optimized on massive parallel computers have been developed for efficient large-scale density functional theory (DFT) calculations of materials and biomolecules. The iterative diagonalization of the Hamiltonian matrix is a computational bottleneck in real-space DFT calculations. Despite the development of various iterative eigensolvers, the absence of efficient real-space preconditioners has hindered their overall efficiency. An efficient preconditioner must satisfy two conditions: appropriate acceleration of the convergence of the iterative process and inexpensive computation. This study proposed a Gaussian-approximated Poisson preconditioner (GAPP) that satisfied both conditions and was suitable for real-space methods. A low computational cost was realized through the Gaussian approximation of a Poisson Green's function. Fast convergence was achieved through the proper determination of Gaussian coefficients to fit the Coulomb energies. The performance of GAPP was evaluated for several molecular and extended systems, and it showed the highest efficiency among the existing preconditioners adopted in real-space codes.

Accelerating self-consistent field iterations in Kohn-Sham density functional theory using a low-rank approximation of the dielectric matrix

We present an efficient preconditioning technique for accelerating the fixed point iteration in real-space Kohn-Sham density functional theory (DFT) calculations. The preconditioner uses a low rank approximation of the dielectric matrix (LRDM) based on G\^ateaux derivatives of the residual of fixed point iteration along appropriately chosen direction functions. We develop a computationally efficient method to evaluate these G\^ateaux derivatives in conjunction with the Chebyshev filtered subspace iteration procedure, an approach widely used in large-scale Kohn-Sham DFT calculations. Further, we propose a variant of LRDM preconditioner based on adaptive accumulation of low-rank approximations from previous SCF iterations, and also extend the LRDM preconditioner to spin-polarized Kohn-Sham DFT calculations. We demonstrate the robustness and efficiency of the LRDM preconditioner against other widely used preconditioners on a range of benchmark systems with sizes ranging from $\sim$ 100-1100 atoms ($\sim$ 500--20,000 electrons). The benchmark systems include various combinations of metal-insulating-semiconducting heterogeneous material systems, nanoparticles with localized $d$ orbitals near the Fermi energy, nanofilm with metal dopants, and magnetic systems. In all benchmark systems, the LRDM preconditioner converges robustly within 20--30 iterations. In contrast, other widely used preconditioners show slow convergence in many cases, as well as divergence of the fixed point iteration in some cases. Finally, we demonstrate the computational efficiency afforded by the LRDM method, with up to 3.4$\times$ reduction in computational cost for the total ground-state calculation compared to other preconditioners.

Built-in electric field-assisted W-C3/X-C3 van der Waals heterogeneous single-atom catalysts for enhanced electrocatalytic nitrogen reduction

Electrocatalytic nitrogen reduction reaction (eNRR) is considered an alternative approach to the Haber-Bosch process for green ammonia synthesis. However, developing NRR electrocatalysts with high activity and selectivity remains a great challenge. Here, we designed a new type of van der Waals heterogeneous single-atom catalysts (vdW-SACs): W-C3/X-C3 (X = TM, Be, B, C, N, O, P, S, and Cl). By means of large-scale density functional theory (DFT) calculations, we built up the full profile of the stability, NRR activity, and selectivity of these vdW-SACs. Five structures exhibit lower overpotential for NRR than the previously reported monolayer W1C3. A volcano-shaped relationship between ΔG∗N and UL was obtained, and particularly, the W-C3/P-C3 locates right on the peak of the volcano with the highest activity. The built-in electric field induced by the asymmetric active sites (W and P) adjusts the binding strength between reactants and catalysts surface and achieves the lowest overpotential. The electronic structure analysis discloses that the high degree of pd hybridization between W 5d orbitals and N2π* antibonding orbitals in W-C3/P-C3 promotes the activation of intermediates. These new insights may open up opportunities for exploring advanced electrocatalysts for NRR and beyond by regulating the built-in electric field of vdW-SACs.

Small Heterocyclic Ligands as Anticancer Agents: QSAR with a Model G-Quadruplex.

G-quadruplexes (GQs) have become valid targets for anticancer studies in recent decades due to their multifaceted biological function. Herewith, we aim to quantify interactions of potential heterocyclic ligands (Ls) with model GQs. For seven 4-aminoquinazolines and three 2-heteroaryl perimidines, seven of this ten-membered group so far unknown, we use routine quantum chemical modeling. As shown in the literature, a preferred mode of interaction of heterocycles with cellular structures is stacking to exposable faces of G-quadruplexes. To exploit the energy of this interaction as a molecular descriptor and achieve the necessary chemical precision, we use state of the art large-scale density functional theory (DFT) calculations of stacked heterocycles to a GQ. Actually, the GQ has been simplified for the computation by stripping it off all pentose phosphate residues into a naked model of stacked guanine quartets. The described model thus becomes computable. The obtained heterocyclic ligand GQ.L stacking energies, that is, their GQ affinities, are the necessary ligand descriptors. Using the ligand biological inhibitory activities (IC50) on a human malignant melanoma A375 cell line, we obtain a good linear relationship between computed ligand stacking affinities to GQ, and experimental log (IC50) values. Based on the latter relationship, we discuss a putative mechanism of anticancer activity of heterocyclic ligands via stacking interactions with GQs and thereby controlling cell regulatory activity. This mechanism may tentatively be applied to other condensed five- and six-membered small heterocycles as well.

Modulating the microscopic lattice distortions through the Al-rich layers for boosting the ferroelectricity in Al:HfO2 nanofilms

Combining the experimental characterization with the large-scale density functional theory calculations based on finite-element discretization (DFT-FE), we address the stabilization of polar orthorhombic phases (o-HfO2) in Al:HfO2 nanofilms by means of the atomic registry distortions and lattice deformation caused by Al substitutional defects (AlHf) and Schottky defects (2AlHf + VO) in tetragonal phases (t-HfO2) or monoclinic phases (m-HfO2). The phase transformation directly from the t-HfO2 into polar o-HfO2 are also elucidated within a heterogeneous distribution of Al dopants in both t-HfO2 bulk crystal structure and Al:HfO2 nanofilm. It is revealed using large-scale DFT calculations that the Al substitutional defects (AlHf) or the Schottky defect (2AlHf + VO) could induce the highly extended atomic registry distortions or lattice deformation in the t- and m-HfO2 phases, but such effects are greatly diminished in ferroelectric orthorhombic phase. By purposely engineering the multiple AlHf defects to form dopant-rich layers in paraelectric t-HfO2 nanofilm or bulk crystal, the induced extended lattice distortions surrounding the defect sites exhibit the shearing-like atomic displacement vector field. The large-scale DFT calculations further predicted that the shearing-like microscopic lattice distortions could directly induce the phase transformation from the t-HfO2 into polar orthorhombic phase in both Al:HfO2 bulk crystal and nanofilms, leading to the large remanent polarization observed in Al:HfO2 nanofilms with the presence of Al-rich layers. The current study demonstrates that the ferroelectricity of HfO2 bulk crystal or thin film can be optimized and tuned by delicately engineering both the distribution and concentration of Al dopants in atomic layer deposition without applying the top capping electrode, providing the extra flexibility for designing the HfO2 based electronic devices in the future.

Large-Scale DFT Methods for Calculations of Materials with Complex Structures

Large-scale density functional theory (DFT) calculations provide a powerful tool to investigate the atomic and electronic structure of materials with complex structures. This article reviews a large-scale DFT calculation method, the multi-site support function (MSSF) method, in the CONQUEST code. MSSFs are linear combinations of the basis functions which belong to a group of atoms in a local region. The method can reduce the computational time while preserving accuracy. The accuracy of MSSFs has been assessed for bulk Si, Al, Fe and NiO and hydrated DNA, which demonstrate the applicability of the MSSFs for varied materials. The applications of MSSFs on large systems with several thousand atoms, which have complex interfaces and non-periodic structures, indicate that the MSSF method is promising for precise investigations of materials with complex structures.

Electric fields control water-gated proton transfer in cytochrome c oxidase

Aerobic life is powered by membrane-bound enzymes that catalyze the transfer of electrons to oxygen and protons across a biological membrane. Cytochrome c oxidase (CcO) functions as a terminal electron acceptor in mitochondrial and bacterial respiratory chains, driving cellular respiration and transducing the free energy from O2 reduction into proton pumping. Here we show that CcO creates orientated electric fields around a nonpolar cavity next to the active site, establishing a molecular switch that directs the protons along distinct pathways. By combining large-scale quantum chemical density functional theory (DFT) calculations with hybrid quantum mechanics/molecular mechanics (QM/MM) simulations and atomistic molecular dynamics (MD) explorations, we find that reduction of the electron donor, heme a, leads to dissociation of an arginine (Arg438)-heme a3 D-propionate ion-pair. This ion-pair dissociation creates a strong electric field of up to 1 V Å-1 along a water-mediated proton array leading to a transient proton loading site (PLS) near the active site. Protonation of the PLS triggers the reduction of the active site, which in turn aligns the electric field vectors along a second, "chemical," proton pathway. We find a linear energy relationship of the proton transfer barrier with the electric field strength that explains the effectivity of the gating process. Our mechanism shows distinct similarities to principles also found in other energy-converting enzymes, suggesting that orientated electric fields generally control enzyme catalysis.

DFT-FE 1.0: A massively parallel hybrid CPU-GPU density functional theory code using finite-element discretization

We present DFT-FE 1.0, building on DFT-FE 0.6 Motamarri et al. (2020) [28], to conduct fast and accurate large-scale density functional theory (DFT) calculations (reaching ∼100,000 electrons) on both many-core CPU and hybrid CPU-GPU computing architectures. This work involves improvements in the real-space formulation—via an improved treatment of the electrostatic interactions that substantially enhances the computational efficiency—as well high-performance computing aspects, including the GPU acceleration of all the key compute kernels in DFT-FE. We demonstrate the accuracy by comparing the ground-state energies, ionic forces and cell stresses on a wide-range of benchmark systems against those obtained from widely used DFT codes. Further, we demonstrate the numerical efficiency of our GPU acceleration, which yields ∼20× speed-up on hybrid CPU-GPU nodes of the Summit supercomputer. Notably, owing to the parallel-scaling of the GPU implementation, we obtain wall-times of 80−140 seconds for full ground-state calculations, with stringent accuracy, on benchmark systems containing ∼6,000−15,000 electrons using 64−224 nodes of the Summit supercomputer. Program summaryProgram Title: DFT-FECPC Library link to program files:https://doi.org/10.17632/c5ghfc6ctn.1Developer's repository link:https://github.com/dftfeDevelopers/dftfeLicensing provisions: LGPL v3Programming language: C/C++External routines/libraries: p4est (http://www.p4est.org/), deal.II (https://www.dealii.org/), BLAS (http://www.netlib.org/blas/), LAPACK (http://www.netlib.org/lapack/), ELPA (https://elpa.mpcdf.mpg.de/), ScaLAPACK (http://www.netlib.org/scalapack/), Spglib (https://atztogo.github.io/spglib/), ALGLIB (http://www.alglib.net/), LIBXC (http://www.tddft.org/programs/libxc/), PETSc (https://www.mcs.anl.gov/petsc), SLEPc (http://slepc.upv.es), NCCL (optional-https://github.com/NVIDIA/nccl).Nature of problem: Density functional theory calculations.Solution method: We employ a local real-space variational formulation of Kohn-Sham density functional theory that is applicable for both pseudopotential and all-electron calculations on periodic, semi-periodic and non-periodic geometries. Higher-order adaptive spectral finite-element basis is used to discretize the Kohn-Sham equations. Chebyshev polynomial filtered subspace iteration procedure (ChFSI) is employed to solve the nonlinear Kohn-Sham eigenvalue problem self-consistently. ChFSI in DFT-FE employs Cholesky factorization based orthonormalization, and spectrum splitting based Rayleigh-Ritz procedure in conjunction with mixed precision arithmetic. Configurational force approach is used to compute ionic forces and periodic cell stresses for geometry optimization.Additional comments including restrictions and unusual features: Exchange correlation functionals are restricted to Local Density Approximation (LDA) and Generalized Gradient Approximation (GGA), with and without spin. The pseudopotentials available are optimized norm conserving Vanderbilt (ONCV) pseudopotentials and Troullier–Martins (TM) pseudopotentials. Calculations are non-relativistic.DFT-FE handles all-electron and pseudopotential calculations in the same framework, while accommodating periodic, non-periodic and semi-periodic boundary conditions.

Mixed magnetic edge states in graphene quantum dots

Graphene quantum dots (GQDs) exhibit abundant magnetic edge states with promising applications in spintronics. Hexagonal zigzag GQDs possess a ground state with an antiferromagnetic (AFM) inter-edge coupling, followed by a metastable state with ferromagnetic (FM) inter-edge coupling. By analyzing the Hubbard model and performing large-scale spin-polarized density functional theory calculations containing thousands of atoms, we predict a series of new mixed magnetic edge states of GQDs arising from the size effect, namely mix-n, where n is the number of spin arrangement parts at each edge, with parallel spin in the same part and anti-parallel spin between adjacent parts. In particular, we demonstrate that the mix-2 state of bare GQDs (C) appears when and the mix-3 state appears when , where N is the number of six-membered-ring at each edge, while the mix-2 and mix-3 magnetic states appear in the hydrogenated GQDs with N = 13 and N = 15, respectively.

Can a node-less wave function have higher energy than node-full ones?

“The energy level of an electron state increases as the number of nodes in its wave function increases.” The preceding statement, often found in textbooks, was challenged by our large-scale DFT (density-functional theory) calculations performed for the decavacancy in Si crystal.

Highly accurate machine learning model for kinetic energy density functional

Inspired by the remarkable ongoing progress of data-driven approaches, a very accurate predictive model is developed to estimate one-dimensional kinetic energy density functionals (KEDF) using Machine Learning (ML). Starting from possible analytical forms of kinetic energy density and by utilizing a variety of solvable models, a simple – yet highly – accurate linear regression model is statistically trained to estimate the kinetic energy as functionals of the density. The mean relative accuracy for even a small number of randomly generated potentials is found to be better than the standard KEDF (Thomas-Fermi (TF) and von Weizsäcker (vW)) by several orders of magnitudes. As more different potentials of model problems are mixed, the coefficients of the linear model significantly approach the known values of Thomas-Fermi and von Weizsäcker, suggesting the reliability of the statistical training approach. This work can provide an important step toward more accurate large-scale orbital free density functional theory (OF-DFT) calculations.

Tensor-structured algorithm for reduced-order scaling large-scale Kohn\u2013Sham density functional theory calculations

We present a tensor-structured algorithm for efficient large-scale density functional theory (DFT) calculations by constructing a Tucker tensor basis that is adapted to the Kohn–Sham Hamiltonian and localized in real-space. The proposed approach uses an additive separable approximation to the Kohn–Sham Hamiltonian and an L1 localization technique to generate the 1-D localized functions that constitute the Tucker tensor basis. Numerical results show that the resulting Tucker tensor basis exhibits exponential convergence in the ground-state energy with increasing Tucker rank. Further, the proposed tensor-structured algorithm demonstrated sub-quadratic scaling with system-size for both systems with and without a gap, and involving many thousands of atoms. This reduced-order scaling has also resulted in the proposed approach outperforming plane-wave DFT implementation for systems beyond 2000 electrons.

Ab initio predictions of graphite-like phase with anomalous grain boundaries and flexoelectricity from collapsed carbon nanotubes.

Although large-radius carbon nanotubes (CNTs) are now available in macroscopic quantities, little is known about their condensed phase. Large-scale density functional theory calculations predict a low energy phase in which the same-diameter "dog-bone" collapsed CNTs form a graphite-like phase with complex, anomalous grain boundaries (GBs). The excess GB volume does not prevent the strong van der Waals coupling of the flattened CNT sides into AB stacking. The associated GB energetics is dominated by the van der Waals energy penalty and high curvature bending of the loop CNT edges, which exhibit reactivity and flexoelectricity. The large density and superior mechanical rigidity of the proposed microstructural organization as well as the GB flexoelectricity are desirable properties for developing ultra-strong composites based on large-radius CNTs.

Parallel Implementation of Large-Scale Linear Scaling Density Functional Theory Calculations With Numerical Atomic Orbitals in HONPAS.

Linear-scaling density functional theory (DFT) is an efficient method to describe the electronic structures of molecules, semiconductors, and insulators to avoid the high cubic-scaling cost in conventional DFT calculations. Here, we present a parallel implementation of linear-scaling density matrix trace correcting (TC) purification algorithm to solve the Kohn–Sham (KS) equations with the numerical atomic orbitals in the HONPAS package. Such a linear-scaling density matrix purification algorithm is based on the Kohn's nearsightedness principle, resulting in a sparse Hamiltonian matrix with localized basis sets in the DFT calculations. Therefore, sparse matrix multiplication is the most time-consuming step in the density matrix purification algorithm for linear-scaling DFT calculations. We propose to use the MPI_Allgather function for parallel programming to deal with the sparse matrix multiplication within the compressed sparse row (CSR) format, which can scale up to hundreds of processing cores on modern heterogeneous supercomputers. We demonstrate the computational accuracy and efficiency of this parallel density matrix purification algorithm by performing large-scale DFT calculations on boron nitrogen nanotubes containing tens of thousands of atoms.