Oak Ridge Leadership Computing Facility Research Articles

Stabilized bases for high-order, interpolation semi-Lagrangian, element-based tracer transport

In a computational fluid model of the atmosphere, the advective transport of trace species, or tracers, can be computationally expensive. For efficiency, models often use semi-Lagrangian advection methods. High-order interpolation semi-Lagrangian (ISL) methods, in particular, can be extremely efficient, if the problem of property preservation specific to them can be addressed. Atmosphere models often use geometrically and logically nonuniform grids for efficiency and, as a result, element-based discretizations. Such grids and discretizations make stability a particular problem for ISL methods. Generally, high-order, element-based ISL methods that use the natural polynomial interpolant associated with a nodal finite-element discretization are unstable. We derive new bases having order of accuracy up to nine, with positive nodal weights, that stabilize the element-based ISL method. We use these bases to construct the linear advection operator in the property-preserving Interpolation Semi-Lagrangian Element-based Transport (Islet) method. Then we discuss key software implementation details. Finally, we show performance results for the Energy Exascale Earth System Model's atmosphere dynamical core, comparing the original and new transport methods. These simulations used up to 27,600 Graphical Processing Units (GPU) on the Oak Ridge Leadership Computing Facility's Summit supercomputer.

Early experiences evaluating the HPE/Cray ecosystem for AMD GPUs

SummaryThe Oak Ridge Leadership Computing Facility (OLCF) has a long history of supporting and promoting GPU‐accelerated computing starting with the deployment of the Titan supercomputer in 2021 and continuing with the Summit supercomputer which has a theoretical peak performance of approximately 200 petaflops. Because the majority of Summit's computational power comes from its 27,972 GPUs, users must port their applications to one of the supported programming models in order to make efficient use of the system. To prepare the transition to Frontier, the OLCF's exascale supercomputer, users will need to adapt to an entirely new ecosystem which will include new hardware and software technologies. First, users will need to familiarize themselves with the AMD Radeon GPU architecture. Furthermore, users who have been previously relying on CUDA will need to transition to the Heterogeneous‐Computing Interface for Portability (HIP) or one of the other supported programming models (e.g., OpenMP, OpenACC). In this work, we describe our initial experiences and lessons learned in porting three applications or proxy apps currently running on Summit to the HPE/Cray ecosystem to leverage the compute power from AMD GPUs: minisweep, GenASiS, and Sparkler. Each one is representative of current production workloads utilized at the OLCF, different programming languages, and different programming models.

Early experiences on the OLCF Frontier system with AthenaPK and Parthenon‐Hydro

SummaryThe Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, “real‐world” user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per‐cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini‐application stressing the core functionality of a performance portable block‐structured adaptive mesh refinement framework, Parthenon‐Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon‐Hydro miniapp reaches a total of zone‐cycles/s on 9216 nodes (73,728 logical GPUs) at 92% weak scaling parallel efficiency (starting from a single node using a second‐order, finite‐volume method).

Exascale Computational Fluid Dynamics in Heterogeneous Systems

Abstract Exascale computing has extended the reach of resolved flow simulations in complex, heterogeneous systems far beyond conventional computational fluid dynamics capabilities. As a result, unprecedented pore and microscale resolution have been achieved in domains that have been traditionally modeled by, and limited to, continuum, effective medium approaches. By making use of computational resources on the new exascale supercomputer, Frontier, at the Oak Ridge Leadership Computing Facility, we performed flow simulations that have pushed the limits of domain-to-resolution ratios by several orders of magnitude for heterogeneous media. Our approach is an incompressible, Navier–Stokes CFD solver based on adaptive, embedded boundary (EB) methods supported by the Chombo software framework for applied partial differential equations (PDEs). The computational workhorse in the CFD application code is an elliptic solver framework in Chombo for pressure-Poisson and viscous, Helmholtz terms that leverages a PETSc-hypre software interface tuned for accelerator-based platforms. We demonstrate scalability of the approach by replicating a unit cylinder packed with microspheres to achieve over 400 × 109 degrees-of-freedom simulated. These simulations model domain lengths of over 20 meters with channel volumes of over 400 cm3 and containing millions of packed spheres with 20 micron grid resolution, challenging current understanding of what it means to be a representative elementary volume (REV) of the continuum scale in heterogeneous media. We also simulate a range of Reynolds numbers to demonstrate wide applicability and robustness of the approach.

A step towards the final frontier: Lessons learned from acceptance testing of the first HPE/Cray EX 3000 system at ORNL

SummaryIn this article, we summarize the deployment of the Air Force Weather (AFW) HPC11 system at Oak Ridge National Laboratory (ORNL) including the process followed to successfully complete acceptance testing of the system. HPC11 is the first HPE/Cray EX 3000 system that has been successfully released to its user community in a federal facility. HPC11 consists of two identical 800‐node supercomputers, Fawbush and Miller, with access to two independent and identical lustre parallel file systems. HPC11 is equipped with Slingshot 10 interconnect technology and relies on the HPE Performance Cluster Manager software for system configuration. ORNL has a clearly defined acceptance testing process used to ensure that every new system deployed can provide the necessary capabilities to support user workloads. We worked closely with HPE and AFW to develop a set of tests that used the United Kingdom's Meteorological Office's Unified Model and 4‐dimensional variational data assimilation. We also included benchmarks and applications from the Oak Ridge Leadership Computing Facility portfolio to fully exercise the HPE/Cray programming environment and evaluate the functionality and performance of the system. Acceptance testing of HPC11 required parallel execution of each element on Fawbush and Miller. In addition, careful coordination was needed to ensure successful acceptance of the newly deployed lustre file systems alongside the compute resources. In this work, we present test results from specific system components and provide an overview of the issues identified, challenges encountered, and the lessons learned along the way.

Two excited-state datasets for quantum chemical UV-vis spectra of organic molecules

We present two open-source datasets that provide time-dependent density-functional tight-binding (TD-DFTB) electronic excitation spectra of organic molecules. These datasets represent predictions of UV-vis absorption spectra performed on optimized geometries of the molecules in their electronic ground state. The GDB-9-Ex dataset contains a subset of 96,766 organic molecules from the original open-source GDB-9 dataset. The ORNL_AISD-Ex dataset consists of 10,502,904 organic molecules that contain between 5 and 71 non-hydrogen atoms. The data reveals the close correlation between the magnitude of the gaps between the highest occupied molecular orbital (HOMO) and the lowest unoccupied molecular orbital (LUMO), and the excitation energy of the lowest singlet excited state energies quantitatively. The chemical variability of the large number of molecules was examined with a topological fingerprint estimation based on extended-connectivity fingerprints (ECFPs) followed by uniform manifold approximation and projection (UMAP) for dimension reduction. Both datasets were generated using the DFTB+ software on the “Andes” cluster of the Oak Ridge Leadership Computing Facility (OLCF).

A SCALABLE TRANSFORMER MODEL FOR REAL-TIME DECISION MAKING IN NEUTRON SCATTERING EXPERIMENTS

The U.S. Department of Energy's (DOE's) neutron research facilities at Oak Ridge National Laboratory (ORNL), including the High Flux Isotope Reactor (HFIR) and the Spallation Neutron Source (SNS), are a state-of-the-art neutron scattering facility that allows researchers to study the structure and dynamics of materials at the atomic scale. At the SNS, neutrons are measured using the time-of-flight (TOF) technique as they move through a neutron beamline to interact with a sample. Large volumes of neutron scattering data are collected and recorded in neutron event mode. Optimal productivity of the TOF instrument is limited due to the lack of real-time data analysis tools. The large amount of data generated by the experiments can be challenging to process and analyze in real time, particularly for experiments that require rapid feedback and adjustment of experimental parameters. The regular computer/workstation cannot keep up with the experiment speed to provide real-time feedback to adjust experimental parameters, so connecting the supercomputers available to the neutron facility is necessary to achieve real-time data analysis and experiment steering. To address this challenge, we exploit the Frontier supercomputer at Oak Ridge Leadership Computing Facility (OLCF) to train a scalable temporal fusion transformer model for real-time decision making of TOF neutron scattering experimentation. In this paper, we present the results using Frontier to provide the processing power needed to rapidly process and analyze large volumes of single-crystal diffraction data collected at TOPAZ, a neutron time-of-flight Laue single-crystal diffractometer at the SNS.

An OpenMP GPU-offload implementation of a non-equilibrium solidification cellular automata model for additive manufacturing

In this paper, performance strategies on GPU-based HPC platforms of a cellular automata (CA) simulation code for non-equilibrium solidification, including nucleation, grain growth, solute partitioning and transport for the metal additive manufacturing (AM) process are investigated using OpenMP 4.5. To accurately report the speed-up for multicore CPUs and GPUs, a rigorous performance analysis employed optimizations appropriate for both CPU-only code (baseline) and GPU offload codes for an isothermal test problem. The performance results on Summit at the Oak Ridge Leadership Computing Facility indicate that using a precomputed list of interface cells significantly decreased the wall-clock time on GPUs. The speedup due to GPU acceleration was evaluated for a full Summit node and measured to be 1.8X when comparing a 6 MPI tasks run with 6 GPUs versus 36 MPI tasks on the CPU only. That speed-up was found to be 7.9X when comparing 6 MPI tasks with 6 GPUs versus the 6 MPI tasks running on the CPU only. Performance measurements showed that system total time is almost constant for runs with more than 96 MPI tasks (or GPUs), indicating that the GPU-accelerated code showed an excellent weak scaling performance.Finally, a rapid directional solidification problem was considered to demonstrate the CA code capability on Summit. It was found that a mesh size of at least 0.05 μm is recommended for the AM-like simulations in order to obtain accurate elongated grain microstructure and elongated subgrain features, which are in qualitative good agreement with experimental data. The results presented in this study indicate that the performance strategies on GPU-based HPC platforms for the CA code are appropriate for novel HPC exascale platforms.

Programming approaches for scalability, performance, and portability of combustion physics codes

This paper presents the process, strategy, and results associated with porting a typical combustion physics flow solver to current state-of-the-art and future massively-parallel computer architectures. Major focus is placed on the distinct algorithmic structure of these types of codes and how it can be integrated with modern programming paradigms for heterogeneous platforms (i.e., distributed many-core systems with accelerators). An end-to-end case study is presented that exemplifies the process in a generic manner, which then serves as a clear guide with respect to the strategy and best practices leading to a robust and adaptable framework that performs well, is durable over time, is portable, and requires minimal human-effort. This end is accomplished beginning with the use of a mature, validated, structured, multiblock code framework optimized for application of both Large Eddy Simulation (LES) and Direct Numerical Simulation (DNS). This code has been ported to a variety of platforms over the past decade, including most recently the Oak Ridge Leadership Computing Facility’s “Summit” Platform. The experience gained on these multiple platforms provides general insights and thus the results presented are not specific to any one code or platform other than the overarching trend toward distributed many-core systems with accelerators in order to move toward exascale performance. The resultant performance and scalability of the ported code is demonstrated on a real-world application; a state-of-the-art rotating detonation rocket engine simulation that matches the complex geometry and boundary conditions imposed as part of a companion experimental campaign.

Scalable training of graph convolutional neural networks for fast and accurate predictions of HOMO-LUMO gap in molecules

Graph Convolutional Neural Network (GCNN) is a popular class of deep learning (DL) models in material science to predict material properties from the graph representation of molecular structures. Training an accurate and comprehensive GCNN surrogate for molecular design requires large-scale graph datasets and is usually a time-consuming process. Recent advances in GPUs and distributed computing open a path to reduce the computational cost for GCNN training effectively. However, efficient utilization of high performance computing (HPC) resources for training requires simultaneously optimizing large-scale data management and scalable stochastic batched optimization techniques. In this work, we focus on building GCNN models on HPC systems to predict material properties of millions of molecules. We use HydraGNN, our in-house library for large-scale GCNN training, leveraging distributed data parallelism in PyTorch. We use ADIOS, a high-performance data management framework for efficient storage and reading of large molecular graph data. We perform parallel training on two open-source large-scale graph datasets to build a GCNN predictor for an important quantum property known as the HOMO-LUMO gap. We measure the scalability, accuracy, and convergence of our approach on two DOE supercomputers: the Summit supercomputer at the Oak Ridge Leadership Computing Facility (OLCF) and the Perlmutter system at the National Energy Research Scientific Computing Center (NERSC). We present our experimental results with HydraGNN showing (i) reduction of data loading time up to 4.2 times compared with a conventional method and (ii) linear scaling performance for training up to 1024 GPUs on both Summit and Perlmutter.

GenASiSBasics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 4)

GenASiSBasics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 4)

Artificial intelligence-driven thermal design for additively manufactured reactor cores

The paper describes application of machine learning (ML), specifically deep learning and Gaussian processes, to optimize the design of complex nuclear engineering systems for which predictive, but computationally intensive, multiphysics solutions are available. The approach combines reduced-order modeling, simulation, and ML for computational design. High-fidelity reactor thermal hydraulics (TH) simulations are utilized effectively, minimizing the computational and physical costs of large-scale simulations and expensive prototyping, by using reduced-order models to explore the design space. Models are bias corrected by ML error correction using information from the high-fidelity simulations. This method draws on the uncertainty quantification intrinsic to Gaussian processes to provide the best predictions of optimal designs with error bounds. Based on the multi-objective loss function, the method proposes a new set of simulations that maximizes reduction of uncertainty in the regions identified as the most promising to minimize loss.This work explores the automatic construction of physics-informed ML methods using emulators validated by very sparse sampling of coupled thermal diffusion and viscous, turbulent flow solutions. The emulators become more accurate as new data are generated by steady-state reduced-order modeling, in which an individual design can be executed on a single graphics processing unit (GPU) in a few minutes. The parallel nature of probing the design space results in almost half a million different reactor designs that can be evaluated in an hour on Summit, an Oak Ridge Leadership Computing Facility (OLCF) supercomputer.

Comparative evaluation of deep learning workloads for leadership-class systems

Deep learning (DL) workloads and their performance at scale are becoming important factors to consider as we design, develop and deploy next-generation high-performance computing systems. Since DL applications rely heavily on DL frameworks and underlying compute (CPU/GPU) stacks, it is essential to gain a holistic understanding from compute kernels, models, and frameworks of popular DL stacks, and to assess their impact on science-driven, mission-critical applications. At Oak Ridge Leadership Computing Facility (OLCF), we employ a set of micro and macro DL benchmarks established through the Collaboration of Oak Ridge, Argonne, and Livermore (CORAL) to evaluate the AI readiness of our next-generation supercomputers. In this paper, we present our early observations and performance benchmark comparisons between the Nvidia V100 based Summit system with its CUDA stack and an AMD MI100 based testbed system with its ROCm stack. We take a layered perspective on DL benchmarking and point to opportunities for future optimizations in the technologies that we consider.

AbacusSummit: a massive set of high-accuracy, high-resolution N-body simulations

ABSTRACT We present the public data release of the AbacusSummit cosmological N-body simulation suite, produced with the Abacus N-body code on the Summit supercomputer of the Oak Ridge Leadership Computing Facility. Abacus achieves $\mathcal {O}(10^{-5})$ median fractional force error at superlative speeds, calculating 70M particle updates per second per node at early times, and 45M particle updates per second per node at late times. The simulation suite totals roughly 60 trillion particles, the core of which is a set of 139 simulations with particle mass $2\times 10^{9}\, h^{-1}\, \mathrm{M}_\odot$ in box size $2\, h^{-1}\, \mathrm{Gpc}$. The suite spans 97 cosmological models, including Planck 2018, previous flagship simulation cosmologies, and a linear derivative and cosmic emulator grid. A subsuite of 1883 boxes of size $500\, h^{-1}\, \mathrm{Mpc}$ is available for covariance estimation. AbacusSummit data products span 33 epochs from z = 8 to 0.1 and include light cones, full particle snapshots, halo catalogues, and particle subsets sampled consistently across redshift. AbacusSummit is the largest high-accuracy cosmological N-body data set produced to date.

Charmed and ϕ meson decay constants from 2+1-flavor lattice QCD * *Supported by the National Key Research and Development Program of China (2017YFB0203200). This work was partially supported by the National Natural Science Foundation of China (NSFC) (11935017). This research used resources of the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory, which is supported by the

On a lattice with 2+1-flavor dynamical domain-wall fermions at the physical pion mass, we calculate the decay constants of , , and . The lattice size is , which corresponds to a spatial extension of fm, with a lattice spacing of fm. For the valence light, strange, and charm quarks, we use overlap fermions at several mass points close to their physical values. Our results at the physical point are MeV, MeV, MeV, MeV, and MeV. The couplings of and to the tensor current ( ) can be derived from ratios and , respectively, which are the first lattice quantum chromodynamics (QCD) results. We also obtain ratios and , which reflect the size of heavy quark symmetry breaking in charmed mesons. Ratios and can be taken as a measure of SU(3) flavor symmetry breaking.

Global adjoint tomography—model GLAD-M25

SUMMARYBuilding on global adjoint tomography model GLAD-M15, we present transversely isotropic global model GLAD-M25, which is the result of 10 quasi-Newton tomographic iterations with an earthquake database consisting of 1480 events in the magnitude range 5.5 ≤ Mw ≤ 7.2, an almost sixfold increase over the first-generation model. We calculated fully 3-D synthetic seismograms with a shortest period of 17 s based on a GPU-accelerated spectral-element wave propagation solver which accommodates effects due to 3-D anelastic crust and mantle structure, topography and bathymetry, the ocean load, ellipticity, rotation and self-gravitation. We used an adjoint-state method to calculate Fréchet derivatives in 3-D anelastic Earth models facilitated by a parsimonious storage algorithm. The simulations were performed on the Cray XK7 ‘Titan’ and the IBM Power 9 ‘Summit’ at the Oak Ridge Leadership Computing Facility. We quantitatively evaluated GLAD-M25 by assessing misfit reductions and traveltime anomaly histograms in 12 measurement categories. We performed similar assessments for a held-out data set consisting of 360 earthquakes, with results comparable to the actual inversion. We highlight the new model for a variety of plumes and subduction zones.

(Invited) Functional Defects By Design—a High-Throughput Approach to Energy Materials Discovery

Defects and impurities introduce localized heterogeneities in solids and decisively control the behavior of a wide range of energy technologies. Fuel cell materials, especially proton conducting fuel cells, are a quintessential example in this regard. Designing and developing solid oxide materials that can selectively transport protons will enable us to develop the next generation proton conducting solid oxide fuel cells. Protons require less activation energy compared to oxygen ions which results in a lower operating temperature, higher operating efficiency and better material reliability. In this work [1,2,3,4] we focus obtaining fundamental insights on how properties of host material structure along with dopants, disorder and strain influences proton transport properties in solid oxides by coupling high-throughput computations with functional imaging, neutron spectroscopy and transport measurements.We initially focus on the perovskite family of compounds (such as doped BaZrO3). We benchmark our calculations against a wide range of experimental measurements such as kelvin probe force microscopy (KPFM), inelastic neutron scattering (INS) and atom probe tomography (APT). To obtain better insights on why certain cubic perovskite/dopant combinations are better at conducting protons compared to others, we developed a high-throughput framework to perform ab initio calculations. The high-throughput framework can scale massively to tens of thousands of nodes to fully exploit the computational capability of the Oak Ridge Leadership Computing facility. We employ this approach to calculate proton transport properties in several cubic perovskite materials with different host atoms and dopants. The results obtained from these calculations enables us to obtain better insights on how material structure – such as atomic properties (electronegativity, ionic radius) and lattice properties (sub-lattice distortion) influences proton transport. The results obtained from this high-throughput analysis is being employed to develop a machine learning framework to predict structure-property correlations on a larger set of perovskites materials. Finally, we explore the role of disorder on proton transport by studying for example fluorite based lanthanum tungstate materials. [1] “Defect Genome of Cubic Perovskites for Fuel Cell Applications”, Journal of Physical Chemistry C, 121, 26637 (2017)[2] “The Influence of Local Distortions on Proton Mobility in Acceptor Doped Perovskites”, Chemistry of Materials, 2018, 30 (15), pp 4919–4925[3]“The Influence of the Local Structure on Proton Transport in a Solid Oxide Proton Conductor La0.8Ba1.2GaO3.9”, J. Mat. Chem. A, 5, 15507 (2017)[4] “Influence of Non-Stoichiometry on Proton Conductivity in Thin Film Yttrium-doped Barium Zirconate”, ACS Appl. Mater. Interfaces, 2018, 10 (5), pp 4816–4823

Targeting GPUs with OpenMP directives on Summit: A simple and effective Fortran experience

We use OpenMP to target hardware accelerators (GPUs) on Summit, a newly deployed supercomputer at the Oak Ridge Leadership Computing Facility (OLCF), demonstrating simplified access to GPU devices for users of our astrophysics code GenASiS and useful speedup on a sample fluid dynamics problem. We modify our workhorse class for data storage to include members and methods that significantly streamline the persistent allocation of and association to GPU memory. Users offload computational kernels with OpenMP target directives that are rather similar to constructs already familiar from multi-core parallelization. In this initial example we ask, “With a given number of Summit nodes, how fast can we compute with and without GPUs?”, and find total wall time speedups of ∼ 12X. We also find reasonable weak scaling up to 8000 GPUs (1334 Summit nodes). We make available the source code from this work at https://github.com/GenASiS/GenASiS_Basics.

GenASiSBasics: Object-oriented utilitarian functionality for large-scale physics simulations (Version 3)

GenASiS Basics provides Fortran 2003 classes furnishing extensible object-oriented utilitarian functionality for large-scale physics simulations on distributed memory supercomputers. This functionality includes physical units and constants; display to the screen or standard output device; message passing; I/O to disk; and runtime parameter management and usage statistics. This revision---Version 3 of Basics---includes a significant name change, some minor additions to functionality, and a major addition to functionality: infrastructure facilitating the offloading of computational kernels to devices such as GPUs.

Lipon-like Electrolyte Powders Made By Scalable Alternative Processing

Lithium phosphorus oxynitride (Lipon) is the most prominent example of glassy oxynitride electrolytes, which have demonstrated high ionic conductivities, stability over a wide voltage window, and resistance to Li dendrite formation.1Perhaps the greatest drawback of glassy oxynitride electrolytes is that expensive, high-vacuum deposition techniques have been necessary to produce electrolyte compositions with high ionic conductivity (> ~10-6S/cm). High-vacuum deposition techniques have low deposition rates (1 – 50 nm/min, which is less than 0.07 g/hr for Lipon at lab scale) and have only been successfully demonstrated for use in thin film batteries. Alternative processing routes for producing Lipon-like electrolytes could enable their wider usage as electrolytes and ionically conductive additives in commercially viable lithium metal and lithium-ion batteries. We have used scalable alternative processing to produce Lipon-like electrolyte nanopowders at a high throughput (5 g/hr at lab scale). This alternative processing method has successfully produced Lipon-like electrolytes that conventional ceramic and glass processing methods have been unable to produce.2,3 The processed powders have a predominantly amorphous structure (x-ray diffraction), a particle size of ~100 nm (SEM), and a Lipon-like composition (energy dispersive x-ray spectroscopy and inductively coupled plasma optical emission spectroscopy). Neutron pair distribution function analysis has been coupled with ab initiomolecular dynamics to verify that the Lipon-like nanopowders have local atomic structures (0 – 20 Å) that are nearly identical to conventionally sputtered Lipon. Impedance spectroscopy was used to measure the ionic conductivity and activation energy of unsintered Lipon-like nanopowder compacts. While interparticle contact resistance likely dominates these values, the unsintered Lipon-like nanopowder compact still demonstrated an ionic conductivity of 5.3 x 10-7S/cm at room temperature. Acknowledgements The information, data, and work presented herein was funded by the Advanced Research Projects Agency – Energy (ARPA-E), U.S. Department of Energy, under Award Number DE-AR0000775. This manuscript has been authored by UT-Battelle, LLC, under contract DE-AC05-00OR22725 with the U.S. Department of Energy. Electron microscopy experiments were conducted at ORNL’s Center for Nanophase Materials Sciences, which is a DOE Office of Science User Facility. Computing resources were provided by the Oak Ridge Leadership Computing Facility at the Oak Ridge National Laboratory. A portion of this research used resources at the Spallation Neutron Source, a DOE Office of Science User Facility operated by the Oak Ridge National Laboratory. Brendan Lewis, Rutuja Samant, Dora Cheung, and Bob Herbeck at Buffalo Manufacturing Works aided in the processing of later batches of Lipon-like material. Special thanks to: Brenda Smith, Chelsea Chen, Dale Hensley, and Robert Sacci at Oak Ridge National Laboratory. References 1 Bates, J. B.; Dudney, N. J.; Gruzalski, G. R.; Zuhr, R. A.; Choudhury, A.; Luck, C. F. Sputtering of lithium compounds for preparation of electrolyte films. Solid State Ionics, 1992, 53-56, 647-654. 2 Muñoz, F.; Durán, A.; Pascual, L.; Montagne, L.; Revel, B.; Rodrigues, A. C. M. Increased electrical conductivity of LiPON glasses produced by ammonolysis. Solid State Ionics, 2008, 179, 574-579. 3 Mascaraque, N.; Fierro, J. L. G.; Durán, A.; Muñoz, F. An interpretation for the increase of ionic conductivity by nitrogen incorporation in LiPON oxynitride glasses. Solid State Ionics, 2013,233, 73-79.