Hessian Calculations Research Articles

Modelling of the Elementary Steps Involved in the Aluminum Electrochemical Deposition from Ionic Liquid Based Solution: The BMImCl/AlCl3 System

The energy hierarchy, of the main chemical species involved in the reaction mechanism relevant to the electrodeposition of aluminum in 1-Butyl-3-methylimidazolium chloride/aluminum trichloride solution (BMImCl/AlCl3), is studied by using ab-initio based theoretical calculations. Eventually, a reasonable theoretical estimate of energies, involved in the principal reactions ruling the aluminum electrodeposition from BMImCl ionic liquid solutions, is obtained. For screening purposes (geometry optimization and Hessian calculations) the CAMB3LYP density functional, DFT, has been used. Then single point (exploiting CAMB3LYP optimized geometries) energy data are obtained at the Møller-Plesset (MP2) level of the theory. They are used to cross-check DFT results. A reaction mechanism emerges in which, although the species AlCl−4 is formed with very high efficiency from the neutral species AlCl3, the competing reaction points to an almost complete conversion of aluminum to the dimeric form into bulk solution. This is observed in the absence and, most importantly, in the presence of a coordinating BMIm+ cation. In this respect, the presence of BMIm+ does not seem to affect significantly the equilibrium between the monomeric and dimeric forms of aluminum. This outcome is very interesting because the dimeric species is directly reduced to yield the metal aluminum. Indeed, a larger concentration of Al2Cl−7 gives due reason for a more effective electrodeposition process, as it is experimentally observed in the ionic liquid medium.

Review of second-order optimization techniques in artificial neural networks backpropagation

Second-order optimization technique is the advances of first-order optimization in neural networks. It provides an addition curvature information of an objective function that adaptively estimate the step-length of optimization trajectory in training phase of neural network. With the additional information, it reduces training iteration and achieves fast convergence with less tuning of hyper-parameter. The current improved memory allocation and computing power further motivates machine learning practitioners to revisit the benefits of second-order optimization techniques. This paper covers the review on second-order optimization techniques that involve Hessian calculation for neural network training. It reviews the basic theory of Newton method, quasi-Newton, Gauss-Newton, Levenberg-Marquardt, Approximate Greatest Descent and Hessian-Free optimization. This paper summarizes the feasibility and performance of optimization techniques in deep neural network training. Comments and suggestions are highlighted for second-order optimization techniques in artificial neural network training in term of advantages and limitations.

Accurate embedding through potential reconstruction: A comparison of different strategies.

Potential reconstruction is a powerful strategy for deriving accurate (sometimes called "exact") embedding potentials in the context of density-dependent embedding methods. It is particularly useful for partitioning covalent bonds in such fragment-based electronic-structure methods. While the general approach is well defined and easily explained, there are a number of choices to be made in practice, concerning, e.g., the specific reconstruction algorithm, the assignment of electrons to subsystems, or the initial guess potential. A general choice to be made is whether "exact" embedding potentials shall be derived for pre-defined target densities (top-down) or for approximate fragment densities that can be iteratively defined (bottom-up). Here, we compare the pros and cons of a variety of different variants of potential reconstruction, both in terms of conceptual issues and concerning their accuracy and efficiency. We also present several algorithmic improvements that can be crucial in critical cases of potential reconstruction, namely, we show (i) that a combination of basis-set and grid-based potential reconstruction schemes can lead to improved resulting densities, (ii) that similarly the combination of real-space and matrix-representation based potential reconstruction gives great advantages, and (iii) that the potential-matrix reconstruction by Zhang and Carter [J. Chem. Phys. 148, 034105 (2018)] can be made much more efficient by avoiding an explicit Hessian calculation. Additionally, we demonstrated (iv) that a double reconstruction, meaning a reconstruction of both the supersystem potential and the subsystem potential, may lead to beneficial error cancellation. We also address the question of consistent energetics derived from such reconstructed potentials.

Dual-Level Approach to Instanton Theory.

Instanton theory is an established method to calculate rate constants of chemical reactions including atom tunneling. Technical and methodological improvements increased its applicability. Still, a large number of energy and gradient calculations is necessary to optimize the instanton tunneling path, and second derivatives of the potential energy along the tunneling path have to be evaluated, restricting the range of suitable electronic structure methods. To enhance the applicability of instanton theory, we present a dual-level approach in which instanton optimizations and Hessian calculations are performed using an efficient but approximate electronic structure method, and the potential energy along the tunneling path is recalculated using a more accurate method. This procedure extends the applicability of instanton theory to high-level electronic structure methods for which analytic gradients may not be available, like local linear-scaling approaches. We demonstrate for the analytical Eckart barrier and three molecular systems how the dual-level instanton approach corrects for the largest part of the error caused by the inaccuracy of the efficient electronic structure method. This reduces the error of the calculated rate constants significantly.

Contingent derivatives and regularization for noncoercive inverse problems

We study the inverse problem of parameter identification in noncoercive variational problems that commonly appear in applied models. We examine the differentiability of the set-valued parameter-to-solution map using the first-order and the second-order contingent derivatives. We explore the inverse problem using the output least-squares and the modified output least-squares objectives. By regularizing the noncoercive variational problem, we obtain a single-valued regularized parameter-to-solution map and investigate its smoothness and boundedness. We also consider optimization problems using the output least-squares and the modified output least-squares objectives for the regularized variational problem. We give a complete convergence analysis showing that for the output least-squares and the modified output least-squares, the regularized minimization problems approximate the original optimization problems suitably. We also provide the first-order and the second-order adjoint method for the computation of the first-order and the second-order derivatives of the output least-squares objective. We provide discrete formulas for the gradient and the Hessian calculation and present numerical results.

Derivative-free Nonlinear Version of Extended Recursive Three-step Filter for State and Parameter Estimation during Mars Entry

Parameter uncertainties which may lead to divergence of traditional Kalman filters during Mars entry are investigated in this paper. To achieve high precision navigation, a Derivative-free Nonlinear version of an Extended Recursive Three-Step Filter (DNERTSF) is introduced, which suits nonlinear systems with arbitrary parameter uncertainties. A DNERTSF can estimate the state and the parameters simultaneously, and Jacobian and Hessian calculations are not necessary for this filter. Considering the uncertainties in atmosphere density, ballistic coefficient and lift-to-drag ratio, a numerical simulation of Mars entry navigation is carried out. Compared with the standard Unscented Kalman Filter (UKF), DNERTSF can effectively reduce the adverse effects of parameter uncertainties and achieve a high navigation accuracy performance, keeping position and velocity estimation errors at a very low level. In all, the DNERTSF in this paper shows good advantages for Mars entry navigation, providing a possible application for a future Mars pinpoint landing.

Fast algorithm for geometrical coding of digital images

A fast algorithm for geometrical coding is presented in the paper. It allows one to perform the image analysis by methods of differential geometry rather than widespread methods based on gradient and Hessian calculations. Nowadays, one of the most important aspects in image analysis is the speed of computations. The algorithm for geometrical coding is comparable or faster than traditional algorithms of image analysis and in many cases it gives better results. Several examples of algorithm working results for contour detection are presented.

An accurate and efficient reliability-based design optimization using the second order reliability method and improved stability transformation method

ABSTRACTThe first order reliability method has been extensively adopted for reliability-based design optimization (RBDO), but it shows inaccuracy in calculating the failure probability with highly nonlinear performance functions. Thus, the second order reliability method is required to evaluate the reliability accurately. However, its application for RBDO is quite challenge owing to the expensive computational cost incurred by the repeated reliability evaluation and Hessian calculation of probabilistic constraints. In this article, a new improved stability transformation method is proposed to search the most probable point efficiently, and the Hessian matrix is calculated by the symmetric rank-one update. The computational capability of the proposed method is illustrated and compared to the existing RBDO approaches through three mathematical and two engineering examples. The comparison results indicate that the proposed method is very efficient and accurate, providing an alternative tool for RBDO of engineering structures.

Global exploration of isomers and isomerization channels on the quantum chemical potential energy surface of H3 CNO3.

Global exploration of isomers and isomerization channels on the quantum chemical potential energy surface (PES) is performed for H3 CNO3 using the Scaled Hypersphere Search-Anharmonic Downward Distortion Following (SHS-ADDF) method. The molecular formula of H3 CNO3 includes functional groups of CH3 , OH, NH2 , COOH, NO, NO2 , and NO3 , which are very important in connection with amino acids and NOx. Geometrical structures and interconversion pathways are disclosed after 18719781 force calculations and 534726 Hessian calculations at the level of B3LYP/6-31G(d). The explored results are confirmed to be valid, especially for the important lower energy regions, by re-optimization at the higher level of B3LYP/6-311++G(d,p). A global reaction route-mapping using SHS-ADDF demonstrates the entire view and undeveloped landscapes on PES of H3 CNO3 . Typical compounds of H3 CNO3 , aminoxy formic acid, hydroxycarbamic acid, aminoperformic acid, hydroxymethyl nitrite, nitromethanol, methyl nitrate, methyl peroxynitrite, and dioxaziridine, are well separated from others by very high energy-barriers. The stable-most conformer of H3 CNO3 is difficult to be determined, because of seven structures existing with nearly the same energies within 5.7 kJ/mol at the level of CCSD(T)/aug-cc-pVTZ. © 2017 Wiley Periodicals, Inc.

An optimized second order stochastic learning algorithm for neural network training

This paper proposes an improved stochastic second order learning algorithm for supervised neural network training. The proposed algorithm, named bounded stochastic diagonal Levenberg–Marquardt (B-SDLM), utilizes both gradient and curvature information to achieve fast convergence while requiring only minimal computational overhead than the stochastic gradient descent (SGD) method. B-SDLM has only a single hyperparameter as opposed to most other learning algorithms that suffer from the hyperparameter overfitting problem due to having more hyperparameters to be tuned. Experiments using the multilayer perceptron (MLP) and convolutional neural network (CNN) models have shown that B-SDLM outperforms other learning algorithms with regard to the classification accuracies and computational efficiency (about 5.3% faster than SGD on the mnist-rot-bg-img database). It can classify all testing samples correctly on the face recognition case study based on AR Purdue database. In addition, experiments on handwritten digit classification case studies show that significant improvements of 19.6% on MNIST database and 17.5% on mnist-rot-bg-img database can be achieved in terms of the testing misclassification error rates (MCRs). The computationally expensive Hessian calculations are kept to a minimum by using just 0.05% of the training samples in its estimation or updating the learning rates once per two training epochs, while maintaining or even achieving lower testing MCRs. It is also shown that B-SDLM works well in the mini-batch learning mode, and we are able to achieve 3.32× performance speedup when deploying the proposed algorithm in a distributed learning environment with a quad-core processor.

Electronically nonadiabatic wave packet propagation using frozen Gaussian scattering.

We present an approach, which allows to employ the adiabatic wave packet propagation technique and semiclassical theory to treat the nonadiabatic processes by using trajectory hopping. The approach developed generates a bunch of hopping trajectories and gives all additional information to incorporate the effect of nonadiabatic coupling into the wave packet dynamics. This provides an interface between a general adiabatic frozen Gaussian wave packet propagation method and the trajectory surface hopping technique. The basic idea suggested in [A. D. Kondorskiy and H. Nakamura, J. Chem. Phys. 120, 8937 (2004)] is revisited and complemented in the present work by the elaboration of efficient numerical algorithms. We combine our approach with the adiabatic Herman-Kluk frozen Gaussian approximation. The efficiency and accuracy of the resulting method is demonstrated by applying it to popular benchmark model systems including three Tully's models and 24D model of pyrazine. It is shown that photoabsorption spectrum is successfully reproduced by using a few hundreds of trajectories. We employ the compact finite difference Hessian update scheme to consider feasibility of the ab initio "on-the-fly" simulations. It is found that this technique allows us to obtain the reliable final results using several Hessian matrix calculations per trajectory.

Non-invasive quality evaluation of confluent cells by image-based orientation heterogeneity analysis

In recent years, cell and tissue therapy in regenerative medicine have advanced rapidly towards commercialization. However, conventional invasive cell quality assessment is incompatible with direct evaluation of the cells produced for such therapies, especially in the case of regenerative medicine products. Our group has demonstrated the potential of quantitative assessment of cell quality, using information obtained from cell images, for non-invasive real-time evaluation of regenerative medicine products. However, image of cells in the confluent state are often difficult to evaluate, because accurate recognition of cells is technically difficult and the morphological features of confluent cells are non-characteristic. To overcome these challenges, we developed a new image-processing algorithm, heterogeneity of orientation (H-Orient) processing, to describe the heterogeneous density of cells in the confluent state. In this algorithm, we introduced a Hessian calculation that converts pixel intensity data to orientation data and a statistical profiling calculation that evaluates the heterogeneity of orientations within an image, generating novel parameters that yield a quantitative profile of an image. Using such parameters, we tested the algorithm's performance in discriminating different qualities of cellular images with three types of clinically important cell quality check (QC) models: remaining lifespan check (QC1), manipulation error check (QC2), and differentiation potential check (QC3). Our results show that our orientation analysis algorithm could predict with high accuracy the outcomes of all types of cellular quality checks (>84% average accuracy with cross-validation).

A vibrational spectroscopy study on anserine and its aqueous solutions

In this study based on vibrational spectroscopic measurements and Density Functional Theory (DFT), we aimed for a reliable interpretation of the IR and Raman spectra recorded for anserine in the solid phase and water (H2O) and heavy water (D2O) solutions. Initial DFT calculations at the B3LYP/6-31G(d) searched possible conformers of the anserine zwitterion using a systematic conformational search. The corresponding equilibrium geometrical parameters and vibrational spectral data were determined for each of the stable conformers (in water) by the geometry optimization and hessian calculations performed at the same level of theory using the polarized continuum model (PCM). The same calculations were repeated to determine the most energetically preferred dimer structure for the molecule and the associated geometry, force field and vibrational spectral data. The harmonic force constants obtained from these calculations were scaled by the Scaled Quantum Mechanical Force Field (SQM) method and then used in the calculation of the refined wavenumbers, potential energy distributions, IR and Raman intensities. These refined theoretical data, which confirm the zwitterion structure for anserine in the solid phase or aqueous solvents, revealed the remarkable effects of intermolecular hydrogen bonding on the structural properties and observed IR and Raman spectra of this molecule.

Using bonding to guide transition state optimization.

Optimization of a transition state typically requires both a good initial guess of the molecular structure and one or more computationally demanding Hessian calculations to converge reliably. Often, the transition state being optimized corresponds to the barrier in a chemical reaction where bonds are being broken and formed. Utilizing the geometries and bonding information for reactants and products, an algorithm is outlined to reliably interpolate an initial guess for the transition state geometry. Additionally, the change in bonding is also used to increase the reliability of transition state optimizations that utilize approximate and updated Hessian information. These methods are described and compared against standard transition state optimization methods.

Efficient implementation of the analytic second derivatives of Hartree–Fock and hybrid DFT energies: a detailed analysis of different approximations

In this paper, various implementations of the analytic Hartree–Fock and hybrid density functional energy second derivatives are studied. An approximation-free four-centre implementation is presented, and its accuracy is rigorously analysed in terms of self-consistent field (SCF), coupled-perturbed SCF (CP-SCF) convergence and prescreening criteria. The CP-SCF residual norm convergence threshold turns out to be the most important of these. Final choices of convergence thresholds are made such that an accuracy of the vibrational frequencies of better than 5 cm−1 compared to the numerical noise-free results is obtained, even for the highly sensitive low frequencies (<100–200 cm−1). The effects of the choice of numerical grid for density functional exchange–correlation integrations are studied and various weight derivative schemes are analysed in detail. In the second step of the work, approximations are introduced in order to speed up the computation without compromising its accuracy. To this end, the accuracy and efficiency of the resolution of identity approximation for the Coulomb terms and the semi-numerical chain of spheres approximation to the exchange terms are carefully analysed. It is shown that the largest performance improvements are realised if either Hartree–Fock exchange is absent (pure density functionals) and otherwise, if the exchange terms in the CP-SCF step of the calculation are approximated by the COSX method in conjunction with a small integration grid. Default values for all the involved truncation parameters are suggested. For vancomycine (176 atoms and 3593 basis functions), the RIJCOSX Hessian calculation with the B3LYP functional and the def2-TZVP basis set takes ∼3 days using 16 Intel® Xeon® 2.60GHz processors with the COSX algorithm having a net parallelisation scaling of 11.9 which is at least ∼20 times faster than the calculation without the RIJCOSX approximation.

Second-order reliability method-based inverse reliability analysis using Hessian update for accurate and efficient reliability-based design optimization

SUMMARY First-order reliability method (FORM) has been mostly utilized for solving reliability-based design optimization (RBDO) problems efficiently. However, second-order reliability method (SORM) is required in order to estimate a probability of failure accurately in highly nonlinear performance functions. Despite accuracy of SORM, its application to RBDO is quite challenging due to unaffordable numerical burden incurred by a Hessian calculation. For reducing the numerical efforts, a quasi-Newton approach to approximate the Hessian is introduced in this study instead of calculating the true Hessian. The proposed SORM with the approximated Hessian requires computations only used in FORM, leading to very efficient and accurate reliability analysis. The proposed SORM also utilizes a generalized chi-squared distribution in order to achieve better accuracy. Furthermore, SORM-based inverse reliability method is proposed in this study. An accurate reliability index corresponding to a target probability of failure is updated using the proposed SORM. Two approaches in terms of finding an accurate most probable point using the updated reliability index are proposed. The proposed SORM-based inverse analysis is then extended to RBDO in order to obtain a reliability-based optimum design satisfying probabilistic constraints more accurately even for a highly nonlinear system. The numerical study results show that the proposed reliability analysis and RBDO achieve efficiency of FORM and accuracy of SORM at the same time. Copyright © 2014 John Wiley & Sons, Ltd.

Erratum: “A finite difference Davidson procedure to sidestep full ab initio hessian calculation: Application to characterization of stationary points and transition state searches” [J. Chem. Phys. 140, 164115 (2014)

First Page

A finite difference Davidson procedure to sidestep full ab initio hessian calculation: Application to characterization of stationary points and transition state searches

The cost of calculating nuclear hessians, either analytically or by finite difference methods, during the course of quantum chemical analyses can be prohibitive for systems containing hundreds of atoms. In many applications, though, only a few eigenvalues and eigenvectors, and not the full hessian, are required. For instance, the lowest one or two eigenvalues of the full hessian are sufficient to characterize a stationary point as a minimum or a transition state (TS), respectively. We describe here a method that can eliminate the need for hessian calculations for both the characterization of stationary points as well as searches for saddle points. A finite differences implementation of the Davidson method that uses only first derivatives of the energy to calculate the lowest eigenvalues and eigenvectors of the hessian is discussed. This method can be implemented in conjunction with geometry optimization methods such as partitioned-rational function optimization (P-RFO) to characterize stationary points on the potential energy surface. With equal ease, it can be combined with interpolation methods that determine TS guess structures, such as the freezing string method, to generate approximate hessian matrices in lieu of full hessians as input to P-RFO for TS optimization. This approach is shown to achieve significant cost savings relative to exact hessian calculation when applied to both stationary point characterization as well as TS optimization. The basic reason is that the present approach scales one power of system size lower since the rate of convergence is approximately independent of the size of the system. Therefore, the finite-difference Davidson method is a viable alternative to full hessian calculation for stationary point characterization and TS search particularly when analytical hessians are not available or require substantial computational effort.

NDL-v2.0: A new version of the numerical differentiation library for parallel architectures

NDL-v2.0: A new version of the numerical differentiation library for parallel architectures

Analytical derivative techniques for TDDFT excited-state properties: Theory and application

We review our recent work on the methodology development of the excited-state properties for the molecules in vacuum and liquid solution. The general algorithms of analytical energy derivatives for the specific properties such as the first and second geometrical derivatives and IR/Raman intensities are demonstrated in the framework of the time-dependent density functional theory (TDDFT). The performance of the analytical approaches on the calculation of excited-state energy Hessian has also been shown. It is found that the analytical approaches are superior to the finite-difference method on the computational accuracy and efficiency. The computational cost for a TDDFT excited-state Hessian calculation is only 2–3 times as that for the DFT ground-state Hessian calculation. With the low computational complexity of the developed analytical approaches, it becomes feasible to realize the large-scale numerical calculations on the excited-state vibrational frequencies, vibrational spectroscopies and the electronic-structure parameters which enter the spectrum calculations of electronic absorption and emission, and resonance Raman spectroscopies for medium-to large-sized systems.