quasi-Newton Method Research Articles

THE NEW RANK ONE CLASS FOR UNCONSTRAINED PROBLEMS SOLVING

One of the most well-known methods for unconstrained problems is the quasi-Newton approach, iterative solutions. The great precision and quick convergence of the quasi-Newton methods are well recognized. In this work, the new algorithm for the symmetric rank one SR1 method is driven. The strong Wolfe line search criteria define the step length selection. We also proved the new quasi-Newton equation and positive definite matrix theorem. Preliminary computer testing on the set of fourteen unrestricted optimization test functions leads to the conclusion that this new method is more effective and durable than the implementation of classical SR1 method in terms of iterations count and functions.

APPLICATION OF THE QUASI-NEWTON IN INTERIOR POINT METHODS FOR SOLVING THE PREDISPATCH PROBLEM

Brazilian energy matrix is essentially based on hydroelectricity with long transmission lines, allowing the exchange of energy produced in all regions of the country, the increased demand for energy and the search for lower costs, the application of more efficient and robust methods to minimize generation and transmission losses is necessary, since these are functions of generated and transmitted power, respectively. The purpose of this work is to implement primal dual interior point method for the predispatchof a hydroelectric system, with partial replacement of the Newton method with the quasi-Newton method in order to compute the system Jacobian matrix and reduce the computational costs of the iterations arising from approximations of the inverses of the Hessian matrix. This means that, in order to obtain a search direction, only a matrix vector product is necessary, which is much more efficient, for example, than the Newton method, in which a linear system has to be solved at each iteration. Computational results prove the efficiency of the approach used.

A Fast and Efficient Estimation of the Parameters of a Model of Accident Frequencies via an MM Algorithm

In this paper, we consider a multivariate statistical model of accident frequencies having a variable number of parameters and whose parameters are dependent and subject to box constraints and linear equality constraints. We design a minorization-maximization (MM) algorithm and an accelerated MM algorithm to compute the maximum likelihood estimates of the parameters. We illustrate, through simulations, the performance of our proposed MM algorithm and its accelerated version by comparing them to Newton-Raphson (NR) and quasi-Newton algorithms. The results suggest that the MM algorithm and its accelerated version are better in terms of convergence proportion and, as the number of parameters increases, they are also better in terms of computation time.

Extrapolation methods as nonlinear Krylov subspace methods

When applied to linear vector sequences, extrapolation methods are equivalent to Krylov subspace methods. Both types of methods can be expressed as particular cases of the multisecant equations, the secant method generalized to higher dimensions. Through these equations, there is also equivalence with a variety of quasi-Newton methods. This paper presents a framework to connect these various methods.

Heart rate variability in different sleep stages is associated with metabolic function and glycemic control in type 2 diabetes mellitus.

Introduction: Autonomic nervous system (ANS) plays an important role in the exchange of metabolic information between organs and regulation on peripheral metabolism with obvious circadian rhythm in a healthy state. Sleep, a vital brain phenomenon, significantly affects both ANS and metabolic function. Objectives: This study investigated the relationships among sleep, ANS and metabolic function in type 2 diabetes mellitus (T2DM), to support the evaluation of ANS function through heart rate variability (HRV) metrics, and the determination of the correlated underlying autonomic pathways, and help optimize the early prevention, post-diagnosis and management of T2DM and its complications. Materials and methods: A total of 64 volunteered inpatients with T2DM took part in this study. 24-h electrocardiogram (ECG), clinical indicators of metabolic function, sleep quality and sleep staging results of T2DM patients were monitored. Results: The associations between sleep quality, 24-h/awake/sleep/sleep staging HRV and clinical indicators of metabolic function were analyzed. Significant correlations were found between sleep quality and metabolic function (|r| = 0.386 ± 0.062, p < 0.05); HRV derived ANS function showed strengthened correlations with metabolic function during sleep period (|r| = 0.474 ± 0.100, p < 0.05); HRV metrics during sleep stages coupled more tightly with clinical indicators of metabolic function [in unstable sleep: |r| = 0.453 ± 0.095, p < 0.05; in stable sleep: |r| = 0.463 ± 0.100, p < 0.05; in rapid eye movement (REM) sleep: |r| = 0.453 ± 0.082, p < 0.05], and showed significant associations with glycemic control in non-linear analysis [fasting blood glucose within 24h of admission (admission FBG), |r| = 0.420 ± 0.064, p < 0.05; glycated hemoglobin (HbA1c), |r| = 0.417 ± 0.016, p < 0.05]. Conclusions: HRV metrics during sleep period play more distinct role than during awake period in investigating ANS dysfunction and metabolism in T2DM patients, and sleep rhythm based HRV analysis should perform better in ANS and metabolic function assessment, especially for glycemic control in non-linear analysis among T2DM patients.

Using Structural Equation Modeling to Reproduce and Extend ANOVA-Based Generalizability Theory Analyses for Psychological Assessments

Generalizability theory provides a comprehensive framework for determining how multiple sources of measurement error affect scores from psychological assessments and using that information to improve those assessments. Although generalizability theory designs have traditionally been analyzed using analyses of variance (ANOVA) procedures, the same analyses can be replicated and extended using structural equation models. We collected multi-occasion data from inventories measuring numerous dimensions of personality, self-concept, and socially desirable responding to compare variance components, generalizability coefficients, dependability coefficients, and proportions of universe score and measurement error variance using structural equation modeling versus ANOVA techniques. We further applied structural equation modeling techniques to continuous latent response variable metrics and derived Monte Carlo-based confidence intervals for those indices on both observed score and continuous latent response variable metrics. Results for observed scores estimated using structural equation modeling and ANOVA procedures seldom varied. Differences in reliability between raw score and continuous latent response variable metrics were much greater for scales with dichotomous responses, thereby highlighting the value of doing analyses on both metrics to evaluate gains that might be achieved by increasing response options. We provide detailed guidelines for applying the demonstrated techniques using structural equation modeling and ANOVA-based statistical software.

Fuzzy modeling and cost optimization for machine repair problem with retrial under admission control F-policy and feedback

In the present paper, the machine repair problem (MRP) with real-life scenarios is investigated, which plays a significant role in the manufacturing process, industries, production lines, etc. An M∖M∖1∖K machine repair problem with retrial attempts under admission control policy F-policy and machine’s feedback is considered in which a single repairman is available to repair the failed machines. The recursive technique is applied to solve the mathematical model and establish the probability distributions for different states, which are further used to develop the performance indices. The parametric non-linear programming (P-NLP) technique is employed to develop the MRP with F-policy in a fuzzy environment using the trapezoidal and sigmoidal fuzzy numbers. Zadeh’s extension principle and parametric non-linear programming technique help us to find the α-cuts of the developed performance predictors of MRP. We frame and minimize the total cost of the system. The harmony search method (HSM) and quasi-Newton method are implemented to achieve the total minimum cost of the system corresponding to the optimal repair rate, which helps the system managers in the formulation of business policies in industries, including the automobile sector. A numerical example is also given, which produces several tables and graphs that show the validity of the model.

Accelerated nonlinear finite element method for analysis of isotropic hyperelastic materials nonlinear deformations

In this work, an approximate Jacobian matrix is proposed based on the total Lagrangian formulation of Finite Element Method for isotropic hyperelastic materials. The approximate Jacobian matrix can take the place of the exact Jacobian matrix in the Newton–Raphson method to avoid frequent construction and factorization of the Jacobian matrix. A new Quasi–Newton method employing the approximate Jacobian matrix is developed to significantly improve the convergence rate. The proposed method was tested on three examples with the combinations of three hyperelastic material models and three element types. The results show that the proposed method is more efficient than ABAQUS and CALCULIX (an open-source FEM software package) on all of the tests without loss of accuracy. It is up to 100 times faster than the traditional Quasi–Newton method, and at least 2.5 times faster than ABAQUS.

A Modified Dai–Liao Conjugate Gradient Method Based on a Scalar Matrix Approximation of Hessian and Its Application

We introduce and investigate proper accelerations of the Dai–Liao (DL) conjugate gradient (CG) family of iterations for solving large-scale unconstrained optimization problems. The improvements are based on appropriate modifications of the CG update parameter in DL conjugate gradient methods. The leading idea is to combine search directions in accelerated gradient descent methods, defined based on the Hessian approximation by an appropriate diagonal matrix in quasi-Newton methods, with search directions in DL-type CG methods. The global convergence of the modified Dai–Liao conjugate gradient method has been proved on the set of uniformly convex functions. The efficiency and robustness of the newly presented methods are confirmed in comparison with similar methods, analyzing numerical results concerning the CPU time, a number of function evaluations, and the number of iterative steps. The proposed method is successfully applied to deal with an optimization problem arising in 2D robotic motion control.

A Combined Conjugate Gradient Quasi-Newton Method with Modification BFGS Formula

The conjugate gradient and Quasi-Newton methods have advantages and drawbacks, as although quasi-Newton algorithm has more rapid convergence than conjugate gradient, they require more storage compared to conjugate gradient algorithms. In 1976, Buckley designed a method that combines the CG method with QN updates, which is better than that observed for conjugate gradient algorithms but not as good as the quasi-Newton approach. This type of method is called the preconditioned conjugate gradient (PCG) method. In this paper, we introduce two new preconditioned conjugate gradient (PCG) methods that combine conjugate gradient with a new update of quasi-Newton methods. The new quasi-Newton method satisfied the positive define, and the direction of the new preconditioned conjugate gradient is descent direction. In numerical results, it is showing the new preconditioned conjugate gradient method is more effective on several high-dimension test problems than standard preconditioning.

A Space-Time Absolute Nodal Coordinate Formulation Cable Element and the Study of Its Accuracy and Efficiency

In this paper, a space-time absolute nodal coordinate formulation cable (SAC) element forming technique based on the Lagrange family of shape functions is proposed. Two distinct SAC elements, each with a distinct spatial shape function, have been generated by this method. Moreover, the external forces such as the bending moment and the air resistance formula have been accounted for. The Lagrange multiplier method, along with the concepts of replacement constraint and supplementary constraint, has been employed to provide a solution for the dynamics of constrained mechanical systems. Additionally, a constraint conversion strategy has been suggested. The solver has been constructed through Hamilton’s law of varying action. The space-time finite element method is used to solve dynamic problems, employing the Newton algorithm and quasi-Newton algorithm. The accuracy and efficiency of the solution has been verified by three simulations and one experiment. The circle-bending static simulation and the double-ended velocity impact dynamic simulation demonstrate the accuracy of the two elements. The correlation between statics and dynamics has been studied for different discretization methods and different solvers’ calculation accuracy and efficiency. Different modeling methods, time steps, order and the application of the quasi-Newton method all have a bearing on the efficiency of the solution. Finally, a comparison with an experiment in the free-pendulum simulation reveals the capability of this model to simulate dynamic problems with air resistance.

A Multi-medium and Multi-mechanism model for CO2 injection and storage in fractured shale gas reservoirs

Under the background of “dual-carbon”, it is strategically significant to use shale gas reservoirs for CO2 injection and storage. Injection of CO2 into shale formations can improve natural gas recovery and achieve effective storage of CO2. However, due to the complex characteristics of shale reservoirs, like complex fractures and a variety of flow mechanisms, few work has been done to comprehensively consider those complex characteristics for accurate reservoir simulation of CO2 injection and storage. Therefore, to improve this situation, this paper establishes a fully coupled model of the matrix, natural fracture and hydraulic fracture, which considers multiple mechanisms, including adsorption/desorption, diffusion, stress sensitivity of fracture and stimulated reservoir volume (SRV). The embedded discrete fracture model (EDFM) is applied to describe the transfer flow between matrix-fracture, fracture-fracture and fracture-well. The finite volume and quasi-Newton iterative methods are used to solve the model. The model's reliability and practicability are verified by matching the production history of commercial software in shale gas reservoirs. The effects of various factors on the production curve and CO2 storage characteristics are analyzed, including adsorption/desorption, diffusion, stress sensitivity, well production pressure, CO2 injection rate, CO2 injection opportunity and fracture parameters. The results show that the model is reliable and practical and can be used to study the dynamic performances of enhanced shale gas recovery and CO2 storage.

ANFIS and metaheuristic optimization for strategic joining policy with re-attempt and vacation

This research work deals with the queueing analysis and strategic behavior of primitive users (PUs) and ordinary users (OUs) in cognitive radio system (CRS). Whenever fusion center (FC) is occupied, the ordinary users must decide whether to join a retrial orbit and make re-attempts or depart from the system. The OUs residing in the retrial orbit try to access the spectrum through FC. The FC becomes inactive and considered on vacation in case if there are no users in the CRS. Returning back from the vacation, the FC renders service to the primitive users. The service of ordinary users (OUs) may be interrupted due to breakdown of FC. The probability generating function approach is used to derive the performance indices related to the queueing characterization. The joining strategic behavior of the primitive as well as ordinary users has been investigated by considering a linear reward cost function. The social welfare by formulating the cooperative strategy is studied. The hybrid soft computing technique viz. adaptive network fuzzy inference system (ANFIS) is employed for the numerical simulation and sensitivity analysis of various system metrics. The optimal service rates are determined by minimizing the total cost involved in various activities using the quasi-Newton method (QNM) and metaheuristic algorithms.

Heart rate variability for the evaluation of patients with disorders of consciousness

ObjectiveClinical responsiveness of patients with a Disorder of Consciousness (DoC) correlates to sympathetic/parasympathetic homeostatic balance. Heart Rate Variability (HRV) metrics result in non-invasive proxies of modulation capabilities of visceral states. In this work, our aim was to evaluate whether HRV measures could improve the differential diagnosis between Unresponsive Wakefulness Syndrome (UWS) and Minimally Conscious State (MCS) with respect to multivariate models based on standard clinical electroencephalography (EEG) labeling only in a rehabilitation setting. MethodsA prospective observational study was performed consecutively enrolling 82 DoC patients. Polygraphic recordings were performed. HRV-metrics and EEG descriptors derived from the American Clinical Neurophysiology Society's Standardized Critical Care terminology were included. Descriptors entered univariate and then multivariate logistic regressions with the target set to the UWS/MCS diagnosis. ResultsHRV measures resulted significantly different between UWS and MCS patients, with higher values being associated with better consciousness levels. Specifically, adding HRV-related metrics to ACNS EEG descriptors increased the Nagelkerke R2 from 0.350 (only EEG descriptors) to 0.565 (HRV-EEG combination) with the outcome set to the consciousness diagnosis. ConclusionsHRV changes across the lowest states of consciousness. Rapid changes in heart rate, occurring in better consciousness levels, confirm the mutual correlation between visceral state functioning patterns and consciousness alterations. SignificanceQuantitative analysis of heart rate in patients with a DoC paves the way for the implementation of low-cost pipelines supporting medical decisions within multimodal consciousness assessments.

Memoryless Quasi-Newton Methods Based on the Spectral-Scaling Broyden Family for Riemannian Optimization

Memoryless Quasi-Newton Methods Based on the Spectral-Scaling Broyden Family for Riemannian Optimization

Regime-oriented causal model evaluation of Atlantic–Pacific teleconnections in CMIP6

Abstract. The climate system and its spatio-temporal changes are strongly affected by modes of long-term internal variability, like the Pacific decadal variability (PDV) and the Atlantic multidecadal variability (AMV). As they alternate between warm and cold phases, the interplay between PDV and AMV varies over decadal to multidecadal timescales. Here, we use a causal discovery method to derive fingerprints in the Atlantic–Pacific interactions and to investigate their phase-dependent changes. Dependent on the phases of PDV and AMV, different regimes with characteristic causal fingerprints are identified in reanalyses in a first step. In a second step, a regime-oriented causal model evaluation is performed to evaluate the ability of models participating in the Coupled Model Intercomparison Project Phase 6 (CMIP6) in representing the observed changing interactions between PDV, AMV and their extra-tropical teleconnections. The causal graphs obtained from reanalyses detect a direct opposite-sign response from AMV to PDV when analyzing the complete 1900–2014 period and during several defined regimes within that period, for example, when AMV is going through its negative (cold) phase. Reanalyses also demonstrate a same-sign response from PDV to AMV during the cold phase of PDV. Historical CMIP6 simulations exhibit varying skill in simulating the observed causal patterns. Generally, large-ensemble (LE) simulations showed better network similarity when PDV and AMV were out of phase compared to other regimes. Also, the two largest ensembles (in terms of number of members) were found to contain realizations with similar causal fingerprints to observations. For most regimes, these same models showed higher network similarity when compared to each other. This work shows how causal discovery on LEs complements the available diagnostics and statistical metrics of climate variability to provide a powerful tool for climate model evaluation.

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

Adaptive sampling quasi-Newton methods for zeroth-order stochastic optimization

The Effect of Innovation on the Company’s Performance in Small and Medium-Sized Businesses with the Mediating Role of Lean: Agile Project Management Office (LAPMO)

Small- and medium-sized enterprises (SMEs) require less formalized project management methods than large corporations. However, project management can play a significant role in implementing innovations. Even though lean-agile project management offices (LAPMO) are becoming increasingly important for SMEs, each company’s performance varies significantly due to varying innovative capabilities and the dynamism of internal and external contexts. Based on a literature study, innovative capabilities, and LAPMO, we have developed a theoretical model with 11 assumption models. As a follow-up, we conducted empirical research, including critical variable metrics, data collection and analyses, validity tests, reliability tests, regression analysis, and structural equation modeling. The model developed in this study considers the many roles that innovation capacity and project agility play in enhancing corporate performance. LAPMO mediates the relationship between innovation and performance in small and medium-sized businesses. Organizational innovation, open innovation, and innovation capabilities affect companies’ performance. In small and medium businesses, they also affect LAPMO. For small and medium-sized businesses, LAPMO mediates the relationship between organizational innovation, open innovation, and innovation capabilities.

Rapid antenna optimization with restricted sensitivity updates by automated dominant direction identification

Meticulous tuning of geometry parameters turns pivotal in improving performance of antenna systems. It is more and more often realized using formal optimization methods, which is demonstrably the most efficient way of handling multiple design variables, objectives, and constraints. Although in some cases a need for launching global search arises, a typical design scenario only requires local optimization, especially when a decent initial design can be rendered using engineering experience or parametric studies. At the same time, antenna optimization is typically conducted using full-wave electromagnetic (EM) simulations, which entails considerable computational expenses. In this paper, we introduce a novel procedure for expedited antenna tuning. Its fundamental mechanism is to restrict the antenna response sensitivity updates to the selected dominant directions within the parameter space, determined based on the problem-specific knowledge, in particular, the estimated changes of antenna characteristics when moving across one-dimensional affine subspaces spanned by these directions. Thus, the said dominant directions affect the most the responses of the antenna structure under design as assessed using the introduced metrics. The decision making process concerning the number of directions to be used relies on quantification of the aggregated system response variability metrics. The proposed approach is demonstrated by means of several antenna structures and benchmarked against conventional trust-region algorithm, but also its accelerated versions. The results indicate considerable (up to over 60%) speedup over the reference procedure without noticeable quality degradation.

Improved Reconstruction Algorithm of Wireless Sensor Network Based on BFGS Quasi-Newton Method

Aiming at the problems of low reconstruction rate and poor reconstruction precision when reconstructing sparse signals in wireless sensor networks, a sparse signal reconstruction algorithm based on the Limit-Broyden-Fletcher-Goldfarb-Shanno (L-BFGS) quasi-Newton method is proposed. The L-BFGS quasi-Newton method uses a two-loop recursion algorithm to find the descent direction dk directly by calculating the step difference between m adjacent iteration points, and a matrix Hk approximating the inverse of the Hessian matrix is constructed. It solves the disadvantages of BFGS requiring the calculation and storage of Hk, reduces the algorithm complexity, and improves the reconstruction rate. Finally, the experimental results show that the L-BFGS quasi-Newton method has good experimental results for solving the problem of sparse signal reconstruction in wireless sensor networks.