Good Convergence Properties Research Articles

On the Adaptive Penalty Parameter Selection in ADMM

Many data analysis problems can be modeled as a constrained optimization problem characterized by nonsmooth functionals, often because of the presence of ℓ1-regularization terms. One of the most effective ways to solve such problems is through the Alternate Direction Method of Multipliers (ADMM), which has been proved to have good theoretical convergence properties even if the arising subproblems are solved inexactly. Nevertheless, experience shows that the choice of the parameter τ penalizing the constraint violation in the Augmented Lagrangian underlying ADMM affects the method’s performance. To this end, strategies for the adaptive selection of such parameter have been analyzed in the literature and are still of great interest. In this paper, starting from an adaptive spectral strategy recently proposed in the literature, we investigate the use of different strategies based on Barzilai–Borwein-like stepsize rules. We test the effectiveness of the proposed strategies in the solution of real-life consensus logistic regression and portfolio optimization problems.

Swarm intelligence hybridized with genetic search in multi-objective design optimization under constrained-Pareto dominance

The inclusion of uncertainty in structural design optimization led to more complex design optimization formulations, where uncertainty quantification has significant influence on solution methods and computing times. Designs maintaining steady levels of performance, under uncertainty, are called robust and the combination of both robustness and performance optimality leads to a methodology called Robust Design Optimization (RDO). In this work a new approach to the RDO of angle-ply composite laminate structures is proposed. The key concept of this methodology is the hybridization between the principles of Particle Swarm Optimization and Genetic Algorithms, through the evolution of multiple populations based on local and global constrain-dominance. The RDO problem is here defined as the bi-objective minimization of the structural weight (optimality) and the determinant of the variance–covariance matrix (robustness) of the system’s response functionals, subject to stress and displacement constraints. A numerical structural design problem, representing a composite laminate engine hood shell, shows the capabilities of the proposed approach. Results display the good convergence properties of the proposed hybridizations both in the search and objective spaces.

A modified Chebyshev–Halley‐type iterative family with memory for solving nonlinear equations and its stability analysis

In this paper, we propose a new iterative scheme with memory for solving nonlinear equations numerically in order to achieve higher order of convergence in comparison to the cubically convergent Chebyshev–Halley‐type method. Several modifications on Chebyshev–Halley‐type methods without memory have been considered in order to extend it to the scheme with memory. We have used self accelerating parameter in order to attain acceleration of convergence speed which is estimated from the current and previous iterations using divided differences. Therefore, the order of convergence increases from 3 to 3.30 without any further functional evaluation. We study the complex and real dynamics of the proposed family. The parameter spaces and dynamical planes are presented. From the parameter spaces, we can detect different members of the proposed family that have good and bad convergence properties. From the dynamical planes, we can analyze the stability of the proposed family in terms of different values of the parameter involved. This study aids in determining the family members with stable behavior which in turn are suitable for practical problems. Numerical examples and comparisons with some of the existing methods are included to confirm the theoretical results. Furthermore, basins of attraction are included to describe a clear picture of the convergence of the proposed as well as some of the existing methods.

Reduced representations of assumed fields for Hu–Washizu solid-shell element

Mixed eight-node (hexahedron) solid-shell elements based on the standard or partial version of the three-field Hu–Washizu (HW) functionals are developed for Green strain. Three reduced representations of the assumed stress/strain fields are selected. They improve effectiveness, yet retaining good accuracy and convergence properties. At the outset, the standard HW functional and the assumed stress/strain representations of the 3D solid element B8-15P (Weissman in Int J Numer Methods Eng 39:2337–2361, 1996) are used to derive a solid-shell element with 51 parameters. To eliminate locking, the ANS method is applied to the thickness strain (Betsch and Stein in Commun Numer Methods Eng 11:899–909, 1995) and to the transverse shear strain (Dvorkin and Bathe in Eng Comput 1:77–88, 1984). It is a correct element which, however, yields too large displacements for coarse meshes and trapezoidal through-thickness shapes. To improve the above formulation, the zeta -independent reduced representations of the assumed stress/ strain fields are selected and the transformations to Cartesian components are modified. The thickness strain is enhanced by the EAS method. The element with 35 parameters is derived from the standard/enhanced HW functional, but, to further reduce the assumed fields, partial/enhanced HW functionals are constructed from the 3D potential energy by applying the Lagrange multiplier method only to selected strain components. In the element with 27 parameters, this is applied to the constant in-plane strain and to the transverse shear strain while in the element with 19 parameters, to the constant in-plane strain only.Two other modifications are implemented to enhance the behavior of these elements: (A) the skew coordinates are used in the reduced representations of the in-plane stress/strain (Wisniewski and Turska in Int J Numer Methods Eng 90:506–536, 2012), and (B) the Residual Bending Flexibility correction of the transverse shear stiffness (MacNeal in Comput Struct 8(2):175–183, 1978) is adapted. Finally, the performance of the proposed solid-shell HW elements is demonstrated on several linear and non-linear examples for the linear elastic material and the hyper-elastic material. The proposed elements are compared to each other and to the best existing elements of this class.

Two modified conjugate gradient methods for solving unconstrained optimization and application

Conjugate gradient methods are a popular class of iterative methods for solving linear systems of equations and nonlinear optimization problems as they do not require the storage of any matrices. In order to obtain a theoretically effective and numerically efficient method, two modified conjugate gradient methods (called the MCB1 and MCB2 methods) are proposed. In which the coefficientβkin the two proposed methods is inspired by the structure of the conjugate gradient parameters in some existing conjugate gradient methods. Under the strong Wolfe line search, the sufficient descent property and global convergence of the MCB1 method are proved. Moreover, the MCB2 method generates a descent direction independently of any line search and produces good convergence properties when the strong Wolfe line search is employed. Preliminary numerical results show that the MCB1 and MCB2 methods are effective and robust in minimizing some unconstrained optimization problems and each of these modifications outperforms the four famous conjugate gradient methods. Furthermore, the proposed algorithms were extended to solve the problem of mode function.

Link prediction and its optimization based on low-rank representation of network structures

Currently, among the link prediction researches based on low-rank representation, almost none of literature has considered how to select an appropriate base-matrix and the impact of the structural characteristics of the reconstructed network on the link prediction. Therefore, in this paper, we use the adjacency matrix of the fully connected network (FCN) as the base-matrix for low-rank representation, and any local structure of the observed networks can be represented by the interactions of FCN structures. To explore the properties of link predictions, the nuclear norm of the adjacency matrix for the reconstructed network is taken as a penalty term in the newly proposed low-rank representation objective function. According to the optimal interactive coefficients achieved by solving the novel objective function, in this paper, we design a novel link prediction algorithm (LRNP algorithm) and its optimized algorithm (OLRNP algorithm). Experimental results based on real networks and synthetic networks lead to several conclusions. (1) The LRNP algorithm has good convergence properties. When changing the parameters of the LRNP algorithm, the changes along prediction performance do not exceed 9.48%. LRNP also performs well for sparse networks. (2) Compared with baseline link prediction algorithms, LRNP also shows excellent performance, and its AUC and Precision can increase by 14.35% and 14.89%. (3) The OLRNP algorithm exhibits better performance than the LRNP algorithm, and its AUC and Precision can rise by up to 7.50% and 6.79%, respectively. The data and codes are publicly available at https://github.com/pinglanchu/LRNP-OLRNP.

A Three-Term Conjugate Gradient Method for Non-Convex Functions with Applications for Heat Transfer

The problem of unconstrained optimization (UOP) has recently gained a great deal of attention from researchers around the globe due to its numerous real-life applications. The conjugate gradient (CG) method is among the most widely used algorithms for solving UOP because of its good convergence properties and low memory requirements. This study investigates the performance of a modified CG coefficient for optimization functions, proof of sufficient descent, and global convergence of the new CG method under suitable, standard Wolfe conditions. Computational results on several benchmark problems are presented to validate the robustness and efficacy of the new algorithm. The proposed method was also applied to solve function estimations in inverse heat transfer problems. Another interesting feature possessed by the proposed modification is the ability to solve problems on a large scale and use different dimensions. Based on the theoretical and computational efficiency of the new method, we can conclude that the new coefficient can be a better alternative for solving unconstrained optimization and real-life application problems. KEYWORDS Computational efficiency, global convergence, inverse heat, low memory, optimization problems, theoretical efficiency

An efficient elastoplastic model for the analysis of reinforced concrete shells

Shell structures made of Reinforced Concrete (RC) are used in many civil engineering applications. However, their analysis can frequently require a high computational cost, since fine mesh discretisations are needed for accurate solutions. In this work, a Finite Element (FE) for the analysis of RC shells is presented. It has four nodes and 24 degrees of freedom and both generalised stresses and displacements are interpolated. In particular, the assumed stress field a-priori satisfies equilibrium equations, thereby ensuring good accuracy and convergence properties. The heterogeneous material is described through a layer-wise approach in which a 3D stress yield function is used for concrete. Results confirm that the proposed FE is accurate on coarse meshes and shows good convergence properties if compared with state-of-art alternatives. The approach can be readily extended to model additional material layers, for example reinforcements applied for retrofitting purposes.

A Generalized Stochastic Approximation for the Recursive System Identification

This paper presents an approach for constructing probabilistic approximation type algorithms used in system identification systems. The proposed approach allows obtaining recursive identification algorithms under fairly mild assumptions about noise and disturbances that distort the system. The obtained algorithm does not require inversion of the Hessian of the identification criterion and is robust to changes in the order of the Hessian. This example demonstrates good convergence properties of the obtained algorithm compared to conventional recursive systems.

A Dai-Liao-type projection method for monotone nonlinear equations and signal processing

Abstract In this article, inspired by the projection technique of Solodov and Svaiter, we exploit the simple structure, low memory requirement, and good convergence properties of the mixed conjugate gradient method of Stanimirović et al. [New hybrid conjugate gradient and broyden-fletcher-goldfarbshanno conjugate gradient methods, J. Optim. Theory Appl. 178 (2018), no. 3, 860–884] for unconstrained optimization problems to solve convex constrained monotone nonlinear equations. The proposed method does not require Jacobian information. Under monotonicity and Lipschitz continuity assumptions, the global convergence properties of the proposed method are established. Computational experiments indicate that the proposed method is computationally efficient. Furthermore, the proposed method is applied to solve the ℓ 1 {\ell }_{1} -norm regularized problems to decode sparse signals and images in compressive sensing.

Stabilization of Generalized Empirical Interpolation Method (GEIM) in presence of noise: A novel approach based on Tikhonov regularization

The Empirical Interpolation Method (EIM), and its generalized version (GEIM), are non-intrusive, reduced-basis model order reduction methods hereby adopted and modified to address the problem of optimal placement of sensors and real-time estimation in thermo-hydraulics systems. These techniques have been used to extract the characteristic spatial modes of the system and select a set of points (or functionals) corresponding to the optimal locations for the sensors. Collecting experimental measurements in the available points allows the construction of an empirical interpolation of the fields employed to estimate the variable of interest. However, when these data are affected by noise, the (G)EIM loses its good convergence properties. In this context, stabilization techniques allow good field reconstruction even with noisy data. This work provides an alternative and effective solution to the problem of reconstructing the system state in the presence of experimental data affected by random noise by using the Tikhonov regularization technique. The developed methods have been tested on a simple thermo-fluid dynamics problem known as “two-sided lid-driven differentially heated square cavity”.

High fidelity coupling methods for blast response of thin shell structures

Computational simulation of structures subjected to blast loads requires integration of computational shock physics and structural response with finite deformations. The authors’ particular application of interest is blast loads on thin shell structures, which often deform concurrently with the shock or pressure wave evolution over the structure. This necessitates two-way coupled algorithms. We combine state-of-the-art shock physics, structural modeling with shell finite elements, and an immersed boundary method that accommodates arbitrarily thin structures. Building on successful techniques in the literature, we focus on accuracy and convergence rates of novel extensions to time step coupling approaches and immersed boundary treatments. The new techniques are developed to easily integrate with typical, industry-standard production analysis codes. The final recommended technique combines centered-difference structural time integration, predictor–corrector fluid time integration, level set surface tracking, and the Half Riemann Immersed Boundary method. Examples are given that show robust and accurate modeling of shock and pressure wave problems, and verification problems highlight good convergence properties for both fluid and fluid–structure applications. • Fluid–structure interaction algorithms for blast loads on thin structures. • Combine shock-physics, shell finite elements, and immersed boundaries. • Explore convergence rates of time step coupling and immersed boundary treatments. • Coupling approach takes advantage of convergence rates of underlying solvers.

Machine Learning Methods for Data-Driven Demand Estimation and Assortment Planning Considering Cross-Selling and Substitutions

This study develops machine learning methods for the data-driven demand estimation and assortment planning problem by addressing three subproblems, that is, demand forecasting simultaneously considering cross-selling and substitutions, estimation of the cross-selling and substitution effects, and assortment optimization. These three subproblems are transformed into three sequentially related machine learning problems: collective demand forecasting, demand inference for cross-selling and substitutions, and assortment rule mining. For collective demand forecasting, related product features are introduced to consider both the cross-selling and substitution effects, and a collaborative coordinate descent method with a good convergence property is developed to make distributed demand forecasting and a global update of related product features. Using the results, demand inference adopts transfer and semisupervised learning methods to tackle the challenge of missing data in quantifying the cross-selling and substitution effects. For assortment rule mining, the assortment rules bridge the gap between prediction and optimization, and the developed heuristics obtain the best assortment using the prior knowledge discovered in demand inference. The computational results on a real-world database and a semisynthetic database show that collective demand forecasting obtained far better results than the standard demand forecasting methods and some popular graph learning methods, and the developed heuristics identified much better assortments than those obtained with the baseline methods.History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning.Funding: This work was supported by the construction base project of discipline innovation and talent introduction plan of Chinese higher educational institutions (111 project) [Grant B16009] and the National Natural Science Foundation of China [Grant 72031002].Supplemental Material: The online appendices are available at https://doi.org/10.1287/ijoc.2022.1251 .

The Convergent Properties of a New Parameter for Unconstrained Optimization

Because of its simplicity, low memory requirement, low computational cost, and global convergence properties, the Conjugate Gradient (CG) method is the most popular iterative mathematical technique for optimizing both linear and nonlinear systems. Some classical CG methods, however, have drawbacks such as poor global convergence and numerical performance in terms of iterations and function evaluations. To address these shortcomings, researchers proposed new CG parameter variants with efficient numerical results and good convergence properties. We present a new conjugate gradient formula based on the memoryless self-scale DFP quasi-Newton (QN) method in this paper. The proposed new formula fulfills the sufficient descent property and the global convergent condition with any proposed line research. When the exact line search is used, the proposed formula is reduced to the classical HS formula. Finally, we conclude that our proposed method is effective.

Sunrise strategy for the continuity of maximal operators

In this paper we address the W1,1-continuity of several maximal operators at the gradient level. A key idea in our global strategy is the decomposition of a maximal operator, with the absence of strict local maxima in the disconnecting set, into “lateral” maximal operators with good monotonicity and convergence properties. This construction is inspired in the classical sunrise lemma in harmonic analysis. A model case for our sunrise strategy considers the uncentered Hardy—Littlewood maximal operator $$\tilde M$$ acting on W rad 1,1 (ℝd), the subspace of W1,1(ℝd) consisting of radial functions. In dimension d ≥ 2 it was recently established by H. Luiro that the map $$f \mapsto \nabla \tilde Mf$$ is bounded from W rad 1,1 (ℝd) to L1(ℝd), and we show that such a map is also continuous. Further applications of the sunrise strategy in connection with the W1,1-continuity problem include non-tangential maximal operators on ℝd acting on radial functions when d ≥ 2 and general functions when d = 1, and the uncentered Hardy—Littlewood maximal operator on the sphere $${^d}$$ acting on polar functions when d ≥ 2 and general functions when d = 1.

Multiplicities and Betti numbers in local algebra via lim Ulrich points

This work concerns finite free complexes with finite length homology over a commutative noetherian local ring $R$. The focus is on complexes that have length $\mathrm{dim}\, R$, which is the smallest possible value, and in particular on free resolutions of modules of finite length and finite projective dimension. Lower bounds are obtained on the Euler characteristic of such short complexes when $R$ is a strict complete intersection, and also on the Dutta multiplicity, when $R$ is the localization at its maximal ideal of a standard graded algebra over a field of positive prime characteristic. The key idea in the proof is the construction of a suitable Ulrich module, or, in the latter case, a sequence of modules that have the Ulrich property asymptotically, and with good convergence properties in the rational Grothendieck group of $R$. Such a sequence is obtained by constructing an appropriate sequence of sheaves on the associated projective variety.

A general method for directly phasing diffraction data from high-solvent-content protein crystals.

A procedure is described for direct phase determination in protein crystallography, applicable to crystals with high solvent content. The procedure requires only the diffraction data and an estimate of the solvent content as input. Direct phase determination is treated as a constraint satisfaction problem, in which an image is sought that is consistent with both the diffraction data and generic constraints on the density distribution in the crystal. The problem is solved using an iterative projection algorithm, the Difference Map algorithm, which has good global convergence properties, and can locate the correct solution without any initial phase information. Computational efficiency is improved by breaking the problem down into two stages; initial approximation of the molecular envelope at low resolution, followed by subsequent phase determination using all of the data. The molecular envelope is continually updated during the phase determination step. At both stages, the algorithm is initiated with many different and random phase sets, which are evolved subject to the constraints. A clustering procedure is used to identify consistent results across multiple runs, which are then averaged to generate consensus envelopes or phase sets. The emergence of highly consistent phase sets is diagnostic of success. The effectiveness of the procedure is demonstrated by application to 42 known structures of solvent fraction 0.60-0.85. The procedure works robustly at intermediate resolutions (1.9-3.5 Å) but is strongly dependent on crystal solvent content, only working routinely with solvent fractions greater than 0.70.

Multi-agent reinforcement learning for long-term network resource allocation through auction: A V2X application

We formulate offloading of computational tasks from a dynamic group of mobile agents (e.g., cars) as decentralized decision making among autonomous agents. We design an interaction mechanism that incentivizes such agents to align private and system goals by balancing between competition and cooperation. In the static case, the mechanism provably has Nash equilibria with optimal resource allocation. In a dynamic environment, this mechanism’s requirement of complete information is impossible to achieve. For such environments, we propose a novel multi-agent online learning algorithm that learns with partial, delayed and noisy state information, thus greatly reducing information need. Our algorithm is also capable of learning from long-term and sparse reward signals with varying delay. Empirical results from the simulation of a V2X application confirm that through learning, agents with the learning algorithm significantly improve both system and individual performance, reducing up to 30% of offloading failure rate, communication overhead and load variation, increasing computation resource utilization and fairness. Results also confirm the algorithm’s good convergence and generalization property in different environments.

Data driven approach in multiphysics framework: Application to coupled electro-mechanical problems

Solving coupled multiphysics problems using finite elements and conventional modeling approach requires building, from coupled finite element formulation often in the same code, the monophysical operators and the multiphysics coupling operators. In this work the data driven, model-free computational approach is fitted in the multiphysics framework adapted for ‘smart’ materials. Thus, in this work we ought to expand the phase space and propose a new norm for the distance based data driven solver adapted to the problem at hands. Given the good material database which naturally encodes the coupling interactions, we show that proposed application of data driven approach enables to avoid the coupling tangent terms altogether. In other words, the data driven approach permits to decouple the different ‘physics’ and to manage the coupling from the data. We illustrate the performance and robustness of the approach on two examples related to lattice model composed of planar, piezoelectric truss network and of finite element discretized linear piezoelectric solids. The numerical tests show good convergence properties related to the number of data points, as well as the choice of material and weighting parameters.

A New Diffusive Representation for Fractional Derivatives, Part II: Convergence Analysis of the Numerical Scheme

Recently, we have proposed a new diffusive representation for fractional derivatives and, based on this representation, suggested an algorithm for their numerical computation. From the construction of the algorithm, it is immediately evident that the method is fast and memory-efficient. Moreover, the method’s design is such that good convergence properties may be expected. In this paper, we commence a systematic investigation of these convergence properties.