Optimal Convergence Research Articles

Model Averaging Under Flexible Loss Functions

To address model uncertainty under flexible loss functions in prediction problems, we propose a model averaging method that accommodates various loss functions, including asymmetric linear and quadratic loss functions as well as many other asymmetric/symmetric loss functions as special cases. The flexible loss function allows the proposed method to average a large range of models such as the quantile and expectile regression models. To determine the weights of the candidate models, we establish a J-fold cross-validation criterion. Asymptotic optimality and weight convergence are proved for the proposed method. Simulations and an empirical application show the superior performance of the proposed method compared with other methods of model selection and averaging. History: Accepted by Ram Ramesh, Area Editor for Data Science and Machine Learning. Funding: This work was supported by the Beijing Natural Science Foundation [Grant Z240004], Japan Society for the Promotion of Science (KAKENHI) [Grant 22H00833 to Q. Liu], the CAS Project for Young Scientists in Basic Research [Grant YSBR-008], and the National Natural Science Foundation of China [Grants 71925007, 72091212 and 72495124]. Supplemental Material: The software that supports the findings of this study is available within the paper and its Supplemental Information ( https://pubsonline.informs.org/doi/suppl/10.1287/ijoc.2023.0291 ) as well as from the IJOC GitHub software repository ( https://github.com/INFORMSJoC/2023.0291 ). The complete IJOC Software and Data Repository is available at https://informsjoc.github.io/ .

A novel method of MPPT of photovoltaic power generation system based on MSINGO

Abstract The speed and accuracy of Maximum Power Point Tracking (MPPT) have a significant impact on photovoltaic power generation. In this paper, a novel method is proposed for fast and accurate MPPT of photovoltaic power generation systems. First, the Northern Goshawk Optimization algorithm (NGO) is introduced in the MPPT. Second, a multi strategy is applied to improve the NGO, and MSINGO is proposed. In the MSINGO algorithm, the optimal individual leadership strategy enhances the optimization accuracy and convergence speed, the enhanced Levy flight strategy leads it to escape from the local optimum, and the pinhole imaging learning strategy guides the population towards the global optimum. Finally, MSINGO was applied to MPPT, and a simulation analysis of a single photovoltaic module and photovoltaic array under different working conditions was conducted and compared with other intelligent optimization algorithms. The results show that the proposed method can achieve MPPT with minimal time consumption and better tracking efficiency.

Multi-Strategy Improved Whale Optimization Algorithm and Its Engineering Applications

The Whale Optimization Algorithm (WOA) is recognized for its simplicity, few control parameters, and effective local optima avoidance. However, it struggles with global search efficiency and slow convergence. This paper introduces the Improved WOA (ImWOA) to overcome these challenges. Initially, ImWOA utilizes a dynamic elastic boundary optimization strategy, which leverages boundary information and the current optimal position to guide solutions that exceed the boundaries back within permissible limits, gradually converging towards the optimal solution. Subsequently, ImWOA integrates an advanced random searching strategy that equilibrates global and local searches by focusing on the current optimal location and the mean position of all individuals. Lastly, a combined mutation mechanism is employed to enhance population diversity, prevent the algorithm from stagnating in local optima, and consequently augment its overall search capability. Performance evaluations on CEC2017 benchmark functions show ImWOA outperforming five metaheuristic algorithms and three WOA variants in optimization accuracy, stability, and convergence speed. ImWOA excelled in 25 out of 29 test functions in 30D and 26 out of 29 in 100D scenarios. Furthermore, its efficacy in addressing complex challenges is corroborated by real-world applications in reducer design, vehicle side impact design, and welded beam design, highlighting its potential utility across various engineering domains.

A time-domain finite element formulation of the equivalent fluid model for the acoustic wave equation

Sound-absorptive materials such as foam can be described by the equivalent fluid (EF) model. The homogenized fluid’s acoustic behavior is thereby described by complex-valued, frequency-dependent acoustic material parameters. When transforming the acoustic wave equation for the EF model from the frequency domain to the time domain, convolution integrals arise. The auxiliary differential equation (ADE) method is used to circumvent the direct calculation of these convolution integrals. The wave equation and the coupled set of ordinary ADEs are solved in the time domain using the finite element (FE) method. The approach relies on approximating the complex-valued frequency response functions of the inverse equivalent bulk modulus and density by a sum of rational functions consisting of real and complex poles. The order of the rational function approximation defines the number of additionally introduced auxiliary variables per nodal degree of freedom. The presented FE formulation includes a narrow-band non-reflecting boundary condition (NRBC) for normal incidence. The implementation in openCFS shows optimal temporal and spatial convergence for a semi-infinite duct based on the analytic plane wave solution for harmonic excitation. The simulation of a pressure pulse propagating in an infinite EF domain with a scatterer demonstrates the capability for multidimensional, actual transient problems.

The non-intrusive reduced basis two-grid method applied to sensitivity analysis

This paper deals with the derivation of Non-Intrusive Reduced Basis (NIRB) techniques for sensitivity analysis, more specifically the direct and adjoint state methods. For highly complex parametric problems, these two approaches may become too costly ans thus Reduced Basis Methods (RBMs) may be a viable option. We propose new NIRB two-grid algorithms for both the direct and adjoint state methods in the context of parabolic equations. The NIRB two-grid method uses the HF code solely as a “black-box”, requiring no code modification. Like other RBMs, it is based on an offline-online decomposition. The offline stage is time-consuming, but it is only executed once, whereas the online stage employs coarser grids and thus, is significantly less expensive than a fine HF evaluation. On the direct method, we prove on a classical model problem, the heat equation, that HF evaluations of sensitivities reach an optimal convergence rate in L∞(0, T ; H10(Ω)), and then establish that these rates are recovered by the NIRB two-grid approximation. These results are supported by numerical simulations. We then propose a new procedure that further reduces the computational costs of the online step while only computing a coarse solution of the state equations. On the adjoint state method, we propose a new algorithm that reduces both the state and adjoint solutions. All numerical results are run with the model problem as well as a more complex problem, namely the Brusselator system.

Optimal convergence analysis of second‐order time‐discrete stabilized scheme for the thermally coupled incompressible MHD system

AbstractIn this paper, we construct an optimal convergence analysis of second‐order stabilized scheme for the thermally coupled incompressible magnetohydrodynamic (MHD) system. First, we construct first‐ and second‐order time‐discrete schemes in which the time derivative term is treated by the first‐order backward Euler method and the second‐order backward difference formulation, respectively. And the nonlinear terms are treated by semi‐implicit method. Importantly, we use the Gauge–Uzawa method to decouple velocity and pressure. The proposed schemes have the following two distinct features: they do not need to give an initial value of the pressure, and they do not require artificial boundary conditions on the pressure. Second, the unconditional energy stability of the two schemes is proved. Then, through rigorous error analysis, we provide optimal convergence orders for all unknowns. Finally, some numerical experiments demonstrate the accuracy and effectiveness of the proposed schemes.

Stabilized Unfitted Finite Element Method for Poroelasticity With Weak Discontinuity

AbstractPoromechanics problems in geotechnical and geological contexts often involve complex formations with numerous boundaries and material interfaces, which significantly complicate numerical analysis and simulation. The traditional finite element method (FEM) encounters substantial challenges in these scenarios because it requires the mesh to conform precisely to each boundary and interface. This requirement complicates preprocessing and necessitates meticulous manual control to achieve a high‐quality mesh. In contrast, unfitted FEMs are well‐suited for these problems as they do not require the mesh to align with the model geometry. We propose a stabilized unfitted FEM that incorporates Nitsche's method and ghost penalty stabilization techniques to address complex poroelasticity problems. This approach treats material interfaces as weak discontinuities and ensures that compatibility conditions are satisfied. The proposed method allows the mesh to be independent of both boundaries and material interfaces. Nitsche's method is used to weakly enforce both Dirichlet boundary conditions and interface compatibility conditions, resulting in a symmetric weak form. Additionally, three types of ghost penalty terms are introduced for elements intersected by boundaries or interfaces, effectively eliminating cut‐induced ill‐conditioning. The proposed methodology has been validated through benchmark and practical problems, demonstrating optimal convergence and exceptional stability. This approach significantly enhances the stability and efficiency of hydro‐mechanical analyses for complex geotechnical and geological problems.

Synthesis of the Large-Scaled 64 × 64 Thinned Array Using the Branch and Bound Technique with Convex Optimization for Satellite Communication

This paper presents the synthesis of a 64 × 64 thinned array, using the branch and bound technique with convex optimization and a low sidelobe level for satellite communication. The branch and bound technology decomposes optimization into several subproblems. The convex optimization transforms the optimized variable binary [0, 1] of the thinned array into a continuous variable, i.e., from 0 to 1. Based on the branch and bound technique with convex optimization, the fast and accurate convergence of the synthesis of large-scale thinned array was achieved, and 2400 elements were selected for a full 64 × 64 array with a sidelobe level that satisfies the system requirements. The proposed thinned array with 2400 elements was simulated via full-wave 3D simulation. A prototype of the proposed thinned array was fabricated and measured. The simulated and measured results verified the effectiveness of the branch and bound technique with convex optimization.

Autonomous Decision-Making for Air Gaming Based on Position Weight-Based Particle Swarm Optimization Algorithm

As the complexity of air gaming scenarios continues to escalate, the demands for heightened decision-making efficiency and precision are becoming increasingly stringent. To further improve decision-making efficiency, a particle swarm optimization algorithm based on positional weights (PW-PSO) is proposed. First, important parameters, such as the aircraft in the scenario, are modeled and abstracted into a multi-objective optimization problem. Next, the problem is adapted into a single-objective optimization problem using hierarchical analysis and linear weighting. Finally, considering a problem where the convergence of the particle swarm optimization (PSO) is not enough to meet the demands of a particular scenario, the PW-PSO algorithm is proposed, introducing position weight information and optimizing the speed update strategy. To verify the effectiveness of the optimization, a 6v6 aircraft gaming simulation example is provided for comparison, and the experimental results show that the convergence speed of the optimized PW-PSO algorithm is 56.34% higher than that of the traditional PSO; therefore, the algorithm can improve the speed of decision-making while meeting the performance requirements.

Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion.

Maximum correntropy criterion (MCC) has been an important method in machine learning and signal processing communities since it was successfully applied in various non-Gaussian noise scenarios. In comparison with the classical least squares method (LS), which takes only the second-order moment of models into consideration and belongs to the convex optimization problem, MCC captures the high-order information of models that play crucial roles in robust learning, which is usually accompanied by solving the non-convexity optimization problems. As we know, the theoretical research on convex optimizations has made significant achievements, while theoretical understandings of non-convex optimization are still far from mature. Motivated by the popularity of the stochastic gradient descent (SGD) for solving nonconvex problems, this paper considers SGD applied to the kernel version of MCC, which has been shown to be robust to outliers and non-Gaussian data in nonlinear structure models. As the existing theoretical results for the SGD algorithm applied to the kernel MCC are not well established, we present the rigorous analysis for the convergence behaviors and provide explicit convergence rates under some standard conditions. Our work can fill the gap between optimization process and convergence during the iterations: the iterates need to converge to the global minimizer while the obtained estimator cannot ensure the global optimality in the learning process.

A Multi-Strategy Improved Honey Badger Algorithm for Engineering Design Problems

A multi-strategy improved honey badger algorithm (MIHBA) is proposed to address the problem that the honey badger algorithm may fall into local optimum and premature convergence when dealing with complex optimization problems. By introducing Halton sequences to initialize the population, the diversity of the population is enhanced, and premature convergence is effectively avoided. The dynamic density factor of water waves is added to improve the search efficiency of the algorithm in the solution space. Lens opposition learning based on the principle of lens imaging is also introduced to enhance the ability of the algorithm to get rid of local optimums. MIHBA achieves the best ranking in 23 test functions and 4 engineering design problems. The improvement of this paper improves the convergence speed and accuracy of the algorithm, enhances the adaptability and solving ability of the algorithm to complex functions, and provides new ideas for solving complex engineering design problems.

Multi-Class Sunflower Disease Detection by Integrating Enhanced Jellyfish Search Algorithm

Objectives: This research aims to classify diseases observed in sunflower flowers and leaves using deep learning algorithms. The goal is to enhance agricultural output by addressing the significant issue of plant diseases, which negatively affect the security of the food supply. Methods: The study utilizes a feature selection approach to identify multi-class sunflower diseases. The Enhanced Jellyfish Search (EJFSOA) algorithm is improved in three ways: (i) optimization capabilities and convergence speed are enhanced through sine and cosine learning variables during Type B motion in the swarm, (ii) a local escape operator is added to avoid the local optimization trap, improving the exploitability of the JS algorithm, and (iii) opposition-based learning and quasi-opposition learning strengthen the population distribution. A deep learning model, Multi-class Sunflower Network (MCSFNet), combines ResNets with TensorFlow for feature extraction, multi-label classification, and a multi-spectral stacking module. Findings: The proposed model is evaluated using several criteria, including accuracy, sensitivity, specificity, and F1-score. On a publicly available dataset, the algorithm achieves an average accuracy of 99.56%, outperforming state-of-the-art approaches with accuracies of 98.34% and 97.39%. Novelty: This work introduces improvements to the EJFSOA algorithm, significantly enhancing the precision and convergence speed of sunflower disease classification models. Combining deep learning with an optimized feature selection method leads to superior results in multi-class classification tasks. Keywords: Sunflower Leaves, Disease Detection, Enhanced jellyfish search, Local escape operator, Multi­class Sunflower Network, Residual Networks

Stochastic First-Order Methods for Average-Reward Markov Decision Processes

We study average-reward Markov decision processes (AMDPs) and develop novel first-order methods with strong theoretical guarantees for both policy optimization and policy evaluation. Compared with intensive research efforts in finite sample analysis of policy gradient methods for discounted MDPs, existing studies on policy gradient methods for AMDPs mostly focus on regret bounds under restrictive assumptions, and they often lack guarantees on the overall sample complexities. Toward this end, we develop an average-reward stochastic policy mirror descent method for solving AMDPs with and without regularizers and provide convergence guarantees in terms of the long-term average reward. For policy evaluation, existing on-policy methods suffer from suboptimal convergence rates as well as failure in handling insufficiently random policies due to the lack of exploration in the action space. To remedy these issues, we develop a variance-reduced temporal difference (VRTD) method with linear function approximation for randomized policies along with optimal convergence guarantees, and design an exploratory VRTD method that resolves the exploration issue and provides comparable convergence guarantees. By combining the policy evaluation and policy optimization parts, we establish sample complexity results for solving AMDPs under both generative and Markovian noise models. It is worth noting that when linear function approximation is utilized, our algorithm only needs to update in the low-dimensional parameter space, and thus can handle MDPs with large state and action spaces. Funding: T. Li and G. Lan were supported in part by the National Science Foundation [Grant CCF-1909298] and the Office of Navel Research [Grant N00014-20-1-2089].

Asymptotic normality and finite-sample robustness of the Fourier spot volatility estimator in the presence of microstructure noise

We study the efficiency and robustness of the Fourier spot volatility estimator by Malliavin and Mancino [2002] when high-frequency prices are contaminated by microstructure noise. Firstly, we show that the estimator is consistent and asymptotically efficient in the presence of additive noise, establishing a Central Limit Theorem (CLT) with the optimal rate of convergence n 1 / 8 . Additionally, we complete the asymptotic theory proved by Mancino and Recchioni [2015] in the absence of noise, obtaining a CLT with the optimal rate of convergence n 1 / 4 . Feasible CLTs with the optimal convergence rate are also obtained, by proving the consistency of the Fourier estimators of the spot volatility of volatility and the quarticity in the presence of noise. Secondly, we introduce a feasible method for selecting the cutting frequencies of the estimator in the presence of noise, based on the optimization of the integrated asymptotic variance. Finally, we provide support to the accuracy and robustness of the method by means of a numerical study and an empirical exercise, which is conducted using tick-by-tick prices of three US stocks with different liquidity.

A fully-hybrid finite element formulation for general compressible-incompressible elasticity problems using de Rham compatible spaces

This work proposes a novel fully-hybrid finite element formulation for general elasticity problems, combining $H(\text{div},\Omega)$ conforming vector functions for displacement and $L^2(\Omega)$ discontinuous scalar functions for pressure. This pair is De Rham compatible, which means that within the incompressible regime, the divergence-free constrain will hold strongly at element level. As the $H(\text{div})$ spaces only present continuity of the normal displacements across elements boundaries, the tangential displacement continuity can be weakly imposed performing a hybridization of the shear stresses using $H^1(\partial\Omega_e)$ functions. From past researches conducted at LabMeC, this approach has demonstrated to pose some numerical difficulties as it leads to a saddle point problem with two constraint variables - the pressure and the shear-stresses. In this work, a second hybridization is done by approximating the tangential component of the primal displacement variable using $H^1(\partial\Omega_e)$ space. Two benchmarks are used to verify the developed numerical scheme - the classical Cook's membrane and a tridimensional cantilever beam subjected to an end shear load. Optimum convergence rate is achieved under compressible, quasi-incompressible and full incompressible scenarios.

Virtual Element Method: An overview of formulations

The virtual element method has been around for about a decade, and it has been through a lot of development during this period. The method generalizes the finite element method for polytopal elements, while retaining optimal convergence properties. This is achieved mainly by implicitly defined function spaces and the use of polynomial projections. This work presents an overview of the development of the method as formulated for elliptic problems in two and three dimensions, here represented by Poisson’s equations. The formulations covered are: its original formulation, the modified formulation enabling three-dimensional elements, the Serendipity formulation, and one of the formulations for self-stabilized elements.

Isogeometric analysis of the Laplace eigenvalue problem on circular sectors: Regularity properties and graded meshes

The Laplace eigenvalue problem on circular sectors has eigenfunctions with corner singularities. Standard methods may produce suboptimal approximation results. To address this issue, a novel numerical algorithm that enhances standard isogeometric analysis is proposed in this paper by using a single-patch graded mesh refinement scheme. Numerical tests demonstrate optimal convergence rates for both the eigenvalues and eigenfunctions. Furthermore, the results show that smooth splines possess a superior approximation constant compared to their C0-continuous counterparts for the lower part of the Laplace spectrum. This is an extension of previous findings about excellent spectral approximation properties of smooth splines on rectangular domains to circular sectors. In addition, graded meshes prove to be particularly advantageous for an accurate approximation of a limited number of eigenvalues. Finally, a hierarchical mesh structure is presented to avoid anisotropic elements in the physical domain and to omit redundant degrees of freedom in the vicinity of the singularity. Numerical results validate the effectiveness of hierarchical mesh grading for simulating eigenfunctions of low and high regularity.

An iterative constraint energy minimizing generalized multiscale finite element method for contact problem

This work presents an Iterative Constraint Energy Minimizing Generalized Multiscale Finite Element Method (ICEM-GMsFEM) for solving the contact problem with high contrast coefficients. The model problem can be characterized by a variational inequality, where we add a penalty term to convert this problem into a non-smooth and non-linear unconstrained minimizing problem. The characterization of the minimizer satisfies the variational form of a mixed Dirilect-Neumann-Robin boundary value problem. So we apply CEM-GMsFEM combined with simismooth Newton method and introduce special boundary correctors along with multiscale spaces to achieve an optimal convergence rate. Numerical results are conducted for different highly heterogeneous permeability fields, validating the fast convergence of the CEM-GMsFEM iteration in handling the contact boundary and illustrating the stability of the proposed method with different sets of parameters. We also prove the fast convergence of the proposed iterative CEM-GMsFEM method and provide an error estimate of the multiscale solution under a mild assumption.

Conservative numerical algorithm for simulating thermoelectrical semiconductor device with unconditional optimal convergence analysis

We propose conservative type numerical method to simulate thermoelectrical semiconductor device problem, in which mixed finite element method used for electric potential equation, conservative characteristic finite element method for electron and hole concentration equations, and standard finite element method for heat conduction equation. By temporal-spatial error splitting argument, the optimal error estimates without certain time step restriction are derived, and low order convergence rate of electrostatic potential and electric field intensity will not affect the accuracy of the electron, hole density and temperature. Numerical tests are performed to validate the theoretical results and application performance of the given method.

Convergence analysis of exponential time differencing scheme for the nonlocal Cahn–Hilliard equation

In this paper, we provide a rigorous proof of the convergence for both first-order and second-order exponential time differencing (ETD) schemes applied to the nonlocal Cahn–Hilliard (NCH) equation. The spatial discretization is executed through the Fourier spectral collocation method, whereas the temporal discretization is implemented using ETD-based multistep schemes. The absence of a higher-order diffusion term in the NCH equation creates a significant challenge for the convergence analysis of numerical schemes. To address this issue, we introduce novel error decomposition formulas and utilize higher-order consistency analysis. These techniques allow us to establish the ℓ∞ bound of the numerical solution under certain natural constraints. By treating the numerical solution as a perturbation of the exact solution, we derive optimal convergence rates in ℓ∞(0,T;Hh−1)∩ℓ2(0,T;ℓ2). Furthermore, we conduct several numerical experiments to validate the accuracy and efficiency of the proposed schemes. These experiments include convergence tests and the observation of long-term coarsening dynamics, thereby confirming the robustness and reliability of our approach.