Strong Convergence Properties Research Articles

A kernel-type regression estimator for NMAR response variables with applications to classification

This work deals with the problem of nonparametric estimation of a regression function when the response variable may be missing according to a not-missing-at-random (NMAR) setup. To assess the theoretical performance of our estimators, we study their strong convergence properties in Lp norms where we also look into their rates of convergence. We also study applications of our results to the problem of statistical classification in semi-supervised learning.

Strong Convergence Properties for Weighted Sums of Extended Negatively Dependent Random Variables Under Sub-linear Expectations with Statistical Applications

Strong Convergence Properties for Weighted Sums of Extended Negatively Dependent Random Variables Under Sub-linear Expectations with Statistical Applications

A Nesterov Type Algorithm with Double Tikhonov Regularization: Fast Convergence of the Function Values and Strong Convergence to the Minimal Norm Solution

We investigate the strong convergence properties of a Nesterov type algorithm with two Tikhonov regularization terms in connection to the minimization problem of a smooth convex function f. We show that the generated sequences converge strongly to the minimal norm element from argminf\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\ ext {argmin}f$$\\end{document}. We also show fast convergence for the potential energies f(xn)-minf\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$f(x_n)-\ ext {min}f$$\\end{document} and f(yn)-minf\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$f(y_n)-\ ext {min}f$$\\end{document}, where (xn),(yn)\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$(x_n),\\,(y_n)$$\\end{document} are the sequences generated by our algorithm. Further we obtain fast convergence to zero of the discrete velocity and some estimates concerning the value of the gradient of the objective function in the generated sequences. Via some numerical experiments we show that we need both Tikhonov regularization terms in our algorithm in order to obtain the strong convergence of the generated sequences to the minimum norm minimizer of our objective function.

Constraint Qualifications and Strong Global Convergence Properties of an Augmented Lagrangian Method on Riemannian Manifolds

Constraint Qualifications and Strong Global Convergence Properties of an Augmented Lagrangian Method on Riemannian Manifolds

Two novel two-step inertial algorithms for a class of bilevel variational inequalities

This paper introduces two novel two-step inertial and self-adaptive step sizes algorithms for solving a strongly monotone variational inequality over the solution set of a split convex feasibility problem with multiple output sets in real Hilbert spaces. We establish strong convergence properties of the proposed method under mild conditions and employ it to solve monotone variational inequalities over the solution set of the split feasibility problem. The method is further applied to the image classification problem via the support vector machine learning. Numerical results are given to demonstrate the accelerating behaviors of our method over other related methods in the literature.

Some strong convergence properties for randomly weighted maximum partial sums of END random variables with statistical applications

This article mainly studies the strong convergence properties for randomly weighted maximum partial sums of arrays of row-wise extended negatively dependent (END, for short) random variables, including the complete moment convergence and complete f-moment convergence. The results obtained in this paper extend the corresponding ones in the literature. As an application, we investigate the complete consistency of the estimators of nonparametric regression models based on END random errors by using the complete convergence that we established. Then we provide comprehensive simulation studies to demonstrate the validity of the theoretical results based on the finite sample performance. Finally, an application to a real data set is illustrated.

A Sufficient Descent 3-Term Conjugate Gradient Method for Unconstrained Optimization Algorithm

In recent years, 3-term conjugate gradient algorithms (TT-CG) have sparked interest for large scale unconstrained optimization algorithms due to appealing practical factors, such as simple computation, low memory requirement, better sufficient descent property, and strong global convergence property. In this study, minor changes were made to the BRB-CG method used for addressing the optimization algorithms discussed. Then, a new 3-term BRB-CG (MTTBRB) was presented. This new method solved large-scale unconstrained optimization problems. Despite the fact that the BRB algorithm achieved global convergence by employing a modified strong Wolfe line search, in this new MTTBRB-CG method the researchers employed the classical strong Wolfe-Powell condition (SWPC). This study also attempted to quantify how much better 3-term efficiency is than 2-term efficiency. As a result, in the numerical analysis, the new modification was compared to an effective 2-term CG- method. The numerical analysis demonstrated the effectiveness of the proposed method in solving optimization problems.

A Novel Two-Step Inertial Viscosity Algorithm for Bilevel Optimization Problems Applied to Image Recovery

This paper introduces a novel two-step inertial algorithm for locating a common fixed point of a countable family of nonexpansive mappings. We establish strong convergence properties of the proposed method under mild conditions and employ it to solve convex bilevel optimization problems. The method is further applied to the image recovery problem. Our numerical experiments show that the proposed method achieves faster convergence than other related methods in the literature.

An accelerated descent CG algorithm with clustering the eigenvalues for large-scale nonconvex unconstrained optimization and its application in image restoration problems

Conjugate gradient methods are widely used for solving large-scale unconstrained optimization problems. Since they have the attractive practical factors of simple computation and low memory requirement, interesting theoretical features of curvature information and strong global convergence property. Based on the analysis of the minimization of the condition number and the positiveness of the corresponding matrix, we propose a choice for the parameter in Dai-Liao method and design a descent conjugate gradient algorithm which owns the sufficient descent property independent of the choices for line search techniques. Under some common conditions, the global convergence property for uniformly convex function and the general nonlinear function are established. In the numerical experiments, we firstly focus on 46 ill-conditioned matrix problems and present the corresponding primal results. Then 450 large-scale unconstrained problems are referred. Finally, we give an accelerated strategy for the proposed algorithm and apply it to some image restoration problems. Numerical results indicate that the algorithm is reliable and much more efficient and effective than the other methods for the test problems.

A class of spectral three-term descent Hestenes-Stiefel conjugate gradient algorithms for large-scale unconstrained optimization and image restoration problems

Conjugate gradient methods are much effective and widely used for large-scale unconstrained optimization problems by their simple computation, low memory requirement and strong global convergence property. Spectral gradient methods are also effective for large-scale problems. In this paper, a class of new descent spectral three-term conjugate gradient algorithms are proposed which automatically have the sufficient descent property and satisfy the Dai-Liao conjugate condition. Under the Wolfe line search technique and some standard conditions, the proposed methods are globally convergent for strongly convex functions and general nonlinear functions with the help of the modified secant equations. In numerical part, 732 problems with dimensions varying from 1500 to 150000 and three image restoration problems with three noise levels are considered. Numerical results indicate that the proposed algorithms are more efficient, reliable and robust than the other methods for the testing problems.

Strong convergence theorems for split variational inequality problems in Hilbert spaces

<abstract><p>In this paper, we consider the variational inequality problem and the split common fixed point problem. Considering the common fixed points of an infinite family of nonexpansive mappings, instead of just the fixed point of one nonexpansive mapping, we generalize the results of Tian and Jiang. By removing a projection operator, we improve the efficiency of our algorithm. Finally, we propose a very simple modification to the extragradient method, which gives our algorithm strong convergence properties. We also provide some numerical examples to illustrate our main results.</p></abstract>

Sparsity enhanced MRF algorithm for automatic object detection in GPR imagery.

This study addressed the problem of automated object detection from ground penetrating radar imaging (GPR), using the concept of sparse representation. The detection task is first formulated as a Markov random field (MRF) process. Then, we propose a novel detection algorithm by introducing the sparsity constraint to the standard MRF model. Specifically, the traditional approach finds it difficult to determine the central target due to the influence of different neighbors from the imaging area. As such, we introduce a domain search algorithm to overcome this issue and increase the accuracy of target detection. Additionally, in the standard MRF model, the Gibbs parameters are empirically predetermined and fixed during the detection process, yet those hyperparameters may have a significant effect on the performance of the detection. Accordingly, in this paper, Gibbs parameters are self-adaptive and fine-tuned using an iterative updating strategy followed the concept of sparse representation. Furthermore, the proposed algorithm has then been proven to have a strong convergence property theoretically. Finally, we verify the proposed method using a real-world dataset, with a set of ground penetrating radar antennas in three different transmitted frequencies (50 MHz, 200 MHz and 300 MHz). Experimental evaluations demonstrate the advantages of utilizing the proposed algorithm to detect objects in ground penetrating radar imagery, in comparison with four traditional detection algorithms.

An approximate inertial manifold (AIM) based closure for turbulent flows

A closure model for turbulent flows is developed based on a dynamical system theory. An appropriately discretized formulation of the governing equations is considered for this process. The key ingredient is an approximation of the system’s attractor, where all the trajectories in phase space are confined. This approximate inertial manifold based approach provides a path to track trajectories of the system in a lower-dimensional subspace. Unlike conventional coarse-graining approaches, the turbulent field is decomposed into resolved and unresolved dynamics using the properties of the governing equations. The novelty of the approach relies on the reconstruction of the unresolved field constrained by the governing equations. A posteriori tests for homogeneous isotropic turbulence and the Kuramoto–Sivashinsky equation show promising results for considerable dimension reduction with strong convergence properties. The proposed model outperforms the dynamic Smagorinsky model, and the computational overhead is competitive with similar approaches.

Homogenization of a nonlinear drift–diffusion system for multiple charged species in a porous medium

We consider a nonlinear drift–diffusion system for multiple charged species in a porous medium in 2D and 3D with periodic microstructure. The system consists of a transport equation for the concentration of the species and Poisson’s equation for the electric potential. The diffusion terms depend nonlinearly on the concentrations. We consider non-homogeneous Neumann boundary condition for the electric potential. The aim is the rigorous derivation of an effective (homogenized) model in the limit when the scale parameter ϵ tends to zero. This is based on uniform a priori estimates for the solutions of the microscopic model. The crucial result is the uniform L∞-estimate for the concentration in space and time. This result exploits the fact that the system admits a nonnegative energy functional which decreases in time along the solutions of the system. By using weak and strong (two-scale) convergence properties of the microscopic solutions, effective models are derived in the limit ϵ→0 for different scalings of the microscopic model.

AN INERTIAL SPLITTING METHOD FOR MONOTONE INCLUSIONS OF THREE OPERATORS

In this article, we consider monotone inclusions of three operators in real Hilbert spaces and suggest an inertial version of a generalized Douglas-Rachford splitting. Under standard assumptions, we prove its weak and strong convergence properties. The newly-developed proof techniques are based on the characteristic operator and thus are more self-contained and less convoluted. Rudimentary experiments demonstrated that our suggested inertial splitting method can efficiently solve some large-scale test problems.

Nonlinear Conjugate Gradient Coefficients with Exact and Strong Wolfe Line Searches Techniques

Nonlinear conjugate gradient (CG) methods are very important for solving unconstrained optimization problems. These methods have been subjected to extensive researches in terms of enhancing them. Exact and strong Wolfe line search techniques are usually used in practice for the analysis and implementation of conjugate gradient methods. For better results, several studies have been carried out to modify classical CG methods. The method of Fletcher and Reeves (FR) is one of the most well‐known CG methods. It has strong convergence properties, but it gives poor numerical results in practice. The main goal of this paper is to enhance this method in terms of numerical performance via a convexity type of modification on its coefficient βk. We ensure that with this modification, the method is still achieving the sufficient descent condition and global convergence via both exact and strong Wolfe line searches. The numerical results show that this modified FR is more robust and effective.

Two self-adaptive inertial projection algorithms for solving split variational inclusion problems

<abstract><p>This paper is to analyze the approximation solution of a split variational inclusion problem in the framework of Hilbert spaces. For this purpose, inertial hybrid and shrinking projection algorithms are proposed under the effect of a self-adaptive stepsize which does not require information of the norms of the given operators. The strong convergence properties of the proposed algorithms are obtained under mild constraints. Finally, a numerical experiment is given to illustrate the performance of proposed methods and to compare our algorithms with an existing algorithm.</p></abstract>

Two adaptive modified subgradient extragradient methods for bilevel pseudomonotone variational inequalities with applications

We consider the bilevel variational inequality problem with a pseudomonotone operator in real Hilbert spaces and investigate two modified subgradient extragradient methods with inertial terms. Our first scheme requires the operator to be Lipschitz continuous (the Lipschitz constant does not need to be known) while the second one only requires it to be uniformly continuous. The proposed methods employ two adaptive stepsizes making them work without the prior knowledge of the Lipschitz constant of the mapping. The strong convergence properties of the iterative sequences generated by the proposed algorithms are obtained under mild conditions. Some numerical tests and applications are given to demonstrate the advantages and efficiency of the stated schemes over previously known ones.

Trial and Error Learning for Dynamic Distributed Channel Allocation in Random Medium

This paper considers the problem of fully distributed channel allocation in clustered wireless networks when the propagation medium is random. We extend here the existing Trial and Error (TE) framework developed in the deterministic case and for which strong convergence properties hold. We prove that using directly this solution in the random context leads to unsatisfactory solutions. Then we propose an adaptation of the original Trial and Error Learning (TEL) algorithm, called Robust TEL (RTEL), assuming that the random channel effects translate into a bounded stochastic disturbance of the utility function. The solution consists in introducing thresholds in the transitions of the TEL’s Finite State Controller (FSC). We prove that this new solution restores the good convergence property inherited from the TEL. Furthermore, we provide analysis of the stochastic utilities in the Rayleigh fading case in order to check the bounded assumption. Finally, we develop an online algorithm that dynamically estimates the optimal threshold values to adapt to the instantaneous disturbance. Numerical results corroborate our theoretical claims.

Strong consistency of regression function estimator with martingale difference errors

Abstract In this paper, we consider the regression model with fixed design: Y i = g ( x i ) + ε i {Y}_{i}=g\left({x}_{i})+{\varepsilon }_{i} , 1 ≤ i ≤ n 1\le i\le n , where { x i } \left\{{x}_{i}\right\} are the nonrandom design points, and { ε i } \left\{{\varepsilon }_{i}\right\} is a sequence of martingale, and g g is an unknown function. Nonparametric estimator g n ( x ) {g}_{n}\left(x) of g ( x ) g\left(x) will be introduced and its strong convergence properties are established.