Weak Convergence Theorems Research Articles

Convergence Analysis and Application for Multi‐Layer Neural Network Based on Fractional‐Order Gradient Descent Learning

AbstractFractional order calculus, with its inheritance and infinite memory properties, is a promising research area in information processing and modeling of certain physical systems, system identification, and control. In this paper, a fractional‐order gradient descent method is proposed for backpropagation training of multilayer feedforward neural networks. In particular, the Caputo derivative is used to define the measurement function and consider the smooth regular term. In addition, the monotonicity of the error function and the strong (weak) convergence theorem of the algorithm are rigorously proved. Finally, numerical experiments prove the correctness and effectiveness of the algorithm theory.

Viscosity approximation method for split best proximity point and monotone variational inclusion problem

ABSTRACT To address the split best proximity point and monotone variational inclusion problems in real Hilbert spaces, we present and investigate projection and viscosity approximation methods. Under a few reasonable assumptions, we prove some weak and strong convergence theorems for the aforementioned methods. The efficiency of the proposed method is demonstrated by some numerical examples. Some well-known recent results in this area have been improved, generalized, and extended as an outcome of this paper.

Convergence analysis of M-iteration for 𝒢-nonexpansive mappings with directed graphs applicable in image deblurring and signal recovering problems

Abstract In this article, weak and strong convergence theorems of the M-iteration method for 𝒢-nonexpansive mapping in a uniformly convex Banach space with a directed graph were established. Moreover, weak convergence theorem without making use of Opial’s condition is proved. The rate of convergence between the M-iteration and some other iteration processes in the literature was also compared. Specifically, our main result shows that the M-iteration converges faster than the Noor and SP iterations. Finally, the numerical examples to compare convergence behavior of the M-iteration with the three-step Noor iteration and the SP-iteration were given. As application, some numerical experiments in real-world problems were provided, focused on image deblurring and signal recovering problems.

Randomized empirical processes by algebraic groups, and tests for weak null hypotheses

Randomization tests are based on a re-randomization of existing data to gain data-dependent critical values that lead to exact hypothesis tests under special circumstances. However, it is not always possible to re-randomize data in accordance with the physical randomization from which the data have been obtained. As a consequence, statistical tests typically cannot control the type I error probability if the available sample is not overly large. Still, similarly as the bootstrap, data re-randomization can be used to improve the type I error control. However, in contrast to bootstrap and permutation test theory, a general asymptotic theory under weak null hypotheses has not yet been developed for randomization tests. It is thus the aim of this paper to provide a conveniently applicable theory on the asymptotic validity of randomization tests with test statistics that are asymptotically normal under weak null hypotheses. This can also be used to justify the asymptotic reliability of randomization-based confidence intervals. To achieve this, the present article creates a link between two well-established fields in mathematical statistics: empirical processes and inference based on randomization via algebraic groups. To this end, a broadly applicable conditional weak convergence theorem is developed for empirical processes that are based on randomized observations. Random elements of an algebraic group are applied to the data vectors from which the randomized version of a statistic is derived. Combining a variant of the functional delta-method with a suitable studentization of the statistic, asymptotically exact hypothesis tests can be deduced, while the finite sample exactness property under group-invariant sub-hypotheses is preserved. The methodology is exemplified with three examples: the Pearson correlation coefficient, a Mann-Whitney effect based on right-censored paired data, and a competing risks analysis. The practical usefulness of the approaches is assessed through simulation studies and an application to data from patients suffering from diabetic retinopathy.

Two new self-adaptive algorithms for solving split common fixed point problems with multiple output sets

In this paper, we study split common fixed point problems with multiple output sets in Hilbert spaces. For solving this problem, we improve Reich and Tuyen's algorithms [10] and propose two new self-adaptive algorithms which do not need any priori information on the norms of the transfer operators. Then, we establish weak convergence theorems for them. It can be seen from our numerical experiments that the proposed methods depict a very significant improvement in terms of the number of iterations and execution time.

On monotone pseudocontractive operators and Krasnoselskij iterations in an ordered Hilbert space

The aim of this work is to establish fixed point results in ordered Hilbert spaces for monotone operators with a pseudocontractive property. We state monotone versions of Theorem 12 in [F. E. Browder, W. V. Petryshyn, Construction of fixed points of nonlinear mappings in Hilbert space, J. Math. Anal. Appl. 20 (1967), 197–228] and Theorem 2.1 in [Berinde, Vasile. Weak and strong convergence theorems for the Krasnoselskij iterative algorithm in the class of enriched strictly pseudocontractive operators, Annals of West University of Timisoara-Mathematics and Computer Science, vol. 56, no. 2, 2018, pp. 13–27], as well as, several related results. Further results, in Hilbert spaces without a partial order, are stated too.

New inertial forward–backward algorithm for convex minimization with applications

Abstract In this work, we present a new proximal gradient algorithm based on Tseng’s extragradient method and an inertial technique to solve the convex minimization problem in real Hilbert spaces. Using the stepsize rules, the selection of the Lipschitz constant of the gradient of functions is avoided. We then prove the weak convergence theorem and present the numerical experiments for image recovery. The comparative results show that the proposed algorithm has better efficiency than other methods.

Inertial projection and contraction methods for solving variational inequalities with applications to image restoration problems

In this paper, we introduce two inertial self-adaptive projection and contraction methods for solving the pseudomonotone variational inequality problem with a Lipschitz-continuous mapping in real Hilbert spaces. The adaptive stepsizes provided by the algorithms are simple to update and their computations are more efficient and flexible. Also we prove some weak and strong convergence theorems without prior knowledge of the Lipschitz constant of the mapping. Finally, we present some numerical experiments to demonstrate the effectiveness of the proposed algorithms by comparisons with related methods and some applications of the proposed algorithms to the image deblurring problem.

Self adaptive inertial relaxed $ CQ $ algorithms for solving split feasibility problem with multiple output sets

<p style='text-indent:20px;'>In this paper, we propose two new self-adaptive inertial relaxed <inline-formula><tex-math id="M2">\begin{document}$ CQ $\end{document}</tex-math></inline-formula> algorithms for solving the split feasibility problem with multiple output sets in the framework of real Hilbert spaces. The proposed algorithms involve computing projections onto half-spaces instead of onto the closed convex sets, and the advantage of the self-adaptive step size introduced in our algorithms is that it does not require the computation of operator norm. We establish and prove weak and strong convergence theorems for the iterative sequences generated by the introduced algorithms for solving the aforementioned problem. Moreover, we apply the new results to solve some other problems. Finally, we present some numerical examples to illustrate the implementation of our algorithms and compared them to some existing results.</p>

Convergence analysis for Suzuki's generalized nonexpansive mappings

In this paper, we study the Picard-Mann hybrid iteration process to approximate fixed points of Suzuki's generalized nonexpansive mappings. We establish some weak and strong convergence theorems for such mappings in uniformly convex Banach space.

Subgradient extragradient method with double inertial steps for quasi-monotone variational inequalities

In this paper, we present a modified subgradient extragradient method with double inertial extrapolation terms and a non-monotonic adaptive step size for solving quasi-monotone and Lipschitz continuous variational inequalities in real Hilbert spaces. Under some suitable conditions, we obtain the weak convergence theorem of our proposed algorithm. Moreover, strong convergence is obtained when the cost operator is strongly pseudo-monotone and Lipschitz continuous. Finally, several numerical results illustrate the effectiveness and competitiveness of our algorithm.

Strong and weak convergence theorems for solutions of equations of Hammerstein-type

In this paper, it is our aim in this paper to introduce a new iterative algorithmfor approximation of a solution of an equation of Hammerstein-type. The proposed scheme does not involve computation of inverse of operators under study; it does not involve passing through computation of a certain set that must contain a solution of the equation of Hammerstein-type before convergence takes place. The proposed scheme requires only one parameter satisfying verifiable mild conditions. Moreover, the mappings involved are neither defined on compact subset of the space under study, nor assumed to be angle bounded. Our theorems complement several results that have been obtained in this direction.

Efficient fixed-point iteration for generalized nonexpansive mappings and its stability in Banach spaces

Abstract The aim of this article is to design a new iteration process for solving certain fixed-point problems. In particular, we prove weak and strong convergence theorems for generalized nonexpansive mappings in the framework of uniformly convex Banach spaces. In addition, we discuss the stability of the solution under mild conditions. Further, we provide some numerical examples to indicate that the proposed method works properly.

Batch Gradient Learning Algorithm with Smoothing L1 Regularization for Feedforward Neural Networks

Regularization techniques are critical in the development of machine learning models. Complex models, such as neural networks, are particularly prone to overfitting and to performing poorly on the training data. L1 regularization is the most extreme way to enforce sparsity, but, regrettably, it does not result in an NP-hard problem due to the non-differentiability of the 1-norm. However, the L1 regularization term achieved convergence speed and efficiency optimization solution through a proximal method. In this paper, we propose a batch gradient learning algorithm with smoothing L1 regularization (BGSL1) for learning and pruning a feedforward neural network with hidden nodes. To achieve our study purpose, we propose a smoothing (differentiable) function in order to address the non-differentiability of L1 regularization at the origin, make the convergence speed faster, improve the network structure ability, and build stronger mapping. Under this condition, the strong and weak convergence theorems are provided. We used N-dimensional parity problems and function approximation problems in our experiments. Preliminary findings indicate that the BGSL1 has convergence faster and good generalization abilities when compared with BGL1/2, BGL1, BGL2, and BGSL1/2. As a result, we demonstrate that the error function decreases monotonically and that the norm of the gradient of the error function approaches zero, thereby validating the theoretical finding and the supremacy of the suggested technique.

Approximating $G$-variational inequality problem by $G$-subgradient extragradient method in Hilbert space endowed with graphs

In this article, we introduce $G$-subgradient extragradient method for solving the $G$- variational inequality problem in Hilbert space endowed with a direct graph. Utilizing our mathematical tools, weak and strong convergence theorem are established for the proposed algorithm. In addition, we provide numerical experiments to illustrate the convergence behavior of our proposed algorithm.

Generalization of Fixed Point Approximation of Contraction and Suzuki Generalized Non-Expansive Mappings in Banach Domain

By principal motivation from the results of the new iterative scheme that produces faster results than K-iteration. In this article, we study generalized results by a new iteration scheme to approximate fixed points of generalized contraction and Suzuki non-expansive mappings. We establish strong convergence results of generalized contraction mappings of closed convex Banach space and also deduce data dependent results. Furthermore, we prove some weak and strong convergence theorems in the sense of generalized Suzuki non-expansive mapping by applying condition (C).

A Convergent Algorithm for Equilibrium Problem to Predict Prospective Mathematics Teachers’ Technology Integrated Competency

Educational data classification has become an effective tool for exploring the hidden pattern or relationship in educational data and predicting students’ performance or teachers’ competency. This study proposes a new method based on machine learning algorithms to predict the technology-integrated competency of pre-service mathematics teachers. In this paper, we modified the inertial subgradient extragradient algorithm for pseudomonotone equilibrium and proved the weak convergence theorem under some suitable conditions in Hilbert spaces. We then applied to solve data classification by extreme learning machine using the dataset comprised of the technology-integrated competency of 954 pre-service mathematics teachers in a university in northern Thailand, longitudinally collected for five years. The flexibility of our algorithm was shown by comparisons of the choice of different parameters. The performance was calculated and compared with the existing algorithms to be implemented for prediction. The results show that the proposed method achieved a classification accuracy of 81.06%. The predictions were implemented using ten attributes, including demographic information, skills, and knowledge relating to technology developed throughout the teacher education program. Such data driven studies are significant for establishing a prospective teacher competency analysis framework in teacher education and contributing to decision-making for policy design.

The method for solving the extension of general of the split feasibility problem and fixed point problem of the cutter

This paper first introduces a new iterative method for weak and strong convergence theorem to demonstrate the estimation potential for a fixed point of the cutter and the finite general split feasibility problem. Consequently, the set of fixed points of a quasi-nonexpansive mapping and the finite general split feasibility problem, the constrained minimization problem, and the general constrained minimization problems are proved using our main results. Finally, we give two numerical examples to advocate our main results.

Some convergence results using a new iteration process for generalized nonexpansive mappings in Banach spaces

In this paper, we introduce a new iterative process for approximation fixed points. We show the stability of our proposed iteration process. We prove some weak and strong convergence theorems for generalized [Formula: see text]-nonexpansive mappings in the framework of uniformly convex Banach spaces. We also provide an example of a generalized [Formula: see text]-nonexpansive mapping to show numerically that our iteration is faster than various prominent iteration processes.

Fixed point approximation for a class of generalized nonexpansive multi-valued mappings in Banach spaces

In this paper, we propose a new iteration process, called multi-valued F-iteration process, for the approximation of fixed points. We introduce a new class of multi-valued generalized nonexpansive mappings satisfying a B_{gamma ,mu } property. Moreover, we establish certain weak and strong convergence theorems in uniformly convex Banach spaces. We also discuss the stability of the modified F-iteration process. Furthermore, a numerical example is presented to illustrate the superiority of our results.