The Heavy ball method regularized by Tikhonov term. Simultaneous convergence of values and trajectories

Abstract

<p style='text-indent:20px;'>Let <inline-formula><tex-math id="M1">\begin{document}$ f: {\mathcal H} \rightarrow \mathbb{R} $\end{document}</tex-math></inline-formula> be a convex differentiable function whose solution set <inline-formula><tex-math id="M2">\begin{document}$ {{\rm{argmin}}}\; f $\end{document}</tex-math></inline-formula> is nonempty. To attain a solution of the problem <inline-formula><tex-math id="M3">\begin{document}$ \min_{\mathcal H}f $\end{document}</tex-math></inline-formula>, we consider the second order dynamic system <inline-formula><tex-math id="M4">\begin{document}$ \;\ddot{x}(t) + \alpha \, \dot{x}(t) + \beta (t) \nabla f(x(t)) + c x(t) = 0 $\end{document}</tex-math></inline-formula>, where <inline-formula><tex-math id="M5">\begin{document}$ \beta $\end{document}</tex-math></inline-formula> is a positive function such that <inline-formula><tex-math id="M6">\begin{document}$ \lim_{t\rightarrow +\infty}\beta(t) = +\infty $\end{document}</tex-math></inline-formula>. By imposing adequate hypothesis on first and second order derivatives of <inline-formula><tex-math id="M7">\begin{document}$ \beta $\end{document}</tex-math></inline-formula>, we simultaneously prove that the value of the objective function in a generated trajectory converges in order <inline-formula><tex-math id="M8">\begin{document}$ {\mathcal O}\big(\frac{1}{\beta(t)}\big) $\end{document}</tex-math></inline-formula> to the global minimum of the objective function, that the trajectory strongly converges to the minimum norm element of <inline-formula><tex-math id="M9">\begin{document}$ {{\rm{argmin}}}\; f $\end{document}</tex-math></inline-formula> and that <inline-formula><tex-math id="M10">\begin{document}$ \Vert \dot{x}(t)\Vert $\end{document}</tex-math></inline-formula> converges to zero in order <inline-formula><tex-math id="M11">\begin{document}$ \mathcal{O} \big( \sqrt{\frac{\dot{\beta}(t)}{\beta (t)}}+ e^{-\mu t} \big) $\end{document}</tex-math></inline-formula> where <inline-formula><tex-math id="M12">\begin{document}$ \mu<\frac{\alpha}2 $\end{document}</tex-math></inline-formula>. We then present two choices of <inline-formula><tex-math id="M13">\begin{document}$ \beta $\end{document}</tex-math></inline-formula> to illustrate these results. On the basis of the Moreau regularization technique, we extend these results to non-smooth convex functions with extended real values.</p>