Online Gradient Descent Research Articles

Design of an Improved Model for Multimodal Data Fusion Using XGBoost-LSTM-CNN and Proximal Policy Optimizations

Applications in healthcare, finance, and real-time sensor applications call for much stronger demands of improving accuracy and efficiency in the analysis of multimodal data. Current methods have deficiencies in fusing multiple types of data such as time-series, spatial, and categorical data, arising due to limitations in capturing sequential dependencies, spatial patterns, and feature importance simultaneously. It also discusses these challenges through a novel solution based on a hybrid ML-DL approach, with an integration of reinforcement learning and advanced probabilistic models. First, the method that comes to mind is the XGBoost-LSTM-CNN Hybrid Model, wherein three drivers of performance improvements come together, namely, XGBoost, with much-appreciated capability for outstanding handling of structured tabular data, LSTM due to its proficiency in capturing long-term temporal dependencies, and the strength of CNN in spatial feature extraction. These results improve the predictive accuracy for multimodal datasets. After that, further enhanced multimodal fusion by the CAMT selectively pays attention to the critical features across the modalities, enhancing the accuracy of the contextual time-series predictions. Subsequently, PPO enables real-time model adaptation by dynamic optimization of model parameters and improvement of predictions through continuous learning. KF-BNN reduces noise and uncertainty in time-series data by fusing the filtering capability of a Kalman filter with probabilistic modeling via Bayesian neural networks to give reliable predictions. It also provides federated learning via FedAvg with online gradient descent for distributed model training in a manner that ensures privacy, continuous model updates without having to centralize sensitive data samples. These approaches show significant improvements along many axes, with accuracy improvements of up to 4.5% and MAE reductions of 12% relative to the baseline models. Proposed models provide a valid framework for analyzing multimodal data, thereby increasing precision, recall, and overall adaptiveness in dynamic real time, hence pushing the state-of-the-art in multimodal fusion and timeseries prediction. This research probably will influence those areas dependent on distributed, multimodal, and sequential data samples.

Parametric estimation of stochastic differential equations via online gradient descent

Parametric estimation of stochastic differential equations via online gradient descent

Adaptive Conformal Inference for Computing Market Risk Measures: An Analysis with Four Thousand Crypto-Assets

This paper investigates the estimation of the value at risk (VaR) across various probability levels for the log-returns of a comprehensive dataset comprising four thousand crypto-assets. Employing four recently introduced adaptive conformal inference (ACI) algorithms, we aim to provide robust uncertainty estimates crucial for effective risk management in financial markets. We contrast the performance of these ACI algorithms with that of traditional benchmark models, including GARCH models and daily range models. Despite the substantial volatility observed in the majority of crypto-assets, our findings indicate that ACI algorithms exhibit notable efficacy. In contrast, daily range models, and to a lesser extent, GARCH models, encounter challenges related to numerical convergence issues and structural breaks. Among the ACI algorithms, Fully Adaptive Conformal Inference (FACI) and Scale-Free Online Gradient Descent (SF-OGD) stand out for their ability to provide precise VaR estimates across all quantiles examined. Conversely, Aggregated Adaptive Conformal Inference (AgACI) and Strongly Adaptive Online Conformal Prediction (SAOCP) demonstrate proficiency in estimating VaR for extreme quantiles but tend to be overly conservative for higher probability levels. These conclusions withstand robustness checks encompassing the market capitalization of crypto-assets, time-series size, and different forecasting methods for asset log-returns. This study underscores the promise of ACI algorithms in enhancing risk assessment practices in the context of volatile and dynamic crypto-asset markets.

Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback

Feasible Online Learning with Gradient Feedback Online gradient descent (OGD) is well-known to be doubly optimal under strong convexity or monotonicity assumptions: (1) in the single-agent setting, it achieves an optimal regret of [Formula: see text] for strongly convex cost functions, and (2) in the multiagent setting of strongly monotone games with each agent employing OGD we obtain last-iterate convergence of the joint action to a unique Nash equilibrium at an optimal rate of [Formula: see text]. Whereas these finite-time guarantees highlight its merits, OGD has the drawback that it requires knowing the strong convexity/monotonicity parameters. In “Adaptive, Doubly Optimal No-Regret Learning in Strongly Monotone and Exp-Concave Games with Gradient Feedback,” M. Jordan, T. Lin, and Z. Zhou design a fully adaptive OGD algorithm, AdaOGD, that does not require a priori knowledge of these parameters. In the single-agent setting, the algorithm achieves [Formula: see text] regret under strong convexity, which is optimal up to a log factor. Further, if each agent employs AdaOGD in strongly monotone games, the joint action converges in a last-iterate sense to a unique Nash equilibrium at a rate of [Formula: see text], again optimal up to log factors. The algorithms are illustrated in a learning version of the classic newsvendor problem, in which, because of lost sales, only (noisy) gradient feedback can be observed. The results immediately yield the first feasible and near-optimal algorithm for both the single-retailer and multiretailer settings.

Sparse online regression algorithm with insensitive loss functions

Online learning is an efficient approach in machine learning and statistics, which iteratively updates models upon the observation of a sequence of training examples. A representative online learning algorithm is the online gradient descent, which has found wide applications due to its low complexity and scalability to large datasets. Kernel-based learning methods have been proven to be quite successful in dealing with nonlinearity in the data and multivariate optimization. In this paper we present a class of kernel-based online gradient descent algorithm for addressing regression problems, which generates sparse estimators in an iterative way to reduce the algorithmic complexity for training streaming datasets and model selection in large-scale learning scenarios. In the setting of support vector regression (SVR), we design the sparse online learning algorithm by introducing a sequence of insensitive distance-based loss functions. We prove consistency and error bounds quantifying the generalization performance of such algorithms under mild conditions. The theoretical results demonstrate the interplay between statistical accuracy and sparsity property during learning processes. We show that the insensitive parameter plays a crucial role in providing sparsity as well as fast convergence rates. The numerical experiments also support our theoretical results.

Limited Memory Online Gradient Descent for Kernelized Pairwise Learning with Dynamic Averaging

Pairwise learning, an important domain within machine learning, addresses loss functions defined on pairs of training examples, including those in metric learning and AUC maximization. Acknowledging the quadratic growth in computation complexity accompanying pairwise loss as the sample size grows, researchers have turned to online gradient descent (OGD) methods for enhanced scalability. Recently, an OGD algorithm emerged, employing gradient computation involving prior and most recent examples, a step that effectively reduces algorithmic complexity to O(T), with T being the number of received examples. This approach, however, confines itself to linear models while assuming the independence of example arrivals. We introduce a lightweight OGD algorithm that does not require the independence of examples and generalizes to kernel pairwise learning. Our algorithm builds the gradient based on a random example and a moving average representing the past data, which results in a sub-linear regret bound with a complexity of O(T). Furthermore, through the integration of O(√T logT) random Fourier features, the complexity of kernel calculations is effectively minimized. Several experiments with real-world datasets show that the proposed technique outperforms kernel and linear algorithms in offline and online scenarios.

B-Spline Wavelet Neural-Network-Based Adaptive Control for Linear-Motor-Driven Systems via a Novel Gradient Descent Algorithm

This paper proposes a gradient descent (GD) algorithm-based B-Spline wavelet neural network (GDBSWNN) learning adaptive controller for linear motor (LM) systems under system uncertainties and actuator saturation constraints. A recursive least squares (RLS) algorithm-based indirect adaptive strategy is used to effectively estimate model parameters, which can guarantee they converge to true values. A novel GDBSWNN compensator is proposed to estimate the remaining complex uncertainties, where weights are updated by online GD training. An auxiliary system is integrated into the control scheme to address the saturation problem, which guarantees stability and satisfactory control performance when saturation occurs. In addition, a stability analysis is presented to prove that all signals of the closed-loop system are bounded using the Lyapunov theory. Experiments have been conducted on an LM-driven motion platform, where different controllers have been tested, demonstrating the effectiveness and advantages of the proposed approach.

No-regret learning for repeated non-cooperative games with lossy bandits

This paper considers no-regret learning for repeated continuous-kernel games with lossy bandit feedback. Since it is difficult to give an explicit model of the utility functions in dynamic environments, the players’ actions can only be learned with bandit feedback. Moreover, due to unreliable communication channels or privacy protection, the bandit feedback may be lost or dropped at random. Therefore, we study the asynchronous online learning strategy of the players to adaptively adjust the next actions for minimizing the long-term regret loss. The paper provides a novel no-regret learning algorithm, called Online Gradient Descent with lossy bandits (OGD-lb). We first give the regret analysis for concave games with differentiable and Lipschitz utilities. Then we show that the action profile converges to a Nash equilibrium with probability 1 when the game is also strictly monotone. We further provide the mean-squared convergence rate ONpi−2k−1/3 when the game is β-strongly monotone, where N denotes the number of players and pi is the update probability. In addition, we extend the algorithm to the case when the loss probability of the bandit feedback is unknown, and prove its almost sure convergence to Nash equilibrium for strictly monotone games. Finally, we take the resource management in fog computing as an application example, and carry out numerical experiments to empirically demonstrate the algorithm performance.

Comparison of Selected Machine Learning Algorithms in the Analysis of Mental Health Indicators

Machine learning is increasingly being used to solve clinical problems in diagnosis, therapy and care. Aim: the main aim of the study was to investigate how the selected machine learning algorithms deal with the problem of determining a virtual mental health index. Material and Methods: a number of machine learning models based on Stochastic Dual Coordinate Ascent, limited-memory Broyden–Fletcher–Goldfarb–Shanno, Online Gradient Descent, etc., were built based on a clinical dataset and compared based on criteria in the form of learning time, running time during use and regression accuracy. Results: the algorithm with the highest accuracy was Stochastic Dual Coordinate Ascent, but although its performance was high, it had significantly longer training and prediction times. The fastest algorithm looking at learning and prediction time, but slightly less accurate, was the limited-memory Broyden–Fletcher–Goldfarb–Shanno. The same data set was also analyzed automatically using ML.NET. Findings from the study can be used to build larger systems that automate early mental health diagnosis and help differentiate the use of individual algorithms depending on the purpose of the system.

Decentralized online convex optimization with compressed communications

Due to the iterative information exchange between agents, decentralized multi-agent optimization algorithms often incur large communication overhead, which is not affordable in many practical systems with scarce resources. To address this communication bottleneck, we compress the exchanged information between neighboring agents to solve decentralized online convex optimization problem, where each agent has a time-varying local loss function and the goal is to minimize the accumulated global loss by choosing actions sequentially. We develop a decentralized online gradient descent algorithm based on compressed communications, where the compression operators can reduce the amount of data to be sent significantly. Error-compensation technique is utilized to mitigate the error accumulation caused by the communication compression. We further analyze the performance of the proposed algorithm. It is shown that the regret is bounded from above by O(T) (T is the time horizon), which is the same as (in order sense) the regret bound for vanilla decentralized online gradient descent with perfect communication. Thus, the proposed algorithm is capable of reducing the communication overhead remarkably without much degradation of the optimization performance. This is further corroborated by numerical experiments, where compressed communication reduces the number of transmitted bits by an order of magnitude without compromising the optimization performance.

Active defect discovery: A human-in-the-loop learning method

Unsupervised defect detection methods are applied to an unlabeled dataset by producing a ranked list based on defect scores. Unfortunately, many of the top-ranked instances by unsupervised algorithms are not defects, which leads to high false-positive rates. Active Defect Discovery (ADD) is proposed to overcome this deficiency, which sequentially selects instances to get the labeling information (defects or not). However, labeling is often costly. Therefore, balancing detection accuracy and labeling cost is essential. Along this line, this article proposes a novel ADD method to achieve the goal. Our approach is based on the state-of-the-art unsupervised defect detection method, namely, Isolation Forest, as the baseline defect detector to extract features. Thereafter, the sparsity of the extracted features is utilized to adjust the defect detector so that it can focus on more important features for defect detection. To enforce the sparsity of the features and subsequent improvement of the detection accuracy, a new algorithm based on online gradient descent, namely, Sparse Approximated Linear Defect Discovery (SALDD), is proposed with its theoretical Regret analysis. Extensive experiments are conducted on real-world datasets including healthcare, manufacturing, security, etc. The performance demonstrates that the proposed algorithm significantly outperforms the state-of-the-art algorithms for defect detection.

The optimal dynamic regret for smoothed online convex optimization with squared [formula omitted] norm switching costs

Online convex optimization (OCO) with switching costs is a key enabler for cloud resource provision, online portfolio optimization, and many other applications. Surprisingly, very little theoretical understanding is known. In this study, we investigate OCO with squared l2 norm switching cost (OCOl2SC) for three kinds of loss functions: (a) generally convex, (b) convex and smooth, and (c) strongly convex and smooth. We design customized gradient descent algorithms for OCOl2SC in these three cases: specifically, SOGD (smoothed online gradient descent) for generally convex loss functions, OOMD (online optimistic mirror descent) for convex and smooth functions, and OMGD (online multiple gradient descent) for strongly convex and smooth loss functions. We theoretically analyze the three algorithms’ dynamic regrets and their upper bounds. By showing that the dynamic regrets match their lower bounds, we conclude that the three algorithms achieve the order optimal or near-optimal dynamic regret bounds in the corresponding case. Numerical studies further verify the remarkable performance of the three algorithms.

Distributed dynamic online learning with differential privacy via path-length measurement

Dynamic online learning has been given great concerns as real-time and non-stationary systems develop and it can be used for solving many practical sequential decision problems like dynamic recommendation systems for online advertisement. Due to the large volume of big data in online learning, dynamic online learning is required to be deployed under the distributed framework where the communication among agents is time-varying and real-time. Meanwhile, since the information sharing among agents is vulnerable to interception by the adversary, the privacy protection problem is significant, but to the best of our knowledge, this work is the first to consider privacy-preserving distributed online learning under dynamic circumstances. Specifically, based on a differentially private distributed online gradient descent algorithm, we minimize the regret of the network. Moreover, via the rigorous mathematical analysis, we achieve the sublinear regret bound under ϵ-differential privacy, that is, we respectively obtain O(T+VT) and O(log⁡T+VT) for convex and strongly convex function, where T is the time horizon and the path-length VT reflects the variation of the minimizer sequence. The experiments performed on the real datasets validate that the presented algorithm achieves acceptable optimization performance even when privacy protection is relatively strict.

Robust Online Support Vector Regression with Truncated ε-Insensitive Pinball Loss

Advances in information technology have led to the proliferation of data in the fields of finance, energy, and economics. Unforeseen elements can cause data to be contaminated by noise and outliers. In this study, a robust online support vector regression algorithm based on a non-convex asymmetric loss function is developed to handle the regression of noisy dynamic data streams. Inspired by pinball loss, a truncated ε-insensitive pinball loss (TIPL) is proposed to solve the problems caused by heavy noise and outliers. A TIPL-based online support vector regression algorithm (TIPOSVR) is constructed under the regularization framework, and the online gradient descent algorithm is implemented to execute it. Experiments are performed using synthetic datasets, UCI datasets, and real datasets. The results of the investigation show that in the majority of cases, the proposed algorithm is comparable, or even superior, to the comparison algorithms in terms of accuracy and robustness on datasets with different types of noise.

Online convex optimization for constrained control of linear systems using a reference governor*

In this work, we propose a control scheme for linear systems subject to pointwise in time state and input constraints that aims to minimize time-varying and a priori unknown cost functions. The proposed controller is based on online convex optimization and a reference governor. In particular, we apply online gradient descent to track the time-varying and a priori unknown optimal steady state of the system. Moreover, we use a λ-contractive set to enforce constraint satisfaction and a sufficient convergence rate of the closed-loop system to the optimal steady state. We prove that the proposed scheme is recursively feasible, ensures that the state and input constraints are satisfied at all times, and achieves a dynamic regret that is linearly bounded by the variation of the cost functions. The algorithm's performance and constraint satisfaction is illustrated by means of a simulation example.

Communication-Efficient Randomized Algorithm for Multi-Kernel Online Federated Learning.

Online federated learning (OFL) is a promising framework to learn a sequence of global functions from distributed sequential data at local devices. In this framework, we first introduce a single kernel-based OFL (termed S-KOFL) by incorporating random-feature (RF) approximation, online gradient descent (OGD), and federated averaging (FedAvg). As manifested in the centralized counterpart, an extension to multi-kernel method is necessary. Harnessing the extension principle in the centralized method, we construct a vanilla multi-kernel algorithm (termed vM-KOFL) and prove its asymptotic optimality. However, it is not practical as the communication overhead grows linearly with the size of a kernel dictionary. Moreover, this problem cannot be addressed via the existing communication-efficient techniques (e.g., quantization and sparsification) in the conventional federated learning. Our major contribution is to propose a novel randomized algorithm (named eM-KOFL), which exhibits similar performance to vM-KOFL while maintaining low communication cost. We theoretically prove that eM-KOFL achieves an optimal sublinear regret bound. Mimicking the key concept of eM-KOFL in an efficient way, we propose a more practical pM-KOFL having the same communication overhead as S-KOFL. Via numerical tests with real datasets, we demonstrate that pM-KOFL yields the almost same performance as vM-KOFL (or eM-KOFL) on various online learning tasks.

Rates of convergence of randomized Kaczmarz algorithms in Hilbert spaces

Recently, the Randomized Kaczmarz algorithm (RK) draws much attention because of its low computational complexity and less requirement on computer memory. Many existing results on analysis focus on the behavior of RK in Euclidean spaces, and typically derive exponential converge rates with the base tending to one, as the condition number of the system increases. The dependence on the condition number largely restricts the application of these estimates. There are also results using relaxation (i.e., small step-sizes tending to zero as the sample size increases) to achieve polynomial convergence rates of RK in Hilbert spaces. In this paper, we prove the weak convergence (with polynomial rates) of RK with constant step-size in Hilbert spaces. As a consequence, the strong convergence of RK in Euclidean spaces is obtained with condition number-free polynomial rates. In the setting with noisy data, we study the relaxation technique and obtain a strong convergence of RK in Hilbert spaces, with the rates arbitrarily close to the minimax optimal ones. We apply the analysis to reproducing kernel-based online gradient descent learning algorithms and improve the state-of-the-art convergence estimate in the literature.

Online optimization of switched LTI systems using continuous-time and hybrid accelerated gradient flows

This paper studies the design of feedback controllers to steer a switching linear dynamical system to the solution trajectory of a time-varying convex optimization problem. We propose two types of controllers: (i) a continuous controller inspired by the online gradient descent method, and (ii) a hybrid controller that can be interpreted as an online version of Nesterov’s accelerated gradient method with restarts of the state variables. By design, the controllers continuously steer the system toward a time-varying optimal equilibrium point without requiring knowledge of exogenous disturbances affecting the system. For cost functions that are smooth and satisfy the Polyak–Łojasiewicz inequality, we demonstrate that the online gradient-flow controller ensures uniform global exponential stability when the time scales of the system and controller are sufficiently separated and the switching signal of the system varies slowly on average. For cost functions that are strongly convex, we show that the hybrid accelerated controller can outperform the continuous gradient descent method. When the cost function is not strongly convex, we show that the hybrid accelerated method guarantees global practical asymptotic stability.

Temporal-Spatial Collaborative Mobile Edge Caching With User Satisfaction Awareness

For mobile proactive edge caching problems, the hit action cannot always reflect the efficiency of the cache strategy unilaterally. Thus, the satisfaction information of users should be used to improve the intelligent cache strategy. Motivated by this, this paper focuses a caching scheme that jointly considers the cache evaluation both from the perspectives of the hit action and satisfaction of users (Algorithm 3). Accordingly, we first propose a requesting behavior prediction model (Algorithm 1) to handle the multiple content features. Then, a requesting behavior factor learning algorithm (Algorithm 2) based on the online gradient descent (OGD) with regulations is proposed to obtain the estimation of the parameters of the model in Algorithm 1. Further, the probabilistic tensor factorization learning (PTFL) model with collaborative filtering and Markov Chain Monte Carlo (MCMC) procedure are introduced for the learning process. Finally, the proposed the caching Algorithm 3 based on the hit action prediction and the collaborative temporal-spatial satisfaction learning scheme can be performed. Moreover, we theoretically combine the upper limit performance of user request prediction errors with the lower limit of user satisfaction of edge cache. The experimental results verify the effectiveness of the proposed algorithm.

Online learning for min-max discrete problems

We study various discrete nonlinear combinatorial optimization problems in an online learning framework. In the first part, we address the computational complexity of designing vanishing regret (and vanishing approximate regret) algorithms. We provide a general reduction showing that many (min-max) polynomial time solvable problems not only do not have a vanishing regret, but also no vanishing approximation α-regret, for some α, unless NP=RP. In particular, for the min-max version of the vertex cover problem, which is solvable in polynomial time in the offline case, our reduction implies that there is no (2−ϵ)-regret online randomized algorithm unless Unique Game is in RP. Besides, we prove that the bound is tight by providing an online efficient algorithm based on the online gradient descent method. In the second part, we turn our attention to online learning algorithms that are based on an offline optimization oracle that, given a set of multiple instances of the problem, is able to compute the optimum static solution that performs best on the set of instances overall. We show that for several min-max (nonlinear) discrete optimization problems, it is strongly NP-hard to solve the offline optimization oracle, even for problems that can be solved in polynomial time in the single-instance static case (e.g. min-max vertex cover, min-max perfect matching, etc.). This also provides a useful insight into the connection between the non-linear nature of some problems and the drastic change of their computational hardness when moved to an online learning setting.