Abstract
The present study addresses the problem of sequential least square multidimensional linear regression, particularly in the case of a data stream, using a stochastic approximation process. To avoid the phenomenon of numerical explosion which can be encountered and to reduce the computing time in order to take into account a maximum of arriving data, we propose using a process with online standardized data instead of raw data and the use of several observations per step or all observations until the current step. Herein, we define and study the almost sure convergence of three processes with online standardized data: a classical process with a variable step-size and use of a varying number of observations per step, an averaged process with a constant step-size and use of a varying number of observations per step, and a process with a variable or constant step-size and use of all observations until the current step. Their convergence is obtained under more general assumptions than classical ones. These processes are compared to classical processes on 11 datasets for a fixed total number of observations used and thereafter for a fixed processing time. Analyses indicate that the third-defined process typically yields the best results.
Highlights
In the present analysis, A0 denotes the transposed matrix of A while the abbreviation “a.s.” signifies almost surely.Let R = (R1,. . .,Rp) and S = (S1,. . .,Sq) be random vectors in Rp and Rq respectively
As a whole the major contributions of this work are to reduce gradient variance by online standardization of the data or use of a “dynamic” batch process, to avoid numerical explosions, to reduce computing time and to better adapt the stochastic approximation processes used to the case of a data stream
A stochastic approximation method with standardized data appears to be advantageous compared to a method with raw data
Summary
A0 denotes the transposed matrix of A while the abbreviation “a.s.” signifies almost surely. If the expectation and the variance of the components of R and S were known, standardization of these variables could be made directly and convergence of the processes obtained using existing theorems These moments are unknown in the case of a data stream and are estimated online in this study. As a whole the major contributions of this work are to reduce gradient variance by online standardization of the data or use of a “dynamic” batch process, to avoid numerical explosions, to reduce computing time and to better adapt the stochastic approximation processes used to the case of a data stream
Sign-in/Register to view highlights & summary Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
