Abstract
We introduce an ordinate method for noisy data analysis, based solely on rank information and thus insensitive to outliers. The method is nonparametric, objective, and the required data processing is parsimonious. Main ingredients are a rank-order data matrix and its transform to a stable form, which provide linear trends in excellent agreement with least squares regression, despite the loss of magnitude information. A group symmetry orthogonal decomposition of the 2D rank-order transform for iid (white) noise is further ordered by principal component analysis. This two-step procedure provides a noise "etalon" used to characterize arbitrary stationary stochastic processes. The method readily distinguishes both the Ornstein-Uhlenbeck process and chaos generated by the logistic map from white noise. Ranking within randomness differs fundamentally from that in deterministic chaos and signals, thus forming the basis for signal detection. To further illustrate the breadth of applications, we apply this ordinate method to the canonical nonlinear parameter estimation problem of two-species radioactive decay, outperforming special-purpose least square software. It is demonstrated that the method excels when extracting trends in heavy-tailed noise and, unlike the Thiele-Sen estimator, is not limited to linear regression. Lastly, a simple expression is given that yields a close approximation for signal extraction of an underlying generally nonlinear signal.
Highlights
We report on a discovery of a rank-based method that appears remarkably versatile and robust with respect to the nature of noise because the method is ordinal, nonparametric, and distribution independent
II, we introduce and motivate the initial construction of our method in a simple setting: We begin by solving for the long-term warming trend buried in a fluctuating time series of daily low temperature
We develop the theoretical basis for error analysis and apply it to linear regression, accounting for the otherwise enigmatic agreement of the linear fits exhibited in Sec
Summary
We report on a discovery of a rank-based method that appears remarkably versatile and robust with respect to the nature of noise because the method is ordinal, nonparametric, and distribution independent. II for long-term trends are unaffected by trends in variance X, two data sets with distributions of infinite mean and variance noise are explored For such distributions, the Theil-Sen nonparametric method is commonly used, but it is limited to linear regression.
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Disclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.