Hurst Parameter Estimation Research Articles

Deep learning the Hurst parameter of linear fractional processes and assessing its reliability

AbstractThis research explores the reliability of deep learning, specifically Long Short‐Term Memory (LSTM) networks, for estimating the Hurst parameter in fractional stochastic processes. The study focuses on three types of processes: fractional Brownian motion (fBm), fractional Ornstein–Uhlenbeck (fOU) process, and linear fractional stable motions (lfsm). The work involves a fast generation of extensive datasets for fBm and fOU to train the LSTM network on a large volume of data in a feasible time. The study analyses the accuracy of the LSTM network's Hurst parameter estimation regarding various performance measures like root mean squared error (RMSE), mean absolute error (MAE), mean relative error (MRE), and quantiles of the absolute and relative errors. It finds that LSTM outperforms the traditional statistical methods in the case of fBm and fOU processes; however, it has limited accuracy on lfsm processes. The research also delves into the implications of training length and valuation sequence length on the LSTM's performance. The methodology is applied to estimating the Hurst parameter in li‐ion battery degradation data and obtaining confidence bounds for the estimation. The study concludes that while deep learning methods show promise in parameter estimation of fractional processes, their effectiveness is contingent on the process type and the quality of training data.

Characterization of I/O Behaviors in Cloud Storage Workloads

As cloud platforms become increasingly popular, accurately understanding I/O behaviors in modern cloud storage is of paramount importance for system design and optimization. This paper sheds new light on the correlation of inter-arrival times of both read and write requests at the block level in four representative cloud storage workloads - AliCloud, Systor'17, MSRC and FIU. Our study reveals that I/O arrivals at the block level are very complex in modern cloud storage. There is a certain degree of correlation in the long-term timescale for request arrival intervals in AliCloud and Systor'17_read. Request arrival intervals in MSRC, FIU and Systor'17_write, however, are almost uncorrelated. The Gaussianity test confirms that I/O burstiness appears to be Gaussian in AliCloud_write and Systor'17_read, but the burstiness is non-Gaussian in other workloads. Importantly, we unfold the existence of self-similarity in cloud storage workloads with a certain degree of correlations, via visual evidence, the autocorrelation structure of the aggregated process of I/O request sequences, and Hurst parameter estimates. We further design an alpha-stable workload model for synthetic I/O generation, and the experimental results demonstrate that our model has an edge over conventional models in terms of accurately emulating I/O burstiness.

On quadratic variations for the fractional-white wave equation

This paper studies the behaviour of quadratic variations of a stochastic wave equation driven by a noise that is white in space and fractional in time. Complementing the analysis of quadratic variations in the space component carried out in [Correlation structure, quadratic variations and parameter estimation for the solution to the wave equation with fractional noise, Electron. J. Stat. 12 (2018), no. 2, 3639–3672] and [Generalized k k -variations and Hurst parameter estimation for the fractional wave equation via Malliavin calculus, J. Statist. Plann. Inference 207 (2020), 155–180], it focuses on the time component of the solution process. For different values of the Hurst parameter a central and a noncentral limit theorems are proved, allowing to construct parameter estimators and compare them to the findings in the space-dependent case. Finally, rectangular quadratic variations in the white noise case are studied and a central limit theorem is demonstrated.

The Delay Measurement and Analysis of Unreachable Hosts of Internet

Delay related metrics are significant quality of service criteria for the performance evaluation of networks. Almost all delay related measurement and analysis studies take into consideration the reachable sources of Internet. However, unreachable sources might also shed light upon some problems such as worm propagation. In this study, we carry out a delay measurement study of unreachable destinations and analyse the delay dynamics of unreachable nodes. 2. Internet Control Message Protocol (ICMP) destination unreachable Internet Control Message Protocol-Destination Unreachable (ICMP T3) packets are considered for the delay measurement according to their code types which shows network un reach ability, host un reach ability, port un reach ability, etc. Measurement results show that unreachable sources exhibit totally different delay behaviour compared to reachable IP hosts. A significant part of the unreachable hosts experiences extra 3 seconds Round Trip Time (RTT) delay compared to accessible hosts mostly due to host un reach ability. It is also seen that, approximately 79% of destination un reach ability causes from host un reach ability. Obtained Hurst parameter estimation results reveal that unreachable host RTTs show lower Hurst degree compared to reachable hosts which is approximately a random behaviour. Unreachable sources exhibit totally different distributional characteristic compared to accessible ones which is best fitted with Phased Bi-Exponential distribution.

A Revised Dynamic Hurst Parameter Estimation Method Based on Generalized Exponential Window Function

Dynamic Hurst parameter estimation is very important in the data filtering and system modelling. It is difficult to accurately estimate the dynamic Hurst parameter of long memory processes. In this research, a generalized exponential window function based dynamic Hurst parameter estimation method is provided, and the effective performance of the method is analyzed using fractional Gaussian noise. In the end of this paper, the application of revised estimation method in sleeping electroencephalogram signal is given. The simulation and sleeping electroencephalogram signal analysis results show that the revised estimation method can effectively analyze the random signals with local correlation characteristics. This method can be widely used in biomedical signal analysis, network traffic analysis, vibration signal analysis, statistical data analysis, etc.

A Note on Wavelet-Based Estimator of the Hurst Parameter.

The signals in numerous fields usually have scaling behaviors (long-range dependence and self-similarity) which is characterized by the Hurst parameter H. Fractal Brownian motion (FBM) plays an important role in modeling signals with self-similarity and long-range dependence. Wavelet analysis is a common method for signal processing, and has been used for estimation of Hurst parameter. This paper conducts a detailed numerical simulation study in the case of FBM on the selection of parameters and the empirical bias in the wavelet-based estimator which have not been studied comprehensively in previous studies, especially for the empirical bias. The results show that the empirical bias is due to the initialization errors caused by discrete sampling, and is not related to simulation methods. When choosing an appropriate orthogonal compact supported wavelet, the empirical bias is almost not related to the inaccurate bias correction caused by correlations of wavelet coefficients. The latter two causes are studied via comparison of estimators and comparison of simulation methods. These results could be a reference for future studies and applications in the scaling behavior of signals. Some preliminary results of this study have provided a reference for my previous studies.

Dynamic Hurst Parameter Estimation of Multi-fractional Processes in Impulse Noise Environment

Hurst parameter estimation is the key issue for long range dependent system modeling and data prediction. Dynamic Hurst parameter estimation of data with impulse noise is difficult. In this research, the Dynamic Hurst parameter of multi-fractional signal is estimated using sliding windowed R/S method and generalized Hurst exponent. Simulation results show that sliding windowed R/S method and generalized Hurst exponent can effectively analyze the Hurst parameter of impulse noise contaminated signals.

Identification of abnormal DNS traffic with Hurst parameter

It is a necessity for effective network management to be aware of the activities taking place on computer networks. Network managers should always be alarmed about what is happening now, what might be, or what will be in the future for the sake of network. To gather information about a computer system or a network, attackers mostly exploit networking tools to gain some privileges and login systems. Penetration testers also use these tools to gather information about systems, but their main concern is to discover the vulnerabilities of the system, and to find out what kind of measures could be applied to make the system more resistant to these vulnerabilities. In this study, we propose an abnormal DNS traffic identification method via utilizing Hurst parameter estimation. To do so, we employ DNS information gathering tools in Kali Linux to generate abnormal DNS flows. Then, we estimate its self-similarity degree to compare the differences between normal DNS traffic flows and abnormal ones. Obtained results show that abnormal DNS traffic show higher self-similarity degrees. Another interesting finding is that abnormal DNS traffic shows different distribution characteristic.

VoIP Anomaly Detection - selected methods of statistical analysis

<p>Self-similarity analysis and anomaly detection in networks are interesting fields of research and scientific work of scientists around the world. Simulation studies have demonstrated that the Hurst parameter estimation can be used to detect traffic anomaly. The actual network traffic is self-similar or long-range dependent. The dramatic expansion of applications on modern networks gives rise to a fundamental challenge to network security. The Hurst values are compared with confidence intervals of normal values to detect anomaly in VoIP.</p>

The Power Approximation of Time Series with Using Fractional Brownian Motion

We propose the approximating sequence and some of characteristics of this sequence to coincide with the increments of the fractional Brownian motion (fractional Browniannoise) for the observed time series. We study the Hurst parameter estimation algorithm and check the quality of the approximation.

A wavelet lifting approach to long-memory estimation

Reliable estimation of long-range dependence parameters is vital in time series. For example, in environmental and climate science such estimation is often key to understanding climate dynamics, variability and often prediction. The challenge of data collection in such disciplines means that, in practice, the sampling pattern is either irregular or blighted by missing observations. Unfortunately, virtually all existing Hurst parameter estimation methods assume regularly sampled time series and require modification to cope with irregularity or missing data. However, such interventions come at the price of inducing higher estimator bias and variation, often worryingly ignored. This article proposes a new Hurst exponent estimation method which naturally copes with data sampling irregularity. The new method is based on a multiscale lifting transform exploiting its ability to produce wavelet-like coefficients on irregular data and, simultaneously, to effect a necessary powerful decorrelation of those coefficients. Simulations show that our method is accurate and effective, performing well against competitors even in regular data settings. Armed with this evidence our method sheds new light on long-memory intensity results in environmental and climate science applications, sometimes suggesting that different scientific conclusions may need to be drawn.

Hurst parameter estimation by wavelet transformation and a filter bank for self-similar traffic

A model of traffic generation with the preset Hurst parameter (self-similarity) followed by its estimation in Matlab using wavelet transformation and a filter banks is described. The error in estimating the given parameter is analyzed depending on the value of this parameter and the number of wavelet transformation scales used in the estimation procedure. This study is experimental, aimed at real-time estimation of the Hurst parameter and the accuracy of this estimation in the case of wavelet transformation with the help of filter banks.

Comparative Studies of Methods for Accurate Hurst Parameter Estimation

Over the last decade the traffic of the packet-based networks shows self-similar or long-range dependent nature. Such behaviour raises issues of accuracy in resource allocation and related guarantee required of QoS parameters for data flow. One of the popular mechanisms for QoS parameters guaranty is Measurement-based Admission Control (MBAC). In it’s turn, the efficiency of the MBAC mechanism is tightly related to the accuracy of traffic parameters estimation. The appropriate buffer size estimation is one of primary tasks in MBAC for QoS parameters guarantee. Hurst parameter which describes the level of self-similarity is used for this purpose. The accuracy of it’s estimation is important as otherwise underutilization or over-saturation of the system may occur that would cause high packet losses. The present paper provides a thorough overview of currently known methods for Hurst parameter estimation and presents research results of accuracy levels of each of them.

Stream response to precipitation variability: a spectral view based on analysis and modelling of hydrological cycle components

AbstractHydrological processes commonly exhibit long‐term persistence, also known as the ‘Hurst phenomenon’. Here, we examine long‐term precipitation and streamflow time series from the Elbe River Basin to quantify differences in the spectral properties and in the Hurst parameter estimates () of the individual hydrological cycle components. Precipitation‐runoff modelling is performed for the Elbe River sub‐catchment Striegis using the Soil and Water Assessment Tool (SWAT). For 38 daily 50 years long streamflow time series from the Elbe River Basin, baseflow separation and spectral analysis is performed. The results show a spectral shift towards low‐frequency scales (>2 years) from precipitation to baseflow, with a parallel increase of from 0.52 (precipitation) to 0.65 (baseflow). The SWAT model is able to reproduce both, the main low‐frequency mode (≈7 yr.) and the (0.62) of the observed Striegis River flow time series. The baseflow appears to be the main component which shapes the low‐frequency response and of streamflow in the Elbe River Basin to the input precipitation. This conclusion is further confirmed through PMWIN‐MODFLOW groundwater modelling of a hypothetic phreatic stream‐connected aquifer system that consists of various soils (sand, loamy sand and silt). A power shift towards lower frequencies and an increase of for the hydraulic heads is obtained, as the aquifer's lateral dimensions increase and its hydraulic conductivity decreases. The average of the groundwater heads is 0.80, 0.90 and 1.0 for sand, loamy sand and silt aquifers, respectively. Copyright © 2014 John Wiley & Sons, Ltd.

User behavior modeling of voice communications: an empirical study

AbstractIn this study, analysis and modeling of arrival and service processes are presented in a comprehensive fashion in order to determine statistical properties of voice traffic from end‐user perspective in accordance with the queueing theory. For the first time in the literature, we introduce a user centric approach and examine these services considering both flow directions of voice traffic, the uplink and the downlink as opposed to existing studies with the network centric approach. In our study, we use experimental data composed of actual phone calls collected from 2G/3G networks. To achieve this, we designed and implemented a data collection system for mobile users and compared the results by using data from an operational cellular network. In order to determine the time correlation of voice calls, Hurst parameter estimation methods are used. On the basis of the outcomes, independency of call arrivals is shown. Additionally, it is shown that calls acquired from user and network centric approaches are both Poisson distributed. Next, looking at the problem from service process perspective, thorough analyses are performed to determine mathematical models that can best characterize call holding times. Maximum likelihood estimation and expectation maximization algorithm are used, and it is shown that the optimum mathematical model for the characterization of call holding times is the lognormal distribution family. Copyright © 2014 John Wiley & Sons, Ltd.

Analysis of wavelet estimation of self-similar traffic parameters in the Simulink model

The queue model P/M/1/K with the Hurst parameter (self-similarity parameter) estimator in the Simulink environment is described in the present paper using wavelet transformation. The estimation error of the parameter is analyzed depending on the parameter’s value, as well as the number of measurements based on which the parameter is estimated. This is an experimental paper aimed at accumulating knowledge on the practicability of estimating the Hurst parameter and on the estimation accuracy in the case of using wavelet analysis.

Long‐term variations and temporal scaling of hydroclimatic time series with focus on the German part of the Elbe River Basin

AbstractLong‐term variations and temporal scaling of mean monthly time series of river flow, precipitation, temperature, relative humidity, air pressure, duration of bright sunshine, degree of cloud cover, short wave radiation, wind speed and potential evaporation within or in vicinity of the German part of the Elbe River Basin are analyzed. Statistically significant correlations between the 2–15 year scale‐averaged wavelet spectra of the hydroclimatic variables and the North Atlantic Oscillation‐ and Arctic Oscillation index are found which suggests that such long‐term patterns in hydroclimatic time series are externally forced. The Hurst parameter estimates (H) based on the Detrended Fluctuation Analysis (DFA) indicate persistence for discharge, precipitation, wind speed, air pressure and the degree of cloud cover, all having an annual cycle and a broad low‐frequency distribution. Also, DFA H parameter estimates are higher for discharge than for precipitation. The major long‐term quasi‐periodic variability modes of precipitation detected using Singular Spectrum Analysis coincide with those detected in the discharge time series. Upon subtraction of these low‐frequency quasi‐periodic modes, the DFA H parameter estimates suggest absence of the persistence for both precipitation and discharge. Copyright © 2013 John Wiley & Sons, Ltd.

Real-Time Estimation of Traffic Self-Similarity Parameter in Simulink with Wavelet Transform

The article describes P/M/1/K queuing model and Hurst parameter estimation unit with Wavelet Transform for real-time estimation of the traffic self-similarity parameter in Simulink. It consists of simulation results, which show the possibility to estimate Hurst parameter in real-time by means of Wavelet transform. It is offered to iteratively calculate the averaged Hurst parameter estimation, which improves the accuracy to the accuracy of the estimation over entire traffic series and in some cases even higher, as it has been shown in simulation results analysis. The estimation deviations have been analyzed as well as variance of such deviations. The results also give directions for further research to improve accuracy of traffic parameters estimation. DOI: http://dx.doi.org/10.5755/j01.eee.19.3.3702

Effects of trends and seasonalities on robustness of the Hurst parameter estimators

Long-range dependence (LRD) is discovered in time series arising from different fields, especially in network traffic and econometrics. Detecting the presence and the intensity of LRD plays a crucial role in time-series analysis and fractional system identification. The existence of LRD is usually indicated by the Hurst parameters. Up to now, many Hurst parameter estimators have been proposed in order to identify the LRD property involved in a time series. Since different estimators have different accuracy and robustness performances, in this study, 13 most popular Hurst parameter estimators are summarised and their estimation performances are investigated. LRD processes with known Hurst parameters are generated as the control data set for the robustness evaluation. In addition, three types of LRD processes are also obtained as the test signals by adding noises in terms of means, trends and seasonalities to the control data set. All 13 Hurst parameter estimators are applied to these LRD processes to estimate the existing Hurst parameters. The estimation results are documented and quantified by the standard errors. Conclusions of the accuracy and robustness performances of the estimators are drawn by comparing the estimation results.

Wavelet Spectrum for Investigating Statistical Characteristics of UDP-Based Internet Traffic

In this paper, we consider statistical characteristics of real User Datagram Protocol (UDP) traffic. Four main issues in the study include(i) the presence of long rangedependence (LRD) in the UDP traffic,(ii) the marginal distribution of the UDP traces,(iii) dependence structure of wavelet coefficients,(iv) and performance evaluation of the Hurst parameter estimation based on different numbers of vanishing moments of the mother wavelet . By analyzing a large set of real traffic data, it is evident that the UDP Internet traffic reveals the LRD properties with considerably high non-stationary processes.Furthermore, it exhibits non-Gaussian marginal distributions. However, by increasing the number of vanishing moments,it is impossible to achieve reduction fromLRD to become a short range dependence. Thus, it can be shown that there is no significant difference in performance estimation of the Hurst parameter for different numbers of vanishing moments of the mother wavelet.