Sum Of Processes Research Articles

Limit Theorems for Partial Sum Processes of Moving Averages Based on Heterogeneous Processes

Limit Theorems for Partial Sum Processes of Moving Averages Based on Heterogeneous Processes

Human-like Learning in Temporally Structured Environments

Natural environments have correlations at a wide range of timescales. Human cognition is tuned to this temporal structure, as seen by power laws of learning and memory, and by spacing effects whereby the intervals between repeated training data affect how long knowledge is retained. Machine learning is instead dominated by batch iid training or else relatively simple nonstationarity assumptions such as random walks or discrete task sequences. The main contributions of our work are: (1) We develop a Bayesian model formalizing the brain's inductive bias for temporal structure and show our model accounts for key features of human learning and memory. (2) We translate the model into a new gradient-based optimization technique for neural networks that endows them with human-like temporal inductive bias and improves their performance in realistic nonstationary tasks. Our technical approach is founded on Bayesian inference over 1/f noise, a statistical signature of many natural environments with long-range, power law correlations. We derive a new closed-form solution to this problem by treating the state of the environment as a sum of processes on different timescales and applying an extended Kalman filter to learn all timescales jointly. We then derive a variational approximation of this model for training neural networks, which can be used as a drop-in replacement for standard optimizers in arbitrary architectures. Our optimizer decomposes each weight in the network as a sum of subweights with different learning and decay rates and tracks their joint uncertainty. Thus knowledge becomes distributed across timescales, enabling rapid adaptation to task changes while retaining long-term knowledge and avoiding catastrophic interference. Simulations show improved performance in environments with realistic multiscale nonstationarity. Finally, we present simulations showing our model gives essentially parameter-free fits of learning, forgetting, and spacing effects in human data. We then explore the analogue of human spacing effects in a deep net trained in a structured environment where tasks recur at different rates and compare the model's behavioral properties to those of people.

Simultaneous Model Change Detection in Multivariate Linear Regression With Application to Indonesian Economic Growth Data

In this paper, we study asymptotic model change detection in multivariate linear regression by using the Kolmogorov–Smirnov function of the partial sum process of recursive residuals. We approximate the rejection region and also the power function of the test by establishing a functional central limit theorem for the sequence of the partial sum processes of the recursive residuals of the observations. When the assumed model is true, the limit process is given by the standard multivariate Brownian motion which does not depend on the regression functions. However, when the assumed model is not true (some models change), the limit process is represented by a vector of deterministic trend plus the standard multivariate Brownian motion. The finite sample size rejection region and the power of the test are investigated by means of Monte Carlo simulation. The simulation study shows evidence that the proposed test is consistent in the sense that it attains the power larger than the size of the test when the hypothesis is not true. We also demonstrate the application of the proposed test method to Indonesian economic growth data in which we test the adequacy of three‐variate low‐order polynomial model. The test result shows that the growth of the Indonesian economy is neither simultaneously constant nor linear. The test has successfully detect the appearance of a change in the model which is mainly caused by the COVID‐19 pandemic in 2020.

Counterbalancing steps at random in a random walk

A random walk with counterbalanced steps is a process of partial sums \check S(n)=\check X_{1}+ \cdots + \check X_{n} whose steps \check X_{n} are given recursively as follows. For each n\geq 2 , with a fixed probability p , \check X_{n} is a new independent sample from some fixed law \mu , and with complementary probability 1-p , \check X_{n}= -\check X_{v(n)} counterbalances a previous step, with v(n) a uniform random pick from \{1, \ldots, n-\nobreak 1\} . We determine the asymptotic behavior of \check S(n) in terms of p and the first two moments of \mu . Our approach relies on a coupling with a reinforcement algorithm due to H. A. Simon, and on properties of random recursive trees and Eulerian numbers, which may be of independent interest. The method can be adapted to the situation where the step distribution \mu belongs to the domain of attraction of a stable law.

Two Monotone Processes of a Replacement Policy for a Repairable System with its Repairman Having Multiple Vacations.

For the maintenance of repairable system with a repairman having multiple vacations using two monotone processes. If the system fails and the repairman is on vacation, it will wait for repair until the repairman is available. Assume the damaged system cannot be repaired “as good as new” and the repairman can be repaired immediately with a probability of p, we optimize replacement policy using increasing alpha series and decreasing partial sum processes. The explicit expression for the expected profit under replacement policy N is derived analytically. And we also a numerical example is given to illustrate the theoretical results of the model mentioned in this paper.

Limit theorems for functionals of long memory linear processes with infinite variance

Let X={Xn:n∈N} be a long memory linear process in which the coefficients are regularly varying and innovations are independent and identically distributed and belong to the domain of attraction of an α-stable law with α∈(0,2). Then, for any integrable and square integrable function K on R, under certain mild conditions, we establish the asymptotic behavior of the partial sum process ∑n=1[Nt][K(Xn)−EK(Xn)]:t≥0as N tends to infinity, where [Nt] is the integer part of Nt for t≥0.

Testing for changes in linear models using weighted residuals

We study methods for detecting change points in linear regression models. Motivated by statistics arising from maximally selected likelihood ratio tests, we provide an asymptotic theory for weighted functionals of the cumulative sum (CUSUM) process of linear model residuals. Special attention is given to standardized quadratic form statistics, leading to Darling–Erdős type limit results, as well as novel heavily weighted CUSUM statistics that increase the power of the tests to detect changes that occur early or late in the sample. We discuss improved finite-sample approaches to estimate the critical values for the proposed statistics, which are shown to work well in a Monte Carlo simulation study. The proposed tests are applied to the environmental Kuznets curve, and a COVID-19 dataset.

Monitoring parameter change for bivariate time series models of counts.

In this study, we consider an online monitoring procedure to detect a parameter change for bivariate time series of counts, following bivariate integer-valued generalized autoregressive heteroscedastic (BIGARCH) and autoregressive (BINAR) models. To handle this problem, we employ the cumulative sum (CUSUM) process constructed from the (standardized) residuals obtained from those models. To attain control limits, we develop limit theorems for the proposed monitoring process. A simulation study and real data analysis are conducted to affirm the validity of the proposed method.

Multiscale change point detection via gradual bandwidth adjustment in moving sum processes

A method for the detection of changes in the expectation in univariate sequences is provided. Moving sum processes (MOSUM) are studied. These rely on the selection of a tuning bandwidth. Here, a framework to overcome bandwidth selection is presented – the bandwidth adjusts gradually. For that, MOSUM are made dependent on both time and the bandwidth: the domain becomes a triangle. On the triangle, paths are constructed which systematically lead to change points. An algorithm is provided that estimates change points by subsequent consideration of paths. Strong consistency for the number and location of change points is shown. Simulation studies corroborate estimation precision and reveal competitiveness with state of the art change point detection methods. A companion R-package mscp is made available on CRAN.

Monitoring anomaly for time series of volatility index(VIX)

This study investigates a monitoring procedure that aims to capture a change of model parameters in time series regarding the volatility index (VIX). To solve this problem, we harness a cumulative sum (CUSUM) process using residuals designed to simultaneously detect changes in both the conditional mean and variance of the time series. To obtain the control limits, limit theorems are used for the CUSUM monitoring process. Thereafter, the scheme of statistical process control (SPC) is applied to the time series of the log-returns of VIX by fitting it with an ARMA-GARCH model to obtain further knowledge regarding the financial market’s participants. Our study indicates the proposed monitoring process’ potential as a tool to allude to imminent significant market crashes.

Correct specification of design matrices in linear mixed effects models: tests with graphical representation

Linear mixed effects models (LMMs) are a popular and powerful tool for analysing grouped or repeated observations for numeric outcomes. LMMs consist of a fixed and a random component, which are specified in the model through their respective design matrices. Verifying the correct specification of the two design matrices is important since mis-specifying them can affect the validity and efficiency of the analysis. We show how to use empirical stochastic processes constructed from appropriately ordered and standardized residuals from the model to test whether the design matrices of the fitted LMM are correctly specified. We define two different processes: one can be used to test whether both design matrices are correctly specified, and the other can be used only to test whether the fixed effects design matrix is correctly specified. The proposed empirical stochastic processes are smoothed versions of cumulative sum processes, which have a nice graphical representation in which model mis-specification can easily be observed. The amount of smoothing can be adjusted, which facilitates visual inspection and can potentially increase the power of the tests. We propose a computationally efficient procedure for estimating p-values in which refitting of the LMM is not necessary. Its validity is shown by using theoretical results and a large Monte Carlo simulation study. The proposed methodology could be used with LMMs with multilevel or crossed random effects.

Monitoring parameter change for time series models with application to location-Scale heteroscedastic models

In this study, we consider an on-line monitoring procedure to detect a parameter change in general time series models, featuring location-scale heteroscedastic time series models and their conditional quantiles. To resolve this statistical process control (SPC) problem, we employ a residual-based cumulative sum (CUSUM) process specially designed to effectively detect both upward and downward changes in the conditional mean, variance, and quantiles of time series. To attain control limits analytically, limit theorems are provided for the proposed CUSUM monitoring process. A simulation study and real data analysis are conducted to illustrate its validity empirically.

Convergence of partial sum processes to stable processes with application for aggregation of branching processes

We provide a generalization of Theorem 1 in Bartkiewicz et al. (2011) in the sense that we give sufficient conditions for weak convergence of finite dimensional distributions of the partial sum processes of a strongly stationary sequence to the corresponding finite dimensional distributions of a non-Gaussian stable process instead of weak convergence of the partial sums themselves to a non-Gaussian stable distribution. As an application, we describe the asymptotic behaviour of finite dimensional distributions of aggregation of independent copies of a strongly stationary subcritical Galton–Watson branching process with regularly varying immigration having index in (0,1)∪(1,4/3) in a so-called iterated case, namely when first taking the limit as the time scale and then the number of copies tend to infinity.

Impact of economic policy uncertainty on the stock markets of the G7 Countries:A nonlinear ARDL approach

This paper employs the linear and nonlinear ARDL models to examine the short-run and long-run effects of Economic Policy Uncertainty (EPU) on the stock prices of the G7 countries using monthly data up to May of 2021. Using the linear ARDL model, the results indicate that EPU has a significant negative short-run effect on the stock prices of all the G7 countries. These negative short-run effects last into the long-run in only Canada and Japan, with no long-run effect in the remaining countries. However, using the nonlinear ARDL model shows that at least one of the partial sum processes of positive and negative changes in EPU has a significant short-run effect on the stock prices of all the G7 countries, and that these short-run effects last into the long-run in all the G7 countries, except in the UK. Moreover, we find evidence of asymmetries in both the short-run and long-run in all the countries, except in the UK. Our findings have important policy implications.

Sequential common change detection, isolation, and estimation in multiple poisson processes

In this article, motivated by detecting the occurrence of an epidemic when the arrival rates of patients increase in a portion of M panels or detecting the deterioration of a system composed of M independent components that causes an increase in failure rates in a portion of components, we consider the detection of a common change when M independent Poisson processes are monitored simultaneously where only a portion of the processes have rate increases after the change time. M individual cumulative sum (CUSUM) processes and Shiryaev-Roberts (S-R) processes are calculated recursively in parallel at each pooled arrival time. A systematic procedure is proposed by using the sum of M S-R processes as the detection process for a common change. After the detection, the M individual CUSUM processes are used to isolate the changed panels with false discovery rate (FDR) control and then the medians of the change time estimates from each individual CUSUM process or S-R process based on the isolated panels are used to estimate the common change time. The model can be generalized to different prechange rates, jittered change time, and unknown postchange rates.

Symmetric or asymmetric: how is Nigeria's bilateral trade balance responding to oil price and exchange rate changes?

PurposeThe experience of rising trade imbalance between Nigeria and its key trading partners in recent years motivated this study. Previous studies on this issue either ignored bilateral level or assumed that the effect of crude oil price and/or exchange rate changes on trade balance is symmetric. Consequently, this study investigates whether Nigeria's bilateral trade balance with Belgium, China, United Kingdom (UK) and USA is responding symmetrically or asymmetrically to changes in oil price and exchange rate.Design/methodology/approachThe authors used nonlinear autoregressive-distributed lag (NARDL) model that decomposed oil price and exchange rate into partial sum processes of positive and negative changes over the period 1999Q1–2019Q4.FindingsThe study finds that the effects of oil price hike and plunge asymmetrically influence Nigeria's trade balance with the UK and USA. Further evidence indicated that oil price plunge exerts greater influence than price hike in all the cases, except the UK in the long run. Furthermore, Nigeria's trade balance responds asymmetrically and significantly to changes in exchange rate with China in the long run and with China and the UK in the short run. Specifically, the depreciation effect is more prominent than appreciation.Originality/valueSignificant contributions to the existing literature in Nigeria include the recognition that the effects of oil price and exchange rate changes on trade are asymmetric and the disaggregation of trade into bilateral level to identify country-specific effect.

High-performance pseudo-anonymization of virtual power plant data on a CPU cluster

The considerable move towards the use of renewable energy resources has been provided by the digitization of energy systems with the help of virtual power plants (VPPs). However, due to the coincidence of this move with the introduction of new technologies in information and communications, joining these systems raises concerns about the privacy of personal data. The only real-world approach widely used in this case is to anonymize or pseudonymize the information associated with individuals in data received from distributed measurement devices. In this paper, we propose the method of classifying received data packets into different flows and assigning different access levels for each flow. This method makes data pseudonymous. Before this step, the received data, which has a different format, is unionized. To implement this idea, a tuple space flow classification algorithm is parallelized on a CPU cluster using MPI and OpenMP according to different scenarios. The CPU cluster consists of one head node and two computational nodes for packet classification operations. In this research, two scenarios have been used to run the CPU algorithm in parallel. The first scenario uses MPI and the second scenario uses a combination of MPI and OpenMP libraries. Also, the Tuple Space algorithm has been implemented on the computing systems using the mentioned libraries in the form of two scenarios using OpenMP and MPI. According to our results, the increase in the number of processor cores is linearly correlated with the increase in the speed of classification. Furthermore, while MPI uses more memory than OpenMP, it helps to achieve a higher speed of classification. In the combined method, the maximum speed of flow classification can be achieved if the number of processes and threads is equal to the number of processor cores. In other words, when the sum of processes and threads does not outnumber CPU cores, the least classification time and memory usage can be achieved.

Gas path parameter prediction of aero-engine based on an autoregressive discrete convolution sum process neural network

In order to improve the approximation ability of neural network to functional and improve the prediction accuracy of time series data, this paper proposes an autoregressive discrete convolution sum process neural network prediction model and applies it to the prediction of aero-engine gas path performance parameters. Coupling threshold wavelet de-noising method is applied to the preprocessing of engine gas path parameters, which can effectively remove the noise in the time series data. Process neurons are applied to artificial neural networks to expand the computational functions of artificial neurons. The process neuron can not only realize the spatial aggregation operation of discrete input data and the connection weight, but also realize the time aggregation operation of the product integral of the continuous input function and the connection weight function. The output of discrete time series is affected by the current input/output, and also by the historical input/output value. Therefore, an autoregressive feedback link is added to the network topology, and the value of the network's output node is used as the input to adjust the network connection weight. The convolution sum operation of discrete time series data can realize the time accumulation effect, so the discrete convolution sum operator is used to replace the integral operator to realize the time aggregation function of the process neuron. Since the integral operation of the continuous function is avoided, the weight training process of the process neural network is effectively simplified, and the accuracy loss in the continuous-time function fitting process of discrete input samples is effectively avoided. Bayesian regularization network weight learning algorithm is used to solve the problems of slow convergence speed and easy to fall into local optimum of back propagation learning algorithm, and improve the generalization ability of neural network. It can be seen from the prediction simulation results of the two engines that the average relative error and root mean square error of the engine gas path parameter prediction based on the autoregressive discrete convolution sum process neural network model can be reduced to 0.67% and 0.35 respectively. The stable and high-precision prediction results show that the network model and weight learning algorithm proposed in this paper have better robustness in functional approximation, and have higher accuracy in time series data prediction.

Capsule networks with non-iterative cluster routing

Capsule networks use routing algorithms to flow information between consecutive layers. In the existing routing procedures, capsules produce predictions (termed votes) for capsules of the next layer. In a nutshell, the next-layer capsule’s input is a weighted sum over all the votes it receives. In this paper, we propose non-iterative cluster routing for capsule networks. In the proposed cluster routing, capsules produce vote clusters instead of individual votes for next-layer capsules, and each vote cluster sends its centroid to a next-layer capsule. Generally speaking, the next-layer capsule’s input is a weighted sum over the centroid of each vote cluster it receives. The centroid that comes from a cluster with a smaller variance is assigned a larger weight in the weighted sum process. Compared with the state-of-the-art capsule networks, the proposed capsule networks achieve the best accuracy on the Fashion-MNIST and SVHN datasets with fewer parameters, and achieve the best accuracy on the smallNORB and CIFAR-10 datasets with a moderate number of parameters. The proposed capsule networks also produce capsules with disentangled representation and generalize well to images captured at novel viewpoints. The proposed capsule networks also preserve 2D spatial information of an input image in the capsule channels: if the capsule channels are rotated, the object reconstructed from these channels will be rotated by the same transformation.

Likelihood ratio test for the mean of asymptotic spatial regression with the Brownian sheet noise

Likelihood ratio test (LR-test) is frequently applied in regression analysis when the observations are normally distributed. However, when the normality assumption is violated, such a test can not be directly adopted in practice. In this work, we propose an approach by firstly transforming the observations to a set-indexed partial sums process. The limit model under this transformation is presented as a deterministic trend plus a random noise given by the set-indexed Brownian sheet. We show that the problem of testing the appropriateness of a model can be handled step-wise by testing the validity of the trend in the limit model by defining an LR-test based on the ratio between the likelihood function under H 0 and under H 0 ∪ H 1. The rejection region as well as the power of the size α LR-test are obtained based on the Cameron-Martin-Girsanov formula of the Radon-Nikodym derivative of the set-indexed partial sums limit process of the observation with respect to the set-indexed Brownian sheet. Simulation study shows that the proposed test behaves as a consistent test in that it maximizes the power when H 1 is true. Application of the method to real data can help in detecting valid regression model describing the variability of the maximum height of corn plants over the experimental region.