Non-stationary Data Research Articles

Learning Dynamic Latent Spaces for Lifelong Generative Modelling

Task Free Continual Learning (TFCL) aims to capture novel concepts from non-stationary data streams without forgetting previously learned knowledge. Mixture models, which add new components when certain conditions are met, have shown promising results in TFCL tasks. However, such approaches do not make use of the knowledge already accumulated for positive knowledge transfer. In this paper, we develop a new model, namely the Online Recursive Variational Autoencoder (ORVAE). ORVAE utilizes the prior knowledge by selectively incorporating the newly learnt information, by adding new components, according to the knowledge already known from the past learnt data. We introduce a new attention mechanism to regularize the structural latent space in which the most important information is reused while the information that interferes with novel samples is inactivated. The proposed attention mechanism can maximize the benefit from the forward transfer for learning novel information without forgetting previously learnt knowledge. We perform several experiments which show that ORVAE achieves state-of-the-art results under TFCL.

Gene regulatory network inference based on a nonhomogeneous dynamic Bayesian network model with an improved Markov Monte Carlo sampling

A nonhomogeneous dynamic Bayesian network model, which combines the dynamic Bayesian network and the multi-change point process, solves the limitations of the dynamic Bayesian network in modeling non-stationary gene expression data to a certain extent. However, certain problems persist, such as the low network reconstruction accuracy and poor model convergence. Therefore, we propose an MD-birth move based on the Manhattan distance of the data points to increase the rationality of the multi-change point process. The underlying concept of the MD-birth move is that the direction of movement of the change point is assumed to have a larger Manhattan distance between the variance and the mean of its left and right data points. Considering the data instability characteristics, we propose a Markov chain Monte Carlo sampling method based on node-dependent particle filtering in addition to the multi-change point process. The candidate parent nodes to be sampled, which are close to the real state, are pushed to the high probability area through the particle filter, and the candidate parent node set to be sampled that is far from the real state is pushed to the low probability area and then sampled. In terms of reconstructing the gene regulatory network, the model proposed in this paper (FC-DBN) has better network reconstruction accuracy and model convergence speed than other corresponding models on the Saccharomyces cerevisiae data and RAF data.

Multichannel joint resolution enhancement for nonstationary prestack data with adaptive structure regularization

Seismic waves always suffer from amplitude dissipation and phase distortion and such attenuation-induced effects will degrade the quality of seismic data. Therefore, a slew of research on attenuation compensation has been proposed to correct the attenuation effects and improve the resolution of the original data. In recent years, geophysicists have paid more attention to prestack compensation because prestack data contain more information about the subsurface medium. However, it is challenging to implement prestack compensation for which the traveltime of prestack data is difficult to be determined unless an accurate velocity model is available. Furthermore, prestack data usually contains more noises and the signal-to-noise ratio (S/N) of prestack data is lower, which is more prone to noise amplification during compensation. To improve the performance of prestack compensation, we have developed a robust structure-oriented multichannel prestack compensation method. We incorporate an additional adaptive structure regularization to stabilize the processing of compensation. We refer to our structure-constrained multichannel prestack compensation method as the SCMPC method. With the help of adaptive structure regularization, our method has a superior ability to suppress those artifacts caused by ambient noises and enhance the consistency of effective signals during compensation. Compared with classic single-channel prestack compensation methods, this SCMPC method can provide a better result with higher accuracy and S/N. We use synthetic and field examples to further validate the good performance of our method.

Mode decomposition based large margin distribution machines for sediment load prediction

Precise suspended sediment load (SSL) prediction is essential for irrigation, hydropower and river management practices. But due to some external factors such as high altitude, heavy monsoon and tropical climate conditions, the data we collect may contain noisy samples. Hence, it becomes a challenging task to accurately predict the SSL in rivers. Therefore, to diminish the influence of noise, the empirical mode decomposition (EMD) -based techniques can be adopted. Moreover, the large margin distribution machine-based regression (LDMR) can deal efficiently with noisy datasets as it simultaneously minimizes the insensitive loss as well as quadratic loss. The significance of this work lies in its contribution to improving the accuracy of suspended sediment load (SSL) prediction in rivers, which has practical implications for various applications such as irrigation, hydropower, and river management. We recognize the challenges posed by external factors, which introduce noise into the collected data, making accurate prediction of SSL difficult. Overall, the significance of this work lies in its novel integration of EMD-based techniques and LDMR-based models, which address the challenges posed by noisy and non-stationary sediment load data. The findings contribute to the field of SSL prediction, offering practical solutions for managing and utilizing river resources more effectively in various domains. The main advantage of the least squares LDMR (LS-LDMR) approach is that it solves a system of linear equations rather than solving a large QPPs unlike SVR, TSVR, LDMR, which makes it computationally efficient. It is well known that the sediment load data is complex and non-stationary in nature. Therefore, for daily SSL prediction, the LDMR and least squares LDMR (LSLDMR) models are embedded with two different decomposition techniques, EMD and ensemble EMD (EEMD), to handle the nonlinear and non-stationary characteristics of sediment load data on the forecasted outputs and to increase prediction ability. The results of the proposed EMD-LDMR, EEMD-LDMR, EMD-LSLDMR and EEMD-LSLDMR are compared with the conventional support vector regression (SVR), twin SVR (TSVR), LDMR and LSLDMR. The performance of the evaluated using the best-fit model using the mean absolute error (MAE), root mean square error (RMSE), symmetric mean absolute percentage error (SMAPE), Willmott’s index (WI), correlation coefficient (CC) and R2. Better or comparable results specify the efficiency of the suggested models. For graphical visualization, scatterplots, prediction error plots and violin plots are shown. Numerical results show that these hybrid models show excellent prediction performance. The overall analysis of the results suggests using EEMD-LSLDMR for SSL estimation.

Self-supervised time-frequency representation based on generative adversarial networks

Time-frequency (TF) transforms are commonly used to analyze local features of nonstationary seismic data and to help uncover structural or geologic information. Traditional TF transforms, such as short-time Fourier transform, continuous wavelet transform, and S-transform, suffer from the Heisenberg uncertainty principle, and their TF resolution is limited. The sparse TF (STF) transform has been proposed to address this disadvantage; however, expensive calculation and parameter selection present difficulties. We have developed a self-supervised TF representation based on a generative adversarial networks (STFR-GANs) model to map a 1D seismic signal into a 2D STF image. This model includes three components: a generator, a discriminator, and a reconstruction module. The generator is used to generate the STF spectrum of the input seismic trace, whereas the discriminator distinguishes if this generated STF spectrum is optimal. The reconstruction module serves as a physical constraint to ensure the accuracy of the generated STF spectrum. When implementing model training, the discriminator learns to identify the ideal STF and guides the generator to produce a TF spectrum closer to the ideal one. After model training, we applied the model to synthetic and field data to demonstrate its effectiveness and stability in characterizing the TF features of seismic data. Our results find that STFR-GAN can effectively provide TF representations with higher readability than those of traditional TF methods, further benefiting fluvial channel delineation.

Comparison of ARIMA and LSTM in Predicting Structural Deformation of Tunnels during Operation Period

Accurately predicting the structural deformation trend of tunnels during operation is significant to improve the scientificity of tunnel safety maintenance. With the development of data science, structural deformation prediction methods based on time-series data have attracted attention. Auto Regressive Integrated Moving Average model (ARIMA) is a classical statistical analysis model, which is suitable for processing non-stationary time-series data. Long- and Short-Term Memory (LSTM) is a special cyclic neural network that can learn long-term dependent information in time series. Both are widely used in the field of temporal prediction. In view of the lack of time-series prediction in the tunnel deformation field, the body of this paper uses historical data of the Xinjian Road and the Dalian Road tunnel in Shanghai to propose a new way of modeling based on single points and road sections. ARIMA and LSTM models are applied in comprehensive experiments, and the results show that: (1) Both LSTM and ARIMA models have great performance for settlement and convergence deformation. (2) The overall robustness of ARIMA is better than that of LSTM, and it is more adaptable to the datasets. (3) The model prediction performance is closely related to the data quality. ARIMA has more stable performance under the lack of data volume, while LSTM has better performance with high-quality data and higher upper limit.

Non-spectral linear depth inversion using drone-acquired wave field imagery

We present an efficient approach for video-based bathymetry survey. The proposed algorithm consists of preprocessing for drone-acquired video, extracting incident gravity wave, estimating celerity vector field, operating linear water depth inversion, and mapping bathymetry. The image preprocessing includes resizing, masking, barrel distortion correction, ortho-rectification, and georeferencing. The challenges posed by noisy, non-stationary, and irregular wave field data was resolved using ensemble empirical mode decomposition (EEMD), which is capable of extracting incident gravity wave structure without arbitrary initial guessing of frequencies. The extracted wave structures were used to estimate their celerity vector field using large-scale particle image velocimetry (LSPIV) technique. Given that wave celerity depends on water depth in nearshore based on linear dispersion relation, spatial distribution of water depths was determined from wave celerity estimates. The proposed algorithm was tested using the wave field videos and ground truth data, collected during the field experiment at Jangsa beach, located in the east coast of South Korea, and good agreement was achieved.

Non-Gaussian feature distribution forecasting based on ConvLSTM neural network and its application to robust machine condition prognosis

The prognosis of a machine condition becomes a hot topic nowadays, as the condition monitoring installations provide a massive amount of Health Index (HI) data that could be used for machine remaining lifetime prognosis. However, existing methodologies might not be effective since in many cases the real HI datasets exhibit non-Gaussian characteristics. Thus, to improve efficiency, there is a need to introduce novel approaches which take into account the possible non-Gaussian distribution of HI data. In the proposed methodology, several innovative components are considered to form an efficient solution for the HI time series prediction. As HI is a mixture of trend and random non-Gaussian, non-homogeneous noise, we propose to forecast the HI distribution rather than the direct HI value. The Skewed Generalized t (SGT) distribution is considered as the general non-Gaussian model. Moreover, the combination of convolutional neural network (CNN) and long short-term memory (LSTM) are used to predict SGT distribution parameters. By incorporating the standardization module and the modified activation function alongside a custom time series standardization method, the architecture of the ConvLSTM neural network is created to be appropriate for non-stationary time series data with non-Gaussian behavior. Finally, a pre-training of the model has been proposed, which improves the efficiency of the designed procedure, as usually the amount of HI data is limited and insufficient to reliably train an artificial intelligence (AI) system. The proposed solution is shown to be robust and more resilient to outliers than the classical Gaussian-based approach. Using simulated and real HI data (known as a benchmark), we may conclude that the superiority of our method increases with the increasing non-Gaussianity level of the analyzed time series.

Gaussian Process Regression and Cooperation Search Algorithm for Forecasting Nonstationary Runoff Time Series

In the hydrology field, hydrological forecasting is regarded as one of the most challenging engineering tasks, as runoff has significant spatial–temporal variability under the influences of multiple physical factors from both climate events and human activities. As a well-known artificial intelligence tool, Gaussian process regression (GPR) possesses satisfying generalization performance but often suffers from local convergence and sensitivity to initial conditions in practice. To enhance its performance, this paper investigates the effectiveness of a hybrid GPR and cooperation search algorithm (CSA) model for forecasting nonstationary hydrological data series. The CSA approach avoids the premature convergence defect in GPR by effectively determining suitable parameter combinations in the problem space. Several traditional machine learning models are established to evaluate the validity of the proposed GPR-CSA method in three real-world hydrological stations of China. In the modeling process, statistical characteristics and expert knowledge are used to select input variables from the observed runoff data at previous periods. Different experimental results show that the developed GPR-CSA model can accurately predict nonlinear runoff and outperforms the developed traditional models in terms of various statistical indicators. Hence, a CSA-trained GPR model can provide satisfying training efficiency and robust simulation performance for runoff forecasting.

Type-2 evolving fuzzy Kalman filter design based on unobservable spectral components space for interval tracking of non-stationary experimental data

In this paper, an interval type-2 evolving fuzzy Kalman filter is designed for processing of unobservable spectral components of uncertain experimental data. The adopted methodology consider the following steps: an initial model of the interval type-2 fuzzy Kalman filter, which is off-line identified from an initial window of the experimental data; the updating of antecedent proposition of interval type-2 fuzzy Kalman filter by using an interval type-2 formulation of evolving Takagi-Sugeno (eTS) clustering algorithm and the updating of consequent proposition by using a type-2 fuzzy formulation of Observer/Kalman Filter Identification (OKID) algorithm, taking into account the multivariable recursive Singular Spectral Analysis of the experimental data. The computational results for tracking the Mackey-Glass chaotic time series illustrate the efficiency of proposed methodology as compared to relevant approaches from literature, and the experimental results for tracking a 2DoF helicopter demonstrate its applicability.

Koopman-inspired approach for identification of exogenous anomalies in nonstationary time-series data

In many scenarios, it is necessary to monitor a complex system via a time-series of observations and determine when anomalous exogenous events have occurred so that relevant actions can be taken. Determining whether current observations are abnormal is challenging. It requires learning an extrapolative probabilistic model of the dynamics from historical data, and using a limited number of current observations to make a classification. We leverage recent advances in long-term probabilistic forecasting, namely Deep Probabilistic Koopman, to build a general method for classifying anomalies in multi-dimensional time-series data. We also show how to utilize models with domain knowledge of the dynamics to reduce type I and type II error. We demonstrate our proposed method on the important real-world task of global atmospheric pollution monitoring, integrating it with NASA’s Global Earth Observing System Model. The system successfully detects localized anomalies in air quality due to events such as COVID-19 lockdowns and wildfires.

A Survey on Deep Learning Based Crop Yield Prediction

Agriculture is the most important sector and the backbone of a developing country’s economy. Accurate crop yield prediction models can provide decision-making tools for farmers to make better decisions. Crop yield prediction has challenged researchers due to dynamic, noisy, non-stationary, non-linear features and complex data. The factors that influence crop yield are changes in temperature and rainfall, plant disease, pests, fertilizer, and soil quality. The paper discusses the factors affecting crop yield, explores the features utilized, and analysis deep learning methodologies and performance metrics utilized in crop yield prediction.

A Complex Empirical Mode Decomposition for Multivariant Traffic Time Series

Data-driven modeling methods have been widely used in many applications or studies of traffic systems with complexity and chaos. The empirical mode decomposition (EMD) family provides a lightweight analytical method for non-stationary and non-linear data. However, a large amount of traffic data in practice are usually multidimensional, so the EMD family cannot be used directly for those data. In this paper, a method to calculate the extremum point and the envelope-like function (series) from the complex function (series) is proposed so that the EMD family can be applied to two-variate traffic time-series data. Compared to the existing multivariate EMD, the proposed method has advantages in computational burden, flexibility and adaptivity. Two-dimensional trajectory data were used to test the method and its oscillatory characteristics were extracted. The decomposed feature can be used for data-driven traffic analysis and modeling. The proposed method also extends the utilization of EMD to multivariate traffic data for applications such as traffic data denoising, pattern recognition, traffic flow dynamic evaluation, traffic prediction, etc.

지역화경제와 도시화경제가 지역 산업의 생산성에 미치는 영향

Purpose: Based on Japan’s regional-industries data from 1994 to 2018, we intend to analyze and present Japanese evidence on the productivity-enhancing effects of localization economies(MAR externality) and urbanization economies(Jacobs externality), and to accumulate research evidence for comparative analysis between Korea and Japan on the effect of agglomeration economies on productivity.
 Research design, data, and methodology: In order to empirically analyze the impact of localization economies and urbanization economies on the labor productivity, RIETI’s R-JIP database, which provides high-quality data on 31 industries in 47 prefectures from 1994 to 2018, was used. We implement a panel co-integration test for non-stationary panel data, and estimate panel co-integration vectors.
 Results: As a result of the panel cointegration test, labor productivity by region in Japan has a long-term equilibrium relationship with the industrial specialization index and industrial diversity index. In the estimation result of the panel cointegration vector, In the case of manufacturing, localization economy improves labor productivity, but urbanization economies does not affect labor productivity. On the other hand, in the case of IT manufacturing, both localization and urbanization economies have a significant impact on improving labor productivity.
 Implications: It is necessary to strengthen manufacturing specialization (localization economies) in order to improve the productivity of regional manufacturing. In addition, in order to improve the productivity of the IT manufacturing, a policy to strengthen the localization economies is important, and in addition, a policy to expand the diversity of related industries.

Mental Pressure Recognition Method Based on CNN Model and EEG Signal under Cross Session

There is an important application value in assessing an operator’s mental pressure (MP) level in human–computer cooperative tasks through continuous asymmetric electroencephalogram (EEG) signals, which can help predict hidden risks. Due to the different distributions of EEG features in different periods, it is particularly challenging to accurately identify brain states by training and testing asymmetric EEG signals with static pattern classifiers. Due to the limitations of non-stationary neurophysiological data capture technology, cross-session MP recognition schemes can only be used as an auxiliary means in practical applications. Deep learning methods can achieve stable feature extraction at a high level. Based on this advantage, this paper proposes a triplet loss (TL)-based CNN model that can automatically update the weights of shallow hidden neurons in cross-session MP classification tasks. Firstly, the generalization ability of the CNN model under both intra-session and cross-session conditions is evaluated. Moreover, the proposed model is compared with the existing MP classifier under different feature selection and noise destruction modes. According to the results, our TL-based CNN model has high performance in processing cross-session EEG features.

Towards high-precision data modeling of SHM measurements using an improved sparse Bayesian learning scheme with strong generalization ability

Central to structural health monitoring (SHM) is data modeling, manipulation, and interpretation on the basis of a sophisticated SHM system. Despite continuous evolution of SHM technology, the precise modeling and forecasting of SHM measurements under various uncertainties to extract structural condition-relevant knowledge remains a challenge. Aiming to resolve this problem, a novel application of a fully probabilistic and high-precision data modeling method was proposed in the context of an improved Sparse Bayesian Learning (iSBL) scheme. The proposed iSBL data modeling framework features the following merits. It can remove the need to specify the number of terms in the data-fitting function, and automatize sparsity of the Bayesian model based on the features of SHM monitoring data, which will enhance the generalization ability and then improve the data prediction accuracy. Embedded in a Bayesian framework which exhibits built-in protection against over-fitting problems, the proposed iSBL scheme has high robustness to data noise, especially for data forecasting. The model is verified to be effective on SHM vibration field monitoring data collected from a real-world large-scale cable-stayed bridge. The recorded acceleration data with two different vibration patterns, that is, stationary ambient vibration data and non-stationary decay vibration data, are investigated, returning accurate probabilistic predictions in both the time and frequency domains.

Causality Analysis with Information Geometry: A Comparison.

The quantification of causality is vital for understanding various important phenomena in nature and laboratories, such as brain networks, environmental dynamics, and pathologies. The two most widely used methods for measuring causality are Granger Causality (GC) and Transfer Entropy (TE), which rely on measuring the improvement in the prediction of one process based on the knowledge of another process at an earlier time. However, they have their own limitations, e.g., in applications to nonlinear, non-stationary data, or non-parametric models. In this study, we propose an alternative approach to quantify causality through information geometry that overcomes such limitations. Specifically, based on the information rate that measures the rate of change of the time-dependent distribution, we develop a model-free approach called information rate causality that captures the occurrence of the causality based on the change in the distribution of one process caused by another. This measurement is suitable for analyzing numerically generated non-stationary, nonlinear data. The latter are generated by simulating different types of discrete autoregressive models which contain linear and nonlinear interactions in unidirectional and bidirectional time-series signals. Our results show that information rate causalitycan capture the coupling of both linear and nonlinear data better than GC and TE in the several examples explored in the paper.

MCSTNet: a memory-contextual spatiotemporal transfer network for prediction of SST sequences and fronts with remote sensing data

Ocean fronts are a response to the variabilities of marine hydrographic elements and are an important mesoscale ocean phenomenon, playing a significant role in fish farming and fishing, sea-air exchange, marine environmental protection, etc. The horizontal gradients of sea surface temperature (SST) are frequently applied to reveal ocean fronts. Up to now, existing spatiotemporal prediction approaches have suffered from low prediction precision and poor prediction quality for non-stationary data, particularly for long-term prediction. It is a challenging task for medium- and long-term fine-grained prediction for SST sequences and fronts in oceanographic research. In this study, SST sequences and fronts are predicted for future variation trends based on continuous mean daily remote sensing satellite of SST data. To enhance the precision of the predicted SST sequences and fronts, this paper proposes a novel memory-contextual spatiotemporal transfer network (MCSTNet) for SST sequence and front predictions. MCSTNet involves three components: the encoder-decoder structure, a time transfer module, and a memory-contextual module. The encoder-decoder structure is used to extract the rich contextual and semantic information in SST sequences and frontal structures from the SST data. The time transfer module is applied to transfer temporal information and fuse low-level, fine-grained temporal information with high-level semantic information to improve medium- and long-term prediction precision. And the memory-contextual module is employed to fuse low-level, spatiotemporal information with high-level semantic information to enhance short-term prediction precision. In the training process, mean squared error (MSE) loss and contextual loss are combined to jointly guide the training of MCSTNet. Extensive experiments demonstrate that MCSTNet predicts more authentic and reasonable SST sequences and fronts than the state-of-the-art (SOTA) models on the SST data.

Economic growth and female labour force participation in an ageing society: evidence from Southeast Asia

ABSTRACT This study examines the impact of female labour participation and life expectancy on economic growth in Southeast Asia economies using non-stationary panel data techniques. The study utilizes balanced panel data spanning from 1992 to 2020, encompassing ten countries in Southeast Asia. The results reveal that economic growth, female labour participation, and other factors are cointegrated. The findings indicate that trade openness has a negative impact on economic growth. However, we find that female labour participation, female longevity, female education, domestic investment, and foreign direct investment have a positive long-term effect on economic growth. Our study also highlights that the effect of female labour participation on growth varies depending on female longevity. We find that female labour participation positively impacts economic growth but changes negatively when countries exhibit a high level of female life expectancy.

PENGARUH INSTRUMEN DERIVATIF MINYAK MENTAH, INDEKS DOLAR AS, INDEKS SAHAM UNGGULAN, SUKU BUNGA FED DAN INFLASI AS TERHADAP HARGA FUTURES EMAS: ANALISIS PERIODE 2012-2021

Gold futures have gained significant attention from investors in recent years, as demonstrated by the increasing trading volumes on the Jakarta Futures Exchange. Gold futures as a popular derivative instrument offers attractive returns and the potential to optimize portfolio allocation, making it one of the most preferred asset classes for investors. This study aims to analyze the influence of key factors such as the US dollar index, S&P 500 stock index, US inflation, FED funds rate, and crude oil futures on gold futures prices. The aim of the study is to provide investors, financial practitioners, and academics with a better understanding of the factors that can influence gold futures prices. This study utilized Error Correction Model as the research method, which effectively addresses the challenges associated with non-stationary time series data and enables the identification of short-term and long-term relationships among variables. The results show that the US dollar index has an inverse relationship with gold futures prices, influencing both the short and long-term. In the long run, an increase in the S&P 500 stock index and US inflation is anticipated to drive up gold futures prices. However, the impact of FED fund rates on gold futures prices is insignificant, primarily due to their low volatility. In addition, crude oil futures, as an alternative asset, have an adverse impact on gold futures prices over the long term.