Long Memory Research Articles

ON THE CONSISTENCY OF THE LEAST SQUARES ESTIMATOR IN MODELS SAMPLED AT RANDOM TIMES DRIVEN BY LONG MEMORY NOISE: THE RENEWAL CASE

In this study, we prove the strong consistency of the least squares estimator in a random sampled linear regression model with long-memory noise and an independent set of random times given by renewal process sampling. Additionally, we illustrate how to work with a random number of observations up to time T = 1. A simulation study is provided to illustrate the behavior of the different terms, as well as the performance of the estimator under various values of the Hurst parameter H.

On the Convergence of Orthogonal/Vector AMP: Long-Memory Message-Passing Strategy

Orthogonal/vector approximate message-passing (AMP) is a powerful message-passing (MP) algorithm for signal reconstruction in compressed sensing. This paper proves the convergence of Bayes-optimal orthogonal/vector AMP in the large system limit. The proof strategy is based on a novel long-memory (LM) MP approach: A first step is a construction of LM-MP that is guaranteed to converge systematically. A second step is a large-system analysis of LM-MP via an existing framework of state evolution. A third step is to prove the convergence of state evolution recursions for Bayes-optimal LM-MP via a new statistical interpretation of existing LM damping. The last is an exact reduction of the state evolution recursions for Bayes-optimal LM-MP to those for Bayes-optimal orthogonal/vector AMP. The convergence of the state evolution recursions for Bayes-optimal LM-MP implies that for Bayes-optimal orthogonal/vector AMP. Numerical simulations are presented to show the verification of state evolution results for damped orthogonal/vector AMP and a negative aspect of LM-MP in finite-sized systems.

BAYESIAN SEMIPARAMETRIC LONG MEMORY MODELS FOR DISCRETIZED EVENT DATA.

We introduce a new class of semiparametric latent variable models for long memory discretized event data. The proposed methodology is motivated by a study of bird vocalizations in the Amazon rain forest; the timings of vocalizations exhibit self-similarity and long range dependence. This rules out Poisson process based models where the rate function itself is not long range dependent. The proposed class of FRActional Probit (FRAP) models is based on thresholding, a latent process. This latent process is modeled by a smooth Gaussian process and a fractional Brownian motion by assuming an additive structure. We develop a Bayesian approach to inference using Markov chain Monte Carlo and show good performance in simulation studies. Applying the methods to the Amazon bird vocalization data, we find substantial evidence for self-similarity and non-Markovian/Poisson dynamics. To accommodate the bird vocalization data in which there are many different species of birds exhibiting their own vocalization dynamics, a hierarchical expansion of FRAP is provided in the Supplementary Material.

A novel hybrid ARFURIMA-APARCH model for modeling interminable long memory and asymmetric effect in time series

In this paper, we introduce a new hybrid model namely Autoregressive Fractional Unit Root Integrated Moving Average-Asymmetric Power Autoregressive Conditional Heteroscedasticity (ARFURIMA-APARCH) model. The Nigeria daily COVID19 records and Bitcoin to EURO exchange rate that exhibit a type of Long Memory (LM) called Interminable LM (ILM), volatility and asymmetric (leverage) effect were used to show the applications of the proposed ARFURIMA-APARCH model. The existing Autoregressive Fractional Integrated Moving Average-Asymmetric Power Autoregressive Conditional Heteroscedasticity (ARFIMA-APARCH) model were estimated and compared with the ARFURIMA-APARCH model. Results showed that the new hybrid model is better based on goodness-of-fit, serial correlation tests and forecast measures of accuracy. As a conclusion, our study showed that the ARFURIMA-APARCH model performed better compared to the ARFIMA-APARCH hybrid model. Therefore, the ARFURIMA-APARCH model is a better option for modeling ILM, volatility and leverage effect of health and financial data. Future study should focus on the application of the developed hybrid ARFURIMA-APARCH model using some major economic indicators, for example, Gross Domestic Product (GDP), currency exchange rate, stock price index, interest rate and other financial data.

Bootstrap long memory processes in the frequency domain

The aim of the paper is to describe a bootstrap, contrary to the sieve bootstrap, valid under either long memory (LM) or short memory (SM) dependence. One of the reasons of the failure of the sieve bootstrap in our context is that under LM dependence, the sieve bootstrap may not be able to capture the true covariance structure of the original data. We also describe and examine the validity of the bootstrap scheme for the least squares estimator of the parameter in a regression model and for model specification. The motivation for the latter example comes from the observation that the asymptotic distribution of the test is intractable.

On the Evaluation and Implementation of LSTM Model for Speech Emotion Recognition using MFCC

Speech Emotion Recognition is an emerging research field and is expected to benefit many application domains by providing effective Human Computer Interface. Researchers are extensively working towards decoding of human emotions through speech signal in order to achieve effective interface and smart response by computers. The perfection of speech emotion recognition greatly depends upon the types of features used and also on the classifier employed for recognition. The contribution of this paper is to evaluate twelve different Long Short Term Memory (LSTM) networks models as classifier based on Mel-Frequency Cepstrum Coefficients (MFCC) feature. The paper presents performance evaluation in terms of important parameters such as: precision, recall, F-measure and accuracy for four emotions like happy, neutral, sad and angry using the emotional speech databases namely Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS). The measurement accuracy obtained is 89% which is 9.5% more than reported in recent literature. The suitable LSTM model is further successfully implemented on Raspberry PI board creating standalone Speech Emotion Recognition system.

Bridge weigh-in-motion through bidirectional Recurrent Neural Network with long short-term memory and attention mechanism

In bridge weigh-in-motion (BWIM), dynamic bridge response is measured during traffic and used to identify overloaded vehicles. Most past studies of BWIM use mechanics-based algorithms to estimate axle weights. This research instead investigates deep learning, specifically the recurrent neural network (RNN), toward BWIM. In order to acquire the large data volume to train a RNN network that uses bridge response to estimate axle weights, a finite element bridge model is built through the commercial software package LS-DYNA. To mimic everyday traffic scenarios, tens of thousands of randomized vehicle formations are simulated, with different combinations of vehicle types, spacings, speeds, axle weights, axle distances, etc. Dynamic response from each of the randomized traffic scenarios is recorded for training the RNN. In this paper we propose a 3-stage Bidirectional RNN toward BWIM. Long short-term memory (LSTM) and attention mechanism are embedded in the BRNN to further improve the network performance. Additional test data indicates that the BRNN network achieves high accuracy in estimating axle weights, in comparison with a conventional moving force identification (MFI) method.

Convolution Recurrent Neural Network for Daily Forecast of PM10 Concentrations in Brunei Darussalam

PM10 is a particulate matter with an aerodynamic diameter less than or equal to 10. It is one of the primary pollutants contributing to the ambient air quality level. Air quality monitoring in Brunei Darussalam is using only the PM10 concentrations to measure the nation's daily Pollutant Standard Index (PSI). This study sheds light on a data centric landscape of air pollution prediction in Brunei Darussalam, highlights potential uses of forecasting daily PM10concentrations, and presents comparisons of prediction models built using several methods, namely: moving average, linear regression, recurrent neural network (RNN), long short term memory (LSTM), LSTM with 1D convolutions, and convolutional recurrent neural network (CRNN). This study is using daily PM10 concentrations obtained from the air quality monitoring stations located at every district in Brunei Darussalam for a period of 15 y (2005-2019).

Flood prediction in the Namgang Dam basin using a long short-term memory (LSTM) algorithm

Flood prediction in the Namgang Dam basin using a long short-term memory (LSTM) algorithm

Investing in mutual funds: are you paying for performance or for the ties of the manager?

This study analyses the performance of US Mutual Funds, from the perspective of Long Memory (LM), exploring if the returns of MFs are systematic due to their active management or they are random. The sample was 200 US equity MFs, from four categories, Large Cap, Middle Cap, Small Cap and World Stock, both 1- and 5-stars rating funds according to Morning Star rating. The time period was starting between 1981 and 2006 and ending 2016. Rescaled Range Analysis (R/S) employed for the Hurst exponent estimation, so to detect LM. Using Surrogate Data Analysis (SDA), the study was extended to Hurst exponent estimation for surrogate time series. The findings suggest that the selection of a MF presents a lot of complexity for investors. The 5-star MFs, with high qualified, and so expensive managers, tend to achieve random returns, while the returns of 1-star MFs, are more systematic. These MFs have higher fees than the 5-star MFs, but the management fees paid are quite inferior. This leads to the conclusion, that it might be preferable to pay for gaining an almost the same, but systematic return than to pay for the ties of the manager.

Using Long Short-Term Memory (LSTM) Neural Networks to Predict Emergency Department Wait Time.

Emergency Department (ED) overcrowding is a major global healthcare issue. Many research studies have been conducted to predict ED wait time using various machine learning prediction models to enhance patient experience and improve care efficiency and resource allocation. In this paper, we used Long Short-Term Memory (LSTM) recurrent neural networks to build a model to predict ED wait time in the next 2 hours using a randomly generated patient timestamp dataset of a typical patient hospital journey. Compared with Linear Regression model, the average mean absolute error for the LSTM model is decreased by 9.7% (3 minutes) (p < 0.01). The LSTM model statistically outperforms the LR model, however, both models could be practically useful in ED wait time prediction.

Using Long Short-Term Memory (LSTM) Neural Networks to Predict Emergency Department Wait Time.

Emergency Department (ED) overcrowding is a major global healthcare issue. In this paper, we used Long Short-Term Memory (LSTM) recurrent neural networks to build a model to predict ED wait time in the next 2 hours using a randomly generated patient timestamp dataset of a typical patient hospital journey. Compared with Linear Regression model, the average mean absolute error for the LSTM model is decreased by 15% (3 minutes) (p<0.001). The LSTM model statistically outperforms the LR model, however, both models could be practically useful in ED wait time prediction.

Character-Level Neural Language Modelling in the Clinical Domain.

Word embeddings have become the predominant representation scheme on a token-level for various clinical natural language processing (NLP) tasks. More recently, character-level neural language models, exploiting recurrent neural networks, have again received attention, because they achieved similar performance against various NLP benchmarks. We investigated to what extent character-based language models can be applied to the clinical domain and whether they are able to capture reasonable lexical semantics using this maximally fine-grained representation scheme. We trained a long short-term memory network on an excerpt from a table of de-identified 50-character long problem list entries in German, each of which assigned to an ICD-10 code. We modelled the task as a time series of one-hot encoded single character inputs. After the training phase we accessed the top 10 most similar character-induced word embeddings related to a clinical concept via a nearest neighbour search and evaluated the expected interconnected semantics. Results showed that traceable semantics were captured on a syntactic level above single characters, addressing the idiosyncratic nature of clinical language. The results support recent work on general language modelling that raised the question whether token-based representation schemes are still necessary for specific NLP tasks.

Possibility analysisof future droughts using long short term memory and standardized groundwater level index

Possibility analysisof future droughts using long short term memory and standardized groundwater level index

Prediction of Baltic Dry Index by Applications of Long Short-Term Memory

Prediction of Baltic Dry Index by Applications of Long Short-Term Memory

Entwicklung und Evaluation eines Wiegesystems für Forstkräne auf Basis von künstlichen neuronalen Netzen

Sowohl in der Hackschnitzel- als auch Rundholzlogistik fehlen oft wichtige Referenzmengen fur den dispositiven Einsatz, welche zumeist mit unzureichend genauen Schatzwerten vervollstandigt werden. Um die Informationsqualitat uber die bewegten Holzmengen zu verbessern, konnen optional Kranwaagen zwischen Teleskopauge und Greifer am Kran des Forwarders montiert werden. Dies beeinflusst jedoch die Krankinematik und Manovrierfahigkeit negativ, bei zeitgleich sinkender Maschinenproduktivitat aufgrund einer zur Messung notwendigen Unterbrechung des Ladeprozesses. In diesem Beitrag wird eine datenbasierte Methode prasentiert, welche einen dynamischen Wiegeprozess im kontinuierlichen Ladevorgang bei modernen Forstkranen ermoglicht, ohne dass eine zusatzliche Hardware-Komponente auf der Maschine verbaut werden muss. Bei dieser Methode wird ein Ladezyklus automatisiert erkannt und die geladene Masse anhand eines kunstlichen neuronalen Netzwerkes (KNN) abgeschatzt. Als Eingangsgrosen dienen Signale von Sensoren, die auf auf modernen Forwardern serienmasig installiert sind. Fur die Verarbeitung von zeitbasierten Sensordaten in einem neuronalen Netz hat sich die Long-Short-Term-Memory(LSTM)-Architektur bewahrt. Basierend auf LSTM-Zellen wurde ein entsprechendes Netzwerk entworfen, trainiert und anschliesend optimiert. Im Test zeigt sich fur einen einzelnen Ladezyklus ein mittlerer Gesamtskalenfehler von 1,5 %, bezogen auf 1.000 kg. Fur einen voll beladenen Forwarder resultiert dies in einem Gesamtmassenfehler kleiner 1,2 %.

Named Entity Recognition in Chinese Electronic Medical Records Based on the Model of Bidirectional Long Short-Term Memory with a Conditional Random Field Layer.

Named entity recognition in electronic medical records is of great significance to the construction of medical knowledge maps. This paper proposes a model of bidirectional Long Short-Term Memory with a conditional random field layer(BiLSTM-CRF). In terms of simultaneously identifying 5 types of clinical entities from CCKS2018 Chinese EHRs corpus, the BiLSTM-CRF model finally achieved better performance than the baseline CRF model (F-score of 84.23% vs 82.49%).

An Empirical Test of GRUs and Deep Contextualized Word Representations on De-Identification.

De-identification aims to remove 18 categories of protected health information from electronic health records. Ideally, de-identification systems should be reliable and generalizable. Previous research has focused on improving performance but has not examined generalizability. This paper investigates both performance and generalizability. To improve current state-of-the-art performance based on long short-term memory (LSTM) units, we introduce a system that uses gated recurrent units (GRUs) and deep contextualized word representations, both of which have never been applied to de-identification. We measure performance and generalizability of each system using the 2014 i2b2/UTHealth and 2016 CEGS N-GRID de-identification datasets. We show that deep contextualized word representations improve state-of-the-art performance, while the benefit of switching LSTM units with GRUs is not significant. The generalizability of de-identification system significantly improved with deep contextualized word representations; in addition, LSTM units-based system is more generalizable than the GRUs-based system.

Recurrent Deep Network Models for Clinical NLP Tasks: Use Case with Sentence Boundary Disambiguation.

Although a number of foundational natural language processing (NLP) tasks like text segmentation are considered a simple problem in the general English domain dominated by well-formed text, complexities of clinical documentation lead to poor performance of existing solutions designed for the general English domain. We present an alternative solution that relies on a convolutional neural network layer followed by a bidirectional long short-term memory layer (CNN-Bi-LSTM) for the task of sentence boundary disambiguation and describe an ensemble approach for domain adaptation using two training corpora. Implementations using the Keras neural-networks API are available at https://github.com/NLPIE/clinical-sentences.

Diabetic retinopathy fundus image generation based on generative adversarial networks

Objective To generate various types of diabetic retinopathy (DR) fundus images automatically by computer vision algorithm. Methods A method based on deep learning to generate fundus images was proposed, which used the vascular vein of the fundus image and the text description of lesions as the constraint conditions to generate fundus image.The text description was encoded by using a long short-term memory (LSTM), and the vascular vein image was encoded by a convolutional neural network (CNN). Then the encoded information was combined and used to generate a fundus image by generative adversarial networks (GAN). Results The results showed that the algorithm can generate realistic fundus images.However, the image detail features were not obvious because the text-encoded recurrent neural network (RNN) loss function did not converge well. Conclusions Using the GAN can generate realistic DR fundus images, which has certain application value in expanding medical data.However, the generation of detail features in small areas still needs improvement. Key words: Generative adversarial networks; Fundus images; Image generation; Convolutional neural network