Sequence Prediction Problem Research Articles

Interval grey number sequence prediction by using non-homogenous exponential discrete grey forecasting model

This paper aims to study a new grey prediction approach and its solution for forecasting the main system variable whose accurate value could not be collected while the potential value set could be defined. Based on the traditional non-homogenous discrete grey forecasting model (NDGM), the interval grey number and its algebra operations are redefined and combined with the NDGM model to construct a new interval grey number sequence prediction approach. The solving principle of the model is analyzed, the new accuracy evaluation indices, i.e. mean absolute percentage error of mean value sequence (MAPEM) and mean percent of interval sequence simulating value set covered (MPSVSC), are defined and, the procedure of the interval grey number sequence based the NDGM (IG-NDGM) is given out. Finally, a numerical case is used to test the modelling accuracy of the proposed model. Results show that the proposed approach could solve the interval grey number sequence prediction problem and it is much better than the traditional DGM(1,1) model and GM(1,1) model.

Analysis of Financial Time Sequence using Hidden Markov Models

Financial time sequence analysis has been a popular research topic in the field of finance, data science and machine learning. It is a highly challenging due to the extreme complexity within the sequences. Mostly existing models are failed to capture its intrinsic information, factor and tendency. To improve the previous approaches, in this paper, we propose a Hidden Markov Model (HMMs) based approach to analyze the financial time sequence. The fluctuation of financial time sequence was predicted through introducing a dual-state HMMs. Dual-state HMMs models the sequence and produces the features which will be delivered to SVMs for prediction. Note that we cast a financial time sequence prediction problem to a classification problem. To evaluate the proposed approach, we use Shanghai Composite Index as the dataset for empirically experiments. The dataset was collected from 550 consecutive trading days, and is randomly split to the training set and test set. The extensively experimental results show that: when analyzing financial time sequence, the mean-square error calculated with HMMs was obviously smaller error than the compared GARCH approach. Therefore, when using HMM to predict the fluctuation of financial time sequence, it achieves higher accuracy and exhibits several attractive advantageous over GARCH approach.

Asymptotic non-learnability of universal agents with computable horizon functions

Finding the universal artificial intelligent agent is the old dream of AI scientists. Solomonoff Induction was one big step towards this, giving a universal solution to the general problem of sequence prediction by defining a universal prior distribution. Hutter defined the AIXI model, which extends the latter to the reinforcement learning framework, where almost all if not all AI problems can be formulated. However, new difficulties arise because the agent is now active, whereas it is only passive in the sequence prediction case. This makes proving AIXI’s optimality difficult. In fact, we prove that the current definition of AIXI can sometimes be suboptimal in a certain sense, but that this behavior is still the most rational one, hence emphasizing the difficulty of universal reinforcement learning.

Boosting-based Sequential Output Prediction

Sequence prediction problem has been traditionally identified in the literature with sequence labeling. This approach typically corresponds to the classification of a label sequence associated to observed input sequence. However, another interpretation of sequence prediction may be considered where a label sequence (sequential output) is classified based only on the independent set of attributes. The paper presents a new, based on boosting, ensemble approach, performing such sequential output prediction. The sequential nature of the classified structure is reflected on the applied cost function. The experimental results reported in the paper revealed a high validity and competitiveness of the proposed approach.

Backbone Statistical Potential from Local Sequence-Structure Interactions in Protein Loops

Native proteins have been optimized by evolution simultaneously for structure and sequence. Structural databases reflect this interdependency. In this paper, we present a new statistical potential for a reduced backbone representation that has both structure and sequence characteristics as variables. We use information from structural data available in the Protein Coil Library, selected on the basis of resolution and refinement factor. In these structures, the nonlocal interactions are randomly distributed and, thus, average out in statistics, so structural propensities due to local backbone-based interactions can be studied separately. We collect data in the form of local sequence-specific phi-psi backbone dihedral pairs. From these data, we construct dihedral probability density functions (DPDFs) that quantify any adjacent phi-psi pair distribution in the context of all possible combinations of local residue types. We use a probabilistic analysis to deduce how the correlations encoded in the various DPDFs as well as in residue frequencies propagate along the sequence and can be cumulated in a statistical potential capable of efficiently scoring a loop by its backbone conformation and sequence only. Our potential is able to identify with high accuracy the native structure of a loop with a given sequence among possible alternative conformations from sets of well-constructed decoys. Conversely, the potential can also be used for sequence prediction problems and is shown to score the native sequence of a given loop structure among the most fit of the possible sequence combinations. Applications for both structure prediction and sequence design are discussed.

Individual Sequence Prediction Using Memory-Efficient Context Trees

Context trees are a popular and effective tool for tasks such as compression, sequential prediction, and language modeling. We present an algebraic perspective of context trees for the task of individual sequence prediction. Our approach stems from a generalization of the notion of margin used for linear predictors. By exporting the concept of margin to context trees, we are able to cast the individual sequence prediction problem as the task of finding a linear separator in a Hilbert space, and to apply techniques from machine learning and online optimization to this problem. Our main contribution is a memory efficient adaptation of the perceptron algorithm for individual sequence prediction. We name our algorithm the shallow perceptron and prove a shifting mistake bound, which relates its performance with the performance of any sequence of context trees. We also prove that the shallow perceptron grows a context tree at a rate that is upper bounded by its mistake rate, which imposes an upper bound on the size of the trees grown by our algorithm.

Predicting non-stationary processes

Suppose we are given two probability measures on the set of one-way infinite finite-alphabet sequences. Consider the question of when one of the measures predicts the other, that is, when conditional probabilities converge (in a certain sense), if one of the measures is chosen to generate the sequence. This question may be considered a refinement of the problem of sequence prediction in its most general formulation: for a given class of probability measures, does there exist a measure which predicts all of the measures in the class? To address this problem, we find some conditions on local absolute continuity which are sufficient for prediction and generalize several different notions that are known to be sufficient for prediction. We also formulate some open questions to outline a direction for finding the conditions on classes of measures for which prediction is possible.

Initial state randomness improves sequence learning in a model hippocampal network.

Randomness can be a useful component of computation. Using a computationally minimal, but still biologically based model of the hippocampus, we evaluate the effects of initial state randomization on learning a cognitive problem that requires this brain structure. Greater randomness of initial states leads to more robust performance in simulations of the cognitive task called transverse patterning, a context-dependent discrimination task that we code as a sequence prediction problem. At the conclusion of training, greater initial randomness during training trials also correlates with increased, repetitive firing of select individual neurons, previously named local context neurons. In essence, such repetitively firing neurons recognize subsequences, and previously their presence has been correlated with solving the transverse patterning problem. A more detailed analysis of the simulations across training trials reveals more about initial state randomization. The beneficial effects of initial state randomization derive from enhanced variation, across training trials, of the sequential states of a network. This greater variation is not uniformly present during training; it is largely restricted to the beginning of training and when novel sequences are introduced. Little such variation occurs after extensive or even moderate amounts of training. We explain why variation is high early in training, but not later. This automatic modulation of the initial-state-driven random variation through state space is reminiscent of simulated annealing where modulated randomization encourages a selectively broad search through state space. In contrast to an annealing schedule, the selective occurrence of such a random search here is an emergent property, and the critical randomization occurs during training rather than testing.

Entorhinal/dentate excitation of CA3: A critical variable in hippocampal models

We investigate how the strength of entorhinal cortical inputs during training affects learned performance using computer simulations of a minimal computational model of hippocampal region CA3. After the model learns two partially overlapping sequences, it is tested on two contradictory prediction problems — disambiguation and goal-finding. Relative to total activity, the activity level of entorhinal inputs during learning profoundly affects performance on each task. The optimal input levels differ for the two sequence prediction problems, but a small region of overlap exists where both tasks can usually be performed successfully. This sensitivity to relative input activity suggests critical tests of the model.

A neural network solution to the transverse patterning problem depends on repetition of the input code.

Using computer simulations, this paper investigates how input codes affect a minimal computational model of the hippocampal region CA3. Because encoding context seems to be a function of the hippocampus, we have studied problems that require learning context for their solution. Here we study a hippocampally dependent, configural learning problem called transverse patterning. Previously, we showed that the network does not produce long local context codings when the sequential input patterns are orthogonal, and it fails to solve many context-dependent problems in such situations. Here we show that this need not be the case if we assume that the input changes more slowly than a processing interval. Stuttering, i.e., repeating inputs, allows the network to create long local context firings even for orthogonal inputs. With these long local context firings, the network is able to solve the transverse patterning problem. Without stuttering, transverse patterning is not learned. Because stuttering is so useful, we investigate the relationship between the stuttering repetition length and relative context length in a simple, idealized sequence prediction problem. The relative context length, defined as the average length of the local context codes divided by the stuttering length, interacts with activity levels and has an optimal stuttering repetition length. Moreover, the increase in average context length can reach this maximum without loss of relative capacity. Finally, we note that stuttering is an example of maintained or introduced redundancy that can improve neural computations.

Predicting the distribution of synaptic strengths and cell firing correlations in a self-organizing, sequence prediction model.

This article investigates the synaptic weight distribution of a self-supervised, sparse, and randomly connected recurrent network inspired by hippocampal region CA3. This network solves nontrivial sequence prediction problems by creating, on a neuron-by-neuron basis, special patterns of cell firing called local context units. These specialized patterns of cell firing--possibly an analog of hippocampal place cells--allow accurate prediction of the statistical distribution of synaptic weights, and this distribution is not at all gaussian. Aside from the majority of synapses that are, at least functionally, lost due to synaptic depression, the distribution is approximately uniform. Unexpectedly, this result is relatively independent of the input environment, and the uniform distribution of synaptic weights can be approximately parameterized based solely on the average activity level. Next, the results are generalized to other cell firing types (frequency codes and stochastic firing) and place cell-like firing distributions. Finally, we note that our predictions concerning the synaptic strength distribution can be extended to the distribution of correlated cell firings. Recent published neurophysiological results are consistent with this extension.

A sequence predicting CA3 is a flexible associator that learns and uses context to solve hippocampal-like tasks.

The model discussed in this paper is, by hypothesis, a minimal, biologically plausible model of hippocampal region CA3. Because cognitive mapping can be viewed as a sequence prediction problem, we qualify this model as a successful sequence predictor. Since the model solves problems which require the use of context, the model is also able to learn and use context. The model also solves configural learning problems of which, at least one, requires a hippocampus. Thus, by solving sequence problems, by solving configural learning problems, and by creating codes for context, this model provides a computational unification of hippocampal functions which are often viewed as disparate.