Sequence Of Items Research Articles

IconShop: Text-Guided Vector Icon Synthesis with Autoregressive Transformers

Scalable Vector Graphics (SVG) is a popular vector image format that offers good support for interactivity and animation. Despite its appealing characteristics, creating custom SVG content can be challenging for users due to the steep learning curve required to understand SVG grammars or get familiar with professional editing software. Recent advancements in text-to-image generation have inspired researchers to explore vector graphics synthesis using either image-based methods (i.e., text → raster image → vector graphics) combining text-to-image generation models with image vectorization, or language-based methods (i.e., text → vector graphics script) through pretrained large language models. Nevertheless, these methods suffer from limitations in terms of generation quality, diversity, and flexibility. In this paper, we introduce IconShop, a text-guided vector icon synthesis method using autoregressive transformers. The key to success of our approach is to sequentialize and tokenize SVG paths (and textual descriptions as guidance) into a uniquely decodable token sequence. With that, we are able to exploit the sequence learning power of autoregressive transformers, while enabling both unconditional and text-conditioned icon synthesis. Through standard training to predict the next token on a large-scale vector icon dataset accompanied by textural descriptions, the proposed IconShop consistently exhibits better icon synthesis capability than existing image-based and language-based methods both quantitatively (using the FID and CLIP scores) and qualitatively (through formal subjective user studies). Meanwhile, we observe a dramatic improvement in generation diversity, which is validated by the objective Uniqueness and Novelty measures. More importantly, we demonstrate the flexibility of IconShop with multiple novel icon synthesis tasks, including icon editing, icon interpolation, icon semantic combination, and icon design auto-suggestion.

Convolutional transformer network for fine-grained action recognition

Fine-grained action recognition is one of the critical problems in video processing, which aims to recognize similar actions of subtle interactions between humans and objects. Inspired by the remarkable performance of the Transformer in natural language processing, Transformer has been applied to the fine-grained action recognition task. However, Transformer needs abundant training data and extra supervision to achieve comparable results with convolutional neural networks (CNNs). To address these issues, we propose a Convolutional Transformer Network (CTN), which integrates the merits of CNN (e.g., sharing weights, capturing low-level features in videos and locality) and the benefits of Transformer (e.g., dynamic attention and learning long-range dependencies). In this paper, we propose two modifications to the original Transformer: (i) We propose a video-to-tokens module that can extract tokens from extracted spatial-temporal features in videos by 3D convolutions instead of the direct token embedding from raw input video clips; (ii) We completely replace the linear mapping in multi-head self-attention layer with depth-wise convolutional mapping, which applies a depth-wise separable convolution operation on embedded token maps. With these two modifications, our approach can extract effective spatial-temporal features from videos and process the long sequences of tokens encountered in videos. Experimental results demonstrate that our proposed CTN can achieve state-of-the-art accuracy on two fine-grained action recognition datasets (i.e., Epic-Kitchens and Diving 48) with a small computational increase.

A multi-range spectral-spatial transformer for hyperspectral image classification

In recent years, the transformer-based approach becoming a hot research topic in hyperspectral image (HSI) classification tasks. However, most of these studies have focused on optimizing the model framework in pursuit of high-accuracy classification, with little attention to the composition of the input token sequence as an important factor affecting the performance of the transformer. Therefore, this paper further explores a novel token structure to strengthen the Transformer's performance for HSI classification tasks, based on which a Multi-Range Spectral-Spatial Transformer (MRSST) framework is developed. Specifically, a convolutional feature pre-encoder with two branches is designed to extract shallow features for each spectral channel separately. Then, a token generator is introduced to combine the shallow features with the raw spectral information to yield the token sequences with multi-range information. Finally, the tokens are input into the transformer encoders enhanced by a module that strengthens the information exchange between its mid-range and short-range semantic features. Experiments conducted on three well-known hyperspectral datasets demonstrate that the proposed multi-range composite token sequences and information exchange mechanisms significantly enhance the transformer's performance. Codes are released at: https://github.com/HyperSystemAndImageProc/Multi-Range-Spectral-Spatial-Transformer.

Protocol for the development and validation of a patient-reported experience measure (PREM) for people with hearing loss: the PREM-HeLP

IntroductionHearing loss is a common chronic health condition and adversely affects communication and social function resulting in loneliness, social isolation and depression. We know little about the patient experience of...

Global and local neural cognitive modeling for student performance prediction

Student performance prediction is a critical task in supporting decision-making for Intelligent Tutoring Systems (ITS). Correct predictions of student performance are prerequisites for ITS to supply intelligent services and optimize learning efficiency, e.g., recommending the most appropriate learning resources for each student. Existing methods mainly include cognitive diagnosis and knowledge tracing, both of which focus on students’ cognitive modeling based on their interactions on a sequence of items and give predictions by assessing if their cognitive states can meet the item requirements. Specifically, cognitive diagnosis only considers students’ global static cognitive states, while knowledge tracing focuses on students’ local dynamics. However, both global and local features are critical for predicting student performance. Therefore, in this paper, we propose a novel Global and Local Neural Cognitive (GLNC) model to capture both global and local features in student-item interactions for more accurate predictions. Specifically, we first learn students’ global cognitive level according to all student-item interactions. Then, we propose a self-attentive encoder based on the scaled dot-product attention mechanism to extract the local cognitive dynamics and the dependencies between students’ recent interactions. Finally, to obtain better predictions, we design a fused gate based on the similarity between students’ recently responded items and the item to be predicted to adaptively combine the global and local features. To evaluate the effectiveness of GLNC, we compare it with both cognitive diagnosis and knowledge tracing methods. All experiments are conducted on three public datasets that contain real student-item interactions on mathematics or language courses from various ITS. The experimental results demonstrate that GLNC achieves an average score of 0.7810 on the AUC metric, 0.7627 on the ACC metric, 0.3987 on the RMSE metric, 0.2023 on the r2 metric, respectively achieving an average improvement of 1.84%, 1.07%, 1.87%, and 11.38% in contrast to existing state-of-the-art methods (i.e., NCD and LPKT). Moreover, we further analyze the performance of GLNC under different probabilities of guessing and slipping, the results indicate that GLNC is more robust against the influence of noisy data while considering both global and local features. Benefiting from the superior accuracy and stability, our proposed GLNC has a wide range of potential implications for ITS, which can be easily applied to improve students’ learning efficiency and experience.

Diabetic Retinopathy Image Classification Using Shift Window Transformer

Diabetic retinopathy is one of the most dangerous complications for diabetic patients, leading to blindness if not diagnosed early. However, early diagnosis can control and prevent the disease from progressing to blindness. Transformers are considered state-of-the-art models in natural language processing that do not use convolutional layers. In transformers, means of multi-head attention mechanisms capture long-range contextual relations between pixels. For grading diabetic retinopathy, CNNs currently dominate deep learning solutions. However, the benefits of transformers, have led us to propose an appropriate transformer-based method to recognize diabetic retinopathy grades. A major objective of this research is to demonstrate that the pure attention mechanism can be used to determine diabetic retinopathy and that transformers can replace standard CNNs in identifying the degrees of diabetic retinopathy. In this study, a Swin Transformer-based technique for diagnosing diabetic retinopathy is presented by dividing fundus images into nonoverlapping batches, flattening them, and maintaining positional information using a linear and positional embedding procedure. Several multi-headed attention layers are fed into the resulting sequence to construct the final representation. In the classification step, the initial token sequence is passed into the SoftMax classification layer, which produces the recognition output. This work introduced the Swin transformer performance on the APTOS 2019 Kaggle for training and testing using fundus images of different resolutions and patches. The test accuracy, test loss, and test top 2 accuracies were 69.44%, 1.13, and 78.33%, respectively for 160*160 image size, patch size=2, and embedding dimension C=64. While the test accuracy was 68.85%, test loss: 1.12, and test top 2 accuracy: 79.96% when the patch size=4, and embedding dimension C=96. And when the size image is 224*224, patch size=2, and embedding dimension C=64, the test accuracy: 72.5%, test loss: 1.07, and test top 2 accuracy: 83.7%. When the patch size =4, embedding dimension C=96, the test accuracy was 74.51%, test loss: 1.02, and the test top 2 accuracy was 85.3%. The results showed that the Swin Transformer can achieve flexible memory savings. The proposed method highlights that an attention mechanism based on the Swin Transformer model is promising for the diabetic retinopathy grade recognition task.

SMiT: symmetric mask transformer for disease severity detection.

The application of deep learning methods to the intelligent diagnosis of diseases has been the focus of intelligent medical research. When dealing with image classification tasks, if the lesion area is small and uneven, the background image involved in the training will affect the ultimate accuracy in determining the extent of the lesion. We did not follow the traditional approach of building an intelligent system to assist physicians in diagnosis from the perspective of CNN models, but instead proposed a pure transformer framework that can be used for diagnostic grading of pathological images. We propose a Symmetric Mask Pre-Training vision Transformer SMiT model for grading pathological cancer images. SMiT performs a symmetrically identical high probability sparsification of the input image token sequence at the first and last encoder layer positions to pre-train visual transformers, and the parameters of the baseline model are fine-tuned after loading the pre-training weights, allowing the model to concentrate more on extracting detailed features in the lesion region, effectively getting rid of the potential feature dependency problem. SMiT achieved 92.8% classification accuracy on 4500 histopathological images of colorectal cancer processed by Gaussian filter denoising. We validated the effectiveness and generalizability of this study's methodology on the publicly available diabetic retinopathy dataset APTOS2019 from Kaggle and achieved quadratic Cohen Kappa, accuracy and F1-score of 91.9%, 86.91% and 72.85%, respectively, which were 1-2% better than previous studies based on CNN models. SMiT uses a simpler strategy to achieve better results to assist physicians in making accurate clinical decisions, which can be an inspiration for making good use of the visual transformers in the field of medical imaging.

User Cold-Start Learning In Recommender Systems Using Monte Carlo Tree Search

We consider the cold-start task for new users of a recommender system whereby a new user is asked to rate a few items with the aim of quickly discovering the user’s preferences. This is a combinatorial stochastic learning task, and so difficult in general. In this paper we study the use of Monte Carlo Tree Search (MCTS) to dynamically select the sequence of items presented to a new user. We find that the MCTS-based cold-start approach is able to consistently quickly identify the preferences of a user with significantly higher accuracy than with either a decision-tree or a state of the art bandit-based approach without incurring higher regret i.e the learning performance is fundamentally superior to that of the state of the art. This boost in recommender accuracy is achieved in a computationally lightweight fashion. The MCTS approach is flexible in the sense that it can readily extended to incorporate different types of user feedback including explicit ratings, ranked comparisons and missing not at random data.

MMNet: A Mixing Module Network for Polyp Segmentation.

Traditional encoder-decoder networks like U-Net have been extensively used for polyp segmentation. However, such networks have demonstrated limitations in explicitly modeling long-range dependencies. In such networks, local patterns are emphasized over the global context, as each convolutional kernel focuses on only a local subset of pixels in the entire image. Several recent transformer-based networks have been shown to overcome such limitations. Such networks encode long-range dependencies using self-attention methods and thus learn highly expressive representations. However, due to the computational complexity of modeling the whole image, self-attention is expensive to compute, as there is a quadratic increment in cost with the increase in pixels in the image. Thus, patch embedding has been utilized, which groups small regions of the image into single input features. Nevertheless, these transformers still lack inductive bias, even with the image as a 1D sequence of visual tokens. This results in the inability to generalize to local contexts due to limited low-level features. We introduce a hybrid transformer combined with a convolutional mixing network to overcome computational and long-range dependency issues. A pretrained transformer network is introduced as a feature-extracting encoder, and a mixing module network (MMNet) is introduced to capture the long-range dependencies with a reduced computational cost. Precisely, in the mixing module network, we use depth-wise and 1 × 1 convolution to model long-range dependencies to establish spatial and cross-channel correlation, respectively. The proposed approach is evaluated qualitatively and quantitatively on five challenging polyp datasets across six metrics. Our MMNet outperforms the previous best polyp segmentation methods.

Predictive Mouse Ultrasonic Vocalization Sequences: Uncovering Behavioral Significance, Auditory Cortex Neuronal Preferences, and Social-Experience-Driven Plasticity.

Mouse ultrasonic vocalizations (USVs) contain predictable sequential structures like bird songs and speech. Neural representation of USVs in the mouse primary auditory cortex (Au1) and its plasticity with experience has been largely studied with single-syllables or dyads, without using the predictability in USV sequences. Studies using playback of USV sequences have used randomly selected sequences from numerous possibilities. The current study uses mutual information to obtain context-specific natural sequences (NSeqs) of USV syllables capturing the observed predictability in male USVs in different contexts of social interaction with females. Behavioral and physiological significance of NSeqs over random sequences (RSeqs) lacking predictability were examined. Female mice, never having the social experience of being exposed to males, showed higher selectivity for NSeqs behaviorally and at cellular levels probed by expression of immediate early gene c-fos in Au1. The Au1 supragranular single units also showed higher selectivity to NSeqs over RSeqs. Social-experience-driven plasticity in encoding NSeqs and RSeqs in adult females was probed by examining neural selectivities to the same sequences before and after the above social experience. Single units showed enhanced selectivity for NSeqs over RSeqs after the social experience. Further, using two-photon Ca2+ imaging, we observed social experience-dependent changes in the selectivity of sequences of excitatory and somatostatin-positive inhibitory neurons but not parvalbumin-positive inhibitory neurons of Au1. Using optogenetics, somatostatin-positive neurons were identified as a possible mediator of the observed social-experience-driven plasticity. Our study uncovers the importance of predictive sequences and introduces mouse USVs as a promising model to study context-dependent speech like communications.SIGNIFICANCE STATEMENT Humans need to detect patterns in the sensory world. For instance, speech is meaningful sequences of acoustic tokens easily differentiated from random ordered tokens. The structure derives from the predictability of the tokens. Similarly, mouse vocalization sequences have predictability and undergo context-dependent modulation. Our work investigated whether mice differentiate such informative predictable sequences (NSeqs) of communicative significance from RSeqs at the behavioral, molecular, and neuronal levels. Following a social experience in which NSeqs occur as a crucial component, mouse auditory cortical neurons become more sensitive to differences between NSeqs and RSeqs, although preference for individual tokens is unchanged. Thus, speech-like communication and its dysfunction may be studied in circuit, cellular, and molecular levels in mice.

Visual Exploration of Relationships and Structure in Low-Dimensional Embeddings.

In this work, we propose an interactive visual approach for the exploration and formation of structural relationships in embeddings of high-dimensional data. These structural relationships, such as item sequences, associations of items with groups, and hierarchies between groups of items, are defining properties of many real-world datasets. Nevertheless, most existing methods for the visual exploration of embeddings treat these structures as second-class citizens or do not take them into account at all. In our proposed analysis workflow, users explore enriched scatterplots of the embedding, in which relationships between items and/or groups are visually highlighted. The original high-dimensional data for single items, groups of items, or differences between connected items and groups are accessible through additional summary visualizations. We carefully tailored these summary and difference visualizations to the various data types and semantic contexts. During their exploratory analysis, users can externalize their insights by setting up additional groups and relationships between items and/or groups. We demonstrate the utility and potential impact of our approach by means of two use cases and multiple examples from various domains.

Dynamic Multi-Behavior Sequence Modeling for Next Item Recommendation

Sequential Recommender Systems (SRSs) aim to predict the next item that users will consume, by modeling the user interests within their item sequences. While most existing SRSs focus on a single type of user behavior, only a few pay attention to multi-behavior sequences, although they are very common in real-world scenarios. It is challenging to effectively capture the user interests within multi-behavior sequences, because the information about user interests is entangled throughout the sequences in complex relationships. To this end, we first address the characteristics of multi-behavior sequences that should be considered in SRSs, and then propose novel methods for Dynamic Multi-behavior Sequence modeling named DyMuS, which is a light version, and DyMuS+, which is an improved version, considering the characteristics. DyMuS first encodes each behavior sequence independently, and then combines the encoded sequences using dynamic routing, which dynamically integrates information required in the final result from among many candidates, based on correlations between the sequences. DyMuS+, furthermore, applies the dynamic routing even to encoding each behavior sequence to further capture the correlations at item-level. Moreover, we release a new, large and up-to-date dataset for multi-behavior recommendation. Our experiments on DyMuS and DyMuS+ show their superiority and the significance of capturing the characteristics of multi-behavior sequences.

Uniform Sequence Better: Time Interval Aware Data Augmentation for Sequential Recommendation

Sequential recommendation is an important task to predict the next-item to access based on a sequence of interacted items. Most existing works learn user preference as the transition pattern from the previous item to the next one, ignoring the time interval between these two items. However, we observe that the time interval in a sequence may vary significantly different, and thus result in the ineffectiveness of user modeling due to the issue of preference drift. In fact, we conducted an empirical study to validate this observation, and found that a sequence with uniformly distributed time interval (denoted as uniform sequence) is more beneficial for performance improvement than that with greatly varying time interval. Therefore, we propose to augment sequence data from the perspective of time interval, which is not studied in the literature. Specifically, we design five operators (Ti-Crop, Ti-Reorder, Ti-Mask, Ti-Substitute, Ti-Insert) to transform the original non-uniform sequence to uniform sequence with the consideration of variance of time intervals. Then, we devise a control strategy to execute data augmentation on item sequences in different lengths. Finally, we implement these improvements on a state-of-the-art model CoSeRec and validate our approach on four real datasets. The experimental results show that our approach reaches significantly better performance than the other 9 competing methods. Our implementation is available: https://github.com/KingGugu/TiCoSeRec.

CF-ViT: A General Coarse-to-Fine Method for Vision Transformer

Vision Transformers (ViT) have made many breakthroughs in computer vision tasks. However, considerable redundancy arises in the spatial dimension of an input image, leading to massive computational costs. Therefore, We propose a coarse-to-fine vision transformer (CF-ViT) to relieve computational burden while retaining performance in this paper. Our proposed CF-ViT is motivated by two important observations in modern ViT models: (1) The coarse-grained patch splitting can locate informative regions of an input image. (2) Most images can be well recognized by a ViT model in a small-length token sequence. Therefore, our CF-ViT implements network inference in a two-stage manner. At coarse inference stage, an input image is split into a small-length patch sequence for a computationally economical classification. If not well recognized, the informative patches are identified and further re-split in a fine-grained granularity. Extensive experiments demonstrate the efficacy of our CF-ViT. For example, without any compromise on performance, CF-ViT reduces 53% FLOPs of LV-ViT, and also achieves 2.01x throughput. Code of this project is at https://github.com/ChenMnZ/CF-V

Learning and Fusing Multi-View Code Representations for Function Vulnerability Detection

The explosive growth of vulnerabilities poses a significant threat to the security of software systems. While various deep-learning-based vulnerability detection methods have emerged, they primarily rely on semantic features extracted from a single code representation structure, which limits their ability to detect vulnerabilities hidden deep within the code. To address this limitation, we propose S2FVD, short for Sequence and Structure Fusion-based Vulnerability Detector, which fuses vulnerability-indicative features learned from the multiple views of the code for more accurate vulnerability detection. Specifically, S2FVD employs either well-matched or carefully extended neural network models to extract vulnerability-indicative semantic features from the token sequence, attributed control flow graph (ACFG) and abstract syntax tree (AST) representations of a function, respectively. These features capture different perspectives of the code, which are then fused to enable S2FVD to accurately detect vulnerabilities that are well-hidden within a function. The experiments conducted on two large vulnerability datasets demonstrated the superior performance of S2FVD against state-of-the-art approaches, with its accuracy and F1 scores reaching 98.07% and 98.14% respectively in detecting the presence of vulnerabilities, and 97.93% and 97.94%, respectively, in pinpointing specific vulnerability types. Furthermore, with regard to the real-world dataset D2A, S2FVD achieved average performance gains of 6.86% and 14.84% in terms of accuracy and F1 metrics, respectively, over the state-of-the-art baselines. This ablation study also confirms the superiority of fusing the semantics implied in multiple distinct code views to further enhance vulnerability detection performance.

A Survey on Sparsity Exploration in Transformer-Based Accelerators

Transformer models have emerged as the state-of-the-art in many natural language processing and computer vision applications due to their capability of attending to longer sequences of tokens and supporting parallel processing more efficiently. Nevertheless, the training and inference of transformer models are computationally expensive and memory intensive. Meanwhile, utilizing the sparsity in deep learning models has proven to be an effective approach to alleviate the computation challenge as well as help to fit large models in edge devices. As high-performance CPUs and GPUs are generally not flexible enough to explore low-level sparsity, a number of specialized hardware accelerators have been proposed for transformer models. This paper provides a comprehensive review of hardware transformer accelerators that have been proposed to explore sparsity for computation and memory optimizations. We classify existing works based on the strategies of utilizing sparsity and identify their pros and cons in those strategies. Based on our analysis, we point out promising directions and recommendations for future works on improving the effective sparse execution of transformer hardware accelerators.

Implicit Processing of Numerical Order: Evidence from a Continuous Interocular Flash Suppression Study

Processing the ordered relationships between sequential items is a key element in many cognitive abilities that are important for survival. Specifically, order may play a crucial role in numerical processing. Here, we assessed the existence of a cognitive system designed to implicitly evaluate numerical order, by combining continuous flash suppression with a priming method in a numerical enumeration task. In two experiments and diverse statistical analysis, targets that required numerical enumeration were preceded by an invisibly ordered or non-ordered numerical prime sequence. The results of both experiments showed that enumeration for targets that appeared after an ordered prime was significantly faster, while the ratio of the prime sequences produced no significant effect. The findings suggest that numerical order is processed implicitly and affects a basic cognitive ability: enumeration of quantities.

A CNN-Transformer Network Combining CBAM for Change Detection in High-Resolution Remote Sensing Images

Current deep learning-based change detection approaches mostly produce convincing results by introducing attention mechanisms to traditional convolutional networks. However, given the limitation of the receptive field, convolution-based methods fall short of fully modelling global context and capturing long-range dependencies, thus insufficient in discriminating pseudo changes. Transformers have an efficient global spatio-temporal modelling capability, which is beneficial for the feature representation of changes of interest. However, the lack of detailed information may cause the transformer to locate the boundaries of changed regions inaccurately. Therefore, in this article, a hybrid CNN-transformer architecture named CTCANet, combining the strengths of convolutional networks, transformer, and attention mechanisms, is proposed for high-resolution bi-temporal remote sensing image change detection. To obtain high-level feature representations that reveal changes of interest, CTCANet utilizes tokenizer to embed the features of each image extracted by convolutional network into a sequence of tokens, and the transformer module to model global spatio-temporal context in token space. The optimal bi-temporal information fusion approach is explored here. Subsequently, the reconstructed features carrying deep abstract information are fed to the cascaded decoder to aggregate with features containing shallow fine-grained information, through skip connections. Such an aggregation empowers our model to maintain the completeness of changes and accurately locate small targets. Moreover, the integration of the convolutional block attention module enables the smoothing of semantic gaps between heterogeneous features and the accentuation of relevant changes in both the channel and spatial domains, resulting in more impressive outcomes. The performance of the proposed CTCANet surpasses that of recent certain state-of-the-art methods, as evidenced by experimental results on two publicly accessible datasets, LEVIR-CD and SYSU-CD.

Development of an embedded molecular structure-based model for prediction of micropollutant treatability in a drinking water treatment plant by machine learning from three years monitoring data

In this study, an autoencoder-based molecular structure embedding model was developed to predict treatability of micropollutant in a drinking water treatment plant (DWTP) by machine learning using 69 micropollutants monitoring data at 18 DWTPs for three years. The molecular structure, which contains physicochemical characteristics, was embedded as a fixed-length vector that is advantageous for data-driven analysis and machine learning. First, the molecular structure of the micropollutants was converted to a sequence of tokens using the simplified molecular-input line-entry system (SMILES) pair encoding tokenizer, a frequency-based tokenization method. It was then compressed into fixed-length vectors using an autoencoder trained on various molecular structures within the Chemical Entities of Biological Interest. To validate the proposed models, a binary classification of micropollutant treatability was performed using the embedded molecular structure of micropollutants with various external features, such as concentration, season, and the presence of specific drinking water treatment processes by machine learning. The accuracy of the developed model for the 69 micropollutants in this study was 0.86, and the molecular structure was determined to be the most important feature. Furthermore, an accuracy of 0.71 was obtained in external validation for pharmaceuticals and personal care products that were not used for training. This shows that the proposed embedding vector can be generalized to unseen molecules during the training process, which means that it reflects the characteristics of the molecular structures.

Generative models for sequential dynamics in active inference

AbstractA central theme of theoretical neurobiology is that most of our cognitive operations require processing of discrete sequences of items. This processing in turn emerges from continuous neuronal dynamics. Notable examples are sequences of words during linguistic communication or sequences of locations during navigation. In this perspective, we address the problem of sequential brain processing from the perspective of active inference, which inherits from a Helmholtzian view of the predictive (Bayesian) brain. Underneath the active inference lies a generative model; namely, a probabilistic description of how (observable) consequences are generated by (unobservable) causes. We show that one can account for many aspects of sequential brain processing by assuming the brain entails a generative model of the sensed world that comprises central pattern generators, narratives, or well-defined sequences. We provide examples in the domains of motor control (e.g., handwriting), perception (e.g., birdsong recognition) through to planning and understanding (e.g., language). The solutions to these problems include the use of sequences of attracting points to direct complex movements—and the move from continuous representations of auditory speech signals to the discrete words that generate those signals.