Sequence Of Symbols Research Articles

Combining Syntactic Methods With LSTM to Classify Soybean Aerial Images

Syntactic methods in computer vision represent visual patterns in a hierarchical and compositional perspective, which is converted to strings. Long short-term memory (LSTM) is able to learn patterns in sequences. In this letter, we propose a syntactic approach to represent visual patterns as sequences of symbols, and we use an LSTM as a classifier to learn the relationship between the symbols in sequences. An extensive experimental evaluation using aerial images from a soybean field captured by unmanned aerial vehicles has been conducted to compare our method with two deep learning architectures, one syntactic method, and one shallow learning algorithm. The results achieved by the proposed method maintain stability even when trained on small data sets, suggesting that representing visual patterns in a compositional way, repeating primitives, may be a viable alternative when there are only a limited number of samples for training.

Correlation function inadequacy in random-sequence entropy measures.

Considering symbolic and numerical random sequences in the framework of the additive Markov chain approach, we establish a relation between their correlation functions and conditional entropies. We express the entropy by means of the two-point probability distribution functions and then evaluate the entropy for the numerical random chain in terms of the correlation function. We show that such approximation gives a satisfactory result only for special types of random sequences. In general case the conditional entropy of numerical sequences obtained in the two-point distribution function approach is lower. We derive the conditional entropy of the additive Markov chain as a sum of the Kullback-Leibler mutual information and give an example of random sequence with the exactly zero correlation function and the nonzero correlations.

Memory-related cognitive load effects in an interrupted learning task: A model-based explanation.

The Cognitive Load Theory provides a well-established framework for investigating aspects of learning situations that demand learners' working memory resources. However, the interplay of these aspects at the cognitive and neural level is still not fully understood. We developed four computational models in the cognitive architecture ACT-R to clarify underlying memory-related strategies and mechanisms. Our models account for human data of an experiment that required participants to perform a symbol sequence learning task with embedded interruptions. We explored the inclusion of subsymbolic mechanisms to explain these data and used our final model to generate fMRI predictions. The final model indicates a reasonable fit for reaction times and accuracy and links the fMRI predictions to the Cognitive Load Theory. Our work emphasizes the influence of task characteristics and supports a process-related view on cognitive load in instructional scenarios. It further contributes to the discussion of underlying mechanisms at a neural level.

Genetic Music System with Synthetic Biology.

This article introduces GeMS, a system for music composition informed by synthetic biology. GeMS generates music with simulations of genetic processes, such as transcription, translation, and protein folding, with which biological systems render chains of amino acids from DNA strands. The system comprises the following components: the Miranda machine, the rhythmator, and the pitch processor. The Miranda machine is an abstract Turing-machine-like processor, which manipulates a sequence of DNA symbols according to a set of programming instructions. This process generates a pool of new DNA strands, which are subsequently translated into rhythms. GeMS represents the musical equivalent of amino acids in terms of rhythms, referred to as rhythmic codons. This enables the rhythmator to convert DNA sequences into rhythmic sequences. The pitch processor generates pitches for such rhythmic sequences. It is inspired by the phenomenon of protein folding. The pitch processor considers orientation information of DNA instructions yielded by the Miranda machine in order to activate algorithms for generating pitches. A musical composition, entitled Artibiotics, for percussion ensemble and electronic instruments, is presented to demonstrate the system.

Rate-Constrained Shaping Codes for Structured Sources

Shaping codes are used to encode information for use on channels with cost constraints. Applications include data transmission with a power constraint and, more recently, data storage on flash memories with a constraint on memory cell wear. In the latter application, system requirements often impose a rate constraint. In this paper, we study rate-constrained fixed-to-variable length shaping codes for noiseless, memoryless costly channels and general i.i.d. sources. The analysis relies on the theory of word-valued sources. We establish a relationship between the code expansion factor – the ratio of the expected codeword length to the length of the input source word – and the minimum average symbol cost. We then determine the expansion factor that minimizes the average cost per source symbol (total cost), corresponding to a conventional optimal source code with cost. An equivalence is established between codes minimizing average symbol cost and codes minimizing total cost, and a separation theorem is proved, showing that optimal shaping can be achieved by a concatenation of optimal compression and optimal shaping for a uniform i.i.d. source. Shaping codes often incorporate, either explicitly or implicitly, some form of non-equiprobable signaling. We use our results to further explore the connections between shaping codes and codes that map a sequence of i.i.d. source symbols into an output sequence of symbols that are approximately independent and distributed according to a specified target distribution, such as distribution matching (DM) codes. Optimal DM codes are characterized in terms of a new performance measure - generalized expansion factor (GEF) - motivated by the costly channel perspective. The GEF is used to study DM codes that minimize informational divergence and normalized informational divergence.

Arrow of time across five centuries of classical music

The concept of time series irreversibility -- the degree by which the statistics of signals are not invariant under time reversal -- naturally appears in non-equilibrium physics in stationary systems which operate away from equilibrium and produce entropy. This concept has not been explored to date in the realm of musical scores as these are typically short sequences whose time reversibility estimation could suffer from strong finite size effects which preclude interpretability. Here we show that the so-called Horizontal Visibility Graph method -- which recently was shown to quantify such statistical property even in non-stationary signals -- is a method that can estimate time reversibility of short symbolic sequences, thus unlocking the possibility of exploring such properties in the context of musical compositions. Accordingly, we analyse over 8000 musical pieces ranging from the Renaissance to the early Modern period and certify that, indeed, most of them display clear signatures of time irreversibility. Since by construction stochastic processes with a linear correlation structure (such as 1/f noise) are time reversible, we conclude that musical compositions have a considerably richer structure, that goes beyond the traditional properties retrieved by the power spectrum or similar approaches. We also show that musical compositions display strong signs of nonlinear correlations, that nonlinearity is correlated to irreversibility, and that these are also related to asymmetries in the abundance of musical intervals, which we associate to the narrative underpinning a musical composition. These findings provide tools for the study of musical periods and composers, as well as criteria related to music appreciation and cognition.

Data representations for audio-to-score monophonic music transcription

This work presents an end-to-end method based on deep neural networks for audio-to-score music transcription of monophonic excerpts. Unlike existing music transcription methods, which normally perform pitch estimation, the proposed approach is formulated as an end-to-end task that outputs a notation-level music score. Using an audio file as input, modeled as a sequence of frames, a deep neural network is trained to provide a sequence of music symbols encoding a score, including key and time signatures, barlines, notes (with their pitch spelling and duration) and rests. Our framework is based on a Convolutional Recurrent Neural Network (CRNN) with Connectionist Temporal Classification (CTC) loss function trained in an end-to-end fashion, without requiring to align the input frames with the output symbols. A total of 246,870 incipits from the Répertoire International des Sources Musicales online catalog were synthesized using different timbres and tempos to build the training data. Alternative input representations (raw audio, Short-Time Fourier Transform (STFT), log-spaced STFT and Constant-Q transform) were evaluated for this task, as well as different output representations (Plaine & Easie Code, Kern, and a purpose-designed output). Results show that it is feasible to directly infer score representations from audio files and most errors come from music notation ambiguities and metering (time signatures and barlines).

A Type-II Phase-Tracking Receiver

We present a new analog-to-digital converter (ADC)-based architecture of a phase-tracking receiver (PT-RX) optimized for ultra-low-power (ULP) and ultra-low-voltage (ULV) operations for the Internet of Things (IoT). The RX employs a type-II loop configuration that offers improved stability compared with the previous type-I PT-RX solutions. In addition, the type-II loop is also very tolerant of long run-lengths of consecutive “1” or “0” symbol sequences. Fabricated in 28-nm CMOS, the prototype PT-RX targets Bluetooth low energy (BLE) standard consuming only 1.5 mW at a supply of ≤0.7 V. It maintains an adjacent-channel rejection (ACR) of ≥-11/3.5/17/27 dB at 0/±1/±2/±3 MHz offset and can tolerate out-of-band (OOB) blockers of minimum -21 dBm across 1.0-3.5 GHz while also offering a best-in-class figure of merit (FoM) of 181 dB, with a 1-Mb/s BLE sensitivity of -93 dBm.

Cyclic alternating pattern estimation based on a probabilistic model over an EEG signal

A probabilistic model for sleep analysis is proposed in this work, modeling the temporal relation between the sleep structure and the presence of the electroencephalogram (EEG) Cyclic Alternating Pattern (CAP) with a Hidden Markov Model (HMM). Sleep scoring is frequently performed by assigning a state to each thirty second epoch. However, this approach does not provide enough time resolution to efficiently detect the CAP since, by definition, the CAP cycles are assessed by applying the scoring rules to one second epochs of the EEG signal. Thus, a clustering analysis was employed, with a one second epoch, over the EEG signal to create clusters that were then encoded using symbolic dynamics to produce words. Two algorithms for clustering were analyzed, specifically the self-organizing map and the Gaussian Mixture Model (GMM). The words were then fed to the HMM to determine the presence of the CAP. Both single-channel and multi-channel (based on sensor fusion) approaches were tested. The best results were attained using the GMM with three Gaussians, achieving an average accuracy, sensitivity, specificity and area under the receiver operating characteristic curve of, respectively, 72%, 66%, 75% and 0.71 for single-channel and 76%, 61%, 85% and 0.73 for multi-channel. This results are in the specialist agreement range with visual analysis. Therefore, the proposed model is capable of providing a new view over the CAP cycles by simplifying the complex EEG signal to a simple sequence of symbols. Such analysis can be significantly challenging to perform in more abstract models.

Mining Evolution Patterns from Complex Trajectory Structures—A Case Study of Mesoscale Eddies in the South China Sea

Real-word phenomena, such as ocean eddies and clouds, tend to split and merge while they are moving around within a space. Their trajectories usually bear one or more branches and are accordingly defined as complex trajectories in this study. The trajectories may show significant spatiotemporal variations in terms of their structures and some of them may be more prominent than the others. The identification of prominent structures in the complex trajectories of such real-world phenomena could better reveal their evolution processes and even shed new light on the driving factors behind them. Methods have been proposed for the extraction of periodic patterns from simple trajectories (i.e., those with linear structure and without any branches) with a focus on mining the related temporal, spatial or semantic information. Unfortunately, it is not appropriate to directly use such methods to examine complex trajectories. This study proposes a novel method to study the periodic patterns of complex trajectories by considering the inherent spatial, temporal and topological information. First, we use a sequence of symbols to represent the various structures of a complex trajectory over its lifespan. We then, on the basis of the PrefixSpan algorithm, propose a periodic pattern mining of structural evolution (PPSE) algorithm and use it to identify the largest and most frequent patterns (LFPs) from the symbol sequence. We also identify potential periodic behaviors. The PPSE method is then used to examine the complex trajectories of the mesoscale eddy in the South China Sea (SCS) from 1993 to 2016. The complex trajectories of ocean eddies in the southeast of Vietnam show are different from other regions in the SCS in terms of their structural evolution processes, as indicated by the LFPs with the longest lifespan, the widest active range, the highest complexity, and the most active behaviors. The LFP in the southeast of Vietnam has the longest lifespan, the widest active range, the highest complexity, and the most active behaviors. Across the SCS, we found seven migration channels. The LFPs of the eddies that migrate through these channels have a temporal cycle of 17–24 years. These channels are also the regions where eddies frequently emerge, as revealed by flow field data.

Abstraction refinement and antichains for trace inclusion of infinite state systems

A generic register automaton is a finite automaton equipped with variables (which may be viewed as counters or, more generally, registers) ranging over infinite data domains. A trace of a generic register automaton is an alternating sequence of alphabet symbols and values taken by the variables during an execution of the automaton. The problem addressed in this paper is the inclusion between the sets of traces (data languages) recognized by such automata. Since the problem is undecidable in general, we give a semi-algorithm based on a combination of abstraction refinement and antichains, which is proved to be sound and complete, but whose termination is not guaranteed. Moreover, we further enhance the proposed algorithm by exploiting a concept of data simulations, i.e., simulation relations aware of the data associated with the words. We have implemented our technique in a prototype tool and show promising results on multiple non-trivial examples.

Algebraic Properties of Parikh Matrices of Binary Picture Arrays

A word is a finite sequence of symbols. Parikh matrix of a word is an upper triangular matrix with ones in the main diagonal and nonnegative integers above the main diagonal which are counts of certain scattered subwords in the word. On the other hand, a picture array, which is a rectangular arrangement of symbols, is an extension of the notion of a word to two dimensions. Parikh matrices associated with a picture array have been introduced, and their properties have been studied. Here, we obtain certain algebraic properties of Parikh matrices of binary picture arrays based on the notions of power, fairness, and a restricted shuffle operator extending the corresponding notions studied in the case of words. We also obtain properties of Parikh matrices of arrays formed by certain geometric operations.

Burrows‐Wheeler post‐transformation with effective clustering and interpolative coding

SummaryLossless compression methods based on the Burrows‐Wheeler transform (BWT) are regarded as an excellent compromise between speed and compression efficiency: they provide compression rates close to the PPM algorithms, with the speed of dictionary‐based methods. Instead of the laborious statistics‐gathering process used in PPM, the BWT reversibly sorts the input symbols, using as the sort key as many following characters as necessary to make the sort unique. Characters occurring in similar contexts are sorted close together, resulting in a clustered symbol sequence. Run‐length encoding and Move‐to‐Front (MTF) recoding, combined with a statistical Huffman or arithmetic coder, is then typically used to exploit the clustering. A drawback of the MTF recoding is that knowledge of the character that produced the MTF number is lost. In this paper, we present a new, competitive Burrows‐Wheeler posttransform stage that takes advantage of interpolative coding—a fast binary encoding method for integer sequences, being able to exploit clusters without requiring explicit statistics. We introduce a fast and simple way to retain knowledge of the run characters during the MTF recoding and use this to improve the clustering of MTF numbers and run‐lengths by applying reversible, stable sorting, with the run characters as sort keys, achieving significant improvement in the compression rate, as shown here by experiments on common text corpora.

Content-Based Document Image Retrieval Based on Document Modeling

Recently, language models have gained importance in the field of information retrieval. In this paper, we propose a generic language model to improve a content-based document retrieval system. In this approach, character images are extracted, clustered, and analyzed to form high-level semantic terms using a statistical document model. This model simulates the long-term relationships between characters. Documents are then indexed according to these terms, and a query document is proposed to retrieve the relevant documents. The query document can be a single keyword, or it can be synthesized from a text string. The aim is to generate a semantic representation from low-level image pixels through pattern matching and document modeling. The conventional approach of generating semantic terms in document retrieval includes every possible symbol sequence in the feature representation. Comparatively, our approach can considerably reduce the dimensions of the feature space while producing retrieval results comparable to those of the conventional and state-of-the-art approaches.

Chaos, border collisions and stylized empirical facts in an asset pricing model with heterogeneous agents

An asset pricing model with chartists, fundamentalists and trend followers is considered. A market maker adjusts the asset price in the direction of the excess demand at the end of each trading session. An exogenously given fundamental price discriminates between a bull market and a bear market. The buying and selling orders of traders change moving from a bull market to a bear market. Their asymmetric propensity to trade leads to a discontinuity in the model, with its deterministic skeleton given by a two-dimensional piecewise linear dynamical system in discrete time. Multiple attractors, such as a stable fixed point and one or more attracting cycles or cycles and chaotic attractors, appear through border collision bifurcations. The multi-stability regions are underlined by means of two-dimensional bifurcation diagrams, where the border collision bifurcation curves are detected in analytic form at least for basic cycles with symbolic sequences $${\hbox {LR}}^{n}$$ and $${\hbox {RL}}^{n}$$ . A statistical analysis of the simulated time series of the asset returns, generated by perturbing the deterministic dynamics with a random walk process, indicates that this is one of the simplest asset pricing models which are able to replicate stylized empirical facts, such as excess volatility, fat tails and volatility clustering.

Disentangling sequential from hierarchical learning in Artificial Grammar Learning: Evidence from a modified Simon Task.

In this paper we probe the interaction between sequential and hierarchical learning by investigating implicit learning in a group of school-aged children. We administered a serial reaction time task, in the form of a modified Simon Task in which the stimuli were organised following the rules of two distinct artificial grammars, specifically Lindenmayer systems: the Fibonacci grammar (Fib) and the Skip grammar (a modification of the former). The choice of grammars is determined by the goal of this study, which is to investigate how sensitivity to structure emerges in the course of exposure to an input whose surface transitional properties (by hypothesis) bootstrap structure. The studies conducted to date have been mainly designed to investigate low-level superficial regularities, learnable in purely statistical terms, whereas hierarchical learning has not been effectively investigated yet. The possibility to directly pinpoint the interplay between sequential and hierarchical learning is instead at the core of our study: we presented children with two grammars, Fib and Skip, which share the same transitional regularities, thus providing identical opportunities for sequential learning, while crucially differing in their hierarchical structure. More particularly, there are specific points in the sequence (k-points), which, despite giving rise to the same transitional regularities in the two grammars, support hierarchical reconstruction in Fib but not in Skip. In our protocol, children were simply asked to perform a traditional Simon Task, and they were completely unaware of the real purposes of the task. Results indicate that sequential learning occurred in both grammars, as shown by the decrease in reaction times throughout the task, while differences were found in the sensitivity to k-points: these, we contend, play a role in hierarchical reconstruction in Fib, whereas they are devoid of structural significance in Skip. More particularly, we found that children were faster in correspondence to k-points in sequences produced by Fib, thus providing an entirely new kind of evidence for the hypothesis that implicit learning involves an early activation of strategies of hierarchical reconstruction, based on a straightforward interplay with the statistically-based computation of transitional regularities on the sequences of symbols.

Structural Machines as Unconventional Knowledge
 Processors

Knowledge systems often have very sophisticated structures depicting cognitive andstructural entities. For instance, representation of knowledge in the form of a text involves thestructure of this text. This structure is represented by a hypertext, which is networks consisting oflinguistic objects, such as words, phrases and sentences, with diverse links connecting them.Current computational machines and automata such as Turing machines process information inthe form of symbol sequences. Here we discuss based the methods of structural machinesachieving higher flexibility and efficiency of information processing in comparison with regularmodels of computation. Being structurally universal abstract automata, structural machines allowworking directly with knowledge structures formed by knowledge objects and connectionsbetween them.

Deep Learning-Based End-to-End Wireless Communication Systems With Conditional GANs as Unknown Channels

In this article, we develop an end-to-end wireless communication system using deep neural networks (DNNs), where DNNs are employed to perform several key functions, including encoding, decoding, modulation, and demodulation. However, an accurate estimation of instantaneous channel transfer function, i.e., channel state information (CSI), is needed in order for the transmitter DNN to learn to optimize the receiver gain in decoding. This is very much a challenge since CSI varies with time and location in wireless communications and is hard to obtain when designing transceivers. We propose to use a conditional generative adversarial net (GAN) to represent channel effects and to bridge the transmitter DNN and the receiver DNN so that the gradient of the transmitter DNN can be back-propagated from the receiver DNN. In particular, a conditional GAN is employed to model the channel effects in a data-driven way, where the received signal corresponding to the pilot symbols is added as a part of the conditioning information of the GAN. To address the curse of dimensionality when the transmit symbol sequence is long, convolutional layers are utilized. From the simulation results, the proposed method is effective on additive white Gaussian noise (AWGN) channels, Rayleigh fading channels, and frequency-selective channels, which opens a new door for building data-driven DNNs for end-to-end communication systems.

The Role of Autonomic Nervous System and Baroreflex in the Pattern of Heart Rate Fragmentation

Although rapid oscillations of the heart rate (HR) are commonly attributed to vagal modulation of the heart, a pattern of ultra‐rapid HR variation has been recently studied. This type of variation, named HR fragmentation (HRF), has been shown to be increased in aging and coronary diseases and seems to be linked to the risk of death and life‐threatening cardiovascular events. However, the mechanisms behind HRF are unknown. Thus, the aim of this study was to evaluate the influence of the autonomic nervous system (ANS) and arterial baroreceptors on HRF. For this, electrodes for electrocardiogram (ECG) and a catheter into femoral vein were implanted in Wistar rats. On the next day, under ECG recordings, the rats were submitted to selective pharmacological blockade of cardiac autonomic receptors with atenolol (4 mg/kg iv in bolus followed by 0.8 mg/kg/h iv) followed by methylatropine (2 mg/kg in bolus followed by 0.4 mg/kg/h iv) or vice‐versa. Another group underwent sinoaortic denervation (SAD), or sham‐operation, and received a catheter in femoral artery. On the next day, SAD and sham rats had its blood pressure directly recorded. All recordings were performed in freely moving animals. Pulse interval or RR‐interval series were generated and transformed into sequence of symbols "−1”, “0” or “1" when the difference between successive values (transitions) was negative, zero, or positive, respectively. In addition, the transitions between symbols "−1" and "1" were classified as "hard" (H) while those between "−1" (or "1") and "0" were classified as "soft" (S). Next, sequences of 4 consecutive symbols were classified according to the number and type of transition, i.e. S, M or H, where M represents mixed transitions (H and S). The total percentage of inflection points (PIP) was also quantified. PIP increased after sympathetic (66±2 to 73±2), parasympathetic (68±1 to 73±1) and double autonomic blockade (67±1 to 73±1). The percentage of sequences with H transitions was lower after parasympathetic (27±3 vs 74±2) and double blockade (43±4 vs 76±1), while sequences with S and M transitions were higher after parasympathetic (S: 28±3 vs 7±1% and M: 44±2 vs 18±1%) and double blockade (S: 15±2 vs 6±1 and M: 41±2 vs 17±1%). The percentage of sequences with M‐type transitions increased after sympathetic blockade (16±1 to 23±3), while no change was found with H and S‐type transitions. PIP was lower in SAD rats (68±2) when compared to the sham counterparts (88±1). Regarding type, the percentage of H transitions was higher in SAD (90±1 vs 20±1), whereas sham rats showed more S (27±1 vs 2±0.1) and M‐type transitions (53±1 vs 7±1). In conclusion, the ANS, specially its parasympathetic branch, attenuates HRF. Moreover, although the removal of arterial baroreceptors represents the loss of an important cardiocirculatory control system, the denervated rats presented lower overall HRF. Nevertheless, the percentage of H‐type transitions is higher in SAD rats, evidencing that both the amount and type of fragmentation represent important indices under physiological and pathophysiological conditions.Support or Funding InformationCNPq, CAPES and FAPESP

Alterations in Heart Rate Fragmentation After Myocardial Infarction in Rats: A Novel Approach in Heart Rate Variability Analysis

Heart rate variability (HRV) has been largely studied by linear and nonlinear approaches, in a variety of clinical scenarios and animal models of diseases. Recently, a new pattern of ultra‐rapid heart rate variation has been proposed and named heart rate fragmentation (HRF). This approach showed that aging and coronary disease, increase the fragmentation of heart rate fluctuations. Since HRF was only evaluated in humans, we aimed to investigate this new HRV index in a rat model of myocardial infarction (MI). For this purpose, Wistar rats (250 a 300g) were submitted to MI by permanent ligation of the left anterior descending coronary artery, or sham‐operation (control animals). At the 4th or 12th weeks after MI, or sham‐operation, animals were instrumented with a polyethylene arterial catheter for arterial pressure recordings which were carried out in freely moving rats. At the end of the protocol, cardiac function was evaluated by echocardiography using a 21 MHZ transducer (VisualSonics‐VEVO2100®). For HRF analysis, the pulse interval (PI) series was transformed into a sequence of symbols named "−1”, “0” or “1", when the difference between successive values (transitions) was negative, zero, or positive, respectively. Next, sequences of 4 consecutive symbols were classified according to the numbers of transitions. The percentage of inflection points (PIP) of PI series were also quantified. As expected, cardiac function was found reduced in rats subjected to MI. Ejection fraction was smaller 4 (28±3 vs 68±2%) and 12 weeks after MI (38±3 vs 70±3%). Similarly, fractional shortening was also smaller 4 (13±2 vs 41±2%) and 12 weeks after MI (20±2 vs 41±3%). On the other hand, PIP values were increased in rats subjected to MI at the 4th (74±2 vs 69±1%) and 12th weeks after surgery (70±2 vs 63±1%), revealing a more fragmented pattern compared to control counterparts. We also found a significant correlation between cardiac functional parameters (ejection fraction and fractional shortening) and HRF (PIP) at both 4 and 12 weeks after MI. These findings reveal that in rats, cardiac ischemia also increases HRF, reinforcing the importance of this new HRV approach as a promising tool to explore physiological and pathophysiological conditions at clinical and experimental levels.Support or Funding InformationFinancial support: CNPq (870308/1997‐1) and FAPESP (2013/20549‐7).