Letter N-grams Research Articles

Spenser’s Spell: Archaism and Historical Stylometrics

In the present essay, on Spenser’s orthography, we address the question of whether the orthography of Spenser’s texts, in early modern editions, is salient and therefore worth preserving in modern editions. Even when we face the bibliographic fact that Spenser’s texts seem not to have been regarded as so valuably idiosyncratic that early modern printers preserved them from adjustment, we may still seek to know whether the first editions of Spenser’s works are demonstrably and articulably idiosyncratic, whether orthographically, lexically, inflectionally, or syntactically. This essay formulates a method for statistically probing Spenser’s orthographic profile against what we demonstrate to be the variant, but coherent background of early printed English. We build a model of early modern orthographic change based on letter n-grams extracted from the 60,000 texts in the EEBO-TCP corpus. The n-grams yield some 30,000 features; we concentrate on a subset of the 200 most variant features, reducing the dimensionality of the data by means of principal component analysis (PCA). Situating Spenser’s texts against contemporary and near-contemporary texts, we demonstrate that, in general, Spenser’s corpus is not orthographically distinctive and propose that the impression of the linguistic distinctiveness of Spenser’s poetry is concentrated elsewhere. We conclude both with proposals for new tests that might enable us to isolate that distinctiveness and with a brief assessment of appropriate editorial responses to our investigations.

Evaluation of text representation schemes and distance measures for authorship linking

Abstract Based on n text excerpts, the authorship linking task is to determine a way to link pairs of documents written by the same person together. This problem is closely related to authorship attribution questions, and its solution can be used in the author clustering task. However, no training information is provided and the solution must be unsupervised. To achieve this, various text representation strategies can be applied, such as characters, punctuation symbols, or letter n-grams as well as words, lemmas, Part-Of-Speech (POS) tags, and sequences of them. To estimate the stylistic distance (or similarity) between two text excerpts, different measures have been suggested based on the L1 norm (e.g. Manhattan, Tanimoto), the L2 norm (e.g. Matusita), the inner product (e.g. Cosine), or the entropy paradigm (e.g. Jeffrey divergence). From those possible implementations, it is not clear which text representation and distance functions produce the best performance, and this study provides an answer to this question. Three corpora, extracted from French and English literature, have been evaluated using standard methodology. Moreover, we suggest an additional performance measure called high precision (HPrec) capable of judging the quality of a ranked list of links to provide only correct answers. No systematic difference can be found between token- or lemma-based text representations. Simple POS tags do not provide an effective solution but short sequences of them form a good text representation. Letter n-grams (with n = 4–6) give high HPrec rates. As distance measures, this study found that the Tanimoto, Matusita, and Clark distance measures perform better than the often-used Cosine function. Finally, applying a pruning procedure (e.g. culling terms appearing once or twice or limiting the vocabulary to the 500 most frequent words) reduces the representation complexity and might even improve the effectiveness of the attribution scheme.

Towards Learning Word Representation

Continuous vector representations, as a distributed representations for words have gained a lot of attention in Natural Language Processing (NLP) field. Although they are considered as valuable methods to model both semantic and syntactic features, they still may be improved. For instance, the open issue seems to be to develop different strategies to introduce the knowledge about the morphology of words. It is a core point in case of either dense languages where many rare words appear and texts which have numerous metaphors or similies. In this paper, we extend a recent approach to represent word information. The underlying idea of our technique is to present a word in form of a bag of syllable and letter n-grams. More specifically, we provide a vector representation for each extracted syllable-based and letter-based n-gram, and perform concatenation. Moreover, in contrast to the previous method, we accept n-grams of varied length n. Further various experiments, like tasks-word similarity ranking or sentiment analysis report our method is competitive with respect to other state-of-theart techniques and takes a step toward more informative word representation construction. Keywords: representation learning, n-gram model, NLP

Textual features for corpus visualization using correspondence analysis

Explorative data analysis in text mining essentially relies on effective visualization techniques which can expose hidden relationships among documents and reveal correspondence between documents and their features. In text mining, the documents are most often represented by feature vectors of very high dimensions, requiring dimensionality reduction to obtain visual projections in two- or three-dimensional space. Correspondence analysis is an unsupervised approach that allows for construction of low-dimensional projection space with simultaneous placement of both documents and features, making it ideal for explorative analysis in text mining. Its present use, however, has been limited to word-based features. In this paper, we investigate how this particular document representation compares to the representation with letter n-grams and word n-grams, and find that these alternative representations yield better results in separating documents of different class. We perform our experimental analysis on a bilingual Croatian-English parallel corpus, allowing us to additionally explore the impact of features in different languages on the quality of visualizations.

Attacking Letter Substitution Ciphers with Integer Programming

We introduce a method for solving substitution ciphers using low-order letter n-gram models. This method enforces global constraints using integer programming, and it guarantees that no decipherment key is overlooked. We carry out extensive empirical experiments showing how decipherment accuracy varies as a function of cipher length and n-gram order. We also make an empirical investigation of Shannon's [12] theory of uncertainty in decipherment.

Identifying Language Origin of Named Entity With Multiple Information Sources

To identify the language origin of a named entity, morphological information associated with its letter spelling, such as letter N-grams, is commonly employed. However, with this information only, named entities with similar spellings but from different language origins are difficult to differentiate. In this paper, a measure of popularity, in terms of frequency or page count of the named entity in language-specific Web search, is proposed for identifying its language origin. Morphological information, including letter or letter-chunk N-grams, is used to enhance the performance of language identification in conjunction with Web-based page counts. Six languages, including English, German, French, Portuguese, Chinese, and Japanese (Chinese and Japanese named entities are shown in their corresponding phonetic alphabets, i.e., Pinyin and Romaji), are tested. Experiments show that when classifying four Latin languages, including English, German, French, and Portuguese, which are written in Latin alphabets, features from different information sources yield substantial performance improvements in the classification accuracy over a letter 4-gram-based baseline system. The accuracy increases from 75.0% to 86.3%, or a 45.2% relative error reduction.

Parametric analysis of EEG and fMRI data reveals the neural basis of visual word processing

Parametric analysis of EEG and fMRI data reveals the neural basis of visual word processing

The time course of visual word recognition as revealed by linear regression analysis of ERP data

EEG correlates of a range of psycholinguistic word properties were used to investigate the time course of access to psycholinguistic information during visual word recognition. Neurophysiological responses recorded in a visual lexical decision task were submitted to linear regression analysis. First, 10 psycholinguistic features of each of 300 stimulus words were submitted to a principal component analysis, which yielded four orthogonal variables likely to reflect separable processes in visual word recognition: Word length, Letter n-gram frequency, Lexical frequency and Semantic coherence of a word's morphological family. Since the lexical decision task required subjects to distinguish between words and pseudowords, the binary variable Lexicality was also investigated using a factorial design. Word–pseudoword differences in the event-related potential first appeared at 160 ms after word onset. However, regression analysis of EEG data documented a much earlier effect of both Word length and Letter n-gram frequency around 90 ms. Lexical frequency showed its earliest effect slightly later, at 110 ms, and Semantic coherence significantly correlated with neurophysiological measures around 160 ms, simultaneously with the lexicality effect. Source estimates indicated parieto-temporo-occipital generators for the factors Length, Letter n-gram frequency and Word frequency, but widespread activation with foci in left anterior temporal lobe and inferior frontal cortex related to Semantic coherence. At later stages (>200 ms), all variables exhibited simultaneous EEG correlates. These results indicate that information about surface form and meaning of a lexical item is first accessed at different times in different brain systems and then processed simultaneously, thus supporting cascaded interactive processing models.

Decoding Substitution Ciphers by Means of Word Matching with Application to OCR

A substitution cipher consists of a block of natural language text where each letter of the alphabet has been replaced by a distinct symbol. As a problem in cryptography, the substitution cipher is of limited interest, but it has an important application in optical character recognition. Recent advances render it quite feasible to scan documents with a fairly complex layout and to classify (cluster) the printed characters into distinct groups according to their shape. However, given the immense variety of type styles and forms in current use, it is not possible to assign alphabetical identities to characters of arbitrary size and typeface. This gap can be bridged by solving the equivalent of a substitution cipher problem, thereby opening up the possibility of automatic translation of a scanned document into a standard character code, such as ASCII. Earlier methods relying on letter n-gram frequencies require a substantial amount of ciphertext for accurate n-gram estimates. A dictionary-based approach solves the problem using relatively small ciphertext samples and a dictionary of fewer than 500 words. Our heuristic backtrack algorithm typically visits only a few hundred among the 26! possible nodes on sample texts ranging from 100 to 600 words.