The Type-token Ratio Research Articles

Back to Basics in Measuring Lexical Diversity: Too Simple to Be True

Abstract Measuring lexical diversity in texts that have different lengths is problematic because length has a significant effect on the number of types a text contains, thus hampering any comparison. Treffers-Daller et al. (2018) recommended a simple solution, namely counting the number of types in a section of a given length that was extracted from the middle of each of the texts to be analysed. By applying this approach to second language essays, the authors observed that using the number of types was slightly more effective for differentiating amongst the Common European Framework of Reference for Languages (CEFR) levels than were the Type-Token Ratio (TTR) and the Guiraud index and that these three indices were more effective than were mathematically more complex ones. However, their conclusions regarding these two points are incorrect, and a less basic approach should be used. The last section addresses two broader issues in applied linguistics that are highlighted by these problems.

EVALUATING L2 VOCABULARY DEVELOPMENT FEATURES USING LEXICAL DENSITY AND LEXICAL DIVERSITY MEASURES

Most of the research on lexical measures was conducted in English and conducted similarly in other languages without accurate adaptations to the language being tested. The first objective of this study is to identify differences between applications of lexical density and lexical diversity when tested as in English and when adapted to the language being tested. The second objective is to inspect the effect of acquisition time on intermediate-level second-language learners' lexical density and diversity. In a test-retest study over one year, Arabic Hebrew bilinguals (n=23) wrote argumentative essays, in which lexical density and diversity were analyzed according to two approaches; a conventional approach as in English and an adapted approach more aligned with Hebrew linguistic features. In both approaches, lexical density was calculated as the number of lexical words in proportion to the total number of words and lexical diversity using the type-token ratio (TTR) and Corrected TTR (CTTR). Findings reveal statistically significant differences between the two approaches and the adapted approach allowed for cross-language and interlanguage comparisons. Although no significant increase in lexical density and diversity over the year was found, various patterns for lexical development were observed; at the second time point, students with the lowest level of lexical diversity substantially improved and more essays fell within the lexical density range of a typically written language.

Image Captioning Model Using Part-of-Speech Guidance Module for Description With Diverse Vocabulary

Image captions aim to generate human-like sentences that describe the image’s content. Recent developments in deep learning (DL) have made it possible to caption images for accurate descriptions and detailed expressions. However, since DL learns the relationship between images and captions, it constructs sentences based on commonly frequented words in the dataset. Although these generated sentences are highly accurate, they have low lexical diversity, unlike humans due to limited vocabulary. Therefore, in this paper, we propose a Part-Of-Speech (POS) guidance module and a multimodal-based image captioning model that determines the intensity of images and word sequences and generates sentences through POS to enhance the lexical diversity of DL. The proposed POS guidance module enables rich expression by controlling the information of images and sequences based on the predicted POS guidance to predict words. Then, the POS multimodal layer adds POS and output vector of Bi-LSTM using the multimodal layer to predict the next caption, considering the grammatical structure. We trained and tested the proposed model on the Flicker 30K and MS COCO datasets and compared them with current state-of-the-art studies. Also, we analyzed the lexical diversity of the caption model through the Type-Token Ratio (TTR) and confirmed that the proposed model generates sentences using several words.

Lexical Diversity Measures’ Review And Classification

This paper reviews various lexical diversity (LD) measures and their classification. The authors define the most significant advantages and disadvantages of the measures and investigate the main scopes of LD application. They include measuring LD in the speech of children and people with aphasia, checking progress in learning a foreign language, and investigating different writing styles of certain authors. Results show that the most frequently used measure is the type-token ratio (TTR), which means the ratio of different words (types) to the total number of words (tokens). The most important problem of TTR and other measures based on TTR is that the more tokens a text has, the less is the TTR value. This has led to the development of other measures; some of them are based on a TTR formula, thus, they do not solve the problem and the calculation result is also affected by the text length. In that case, the texts with different length cannot be compared. Another group of measures rests upon the TTR formula supplemented by a principle of sample forming. These measures solve the problem of the TTR partially or completely, though they often require some extra instruments. Fortunately, these instruments are available on the Internet and demand no particular knowledge on their working principle or in programming. Contemporary researchers tend to use independent measures, because texts mostly have different length and the dependent measures cannot give proper results.

Effects of Linguistic and Demographic Features of Chinese International Students on Placement Test Levels in Higher Education: Logistic Regression

Higher education institutions in the United States provide placement essay tests to ensure international students’ readiness for college courses. The high-stakes nature of placement tests makes educators and researchers seek significant components of differentiating levels of placement tests. This study investigated the prediction of two levels (i.e., low vs. intermediate) of 411 placement test essays written by Chinese international students and examined the influence of linguistic and demographic features on placement test levels through logistic regression. The results show that the type-token ratio (TTR), tokens, college type, and graduate status were significant indicators to differentiate students’ placement test essays. However, several demographic features were not statistically significant. The results may shed light on improving writing skills of Chinese international students who scored intermediate or low in the placement tests.

Types e tokens na aquisição típica de linguagem por sujeitos de 18 a 32 meses falantes do português brasileiro

OBJETIVO: analisar comparativamente a mudança em types e tokens e na taxa de type/token em crianças, de ambos os sexos, entre 18 e 36 meses, falantes nativos do português brasileiro, quanto à classe gramatical e à medida total e segmentar. MÉTODO: foram gravadas e transcritas as falas de 60 crianças com desenvolvimento típico de linguagem em atividades lúdicas com as mães ou professora. A seguir, efetuaram-se os cálculos da taxa de type/token (TTR) e de types e tokens (ty/to), na forma total e segmentar, com sua distribuição em classes gramaticais. Os resultados foram analisados estatisticamente, por meio dos testes t-student e análise de variância (ANOVA). RESULTADOS: o número de types e tokens totais apresentou vantagens sobre o segmentar em termos de descrição de classes gramaticais e na diferenciação estatística das faixas etárias de 18, 24 e 32 meses. Evolutivamente substantivos surgem primeiro do que verbos, advérbios e adjetivos e demais classes gramaticais que se completam até 32 meses. CONCLUSÃO: não houve diferença estatística quanto ao gênero. A análise isolada dos types e tokens é mais efetiva do que a divisão dos mesmos (TTR), tendo validade estatística na diferenciação das faixas etárias de 18, 24 e 32 meses. O número de types e tokens segmentar é menos efetivo na descrição gramatical considerando faixa etária e gênero.

Investigating accommodation in language proficiency interviews using a new measure of lexical diversity

Lexical diversity is an important indicator of language learners’ active vocabulary and how it is deployed. Traditionally it has been measured by the Type-Token Ratio (TTR), the ratio of different words to total words used. Unfortunately, TTR is a function of sample size: larger samples of words will give a lower TTR and even commonly used measures derived from TTR which are claimed to be independent of sample size are problematic. To overcome this, the authors have developed an innovative measure of vocabulary diversity, D, based on mathematically modelling how new words are introduced into larger and larger language samples, and have produced software (vocd) to calculate it. Previous research by the authors into language proficiency interviews (Richards and Malvern, 2000) investigated linguistic and discourse accommodation of teacher-testers using a wide range of student and teacher variables. In a study of teenage learners of French, the aspect of teachers’ language in oral interviews that was most responsive to the ability of their students was lexical diversity. The analysis reported here focuses on this finding in greater depth using the new measure, D. The relationship between D and other measures of foreign language proficiency is investigated, the Ds of students and teachers are compared and the correlations between teachers’ D and students’ proficiency are computed. Results firstly demonstrate the validity of D as a measure of vocabulary diversity and the effectiveness of vocd as a tool to analyse language data. Secondly, with regard to accommodation processes in oral testing, the two teachers did not finely tune their vocabulary diversity to the proficiency of individual students. Instead, each teacher roughly adjusted his or her language to the ability of the class they examined.

Measuring vocabulary diversity using dedicated software

This paper describes software (vocd) that implements a solution to problems encountered in quantifying vocabulary diversity. Researchers in various fields of linguistic enquiry have calculated vocabulary diversity using the ratio of different words (Types) to total words (Tokens)-the Type-Token Ratio (TTR)-or measures derived from it. Such measures are flawed, however, because the values obtained are related to the number of words in the sample. The paper shows how the relationship between TTR and sample size can be described by a new mathematical model, which in turn leads to an innovative method of measuring vocabulary diversity. The software automates measurement from transcripts prepared in a widely used computer-readable set of conventions: the CHAT format of the CHILDES project. Options in vocd are described to show how the user can determine which linguistic items will count as valid types and tokens in the analysis. The new measure is calculated by, first, randomly sampling words from the transcript to produce a curve of the TTR against Tokens for the empirical data. Then the software finds the best fit between this empirical curve and theoretical curves calculated from the model by adjusting the value of a parameter. The parameter, D, is shown to be a valid and reliable measure of vocabulary diversity without the problems of sample size found with previous methods.

Coming to grips with lexical richness in spontaneous speech data

In spontaneous speech data, lexical richness is generally operationalized by measures in which the relation between the number of types and tokens plays a role, of which the Type/Token Ratio (TTR) is the most famous. This article discusses the reliability and validity of different measures of lexical richness in various language data research and computer simulations, and examines the behaviour of these measures in spontaneous speech data of first language and second language children learning Dutch, aged four to seven, compared with their lexical abilities as measured by tests. The results show that neither the validity nor the reliability of the measures were satisfactory, especially the widely applied TTR. Initially, the number of types, or lemmas, and the Guiraud and Uber indexes seem to be adequate measures. However, in later stages of vocabulary acquisition (from 3000 words on) neither is valid. It is suggested that more effective measures of lexical richness might be based not on the distribution of or the relation between the types and tokens, but on the degree of difficulty of the words used, as measured by their (levels of) frequency in daily language input.

Repetition in schizophrenic speech.

Previous research on language with the Type-Token Ratio (TTR) indicates an increased repetitiousness in schizophrenics, specifically in thought-disordered schizophrenics. The present investigation involves an analysis of oral language samples utilizing a variety of repetition measures. Subjects were 40 schizophrenics, 20 affective disorder controls and 22 normal controls. Phrase repetitions were found to be more prevalent among the schizophrenic group, and especially among the TD subgroup, and indices of phrase repetition were positively correlated with dimensions of formal thought disorder. The frequency patterns of repeated words in schizophrenic language reflect a less common, but not unusual lexicon compared to controls, thereby supporting the view that word repetition in schizophrenic utterance is not a consequence of a restricted vocabulary range. The intervals between phrase repetitions were not different for schizophrenics and controls, indicating that simple perseveration is also not a likely explanation. Rather, repetitiousness, a major characteristic of schizophrenic speech, may be due to a unique, yet undetermined mechanism responsible for the language disturbances seen in this disorder.

The Type-Token Ratio and Vocabulary Performance

The Type-Token Ratio (TTR), a measure of lexical diversity, was correlated with four measures of vocabulary performance: the Peabody Picture Vocabulary Test-Revised, the Oral and Picture subtests of the Test of Language Development, and the Test of Written Language. The vocabulary tests were administered to 21 6-, 7-, and 8-yr.-old children from whom language samples were obtained for the application of three Type Token Ratio measures: the TTR-Total for the whole language sample, the TTR-100 for language samples of 100 words, and the CTTR, a procedure intended to be independent of language sample size. One correlation of .45 (between the CTTR and the Oral-TOLD) was significant. More research on validity is necessary to clarify the measurement domain of the TTR, if it is to be useful clinically.

The type--token ratio in schizophrenic disorders: clinical and research value.

Prior research has indicated that the type-token ratio (TTR), a measure of repetition in language, correlates with clinical judgements of thought disorder when spoken language was examined, and differentiates statistically thought-disordered from non-thought-disordered schizophrenics and psychiatric and normal controls. We replicated this finding and examined the clinical sensitivity and specificity of the TTR measure in the diagnosis and in the assessment of thought disorder. The current clinical value of the TTR is limited, but further investigations of the nature of repetition in schizophrenic language are warranted.

Formal thought disorder, the type-token ratio and disturbed voluntary motor movement in schizophrenia.

Little work has been done to determine objective, reliable differences in formal characteristics of the actual utterances of thought-disordered and non-thought-disordered subjects. The type-token ratio (TTR), a quantitative measure of repetition in language, correlated highly with clinical judgments of thought disorder when spoken language was examined, and statistically differentiated thought-disordered from non-thought-disordered schizophrenics and psychiatric and normal controls. Elicited and spontaneous motor abnormalities were associated with reduced TTRs both in schizophrenics and in affective subjects with motor disturbance. The TTR is a reliable, objective indicator of language deviance and thought disorder, and strongly associated with motor disturbances.

Stylistic concomitants of persuasion in conversations

This study focused on differences in verbal style between persuasive conversation and nonpersuasive conversation. It was predicted that verbal choices in the persuasive setting would show reduced diversity and increased qualification. “Diversity” was operationalized with the type‐token ratio (TTR) and mean word length (MWL). “Qualification” was operationalized with the noun‐verb/adjective‐adverb (NVIAA) ratio and the verb/auxiliary verb (VAV) ratio. Significant TTR and directional MWL results were consistent with the prediction of decreased diversity. Significant VAV ratio results supported the prediction of increased modification. Directional NV/AA ratio were not consistent with predictions but were interpreted as indicating that subjects achieved qualification in a lexically restricted way.

Diversity of word usage as a stress indicator in an interview situation

Two experiments testing stress in interview situations were carried out. In the first, 40 college student subjects were asked to tell two different tales, 16 of them were frustrated by the experimentor in between the two tales; 24 told them without interruption. The type-token-ratio (T.T.R.) of the first group was significantly higher before than after treatment, while the second group showed a tendency toward the reverse. In the second experiment, 38 subjects were asked about their motives for choosing psychology as a major; 19 were treated normally; 19 in an unfriendly manner. The T.T.R. of the second group was significantly lower than that of the first group. Both experiments confirmed the following hypothesis: in an interview situation, a positive relationship evokes more differentiated word (nouns) usage than does a negative one.