A construct validation study of time-sensitive word-knowledge measures

  • Abstract
  • References
  • Similar Papers
Abstract
Translate article icon Translate Article Star icon
Take notes icon Take Notes

Abstract Researchers increasingly use response time tasks, along with untimed, accuracy-based tasks, to measure lexical knowledge. While these tasks are set up differently and are inherently time-sensitive, there is currently no empirical evidence on whether they also tap into qualitatively different dimensions of vocabulary knowledge. In this article, we report a validation study involving five timed and untimed measures for assessing 40, 2K–5K frequency English words. One hundred and forty-five learners took (1) an untimed meaning recognition test, (2) an untimed form recall test, (3) a Yes–No response time (RT) test (affording both accuracy and RT measures), and (4) a masked repetition priming task. Confirmatory factor analysis suggests that these measures can be placed on one or two psychometric dimensions, whereby the one-factor solution (suggesting the tests measure one construct) is preferred for parsimony. However, the two-factor model had a marginally stronger predictive validity for explaining self-reported proficiency. Our results highlight the value of incorporating response time measures in vocabulary research although the jury is still out on the theorization of vocabulary knowledge as a one- or two-dimensional construct.

ReferencesShowing 10 of 45 papers
  • Cite Count Icon 282
  • 10.1080/14640748108400797
The Use of Abstract Graphemic Information in Lexical Access
  • Nov 1, 1981
  • The Quarterly Journal of Experimental Psychology Section A
  • Lindsay J Evett + 1 more

  • Open Access Icon
  • PDF Download Icon
  • Cite Count Icon 214
  • 10.3758/s13428-020-01501-5
Realistic precision and accuracy of online experiment platforms, web browsers, and devices
  • Nov 2, 2020
  • Behavior Research Methods
  • Alexander Anwyl-Irvine + 3 more

  • Cite Count Icon 102
  • 10.1037/0278-7393.29.6.1256
Masked Repetition and Phonological Priming Within and Across Modalities.
  • Jan 1, 2003
  • Journal of Experimental Psychology: Learning, Memory, and Cognition
  • Jonathan Grainger + 4 more

  • Cite Count Icon 102
  • 10.1177/0265532212438053
Scoring Yes–No vocabulary tests: Reaction time vs. nonword approaches
  • Jul 15, 2012
  • Language Testing
  • Ana Pellicer-Sánchez + 1 more

  • Cite Count Icon 690
  • 10.1017/cbo9781139858656
Learning Vocabulary in Another Language
  • Oct 31, 2013
  • I S P Nation + 1 more

  • Cite Count Icon 1072
  • 10.1037/0278-7393.10.4.680
Repetition priming and frequency attenuation in lexical access.
  • Jan 1, 1984
  • Journal of Experimental Psychology: Learning, Memory, and Cognition
  • Kenneth I Forster + 1 more

  • Open Access Icon
  • PDF Download Icon
  • Cite Count Icon 11
  • 10.1017/s0272263122000079
A closer look at a marginalized test method: Self-assessment as a measure of speaking proficiency
  • Apr 28, 2022
  • Studies in Second Language Acquisition
  • Paula Winke + 2 more

  • Cite Count Icon 51
  • 10.1177/0265532220932481
A meta-analysis of self-assessment and language performance in language testing and assessment
  • Jun 14, 2020
  • Language Testing
  • Minzi Li + 1 more

  • Cite Count Icon 57
  • 10.1111/ijal.12080
What works better for the learning of concrete and abstract words: teachers'L1 use orL2‐only explanations?
  • Oct 8, 2014
  • International Journal of Applied Linguistics
  • Ting Zhao + 1 more

  • Open Access Icon
  • Cite Count Icon 103
  • 10.1017/s0261444819000326
Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation
  • Sep 25, 2019
  • Language Teaching
  • Norbert Schmitt + 2 more

Similar Papers
  • Research Article
  • Cite Count Icon 12
  • 10.1016/0160-2896(87)90012-2
Comparing subtest profiles of g loadings and correlations with RT measures
  • Oct 1, 1987
  • Intelligence
  • Glen A Smith + 1 more

Comparing subtest profiles of g loadings and correlations with RT measures

  • PDF Download Icon
  • Research Article
  • Cite Count Icon 4
  • 10.7202/1060907ar
L2 Vocabulary Knowledge and L2 Listening Comprehension: A Structural Equation Model
  • Jun 18, 2019
  • Canadian Journal of Applied Linguistics
  • Yang Li + 1 more

The current study investigates how second language (L2) listening comprehension is associated with three dimensions of L2 vocabulary knowledge: size, depth, and fluency. Vocabulary knowledge tests administered to 290 participants measured L2 auditory vocabulary size, depth, and fluency. Afterward, participants took an International English Language Testing System (IELTS) listening test that measured L2 listening comprehension. Using a structural equation modeling technique, we found that all three dimensions of vocabulary knowledge are significant predictors of L2 listening comprehension. Size of auditory vocabulary in the L2 has the strongest predictive power over L2 listening comprehension. The results of the current study offer useful pedagogical implications for improving L2 listening comprehension.

  • PDF Download Icon
  • Research Article
  • Cite Count Icon 4
  • 10.3390/ijerph20021204
Quick on Your Feet: Modifying the Star Excursion Balance Test with a Cognitive Motor Response Time Task
  • Jan 10, 2023
  • International Journal of Environmental Research and Public Health
  • Russell K Lowell + 8 more

The Star Excursion Balance Test (SEBT) is a common assessment used across clinical and research settings to test dynamic standing balance. The primary measure of this test is maximal reaching distance performed by the non-stance limb. Response time (RT) is a critical cognitive component of dynamic balance control and the faster the RT, the better the postural control and recovery from a postural perturbation. However, the measure of RT has not been done in conjunction with SEBT, especially with musculoskeletal fatigue. The purpose of this study is to examine RT during a SEBT, creating a modified SEBT (mSEBT), with a secondary goal to examine the effects of muscular fatigue on RT during SEBT. Sixteen healthy young male and female adults [age: 20 ± 1 years; height: 169.48 ± 8.2 cm; weight: 67.93 ± 12.7 kg] performed the mSEBT in five directions for three trials, after which the same was repeated with a response time task using Blazepod™ with a random stimulus. Participants then performed a low-intensity musculoskeletal fatigue task and completed the above measures again. A 2 × 2 × 3 repeated measures ANOVA was performed to test for differences in mean response time across trials, fatigue states, and leg reach as within-subjects factors. All statistical analyses were conducted in JASP at an alpha level of 0.05. RT was significantly faster over the course of testing regardless of reach leg or fatigue state (p = 0.023). Trial 3 demonstrated significantly lower RT compared to Trial 1 (p = 0.021). No significant differences were found between fatigue states or leg reach. These results indicate that response times during the mSEBT with RT is a learned skill that can improve over time. Future research should include an extended familiarization period to remove learning effects and a greater fatigue state to test for differences in RT during the mSEBT.

  • Research Article
  • Cite Count Icon 29
  • 10.1097/00003446-199904000-00005
A comparison of response time and word recognition measures using a word-monitoring and closed-set identification task.
  • Apr 1, 1999
  • Ear and hearing
  • Carol Mackersie + 2 more

The primary purpose of this study was to investigate the possibility of improving speech recognition testing sensitivity by incorporating response time measures as a metric. Two different techniques for obtaining response time were compared: a word-monitoring task and a closed-set identification task. Recordings of the Modified Rhyme Test were used to test 12 listeners with normal hearing. Data were collected using a word-monitoring and a closed-set identification task. Response times and percent correct scores were obtained for each task using signal to noise ratios (SNRs) of -3, 0, +3, +6, +9, and +12 dB. Both response time and percent correct measures were sensitive to changes in SNR, but greater sensitivity was found with the percent correct measures. Individual subject data showed that combining response time measures with percent correct scores improved test sensitivity for the monitoring task, but not for the closed-set identification task. The best test sensitivity was obtained by combining percent correct and response time measures for the monitoring task. Such an approach may hold promise for future clinical applications.

  • Research Article
  • 10.59271/s45741.025.0346.3
أهمية الوعي المورفولوجي في تطوير المعرفة المعجمية لدى تلاميذ السلك الإعدادي: مقاربة لسانية - معرفية
  • Jan 1, 2025
  • International Uni-Scientific Research Journal of Humanities and Social Sciences
  • Hamza Ghoura + 2 more

This study aims to determine the nature of the correlational relationship between morphological awareness and vocabulary knowledge. Its sub-objectives are firstly, to verify the existence of a statistically significant correlation between the components of morphological awareness and the dimensions of vocabulary knowledge, and secondly, to identify the nature of the impact of morphological awareness on learners' vocabulary size. The researchers administered a test kit consisting of three main tests for morphological awareness and three other tests for the dimensions of vocabulary knowledge. This was done to measure the interactive relationship between morphological awareness and its impact on developing the vocabulary size of third-year preparatory school students. A descriptive approach was used to achieve the research objectives. The results of the study showed statistically significant correlations between the components of morphological awareness and the dimensions of vocabulary knowledge among learners. The findings also emphasized the effective role that morphological awareness, in its various forms, plays in developing and enhancing learners' vocabulary size. The study tests revealed that understanding the established relationship between the components of morphological awareness and the dimensions of vocabulary knowledge, along with realizing their relationships and manifestations, can provide a correct direction for formulating educational curricula and programs. This understanding can also inform the building of appropriate and sound foundations and strategies for the educational content adopted in school books, in a manner consistent with cognitive linguistic strategies to enhance the vocabulary size of preparatory school students.

  • Research Article
  • Cite Count Icon 32
  • 10.1080/13803395.2015.1089979
Dispersion in cognitive functioning: Age differences over the lifespan
  • Nov 30, 2015
  • Journal of Clinical and Experimental Neuropsychology
  • Nathalie Mella + 2 more

ABSTRACTIntroduction: A growing body of research suggests that intraindividual variability (IIV) may bring specific information on cognitive functioning, additional to that provided by the mean. The present paper focuses on dispersion, that is IIV across tasks, and its developmental trend across the lifespan. Method: A total of 557 participants (9–89 years) were administered a battery of response time (RT) tasks and of working memory (WM) tasks. Dispersion was analyzed separately for the two types of tasks. Results: Dispersion across RT tasks showed a U-shaped age differences trend, young adults being less variable than both children and older adults. Dispersion across WM tasks (using accuracy scores) presented an opposite developmental trend. A cluster analysis revealed a group of individuals showing relatively little dispersion and good overall performance (faster in RTs and better in WM), contrasted with a group of individuals showing a large dispersion in the RT tasks as well as poorer overall performance. All young adults were grouped in the first cluster; children and older adults were distributed in both clusters. Conclusion: It is concluded that (a) across-task IIV is relatively large in the entire sample and should not be neglected, (b) children and older adults show a larger dispersion than young adults, but only as far as the RT tasks are concerned, (c) variability in RTs and variability in WM performance do not reflect the same phenomenon.

  • Research Article
  • Cite Count Icon 29
  • 10.1111/1469-8986.3850777
The lateralized readiness potential and response kinetics in response‐time tasks
  • Sep 1, 2001
  • Psychophysiology
  • J Toby Mordkoff + 1 more

Previous studies have found that the magnitude of the lateralized readiness potential (LRP) at the time of response initiation is constant across spontaneous variations in response time in both cued and uncued, speeded tasks. Other studies have found that the LRP is also unaffected by instructed changes in peak response force and time to peak force in cued, self-paced tasks, but that the LRP is sensitive to instructed changes in force gain rate in uncued, self-paced tasks. The present study examined the LRP in an uncued, speeded task as a function of response time and several measures of response kinetics. The magnitude of the LRP at the time of electromyographic onset was constant across spontaneous variations in all measures. The peak of the contingent negative variation did vary as a function of peak response force and integrated force to peak, but not response time. These findings support the idea that the LRP in speeded tasks is a selective, on-line index of the preparation associated with using a particular hand, and is not an index of the elements of motor programming that determine subsequent response kinetics.

  • PDF Download Icon
  • Research Article
  • Cite Count Icon 13
  • 10.1038/s41398-018-0262-z
Genome-wide association study reveals novel genetic locus associated with intra-individual variability in response time
  • Oct 4, 2018
  • Translational Psychiatry
  • Ari Pinar + 11 more

Intra-individual response time variability (IIRTV) is proposed as a viable endophenotype for many psychiatric disorders, particularly attention-deficit hyperactivity disorder (ADHD). Here we assessed whether IIRTV was associated with common DNA variation genome-wide and whether IIRTV mediated the relationship between any associated loci and self-reported ADHD symptoms. A final data set from 857 Australian young adults (489 females and 368 males; Mage = 22.14 years, SDage = 4.82 years) who completed five response time tasks and self-reported symptoms of ADHD using the Conners’ Adult ADHD Rating Scale was used. Principal components analysis (PCA) on these response time measures (standard deviation of reaction times and the intra-individual coefficient of variation) produced two variability factors (labelled response selection and selective attention). To understand the genetic drivers of IIRTV we performed a genome-wide association analysis (GWAS) on these PCA-derived indices of IIRTV. For the selective attention variability factor, we identified one single-nucleotide polymorphism (SNP) attaining genome-wide significance; rs62182100 in the HDAC4 gene located on chromosome 2q37. A bootstrapping mediation analysis demonstrated that the selective attention variability factor mediated the relationship between rs62182100 and self-reported ADHD symptoms. Our findings provide the first evidence of a genome-wide significant SNP association with IIRTV and support the potential utility of IIRTV as a valid endophenotype for ADHD symptoms. However, limitations of this study suggest that these observations should be interpreted with caution until replication samples become available.

  • Research Article
  • Cite Count Icon 8
  • 10.1080/23279095.2018.1555161
Establishing supplementary response time validity indicators in the Word Memory Test (WMT) and directions for future research
  • Feb 6, 2019
  • Applied Neuropsychology: Adult
  • Tomer Elbaum + 4 more

Response time (RT) measures in the Word Memory Test (WMT) offer to complement information derived from conventional accuracy measures. The current study aimed to validate the findings of Lupu, Elbaum, Wagner, and Braw in which RT variability was assessed, for the first time, in the WMT. A secondary aim was to suggest directions for the future research of RT measures in Forced-Choice Recognition Memory Performance Validity Tests (FCRM-PVTs). The study utilized a simulation research design, with participants performing the WMT’s immediate recognition (IR) subtest (N = 59). Mean RTs and a scale which combines accuracy and RT measures, but not variability in RTs, possessed adequate discrimination capacity. Enhanced discrimination capacity was found after discarding the first items’ RTs, though the IR-subtest‘s accuracy measure still showed superiority as a stand-alone validity indicator. The promise of RT measures may, therefore, lie in their ability to illuminate speed-accuracy tradeoffs among examinees with border-zone accuracy scores in FCRM-PVTs. They should, therefore, be regarded as complementary to conventional accuracy measures.

  • Research Article
  • 10.1016/j.apm.2014.09.004
Online dispatching and parallel processing algorithms for saving money in systems with heterogeneous, single-buffered, speed-scalable processors
  • Sep 16, 2014
  • Applied Mathematical Modelling
  • Rashid Khogali + 2 more

Online dispatching and parallel processing algorithms for saving money in systems with heterogeneous, single-buffered, speed-scalable processors

  • Research Article
  • Cite Count Icon 40
  • 10.1016/j.ijhydene.2010.04.139
A comparison of test methods for the measurement of hydrogen sensor response and recovery times
  • Jun 3, 2010
  • International Journal of Hydrogen Energy
  • L Boon-Brett + 3 more

A comparison of test methods for the measurement of hydrogen sensor response and recovery times

  • Research Article
  • Cite Count Icon 111
  • 10.1111/j.1469-8986.2004.00147.x
Attention allocation in the dual-task paradigm as measured through behavioral and psychophysiological responses.
  • Dec 17, 2003
  • Psychophysiology
  • Canan Karatekin + 2 more

We investigated attention allocation in a dual-task paradigm using behavioral and pupillary measures. We used an auditory digit span (DS) and a simple visual response time (RT) task. Participants were administered four conditions in which they performed neither task (no-task), a single task (DS or RT only), or both tasks (dual). Dependent variables were DS accuracy, RT, and task-evoked pupillary responses (TEPRs) to digits as estimates of mental effort. Participants maintained almost the same level of DS accuracy on dual as on DS only and sacrificed speed on the RT task. As expected, TEPRs increased linearly with memory load in both DS only and dual. Although TEPRs were initially higher in dual than in DS only, the slope of the increase was shallower in dual. Results suggest that TEPRs can elucidate mechanisms of attention allocation by distinguishing between effectiveness (level of behavioral performance) and efficiency (the costs of that performance in mental effort).

  • Research Article
  • 10.29333/aje.2016.125a
The Impact of Incidental and Intentional L2 Vocabulary Learning on Depths and Breadth Dimensions of Vocabulary Knowledge
  • Oct 1, 2016
  • Anatolian Journal of Education
  • Seyyed Rasool Mirghasempour Ahmadi

Through the introduction of different dimensions of vocabulary knowledge, various studies attempted to examine numerous effective factors on these dimensions. The present study aimed to show the effects of different vocabulary learning styles through extensive and intensive reading programs on depth and breadth aspects of vocabulary knowledge. To achieve this goal, 45 sophomore undergraduate students of English language teaching and translation in Payam-e-Noor University participated in this study. Initially, in order to homogenize the learners based on their level of language proficiency, MEPT was administered. Then, by measuring the mean and standard deviation of participants’ scores, the number of participants reduced to 35. The ultimate subjects’ scores on the reading comprehension items of MEPT show that they are all at the intermediate level of reading ability. Participants divided into three experimental groups randomly: two groups were in the extensive reading program with different form-focused and meaning-focused tasks as incidental vocabulary learning style. And the third group was in the intensive reading program as intentional vocabulary learning style. Participants in these experimental groups read long stories or passages per week with ten goal-oriented words. After 8 weeks, Word Associates Test (WAT) and Vocabulary Knowledge Scale (VKS) were administered to measure the acquired knowledge of new words and also, determining the effects of various learning styles on different dimensions of vocabulary knowledge. The results of Paired-samples and Independent T-tests revealed that both incidental and intentional groups developed in the period between the pre- and post-test, but, there was a significant difference between the effects of incidental vocabulary learning in the form of ER program and intentional vocabulary learning in the form of IR program. Moreover, there was a significant difference between the effects of the form-focused and meaning-focused task.

  • Research Article
  • 10.5750/bjll.v10i0.1270
The Impact of Incidental and Intentional L2 Vocabulary Learning on Depths and Breadth Dimensions of Vocabulary Knowledge
  • Dec 13, 2017
  • The Buckingham Journal of Language and Linguistics
  • Seyyed Rasool Mirghasempoor Ahmadi

Through the introduction of different dimensions of vocabulary knowledge, various studies attempted to examine numerous effective factors on these dimensions. The present study aimed to show the effects of different vocabulary learning styles through extensive and intensive reading programs on depth and breadth aspects of vocabulary knowledge. To achieve this goal, 45 sophomore undergraduate students of English language teaching and translation in Payam-e-Noor University participated in this study. Initially, in order to homogenize the learners based on their level of language proficiency, MEPT was administered. Then, by measuring the mean and standard deviation of participants’ scores, the number of participants reduced to 35. The ultimate subjects’ scores on the reading comprehension items of MEPT show that they are all at the intermediate level of reading ability. Participants divided into three experimental groups randomly: two groups were in the extensive reading program with different form-focused and meaning-focused tasks as incidental vocabulary learning style. And the third group was in the intensive reading program as intentional vocabulary learning style. Participants in these experimental groups read long stories or passages per week with ten goal-oriented words. After 8 weeks, Word Associates Test (WAT) and Vocabulary Knowledge Scale (VKS) were administered to measure the acquired knowledge of new words and also, determining the effects of various learning styles on different dimensions of vocabulary knowledge. The results of Paired-samples and Independent T-tests revealed that both incidental and intentional groups developed in the period between the pre- and post-test, but, there was a significant difference between the effects of incidental vocabulary learning in the form of ER program and intentional vocabulary learning in the form of IR program. Moreover, there was a significant difference between the effects of the form-focused and meaning-focused task.

  • Research Article
  • Cite Count Icon 1
  • 10.5430/wjel.v7n1p50
The Impact of Incidental and Intentional L2 Vocabulary Learning on Depths and Breadth Dimensions of Vocabulary Knowledge
  • Mar 31, 2017
  • World Journal of English Language
  • Seyyed Rasool Mirghasempoor Ahmadi

Through the introduction of different dimensions of vocabulary knowledge, various studies attempted to examinenumerous effective factors on these dimensions. The present study aimed to show the effects of different vocabularylearning styles through extensive and intensive reading programs on depth and breadth aspects of vocabularyknowledge. To achieve this goal, 45 sophomore undergraduate students of English language teaching and translationin Payam-e-Noor University participated in this study. Initially, in order to homogenize the learners based on theirlevel of language proficiency, MEPT was administered. Then, by measuring the mean and standard deviation ofparticipants’ scores, the number of participants reduced to 35. The ultimate subjects’ scores on the readingcomprehension items of MEPT show that they are all at the intermediate level of reading ability. Participantsdivided into three experimental groups randomly: two groups were in the extensive reading program with differentform-focused and meaning-focused tasks as incidental vocabulary learning style. And the third group was in theintensive reading program as intentional vocabulary learning style. Participants in these experimental groups readlong stories or passages per week with ten goal-oriented words. After 8 weeks, Word Associates Test (WAT) andVocabulary Knowledge Scale (VKS) were administered to measure the acquired knowledge of new words and also,determining the effects of various learning styles on different dimensions of vocabulary knowledge. The results ofPaired-samples and Independent T-tests revealed that both incidental and intentional groups developed in the periodbetween the pre- and post-test, but, there was a significant difference between the effects of incidental vocabularylearning in the form of ER program and intentional vocabulary learning in the form of IR program. Moreover, therewas a significant difference between the effects of the form-focused and meaning-focused task.

More from: Applied Linguistics
  • Research Article
  • 10.1093/applin/amaf072
Generative Artificial Intelligence and Language Teaching
  • Nov 4, 2025
  • Applied Linguistics
  • Zeynep Arslan

  • Addendum
  • 10.1093/applin/amaf070
Correction to: A Sociopolitical Agenda for TESOL Teacher Education
  • Oct 7, 2025
  • Applied Linguistics

  • Research Article
  • 10.1093/applin/amaf068
Sociocultural Theory and Second Language Developmental Education
  • Oct 7, 2025
  • Applied Linguistics
  • Ehean Kim

  • Research Article
  • 10.1093/applin/amaf067
Motivation, Autonomy, and Emotions in Foreign Language Learning: A Multi-Perspective Investigation in Hungary
  • Oct 6, 2025
  • Applied Linguistics
  • Guangxiang Liu

  • Research Article
  • 10.1093/applin/amaf062
Rethinking translanguaging: (Trans)bordering, spatiality, and academic discourse socialization in a graduate TESOL classroom
  • Oct 3, 2025
  • Applied Linguistics
  • Gengqi Xiao

  • Research Article
  • 10.1093/applin/amaf055
Claims of teacher learning and retrospective tracing of claimed events in video-mediated transnational virtual exchange interactions
  • Oct 1, 2025
  • Applied Linguistics
  • Gülşah Uyar + 1 more

  • Research Article
  • 10.1093/applin/amaf064
Exploring artificial intelligence appraisal: Appraisal patterns in GPT-generated and human-authored book reviews
  • Sep 30, 2025
  • Applied Linguistics
  • Guangyuan Yao + 1 more

  • Research Article
  • 10.1093/applin/amaf053
‘She’s [the] only fish!’: Investigating the idiom topic transitions of elementary-to-intermediate proficiency Japanese learners of English
  • Sep 29, 2025
  • Applied Linguistics
  • David O’Reilly

  • Research Article
  • 10.1093/applin/amaf056
An exploration of psychological capital resources, mindfulness, L2 grit, and L2 achievement: An explanatory mixed-methods study
  • Sep 22, 2025
  • Applied Linguistics
  • Mirosław Pawlak + 3 more

  • Research Article
  • 10.1093/applin/amaf050
Analysis of sociodemographic and linguistic influences on verbal fluency performance in Basque and Catalan bilingual adults
  • Aug 11, 2025
  • Applied Linguistics
  • Laiene Olabarrieta-Landa + 5 more

Save Icon
Up Arrow
Open/Close
  • Ask R Discovery Star icon
  • Chat PDF Star icon

AI summaries and top papers from 250M+ research sources.

Search IconWhat is the difference between bacteria and viruses?
Open In New Tab Icon
Search IconWhat is the function of the immune system?
Open In New Tab Icon
Search IconCan diabetes be passed down from one generation to the next?
Open In New Tab Icon