Use Of Response Times Research Articles

Reflecting on the use of response times to index linguistic knowledge in SLA

Abstract Response times (RTs) have become ubiquitous in second language acquisition (SLA) research, providing empirical evidence for the theorization of the language learning process. Recently, there have been discussions of some fundamental psychometric properties of RT data, including, but not limited to, their reliability and validity. In this light, we take a step back to reflect on the use of RT data to tap into linguistic knowledge in SLA. First, we offer a brief overview of how RT data are most commonly used as vocabulary and grammar measures. We then point out three key limitations of such uses, namely that (a) RT data can lack substantive importance without considerations of accuracy, (b) RT differences may or may not be a satisfactory psychometric individual difference measure, and (c) some tasks designed to elicit RT data may not be sufficiently fine-grained to target specific language processes. Our overarching goal is to enhance the awareness among SLA researchers of these issues when interpreting RT results and stimulate research endeavors that delve into the unique properties of RT data when used in our field.

What insights can response times provide for education research?

ABSTRACT The main objective of this methodological article is to discuss the contribution of response times as a tool in education research. The use of response times in research is largely a legacy of the work carried out in cognitive psychology, which has made it possible to describe the cognitive processes involved in information processing. In education, research that incorporates response times into its methodological design often has two main objectives: 1) evaluate the automation of basic learning and 2) infer the cognitive processes involved in academic learning, such as working memory and inhibitory control. This article addresses and discusses specific research designs that use response times across various academic disciplines and instructional levels, offering a comprehensive overview of the potential applications of response times in education research. In light of this overview, utilizing response times in education research not only emerges as relevant but also serves to complement existing methodological tools. For instance, such designs can aid in identifying the relative difficulty of different types of learning, understanding the underlying reasons for such variations, and offering valuable insights for developing learning sequences and pedagogical interventions that are more consistent with learning processes.

Predicting response time on self-report rating scale assessments of noncognitive constructs.

Methodological studies of response time on noncognitive assessments have separately demonstrated the relevance of content trait level and response styles as predictive factors. In this paper we examine the simultaneous relevance of both factors as well as the potential for omitted predictor bias when ignoring either factor. Using response time data from several different noncognitive assessments, we demonstrate how a multilevel regression model that attends simultaneously to content and response style factors leads to consistent findings that support the simultaneous relevance of both factors. The average effects of response style consistently emerge as stronger, although also show greater respondent-level variability, possibly due to the multiple different underlying causes of response style behavior. Some implications for the use of response times in noncognitive measurement are considered.

Comparison of the response time-based effort-moderated IRT model and three-parameter logistic model according to computerized adaptive test performances: a simulation study

Depending on the developments in technology and information, paper-pencil tests leave their place for computerized adaptive tests (CATs). CAT is widely used in the field of health, mainly in psychiatry. Many item response theory models have been proposed in the literature regarding the use of response time focusing on item difficulty and personal characteristics by ignoring the multidimensional interactions, therefore these results may cause bias in estimates of individual ability levels. The present simulation study was conducted to compare the performance of CAT applications of the effort-moderated item response theory (EM-IRT) model, which is based on response time, and the three-parameter logistic (3PL) model. While simulating CAT with the EM-IRT model and the 3PL model, the hybrid method was used for ability estimation and maximum Fisher information (MFI) was used for item selection. The CAT process proceeded until the standard error of the estimation was <0.3 and <0.5, or all items in the item bank were used. The number of individuals was specified as 1000, while the number of items was changed to 50, 100, and 250. All six scenarios were repeated 1000 times. With the increase in the number of items and the decrease in the standard error as a stopping criterion, consistent results were obtained with true ability levels in both methods. The CAT with the EM-IRT model estimated true ability level slightly lower than CAT with the 3PL model. The EM-IRT model enables measuring the response time that could yield additional data to the physician about the mental and cognitive condition of the patient. The CAT method can be a promising method of telemedicine in the era of the pandemic.

Understanding examinees’ item responses through cognitive modeling of response accuracy and response times

Understanding the cognitive processes, skills and strategies that examinees use in testing is important for construct validity and score interpretability. Although response processes evidence has long been included as an important aspect of validity (i.e., Standards for Educational and Psychological Tests, 1999), relevant studies are often lacking, especially in large scale educational and psychological testing. An important method for studying response processes involves explanatory mathematical modeling of item responses and item response times from variables that represent sources of cognitive complexity. For many item types, examinees may differ in strategies applied to responding to items. Mixture class item response theory models can identify latent classes of examinees with different processes, skills and strategies based on their pattern of item responses. This study will illustrate the use of response times in conjunction with explanatory item response theory models and mixture models, to provide information relevant to test validity and, hence, to score interpretations.

Changes in the Speed-Ability Relation Through Different Treatments of Rapid Guessing.

As researchers in the social sciences, we are often interested in studying not directly observable constructs through assessments and questionnaires. But even in a well-designed and well-implemented study, rapid-guessing behavior may occur. Under rapid-guessing behavior, a task is skimmed shortly but not read and engaged with in-depth. Hence, a response given under rapid-guessing behavior does bias constructs and relations of interest. Bias also appears reasonable for latent speed estimates obtained under rapid-guessing behavior, as well as the identified relation between speed and ability. This bias seems especially problematic considering that the relation between speed and ability has been shown to be able to improve precision in ability estimation. For this reason, we investigate if and how responses and response times obtained under rapid-guessing behavior affect the identified speed-ability relation and the precision of ability estimates in a joint model of speed and ability. Therefore, the study presents an empirical application that highlights a specific methodological problem resulting from rapid-guessing behavior. Here, we could show that different (non-)treatments of rapid guessing can lead to different conclusions about the underlying speed-ability relation. Furthermore, different rapid-guessing treatments led to wildly different conclusions about gains in precision through joint modeling. The results show the importance of taking rapid guessing into account when the psychometric use of response times is of interest.

Comparing Test‐Taking Behaviors of English Language Learners (ELLs) to Non‐ELL Students: Use of Response Time in Measurement Comparability Research

In this report, we demonstrate use of differential response time (DRT) methodology, an extension of differential item functioning methodology, for examining differences in how students from different backgrounds engage with assessment tasks. We analyze response time data from a digitally delivered mathematics assessment to examine timing differences between English language learner (ELL) and non‐ELL student groups. When matched on the total sum scores of the studied item form, results showed that ELLs spent a significantly longer time on most items compared to the non‐ELLs who performed similarly on the test form. When matched on the total response time, results showed that ELL students spent a significantly longer time on items in the first half of the form but a shorter time on items in the second half. This research demonstrates the usefulness of DRT methodology in gaining insights about the differential engagement of students with assessment tasks.

Targeting data collection in games based assessment

Educational game performance data has the potential to allow new types of complex, procedural skills to be assessed. However, prior work has shown that gameplay data do not readily align to existing assessment validation paradigms, and game performance scores are difficult to use for proficiency testing. A new assessment paradigm that can cope with the nature of gameplay data has not emerged. In this paper, we uncovered a range of structural issues in data collection caused by, and potentially solved by, the engineered environments in games. Choice and the iterative nature of games were found to allow curriculum specialisation. We found evidence that early attempts at new games are less reliable and perhaps best discarded, and we propose a solution to weight scores to reflect novelty in repeated tasks. We found capturing the effect of competitor or collaborator ability on performance challenging but propose the potential for bots to resolve this. Finally, we also investigated the use of response time as a proxy for ability. The physical measure of time proved difficult and potentially unfair to use, but we propose a possible stochastic treatment of speed that could allow scoring some skills in some games using response time.

Maximum information per time unit designs for continuous online item calibration.

Previous designs for online calibration have only considered examinees' responses to items. However, the use of response time, a useful metric that can easily be collected by a computer, has not yet been embedded in calibration designs. In this article we utilize response time to optimize the assignment of new items online, and accordingly propose two new adaptive designs. These are the D-optimal per expectation time unit design (D-ET) and the D-optimal per time unit design (D-T). The former method uses the conditional maximum likelihood estimation (CMLE) method to estimate the expected response times, while the latter employs the nonparametric k-nearest-neighbour method to predict the response times. Simulations were conducted to compare the two new designs with the D-optimal online calibration design (D design) in the context of continuous online calibration. In addition, a preliminary study was carried out to evaluate the performance of CMLE prior to its application in D-ET. The results showed that, compared to the D design, the D-ET and D-T designs saved response time and accrued larger calibration information per time unit, without sacrificing item calibration precision.

Male Smokers’ Behavioral and Brain Responses to Deviant Cigarette-Related Stimuli in a Two-Choice Oddball Paradigm

Abstract. Experimental studies on smoking and response-inhibition capacity have revealed inconsistent findings, which might be due to differences in sensitivity of the behavioral paradigms used. Here we aimed to replicate the impaired response inhibition in male smokers that was found in a previous study using a two-choice oddball task. This task enables the use of response times as index of inhibition capacity and equalizes the response requirement for the different trial types. In addition, we measured event-related brain potentials to explore the nature of the cognitive processes underlying the behavioral difference. Smokers (n = 19) and non-smokers (n = 19) were asked to make a different response to frequent standard stimuli (cigarette-unrelated pictures) than to infrequent deviant stimuli (cigarette-related pictures). Compared to non-smokers, smokers took a longer time to respond to deviant but not standard stimuli. In addition, smokers, but not non-smokers, displayed a smaller N2 amplitude to deviant than standard stimuli, and only the non-smokers showed larger P3 amplitudes to deviant compared to standard stimuli. Moreover, the response time (RT) measure was differentially correlated with N2 and P3 amplitudes in smokers and non-smokers. The joint results support the notion of deviant cognitive processes in smokers compared to non-smokers that are either directly or indirectly related to response-inhibition capacity.

On the Strategic Use of Response Times

People naturally respond quickly when they have a clear preference and slowly when they are close to indifference. The question is whether others exploit this tendency to infer private information. In two-stage bargaining experiments, we observe that the speed with which buyers reject sellers’ first-stage offers decreases with the size of the immediate foregone surplus. This should allow sellers to infer buyers’ values from observable response times (RT), creating an incentive for buyers to manipulate their RT. We identify distinct conditions under which subjects do, and do not, exhibit such strategic behavior. These results provide the first insight into the possible use of RT as a strategic variable.

Optimizing the Use of Response Times for Item Selection in Computerized Adaptive Testing

Despite common operationalization, measurement efficiency of computerized adaptive testing should not only be assessed in terms of the number of items administered but also the time it takes to complete the test. To this end, a recent study introduced a novel item selection criterion that maximizes Fisher information per unit of expected response time (RT), which was shown to effectively reduce the average completion time for a fixed-length test with minimal decrease in the accuracy of ability estimation. As this method also resulted in extremely unbalanced exposure of items, however, a-stratification with b-blocking was recommended as a means for counterbalancing. Although exceptionally effective in this regard, it comes at substantial costs of attenuating the reduction in average testing time, increasing the variance of testing times, and further decreasing estimation accuracy. Therefore, this article investigated several alternative methods for item exposure control, of which the most promising was a simple modification of maximizing Fisher information per unit of centered expected RT. The key advantage of the proposed method is the flexibility in choosing a centering value according to a desired distribution of testing times and level of exposure control. Moreover, the centered expected RT can be exponentially weighted to calibrate the degree of measurement precision. The results of extensive simulations, with item pools and examinees that are both simulated and real, demonstrate that optimally chosen centering and weighting values can markedly reduce the mean and variance of both testing times and test overlap, all without much compromise in estimation accuracy.

Use of Response Time for Measuring Cognitive Ability

The purpose of this paper is to review some of the key literature on response time as it has played a role in cognitive ability measurement, providing a historical perspective as well as covering current research. We discuss the speed-level distinction, dimensions of speed and level in cognitive abilities frameworks, speed–accuracy tradeoff, approaches to addressing speed–accuracy tradeoff, analysis methods, particularly item response theory-based, response time models from cognitive psychology (ex-Gaussian function, and the diffusion model), and other uses of response time in testing besides ability measurement. We discuss several new methods that can be used to provide greater insight into the speed and level aspects of cognitive ability and speed–accuracy tradeoff decisions. These include item-level time limits, the use of feedback (e.g., CUSUMs), explicit scoring rules that combine speed and accuracy information (e.g., count down timing), and cognitive psychology models. We also review some of the key psychometric advances in modeling speed and level, which combine speed and ability measurement, address speed–accuracy tradeoff, allow for distinctions between response times on items responded to correctly and incorrectly, and integrate psychometrics with information-processing modeling. We suggest that the application of these models and tools is likely to advance both the science and measurement of human abilities for theory and applications.

Spatial analysis of aquifer response times for radial flow processes: Nondimensional analysis and laboratory-scale tests

• Nondimensional model of groundwater pumping, injection and recovery is analyzed. • Time scales for pumping, injection and recovery processes to reach steady state is considered. • Closed form expressions for the time scale, including spatial dependence, are given. • Time scales are equivalent for pumping, injection and recovery. • Theoretical predictions are validated with new laboratory measurements. A fundamental concept in groundwater hydrology is the notion of steady state, or equilibrium conditions. When a system at some initial steady state condition is perturbed by pumping, a transient cone of depression will develop and the system will approach a new steady state condition. Understanding the time scale required for the transient process to occur is of practical interest since it would help practitioners decide whether to use a steady state model or a more complicated transient model. Standard approaches to estimate the response time use simple scaling relationships which neglect spatial variations. Alternatively, others define the response time to be the amount of time taken for the difference between the transient and steady state solutions to fall below some arbitrary tolerance level. Here, we present a novel approach and use the concept of mean action time to predict aquifer response time scales in a two-dimensional radial geometry for pumping, injection and recovery processes. Our approach leads to relatively simple closed form expressions that explicitly show how the time scale depends on the hydraulic parameters and position. Furthermore, our dimensionless framework allows us to predict the response time scales for a range of applications including small scale laboratory problems and large scale field problems. Our analysis shows that the response time scales vary spatially, but are equivalent for pumping, injection and associated recovery processes. Furthermore, the time scale is independent of the pumping or injection flow rate. We test these predictions in a laboratory scale aquifer and find that our physical measurements corroborate the theoretical predictions.

웹 기반 농업생산환경 모니터링 시스템 시범구축 및 성능평가

In developed countries such as USA and Europe, agricultural monitoring system is developed and utilized in various fields in order to predict crop yield, observe weather conditions and anomaly, categorize crop fields, and calculate areas for each crop. These system is Web Map Service(WMS) which utilizes open source and commercial softwares, and various information collected from remote sensing data are provided. This study will utilize tools such as Geoserver, ArcGIS server, which are widely used to monitor agricultural production, to publish Map Server and Web Application Server. This enables performance test study for future agricultural production monitoring system by making use of response time and data transfer test. When tested in identical condition Geoserver showed a better result in response time and data transfer for performance test.

How forgiveness affects processing time: Mediation by rumination about the transgression

Previous work has demonstrated that individuals high (vs. low) in forgiveness are faster to rate their current thoughts and feelings toward their transgressor (i.e., their state forgiveness), but the underlying mechanism is still unclear. The present study examined whether individual differences in rumination about the transgression would mediate the association between forgiveness and response time. Participants (N=767) completed measures of trait forgiveness, rumination about the transgression, and state forgiveness (while response time was unobtrusively recorded). Trait forgiveness was significantly negatively associated with response time and this effect was mediated by lower rumination about the transgression. Results support the use of response time as an unobtrusive measure of forgiveness-related processes and further clarify the role of rumination about the transgression in the forgiveness–response time link.

Aging effects in cueing tasks as assessed by the ideal observer: peripheral cues.

Previous aging and cueing studies suggest that automatic orienting driven by peripheral cues is preserved with aging; however, inconsistencies can be found. One issue might be the use of response times (RT) to assess cueing effects (invalid RT--valid RT), which, in many cases, may not have clear quantitative predictions. We propose an ideal observer (IO) analysis of accuracy estimating participants' internal value of cue validity, or weight, which should equal the actual cue validity. The weight measures the use of information provided by the cue and is insensitive to variations in set size and difficulty, thus potentially providing advantages to RT. Older (n = 54) and younger (n = 58) participants performed a yes/no detection task of a two-dimensional (2-D) Gaussian (60 ms). Square peripheral precues (150 ms) indicated likely target locations (70% valid) across two or six locations (set sizes). For cueing effects, (valid--invalid hit rates), younger participants had set-size effects (larger cueing effects for set size 6), while older participants did not. The opposite pattern was found for weights (younger: no set-size effects, older: set-size effects) due to the IO predicting larger cueing effects for larger set sizes. Comparisons to the ideal weight (cue validity) suggested that older participants used the cue information effectively with set size 2 (as or more so than younger participants), but not with set size 6. These results suggest that attentional deficits from aging in peripheral cueing tasks may only arise as difficulty increases, such as larger set sizes.

Response Time to Craving-Item Ratings as an Implicit Measure of Craving-Related Processes.

Drug craving is typically measured through explicit ratings of craving levels. We examined response time to craving ratings as an implicit measure of craving processes in cigarette smokers. Response time and inter-item variability were investigated as potential indices of certainty in craving ratings. Cigarette smokers, categorized as tobacco dependent or nondependent, completed multiple cue-reactivity sessions with smoking and neutral cues. After each cue, craving level and response time was assessed. Significant inverted-U relationships emerged between craving level and both response time and inter-item variability across conditions, sessions, and groups. Faster response times and less inter-item variability emerged after neutral relative to smoking cues for nondependent smokers and after smoking relative to neutral cues for dependent smokers. Response time provided incremental validity beyond craving level in predicting dependence. Results support use of response time as an implicit measure of craving processes and further distinguish craving experiences between dependent and nondependent smokers.

Liking More Means Doing More: Dispositional Attitudes Predict Patterns of General Action.

Dispositional attitudes are an individual difference in the tendency to form positive versus negative attitudes. As positive (negative) attitudes promote active (inactive) responses to stimuli, we predicted that dispositional attitudes would be positively correlated with patterns of general action. In Study 1, participants reported all activities they engaged in during a 1-week period using a structured time use survey. Dispositional attitudes were positively correlated with the number of unique behaviors participants engaged in and with the total number of behaviors reported for the entire week. Study 2 replicated Study 1 using a free response time use survey. Overall, the results demonstrated that dispositional attitudes predict general action, such that the tendency to form positive (negative) attitudes predicts the tendency to engage in many (few) behaviors in daily life. This pattern occurred for both low effort and high effort behaviors. Implications for understanding activity patterns are discussed.

How do you fake a personality test? An investigation of cognitive models of impression-managed responding

Because faking poses a threat to the validity of personality measures, research has focused on ways of detecting faking, including the use of response times. However, the applicability and validity of these approaches are dependent upon the actual cognitive process underlying faking. This study tested three competing cognitive models in order to identify the process underlying faking and to determine whether response time patterns are a viable method of detecting faking. Specifically, we used a within-subjects manipulation of instructions (respond honestly, make a good impression, make a specific impression) to examine whether the distribution of response times across response scale options (e.g., disagree, agree) could be used to identify faking on the NEO PI-R. Our results suggest that individuals reference a schema of an ideal respondent when faking. As a result, response time patterns such as the well-known inverted-U cannot be used to identify faking.