Non-adaptive Testing Research Articles

Meta-Analysis of the Effects of Computerized Adaptive Testing on the Motivation and Emotion of Examinees

Abstract: Findings on the motivational and emotional effects of computerized adaptive testing (CAT) compared to nonadaptive testing (NCAT) have been heterogeneous and inconclusive so far. This meta-analysis aims to determine the specific effects of CAT better by analyzing several moderator variables related to the test, the examinee, and the test setting. To be included in the meta-analysis, studies needed to be original, correlational, quasi-experimental, or experimental studies in English, German, or Chinese, and they needed to compare CAT to NCAT regarding motivational and/or emotional states. A three-level random-effects model with motivation, negative emotion, and positive emotion as dependent variables was estimated. The risk of bias and publication bias were examined. Twenty-seven studies with a total of 190 effect sizes were analyzed. The overall effect of CAT was not significant for motivation and negative emotion and significant for positive emotion after publication bias correction. The moderator analysis showed significant moderator effects for “test speededness” on motivation and positive emotion, “instantaneous feedback on given responses” and “possibility of response revision” on positive emotion, and “average difficulty of presented items” on negative emotion. For CAT configurations, it is recommended to avoid speededness, present easy items, allow for response revision, and provide feedback.

Medical students' perceptions of a novel international adaptive progress test

Between 2018–2021, eight European medical schools took part in a study to develop a medical knowledge Online Adaptive International Progress Test. Here we discuss participants’ self-perception to evaluate the acceptability of adaptive vs non-adaptive testing. Study participants, students from across Europe at all stages of undergraduate medical education with varying levels of prior experience with progress testing, sat remotely invigilated tests using the online QuizOne® platform. Participants completed online feedback questionnaires on their experiences and perceptions of adaptive and non-adaptive tests. Overall satisfaction with the organisation and delivery of remote online tests was high regardless of previous experience with progress testing, differences in stages, programmes, and to some degree language. In statements probing the appropriateness of the level and the length of testing, differences were observed between adaptive and non-adaptive tests. There was a high level of agreement that the adaptive test was a good measure of personal knowledge and increased participants’ motivation for study. Students’ self-perception of the assessment is an important factor in evaluation of acceptability of the exam and its further development. In our study, the adaptive test algorithm adjusted the level of difficulty for the individual student in real-time, leading to positive perceptions of the length of the test and promoting students’ engagement. The assessment increases student motivation for learning and in turn, has the potential to improve their performance.

A negative binomial approximation in group testing

AbstractWe consider the problem of group testing (pooled testing), first introduced by Dorfman. For nonadaptive testing strategies, we refer to a nondefective item as “intruding” if it only appears in positive tests. Such items cause misclassification errors in the well-known COMP algorithm and can make other algorithms produce an error. It is therefore of interest to understand the distribution of the number of intruding items. We show that, under Bernoulli matrix designs, this distribution is well approximated in a variety of senses by a negative binomial distribution, allowing us to understand the performance of the two-stage conservative group testing algorithm of Aldridge.

Property Testing of the Boolean and Binary Rank

We present algorithms for testing if a (0,1)-matrix M has Boolean/binary rank at most d, or is 𝜖-far from having Boolean/binary rank at most d (i.e., at least an 𝜖-fraction of the entries in M must be modified so that it has rank at most d). For the Boolean rank we present a non-adaptive testing algorithm whose query complexity is \(\tilde {O}\left (d^{4}/ \epsilon ^{6}\right )\). For the binary rank we present a non-adaptive testing algorithm whose query complexity is O(22d/𝜖2), and an adaptive testing algorithm whose query complexity is O(22d/𝜖). All algorithms are 1-sided error algorithms that always accept M if it has Boolean/binary rank at most d, and reject with probability at least 2/3 if M is 𝜖-far from having Boolean/binary rank at most d.

Rapid, large-scale, and effective detection of COVID-19 via non-adaptive testing

Pooling of samples can increase lab capacity when using Polymerase chain reaction (PCR) to detect diseases such as COVID-19. However, pool testing is typically performed via an adaptive testing strategy which requires a feedback loop in the lab and at least two PCR runs to confirm positive results. This can cost precious time. We discuss a non-adaptive testing method where each sample is distributed in a prescribed manner over several pools, and which yields reliable results after one round of testing. More precisely, assuming knowledge about the overall incidence rate, we calculate explicit error bounds on the number of false positives which scale favourably with pool size and sample multiplicity. This allows for hugely streamlined PCR testing and cuts in detection times for a large-scale testing scenario. A viable consequence of this method could be real-time screening of entire communities, frontline healthcare workers and international flight passengers, for example, using the PCR machines currently in operation.

Latent Variable Modeling and Adaptive Testing for Experimental Cognitive Psychopathology Research.

The adaptation of experimental cognitive tasks into measures that can be used to quantify neurocognitive outcomes in translational studies and clinical trials has become a key component of the strategy to address psychiatric and neurological disorders. Unfortunately, while most experimental cognitive tests have strong theoretical bases, they can have poor psychometric properties, leaving them vulnerable to measurement challenges that undermine their use in applied settings. Item response theory-based computerized adaptive testing has been proposed as a solution but has been limited in experimental and translational research due to its large sample requirements. We present a generalized latent variable model that, when combined with strong parametric assumptions based on mathematical cognitive models, permits the use of adaptive testing without large samples or the need to precalibrate item parameters. The approach is demonstrated using data from a common measure of working memory-the N-back task-collected across a diverse sample of participants. After evaluating dimensionality and model fit, we conducted a simulation study to compare adaptive versus nonadaptive testing. Computerized adaptive testing either made the task 36% more efficient or score estimates 23% more precise, when compared to nonadaptive testing. This proof-of-concept study demonstrates that latent variable modeling and adaptive testing can be used in experimental cognitive testing even with relatively small samples. Adaptive testing has the potential to improve the impact and replicability of findings from translational studies and clinical trials that use experimental cognitive tasks as outcome measures.

An App for Identifying Children at Risk for Developmental Problems Using Multidimensional Computerized Adaptive Testing: Development and Usability Study.

BackgroundThe use of multidomain developmental screening tools is a viable strategy for pediatric professionals to identify children at risk for developmental problems. However, a specialized multidimensional computer adaptive testing (MCAT) tool has not been developed to date.ObjectiveWe developed an app using MCAT, combined with Multidimensional Screening in Child Development (MuSiC) for toddlers, to help patients and their family members or clinicians identify developmental problems at an earlier stage.MethodsWe retrieved 75 item parameters from the MuSiC literature item bank for 1- to 3-year-old children, and simulated 1000 person measures from a normal standard distribution to compare the efficiency and precision of MCAT and nonadaptive testing (NAT) in five domains (ie, cognitive skills, language skills, gross motor skills, fine motor skills, and socioadaptive skills). The number of items saved and the cutoff points for the tool were determined and compared. We then developed an app for a Web-based assessment.ResultsMCAT yielded significantly more precise measurements and was significantly more efficient than NAT, with 46.67% (=(75-40)/75) saving in item length when measurement differences less than 5% were allowed. Person-measure correlation coefficients were highly consistent among the five domains. Significantly fewer items were answered on MCAT than on NAT without compromising the precision of MCAT.ConclusionsDeveloping an app as a tool for parents that can be implemented with their own computers, tablets, or mobile phones for the online screening and prediction of developmental delays in toddlers is useful and not difficult.

Adaptive Minimax Testing for Circular Convolution

Given observations from a circular random variable contaminated by an additive measurement error, we consider the problem of minimax optimal goodness-of-fit testing in a non-asymptotic framework. We propose direct and indirect testing procedures using a projection approach. The structure of the optimal tests depends on regularity and ill-posedness parameters of the model, which are unknown in practice. Therefore, adaptive testing strategies that perform optimally over a wide range of regularity and ill-posedness classes simultaneously are investigated. Considering a multiple testing procedure, we obtain adaptive i.e. assumption-free procedures and analyse their performance. Compared with the non-adaptive tests, their radii of testing face a deterioration by a log-factor. We show that for testing of uniformity this loss is unavoidable by providing a lower bound. The results are illustrated considering Sobolev spaces and ordinary or super smooth error densities.

Optimal tests for parameter breaking process in conditional quantile models

This paper proposes efficient tests for quantile parameter instability in parametric and semiparametric setups. In each setup, various types of unstable parameter processes are examined such as single structural break, multiple structural breaks, and random parameters, and the optimal test is suggested for each unstable process. In a parametric model, tick-exponential family of distributions is used to construct the likelihood ratio tests. The suggested tests have the best asymptotic weighted average power if the likelihood function is correctly specified and are asymptotically correct-sized even under misspecification. In a semiparametric setup in which the underlying distribution is unknown but is treated as an infinite-dimensional nuisance parameter, we show that semiparametric efficient tests are adaptive if the error term is conditionally iid. Non-adaptive efficient tests are suggested under weaker conditions as well. Monte Carlo simulation shows that the proposed tests have better finite sample powers than the existing tests under various circumstances.

Application of a multidimensional computerized adaptive test for a Clinical Dementia Rating Scale through computer-aided techniques

BackgroundWith the increasingly rapid growth of the elderly population, individuals aged 65 years and above now compose 14% of Taiwanese citizens, thereby making Taiwanese society an aged society. A leading factor that affects the elderly population is dementia. A method of precisely and efficiently examining patients with dementia through multidimensional computer adaptive testing (MCAT) to accurately determine the patients’ stage of dementia needs to be developed. This study aimed to develop online MCAT that family members can use on their own computers, tablets, or smartphones to predict the extent of dementia for patients responding to the Clinical Dementia Rating (CDR) instrument.MethodsThe CDR was applied to 366 outpatients in a hospital in Taiwan. MCAT was employed with parameters for items across eight dimensions, and responses were simulated to compare the efficiency and precision between MCAT and non-adaptive testing (NAT). The number of items saved and the estimated person measures was compared between the results of MCAT and NAT, respectively.ResultsMCAT yielded substantially more precise measurements and was considerably more efficient than NAT. MCAT achieved 20.19% (= [53 − 42.3]/53) saving in item length when the measurement differences were less than 5%. Pearson correlation coefficients were highly consistent among the eight domains. The cut-off points for the overall measures were − 1.4, − 0.4, 0.4, and 1.4 logits, which was equivalent to 20% for each portion in percentile scores. Substantially fewer items were answered through MCAT than through NAT without compromising the precision of MCAT.ConclusionsDeveloping a website that family members can use on their own computers, tablets, and smartphones to help them perform online screening and prediction of dementia in older adults is useful and manageable.

Testing Is More Desirable When It Is Adaptive and Still Desirable When Compared to Note-Taking.

Testing is a well-established desirable difficulty. Yet there are still some open issues regarding the benefits of testing that need to be addressed. First, the possibility to increase its benefits by adapting the sequence of test questions to the learners’ level of knowledge has scarcely been explored. In view of theories that emphasize the benefits of adapting learning tasks to learner knowledge, it is reasonable to assume that the common practice of providing all learners with the same test questions is not optimal. Second, it is an open question as to whether the testing effect prevails if stronger control conditions than the typical restudy condition are used. We addressed these issues in an experiment with N = 200 university students who were randomly assigned to (a) adaptive testing, (b) non-adaptive testing, or note-taking (c) without or (d) with focus guidance. In an initial study phase, all participants watched an e-lecture. Afterward, they processed its content according to their assigned conditions. One week later, all learners took a posttest. As main results, we found that adaptive testing yielded higher learning outcomes than non-adaptive testing. These benefits were mediated by the adaptive learners’ higher testing performance and lower perceived cognitive demand during testing. Furthermore, we found that both testing groups outperformed the note-taking groups. Jointly, our results show that the benefits of testing can be enhanced by adapting the sequence of test questions to learners’ knowledge and that testing can be more effective than note-taking.

The Effect of Increased Cognitive Demand on Auditory Processing Assessment.

Auditory processing (AP) is commonly regarded as the perceptual processing of auditory information in the central nervous system. However, the degree to which higher level cognitive processes are involved in AP or its disorders is contentious. Furthermore, there is little evidence regarding the effects of nonauditory cognitive processes on the various tests of AP in common clinical usage and thus on clinical diagnoses of auditory processing disorder. To determine the effects of increased cognitive demand, generated by using a dual-task paradigm, on performance on different AP tests and types of AP tests in common clinical usage. In addition, to investigate the relationship between executive function and changes in AP test performance associated with increased cognitive demand. Counterbalanced repeated measures design, with assessment of AP test performance both on its own and in a dual-task paradigm designed to increase cognitive demand. Twenty-nine young adults, with no reported hearing, learning, language or attention difficulties, English as first language, and hearing and middle-ear status within normal limits. Testing was completed within a single 90-min session. A selection of standard AP tests, representing both adaptive and nonadaptive tests, as well as tests employing difference scores, was administered. These were Competing Sentences Test, Dichotic Digits Test, Frequency Pattern Test (nonadaptive tests); and Listening in Spatialized Noise-Sentences test, conditions "same-voice, 0°", "different-voice, 0°", and "same-voice, 90°" (adaptive tests), from which the difference scores "talker advantage" and "spatial advantage" were also derived. Each AP test was completed on its own (alone condition), and simultaneously with a visually presented task (dual-task condition). Executive function was assessed using the phonemic subtest of the Verbal Fluency Test. Nonparametric statistical test procedures were used. All five AP measures obtained from the nonadaptive tests showed a significant performance decrement in the dual-task condition compared with the alone condition, with one exception because of a strong ceiling effect. By contrast, none of the three AP measures obtained from the adaptive tests showed a significant performance decrement in the dual-task condition. Furthermore, neither of the two AP measures based on difference scores showed a significant performance decrement, but this finding simply reflects the lack of significant decrements in the relevant raw scores. Consistent with past reports of associations between executive function and AP performance, a significant positive correlation was found between executive function scores and performance on the Dichotic Digits Test. However, there were no significant correlations between executive function scores and changes in AP test scores between alone and dual-task conditions. Performance on commonly used nonadaptive tests of AP was significantly compromised by the increased cognitive demand resulting from the dual-task paradigm. By contrast, performance on AP measures obtained by adaptive test procedures was not significantly affected. Further investigation of the resilience to increased cognitive demand of the adaptive tests used here, and other adaptive tests of AP, is warranted. Results from this study support the further development of computerized adaptive tests of AP for use in clinical test batteries.

Adaptive versus non-adaptive MRI-guided radiation treatments for pancreatic cancer: A dosimetric study.

336 Background: Combination MRI and radiation therapy systems enable magnetic resonance image guided radiation therapy (MR-IGRT). MR-IGRT allows clear visualization of the target and organs at risk (OARs) allowing for dose adaptation. Using adaptive MR-IGRT with Cobalt-60 for stereotactic body radiation therapy (SBRT) in locally advanced pancreatic cancer (LAPC), we hypothesized that MR-IGRT would improve dose to pancreatic tumor without increasing doses to OARs. Methods: Ten LAPC patients received five fraction SBRT with a total dose of 33-40 Gy. For each fraction, the original plan dose was compared to the dose that would be delivered if the original radiotherapy plan was applied to the anatomy that day (non-adaptive). An adaptive plan was then created for each fraction. The plan was re-optimized based on the anatomy as seen on the daily MRI and re-normalized so the volume of the PTV receiving 100% of the prescription (PTV100) would be 90%. Both the non-adaptive and adaptive doses to the target volume and the OARs were recorded to evaluate the value of adaptation. We used a paired t-test to compare PTV100 between the adaptive and non-adaptive techniques and Chi2 tests to compare the probability of dose constraint failures for OARs. Results: Adaptive MR-IGRT improved target coverage. Mean PTV100 for adaptive and non-adaptive techniques was 89.9% [88.4-90.4] and 78.4% [27.3-96.6] respectively, p = 0.0017. There were no statistically significant differences for violations of dose constraints of OARs using adaptive vs. non-adaptive techniques. Point maxima violations above 35 Gy to duodenum occurred in 6 adaptive fractions (renormalized to 90%) vs. 12 non-adaptive fractions (p = 0.118); to stomach in 8 adaptive fractions vs. 9 non-adaptive fractions (p = 0.790), and to bowel in 9 adaptive fractions vs. 6 non-adaptive fractions (p = 0.401). When adapting, attention must be paid to other OARs in the area: Spinal cord point maxima were violated in 4 adaptive fractions. Conclusions: This study demonstrates that adaptive techniques significantly increase SBRT dose delivered to LAPC without significantly increasing dose constraint violations to OARs. Adaptive MR-IGRT may allow for further SBRT dose escalations.

On Finding a Subset of Non-Defective Items from a Large Population

In this paper, we derive mutual information based upper bounds on the number of nonadaptive group tests required to identify a given number of “non-defective” items from a large population containing a small number of “defective” items. In the asymptotic regime with the population size $N \rightarrow \infty$ , to identify $L$ nondefective items out of a population containing $K$ defective items, our results show that $\frac{C_s K}{1-o(1)} (\Phi (\alpha _0, \beta _0) + o(1))$ measurements are sufficient when the tests are reliable. Here, $C_s$ is a constant independent of $N, K$ , and $L$ , and $\Phi (\alpha _0, \beta _0)$ is a bounded function of $\alpha _0 \triangleq \lim _{N\rightarrow \infty } \frac{L}{N-K}$ and $\beta _0 \triangleq \lim _{N\rightarrow \infty } \frac{K}{N-K}$ . In contrast, the necessary number of tests using the conventional approach of first identifying the $K$ defective items and picking the required number of nondefective items from the complement set grows with $N$ as $O\left(K \log N \right)$ . We also derive upper bounds on the number of tests under both dilution and additive noise models. Our results are obtained under a very general sparse signal model, by virtue of which, they are also applicable to other important sparse signal based applications such as compressive sensing.

A new technique to measure online bullying: online computerized adaptive testing

BackgroundWorkplace bullying has been measured in many studies to investigate mental health issues. None uses online computerized adaptive testing (CAT) with cutting points to report bully prevalence at workplace.ObjectiveTo develop an online CAT to examine person being bullied and verify whether item response theory-based CAT can be applied online for nurses to measure exposure to workplace bullying.MethodsA total of 963 nurses were recruited and responded to the 22-item Negative Acts Questionnaire-Revised (NAQ-R). All non-adaptive testing (NAT) items were calibrated with the Rasch rating scale model. Three scenarios (i.e., NAT, CAT, and the randomly selected method to NAT) were manipulated to compare their response efficiency and precision by comparing (i) item length for answering questions, person measure, (ii) correlation coefficients, (iii) paired t tests, and (iv) estimated standard errors (SE) between CAT and the random to its counterpart of NAT.ResultsThe NAQ-R is a unidimensional construct that can be applied for nurses to measure exposure to workplace bullying on CAT. CAT required fewer items (=8.9) than NAT (=22, an efficient gain of 60% =1–8.9/22). Nursing measures derived from both tests (CAT and the random to NAT) were highly correlated (r = 0.93 and 0.96) and their measurement precisions were not statistically different (the percentage of significant count number less than 5%) as expected, but CAT earns smaller person measure SE than the random scenario. The prevalence rate for nurses was 1.5% (=15/963) when cutting points set at −0.7 and 0.7 logits.ConclusionThe CAT-based NAQ-R reduces respondents’ burden without compromising measurement precision and increases endorsement efficiency. The online CAT is recommended for assessing nurses using the criteria at −0.7 and 0.7 (or <30 and <60 in summed score) to identify bully grade as one of the three levels (high, moderate, and low). The bullied nurse can get help from a psychiatrist or a mental health expert at an earlier stage.

Efficient Algorithms for Noisy Group Testing

Group-testing refers to the problem of identifying (with high probability) a (small) subset of $D$ defectives from a (large) set of $N$ items via a “small” number of “pooled” tests (i.e., tests that have a positive outcome if at least one of the items being tested in the pool is defective, else have a negative outcome). For ease of presentation in this paper, we focus on the regime when $D = \mathcal {O}(N^{1- {\delta }})$ for some $ {\delta }> 0$ . The tests may be noiseless or noisy , and the testing procedure may be adaptive (the pool defining a test may depend on the outcome of a previous test), or non-adaptive (each test is performed independent of the outcome of other tests). A rich body of the literature demonstrates that $\Theta (D\log (N))$ tests are information-theoretically necessary and sufficient for the group-testing problem, and provides algorithms that achieve this performance. However, it is only recently that reconstruction algorithms with computational complexities that are sub-linear in $N$ have started being investigated. In the scenario with adaptive tests with noisy outcomes, we present the first scheme that is simultaneously order-optimal (up to small constant factors) in both the number of tests and the decoding complexity ( $ \mathcal {O}\left ({D\log (N)}\right )$ in both the performance metrics). The total number of stages of our adaptive algorithm is “small” ( $ \mathcal {O}\left ({\log (D)}\right )$ ). Similarly, in the scenario with non-adaptive tests with noisy outcomes, we present the first scheme that is simultaneously near-optimal in both the number of tests and the decoding complexity (via an algorithm that requires $ \mathcal {O}\left ({D\log (D)\log (N)}\right )$ tests and has a decoding complexity of ${\mathcal{ O}}(D(\log N+\log ^{2}D))$ . Finally, we present an adaptive algorithm that only requires two stages, and for which both the number of tests and the decoding complexity scale as ${\mathcal{ O}}(D(\log N+\log ^{2}D))$ . For all three settings, the probability of error of our algorithms scales as $ \mathcal {O}\left ({1/(poly(D)}\right )$ . For each of the statements mentioned earlier about the order of the number of measurements, decoding complexity, and probability of error, we provide explicitly computed “small” universal factors in our theorem statements.

Which Statistic Should Be Used to Detect Item Preknowledge When the Set of Compromised Items Is Known?

Benefiting from item preknowledge is a major type of fraudulent behavior during educational assessments. Belov suggested the posterior shift statistic for detection of item preknowledge and showed its performance to be better on average than that of seven other statistics for detection of item preknowledge for a known set of compromised items. Sinharay suggested a statistic based on the likelihood ratio test for detection of item preknowledge; the advantage of the statistic is that its null distribution is known. Results from simulated and real data and adaptive and nonadaptive tests are used to demonstrate that the Type I error rate and power of the statistic based on the likelihood ratio test are very similar to those of the posterior shift statistic. Thus, the statistic based on the likelihood ratio test appears promising in detecting item preknowledge when the set of compromised items is known.

Mobile Online Computer-Adaptive Tests (CAT) for Gathering Patient Feedback in Pediatric Consultations

Background: Few studies have used online patient feedback from smartphones for computer adaptive testing (CAT). Objective: We developed a mobile online CAT survey procedure and evaluated whether it was more precise and efficient than traditional non-adaptive testing (NAT) when gathering patient feedback about their perceptions of interaction with a physician after a consultation. Method: Two hundred proxy participants (parents or guardians) were recruited to respond to twenty 5-point questions (the P4C_20 scale) about perceptions of doctor-patient and doctor-family interaction in clinical pediatric consultations. Through the parameters calibrated using a Rasch partial credit model (PCM) and a Rasch rating scale model (RSM), two paired comparisons of empirical and simulation data were administered to calculate and compare the efficiency and precision of CAT and NAT in terms of shorter item length and fewer counts of difference number ratio (< 5%) using independent t tests. An online CAT was designed using two modes of PCM and RSM for use in clinical settings. Results: The graphical online CAT for smartphones used by the parents or guardians of pediatric hospital patients was more efficient and no less precise than NAT. Conclusions: CAT-based administration of the P4C_20 substantially reduced respondent burden without compromising measurement precision.

Testlet-Based Multidimensional Adaptive Testing.

Multidimensional adaptive testing (MAT) is a highly efficient method for the simultaneous measurement of several latent traits. Currently, no psychometrically sound approach is available for the use of MAT in testlet-based tests. Testlets are sets of items sharing a common stimulus such as a graph or a text. They are frequently used in large operational testing programs like TOEFL, PISA, PIRLS, or NAEP. To make MAT accessible for such testing programs, we present a novel combination of MAT with a multidimensional generalization of the random effects testlet model (MAT-MTIRT). MAT-MTIRT compared to non-adaptive testing is examined for several combinations of testlet effect variances (0.0, 0.5, 1.0, and 1.5) and testlet sizes (3, 6, and 9 items) with a simulation study considering three ability dimensions with simple loading structure. MAT-MTIRT outperformed non-adaptive testing regarding the measurement precision of the ability estimates. Further, the measurement precision decreased when testlet effect variances and testlet sizes increased. The suggested combination of the MTIRT model therefore provides a solution to the substantial problems of testlet-based tests while keeping the length of the test within an acceptable range.

Simulation study of activities of daily living functions using online computerized adaptive testing.

BackgroundComputer adaptive testing (CAT) of the activities of daily living (ADL) functions is required (i) to reveal the advantages of using an efficient and accurate estimation method, (ii) to determine the cutpoint for classifying ADL strata in patients with stroke, and (iii) to evaluate the feasibility of online CAT used in clinical settings for smartphones.MethodsNormally standardized distributions of ADL measurements were simulated using item parameters from published papers. We retrieved item parameters of the combined Barthel Index and Frenchay Activities Index from the literature (the 23-item comprehensive ADL [CADL] and 34-item ADL scales) and simulated three 1000-person measures from a normal standard CAT distribution: [i] CADL (CADL-CAT), [ii] ADL (ADL-CAT), and [iii] NAT (Non-Adaptive Testing). The cutpoints of ADL person strata were determined using a norm-reference method. Maximum a posteriori estimation, expected a posteriori estimation, and maximum likelihood estimation (MAP) were used to compare the Pearson correlation coefficients and different number ratios of paired measures yielded by CAT and NAT. The number of items and the cutpoints for the scale were separately determined.ResultsWe found that (i) correlation coefficients for the three CAT-estimated measures were 0.77 (CADL), 0.93 (Male ADL), and 0.93 (Female ADL) compared with their NAT counterparts. Different number ratios of person-paired measures between CAT and NAT for the three scales were all less than 5 %, indicating no difference exists between CAT and NAT. However, CAT might be 66 % more efficient than NAT. (ii) The estimated cutpoints of T scores (i.e., with a mean of 50 and a standard deviation of 10) were 45, 55, and 65 (e.g., separating person ADL function to four strata with not active, fairly active, active, and very active). (iii) An available-for-download online ADL-CAT APP for clinical practice was demonstrated.ConclusionsAn online ADL-CAT APP using the MAP method was created and used on smartphones to classify ADL strata in patients with stroke.Electronic supplementary materialThe online version of this article (doi:10.1186/s12911-016-0370-8) contains supplementary material, which is available to authorized users.