Difficult Items Research Articles

Physical body experiences questionnaire simplified for active aging (PBE-QAG): Rasch validation.

To validate the Physical Body Experiences Questionnaire simplified for Active aGing (PBE-QAG) with Rasch Measurement Theory. PBE-QAG measures body awareness during physical activity. Community-dwelling adults were recruited at the Minnesota State Fair, Highland Fest, and in the Brain Body Mind Lab (University of Minnesota). They completed demographic, clinical, and behavioral questionnaires and the PBE-QAG, which has 12 items, with scoring options ranging between 0 (totally true) and 4 (totally false). A lower total PBE-QAG score on reflects better body awareness. We validated the structural validity of PBE-QAG in community-dwelling older adults, and in community-dwelling adults (18-99 years of age). We also performed a pilot structural validity in community-dwelling adults with chronic stroke. We evaluated item and person fit, targeting, unidimensionality, person separation reliability, differential item functioning for demographic and clinical characteristics, principal component of residuals, and local item dependence. We obtained unidimensionality and item fit after deleting and rescoring items in older adults (n = 133), adults (n = 530), and adults with chronic stroke (n = 36). In community-dwelling adults, 7 participants did not fit the model (1.13%). There was minimal floor (5.28%), no ceiling effect (0.00%), and no local item dependence or differential item functioning. The person mean location was -1.77±1.22 logits. PBE-QAG demonstrated good item and person fit, but the targeting is off. Therefore, the current version of PBE-QAG is not recommended for use in community-dwelling adults. We encourage further validation of PBE-QAG by adding more difficult items. We also recommend evaluating the PBE-QAG in a larger group of adults with stroke.

The Contribution of Visual and Auditory Working Memory and Non-Verbal IQ to Motor Multisensory Processing in Elementary School Children.

Although cognitive abilities have been shown to facilitate multisensory processing in adults, the development of cognitive abilities such as working memory and intelligence, and their relationship to multisensory motor reaction times (MRTs), has not been well investigated in children. Thus, the aim of the current study was to explore the contribution of age-related cognitive abilities in elementary school-age children (n = 75) aged 5-10 years, to multisensory MRTs in response to auditory, visual, and audiovisual stimuli, and a visuomotor eye-hand co-ordination processing task. Cognitive performance was measured on classical working memory tasks such as forward and backward visual and auditory digit spans, and the Raven's Coloured Progressive Matrices (RCPM test of nonverbal intelligence). Bayesian Analysis revealed decisive evidence for age-group differences across grades on visual digit span tasks and RCPM scores but not on auditory digit span tasks. The results also showed decisive evidence for the relationship between performance on more complex visually based tasks, such as difficult items of the RCPM and visual digit span, and multisensory MRT tasks. Bayesian regression analysis demonstrated that visual WM digit span tasks together with nonverbal IQ were the strongest unique predictors of multisensory processing. This suggests that the capacity of visual memory rather than auditory processing abilities becomes the most important cognitive predictor of multisensory MRTs, and potentially contributes to the expected age-related increase in cognitive abilities and multisensory motor processing.

Exploring the Role of Process Data Analysis in Understanding Student Performance and Interactive Behavior in a Game-Based Argument Task

In this research, we use a process data analysis approach to gather additional evidence about students’ argumentation skills beyond their performance scores in a computer-based assessment. This game-enhanced scenario-based assessment (named Seaball) included five activities that require students to demonstrate their argumentation skills within a scenario about whether junk food should be sold to students. Our research sample included 104 middle school students. Process data analyses focused on an “Interview” activity in which students explored different locations and interviewed various characters to identify their opinions on the junk food issue and categorize each opinion as pro or con. Students could take various paths to complete the activity. Results indicated that the number of trials students made in the Interview activity predicted their performance on the Interview activity as well as the total Seaball scores. It was also found that most students improved their answers in the Interview activity after receiving automated feedback and making corresponding changes. Besides the connections between student activities and performance, results from analyzing the process data helped us to identify difficult items in the task. We conclude with implications for conducting process data analysis to better assess students’ argumentation skills and to inform task design.

N01 Using Item Response Theory to improve the efficiency of the Crohn and Colitis Knowledge score for patients with Inflammatory Bowel Disease

Abstract Background Patient knowledge varies widely in inflammatory bowel disease (IBD), but knowledge of the disease and its management is essential for patients with chronic disorders and can improve health-related quality of life, coping ability, self-care skills, and adherence and can positively impact several outcomes including disease activity and direct and indirect costs. To assess patients’ knowledge is available the Crohn’s and Colitis Knowledge Score (CCKNOW), but can be biased by subjects’ abilities, attitudes, or other latent traits not considered in its validation process. The purpose of this study was to reassess the index developed for measuring IBD patients’ knowledge using the Item Response Theory (IRT). Methods IBD patients with a recent (<18 months) diagnosis, stable therapy, and no previous IBD-related surgery have been asked to answer the 30 items described in the validation of the CCKNOW questionnaire. Measures of between-item relationships (scalability) and unidimensionality (single major trait) were estimated prior to applying a 2-PL logistic IRT model across all 30 items. To evaluate a questionnaire with improved item information, low discriminatory items were removed by order of discrimination value, with the lowest discriminating item<1.7 removed before the 2-PL model was re-estimated with the remaining items. The process was then repeated twice to remove poorly discriminating and inefficient scaling items. The difficulty parameters have also been estimated in the 2-PL logistic IRT model. Cronbach’s alpha (α) was used as a guide to measuring whole-scale reliability. Results 49 patients (17 females, 21 Crohn’s Disease) were enrolled, with a median (IQR) age of 37 (29-47) years, a median disease duration of 334 (212-426) days and a median number of outpatients visit performed of 1 (1-3). Only 6 subjects involved in the study were graduated; the median level of health-related quality of life (S-IBDQ) was 47,7 (30,3-60,9). About 50% of subjects in both groups were in disease remission. Based on discrimination ability, item number 4, 10, 11, 22, 29, and 30 could be excluded because demonstrated a discrimination ability below 1.7. The Cronbach’s alpha for the 24 items questionnaire indicates a suitable internal consistency (α = 0.84). Based on difficulty parameters, the most difficult items were the 25th and 14th, and the easiest were the 19th and 21st. Conclusion This is the first study systematically to interrogate a commonly used index for patients’ IBD knowledge, using robust psychometric and statistical methodologies. IRT is a valid and robust psychometric methodology, which may be used to analyse patient questionnaires. The revised CCKNOW questionnaire provides optimum trait information.

Item Analysis of Functional Measure Indicators in the National Health and Nutrition Examination Survey: A Cross-sectional Study.

This study aimed to develop a function, activities, and participation measurement scale using the questions on the National Health and Nutrition Examination Survey (NHANES) by applying item response theory approaches. This was a cross-sectional study. Survey data were collected from 2512 participants within the NHANES. Twenty questions from the NHANES physical functioning section were included. Exploratory and confirmatory factor analyses were conducted to evaluate the factorial structure and model fit of the NHANES instrument. A multidimensional Rasch modeling was used to model the response data after confirming its model fit. To assess validity across subgroups, differential item functioning test was conducted with respect to sex and race using the three-dimensional partial credit model. Item fit indices indicated a good fit (root-mean-square error of approximation = 0.049, standardized root-mean-square residual = 0.052, Tucker-Lewis index = 0.975). Rasch analysis indicated that "stoop/crouch/kneel," "stand for long periods," and "push/pull large objects" were the three most difficult items, whereas "walk between rooms," "use fork/knife," and "leisure at home" were the easiest. The measure demonstrated good internal consistency overall (Cronbach α = 0.90). The NHANES function, activities, and participation measure demonstrates sufficient evidence of reliability, internal consistency, and validity in noninstitutionalized community-dwelling population. Complete the self-assessment activity and evaluation online at http://www.physiatry.org/JournalCME. Upon completion of this article, the reader should be able to (1) Recognize the potential benefits of using validated patient-reported functional outcome measures within the publically available National Health and Nutrition Examination Survey (NHANES) data for rehabilitation research; (2) Describe the process of scale development and validation; (3) Identify item difficulty based on the item difficulty distribution; and (4) Identify the applicability of the measure to different population groups based on differential item functioning. Advanced. The Association of Academic Physiatrists is accredited by the Accreditation Council for Continuing Medical Education to provide continuing medical education for physicians.The Association of Academic Physiatrists designates this Journal-based CME activity for a maximum of 1.0 AMA PRA Category 1 Credit(s) ™. Physicians should only claim credit commensurate with the extent of their participation in the activity.

The benefit of retrieval practice on cued recall under stress depends on item difficulty

Retrieval practice, the act of recalling previously studied information, yields greater memory retention than repeated studying (retrieval practice effect). Retrieval practice can also protect memories against the negative effects of stress at retrieval in free recall tests. In cued-recall tests, however, retrieval practice may not provide complete protection against stress-related memory impairments. Here we ask whether these conflicting results may be explained by the relative difficulty of the study materials. Participants (59 men) first studied 40 Swahili-Portuguese word pairs. Half of the pairs were then repeatedly studied while the other half were repeatedly recalled. In each condition, half of the pairs were easy to remember (high in memorability) while the other half were more difficult to remember (low in memorability). Participants returned 1 week later for a final cued-recall test. Half of the participants underwent a stress-induction protocol (modified Socially Evaluated Cold Pressor Test) 25 min before the final test for all 40-word pairs; the other half underwent a control condition. Salivary cortisol and questionnaire responses were used to assess the efficacy of stress induction. Participants recalled more word pairs learned via retrieval practice than via repeated study, replicating previous research. More importantly, the results revealed an interaction such that stress decreased recall for easy items, but increased recall for difficult items that had been successfully retrieved during encoding (conditional analysis). The results suggest that the impact of stress on cued recall depends both on the learning strategy and on the intrinsic difficulty of the to-be-learned materials.

ANALISIS BUTIR SOAL MATERI GEOMETRI KELAS V SD

This research is a descriptive research that aims to determine the quality of the items in the mathematics subject subject Geometry students at SDN 012 Tambak. The subjects in this study were practice questions in class V mathematics. The data from this research were obtained by collecting data in the form of daily math practice questions and answer sheets for class V students. The data analysis technique was quantitative descriptive data analysis. Based on the results of data analysis, in terms of content validity it is a valid item because it has rcount greater than rtabel at a significant level of 5%, in terms of overall reliability it is reliable with a distribution of 4 items of moderate reliability and 1 item of high reliability, in terms of difficulty level has a disproportionate distribution of difficulty level comparisons because there are more easy questions than moderate questions and there are no difficult items so that they are not in accordance with the comparison between easy-medium-difficult questions, and in terms of discriminating power most of them have discriminating power very good item.

The role of daily spoken language on the performance of language tests: The Indonesian experience

AbstractThe performance of cognitive tests is highly dependent on the proficiency of the language in which the tests are administered. Not all Indonesians speak Indonesian daily and many are bilingual. We investigate whether language(s) spoken affects the performance on three language tests in 840 participants ranging in age (16–80) and education (6–22 years). Analysis of covariance followed by Helmert contrasts showed a disadvantage on the Boston Naming Test (BNT) and on the most difficult items of the Token Test for those who do not speak Bahasa daily. Bilinguals did worse on the BNT. Education had a large positive effect on the language tests, age a smaller negative effect. This suggests that besides age and education, the factor of language spoken, either in public or at home, needs to be taken into account when a participant's test scores on the BNT and TT are interpreted and compared with normative data.

Hierarchical Properties and Functional Staging of the Fugl-Meyer Assessment Lower Extremity Scale

<h3>Research Objectives</h3> This study aimed (1) to investigate the hierarchical properties of FMA-LE and Brunnstrom stages, and (2) to develop a functional staging, a graphical plot, of FMA-LE and its relationships to low (0–21), high (22–34) FMA-LE cut-off scores. <h3>Design</h3> Secondary analysis of cross-sectional data. <h3>Setting</h3> Outpatient rehabilitation settings. <h3>Participants</h3> The Locomotor Experience Applied Post-stroke (LEAPS) trial data included 408 patients with stroke with a mean age of 62 years old (SD = 12.7, range = 25 to 98). The mean onset of stroke was 64 days (SD = 8.5, range = 43 to 112). About 55% of the sample were male. The majority (80%) of the sample had an ischemic stroke. Approximately half of the sample had brain lesions in the right hemisphere, followed by the left hemisphere (35%), brainstem (15%), and bilateral (2%). <h3>Interventions</h3> N/A. <h3>Main Outcome Measures</h3> The 17-item FMA-LE assesses lower extremity motor function including reflex activity, movement within and outside synergy patterns, and speed/coordination. It has two subscales: Lower Extremity and Speed/Coordination, which are scored on a 3-level ordinal scale (0 = none; 1 = partial; 2 = full). The total summed score of 34 points indicates normal function. <h3>Results</h3> Overall, the item difficulty hierarchical order of the FMA-LE reflected the Brunnstrom stages. The items from easy to difficult were reflex activity items, followed by items assessing flexor and extensor synergy, and movement with little/no synergies. The ‘normal reflex activity', ‘knee flexion to 90° (hip at 0°)', ‘coordination/speed' items represented the most difficult items. The two-level functional staging was judged to be clinically logical and to provide insight for clinical interpretation of patient progress. We further provided the functional staging for the clinical interpretation across all FMA-LE scores. <h3>Conclusions</h3> Our findings suggest that the FMA-LE items reflect the lower extremity motor recovery stage. This study demonstrates how functional staging can be used to translate a standardized assessment into a useful, evidence-based tool for making clinical practice decisions. <h3>Author(s) Disclosures</h3> The authors declare that there is no conflict of interest.

Analyzing the science achievement test: Perspective of classical test theory and Rasch analysis

Assessment in teaching and learning allows both instructor and students to evaluate the achievement, performance and improve upon it. The achievement test appears as one of the most commonly used assessment tools to determine students' gains in the cognitive domain. The study analysed science achievement test (SAT), a multiple-choice test for grade 10 senior secondary school students. The 50 SAT items were developed in compliance with the Malaysian science curriculum specification. The face and content validity of SAT were validated by experts in the science field. The SAT was administered with 50 students in a pilot study. The collected data was analysed using item analysis based on classical test theory (CTT) and Rasch analysis. In terms of difficulty index (p), item analysis showed that there were five difficult items, 33 moderately difficult items, and 12 easy items. Findings also reported eight items presented as poor items due to the poor discriminating power. The SAT showed a good coefficient of 0.862 for Kuder-Richardson 20 (KR-20) and 0.851 for the split half method. While the Rasch analysis showed a reliable and good separation for both item and person. Besides, the Rasch analysis displayed that there are two items should be refreshed or removed. The study revealed that the SAT was a solid, valid, and reliable tool, which is suitable to be used to measure the student’s achievement in science.

Hierarchical Properties of the Fugl-Meyer Assessment Upper Extremity Scale for People After Stroke

<h3>Research Objectives</h3> This study aimed (1) to investigate the hierarchical properties of FMA-UE and Brunnstrom stages, and (2) to develop a functional staging, a graphical plot, of FMA-UE and its relationships to severe (0–28), moderate (29–42), and mild (43–66) FMA-UE cut-off scores. <h3>Design</h3> Secondary analysis of cross-sectional data. <h3>Setting</h3> Outpatient rehabilitation settings. <h3>Participants</h3> The Locomotor Experience Applied Post-stroke (LEAPS) trial data included 408 patients with stroke with a mean age of 62 years old (SD = 12.7, range = 25 to 98). The mean onset of stroke was 64 days (SD = 8.5, range = 43 to 112). About 55% of the sample were male. The majority (80%) of the sample had an ischemic stroke. Approximately half of the sample had brain lesions in the right hemisphere, followed by the left hemisphere (35%), brainstem (15%), and bilateral (2%). Although most of the sample was White (58%), other racial groups were represented: Black (22%), Asian (13%), Native Hawaiian or other Pacific Islander (5%), and American Indian/Alaska Native (1%). <h3>Interventions</h3> N/A. <h3>Main Outcome Measures</h3> The 33-item FMA-UE has four subsections: (1) shoulder-arm, (2) wrist, (3) hand, and (4) coordination and speed designed to measure impairment from proximal to distal, reflexive to voluntary, and synergistic to isolated voluntary movement. <h3>Results</h3> Overall, the item difficulty hierarchical order of the FMA-UE reflected the Brunnstrom stages. The items from easy to difficult were reflex activity items, followed by items assessing flexor and extensor synergy, movement combining synergies, and movement out of synergy. The ‘normal reflex activity', ‘shoulder flexion 90-180°', ‘coordination/speed' items represented the most difficult items. The three-level functional staging was judged to be clinically logical and to provide insight for clinical interpretation of patient progress. <h3>Conclusions</h3> Our findings suggest that the FMA-UE items reflect the upper extremity motor recovery stage. This study demonstrates how functional staging can be used to translate a standardized assessment into a useful, evidence-based tool for making clinical practice decisions. <h3>Author(s) Disclosures</h3> The authors declare that there is no conflict of interest.

Can the balance evaluation systems test be used to identify system-specific postural control impairments in older adults with chronic neck pain?

Older adults with chronic neck pain (CNP) demonstrate impaired postural control. The Balance Evaluation Systems Test (BESTest) is used to assess systems underlying postural control impairments, but its use in CNP has not been reported. This study assessed whether the BESTest can identify postural control impairments in CNP as well as the level of BESTest item difficulty by Rasch analysis. This cross-sectional study recruited thirty young adults (YOUNG) aged 20-40 years and eighty older adults aged 60 years or older [without neck pain (OLD) = 60, with chronic neck pain (CNP) = 20]. Questionnaires were administered to collect demographic data, intensity of neck pain (VAS), patient's self-rated neck pain and disability (NDI), and balance confidence in daily activities (ABC). The BESTest was used to assess postural control. The CNP group showed the lowest ABC scores. Compared to the YOUNG group, the BESTest score was significantly lower in the OLD group, while the CNP group showed the lowest score, suggesting that balance control deteriorated from the normal aging process and further declined in the CNP group, especially in biomechanical constraints, transitions-anticipatory postural adjustment, and reactive postural response (p < 0.05). Using scores from these three sections, the BESTest was accurate at the cutoff score of 48.5 out of 51 for differentiating the older adults whose daily life are affected by neck problems (using the NDI as a reference) with a high AUC (0.79), sensitivity (72%), and specificity (69%). The Rasch analysis revealed that the Timed Up and Go with dual task test was the most difficult BESTest item for all groups, whereas 14 items showed more difficulty for the CNP group. The BESTest can be used to identify postural control impairments in CNP patients, even those with moderate pain and mild disability with a high level of physical functioning. The combined score of biomechanical constraints, transitions-anticipatory postural adjustment, and reactive postural response domains was suggested for the detection of older adults whose daily lives are affected by neck problems. This will also help clinicians consider the management of neck pain to prevent falls in CNP.

“Development and validation of the ICAP Technology Scale to measure how teachers integrate technology into learning activities”

Previous research investigating the use of technology in school has focused mainly on the frequency of use of digital tools during lessons rather than investigating how technology is integrated with respect to different kinds of learning activities.Since the impact of technology use on learning depends on how it is used and on what activities supported by technology are implemented in lessons, a measurement instrument assessing how technology is integrated into learning activities is necessary to investigate its impact on teaching and learning processes. According to the interactive, constructive, active, and passive (ICAP) framework, which distinguishes four different learning activities based on the level of students' cognitive engagement, we developed the 12-item ICAP Technology Scale (ICAP-TS) that accounts for all four dimensions of technology integration in lessons. We used confirmatory factor analysis to validate the four-factor structure of the ICAP-TS with a sample of 1059 upper-secondary school teachers from Switzerland. We also examined reliability using classical test theory and Rasch model analysis to assess the scale's psychometric characteristics. We then analyzed the associations between the ICAP-TS and a general use frequency measure of 12 educational technologies to test the criterion validity. The results confirmed the four-factor structure of the ICAP-TS and revealed good instrument accuracy. The most difficult items to endorse are those describing the integration of technology into interactive learning activities. Furthermore, all 12 items significantly correlated with the frequency of use of 12 educational technologies. We recommend the ICAP-TS as a short and reliable measurement scale for assessing how technology is integrated into lessons, considering different learning activities based on the ICAP theoretical model.

Development of a Problematic Khat Use Screening Test (PKUST-17) in Ethiopia: Classical Test Theory and Item Response Theory Analysis

Problematic khat use has been poorly defined and measured. This study aimed to develop a problematic khat use screening tool in the Gurage Community, South-central Ethiopia. We have used a series of methods (systematic review, qualitative study and cognitive interviewing) to generate a pool of items for problematic khat use screening tool. Classical test theory (inter-item, and item-total correlation, intraclass correlation, Cronbachalpha and exploratory factor analysis) and a 2-parameter item response theory statistics were used for initial psychometric evaluation of the scale. Initially, we developed a pool of 50 items. IRT analysis indicated that the item, ’’during the past three months, how often have you experienced irritability when you didn’t chew khat?” showed the highest discrimination (α thresholds =2.44). The item about “during the past three months, to what extent have you failed to do what was normally expected of you because of your use of khat? was the most difficult (first β thresholds =0.25) item, which was the only positive coefficient. The test information function graph depicts that the scale provided a lot of information in the latent trait’s moderate range. We found that employing series of methods is very important for item selection and refinement. We developed a Problematic Khat Use Screening Test with 17 items which will have a good utility for the general population or primary health care setting more than for the clinical population. Future studies should do full-scale validation of the scale.

Psychometric properties of two mental health screening tools in southeast Liberia: The Liberian Distress Screener and Patient Health Questionnaire.

Evidence suggests that locally developed and/or adapted screening tools for mental ill-health can have higher validity than directly translated tools developed in other settings. We administered the locally developed Liberian Distress Screener (LDS) and the Liberian-adapted Patient Health Questionnaire-9 (PHQ-9L) to a random sample of 142 outpatients at a regional hospital in Maryland County, Liberia. In the LDS, seven items demonstrated poor model fit and were excluded, resulting in an 11-item screener (LDS-11). Exploratory factor analysis of the 11-item screener (LDS-11) showed a single latent variable construct with significant factor loadings. Cronbach's alpha revealed good internal consistency (α = 0.81). Rasch analyses showed that "brain hot" and "heart fall down" were the most difficult idioms of distress to endorse while "things playing on the mind" was the easiest. All LDS-11 elements were associated with elevated function impairment, with "things playing on the mind," "worry too much," "head is hurting," and "heart cut/beat fast" achieving statistical significance. One item in the PHQ-9L demonstrated poor model fit and was excluded from psychometric analyses. The resultant eight-item PHQ demonstrated internal consistency (α = 0.76) and Rasch analysis revealed that "moving/talking too slowly/fast" was the most difficult item to endorse, while "not happy when doing things" was the easiest. Twelve items were significantly associated with functional impairment. Exploratory analyses reveal items that demonstrate ease and appropriateness of use for assessing mental distress in this population. Implementation research is needed to incorporate idioms of distress and screeners into Liberia's mental healthcare system.

Test of Gross Motor Development-3: Item Difficulty and Item Differential Functioning by Gender and Age with Rasch Analysis.

The assessment of motor proficiency is essential across childhood to identify children’s strengths and difficulties and to provide adequate instruction and opportunities; assessment is a powerful tool to promote children’s development. This study aimed to investigate the hierarchal order of the Test of Gross Motor Development-Third Edition (TGMD-3) items regarding difficulty levels and the differential item functioning across gender and age group (3 to 5, 6 to 8, and 9 to 10 years old). Participants are 989 children (3 to 10.9 years; girls n = 491) who were assessed using TGMD-3. For locomotor skills, appropriate results reliability (alpha = 1.0), infit (M = 0.99; SD = 0.17), outfit (M = 1.18; SD = 0.64), and point-biserial correlations (rpb values from 0.14 to 0.58) were found; the trend was similar for ball skills: reliability (alpha = 1.0), infit (M = 0.99; SD = 0.13), outfit (M = 1.08; SD = 0.52); point-biserial correlations (rpb values from 0.06 to 0.59) were obtained. Two motor criteria: gallop, item-1, and one-hand forehand strike, item-4, were the most difficult items; in contrast, run, item-2, and two-hand catch, item-2, were the easiest items. Differential item functioning for age was observed in nine locomotor and ten ball skills items. These items were easier for older children compared to younger ones. The TGMD-3 has items with different difficulty levels capable of differential functioning across age groups.

Validated tests for language research with university students whose native language is English: Tests of vocabulary, general knowledge, author recognition, and reading comprehension.

We present five studies aimed at developing an L1 vocabulary test for English-speaking university students. Such a test is useful as an indicator of crystallized intelligence and because vocabulary size correlates well with reading comprehension. In the first study, we tested 100 written words with four answer alternatives, based on Nation's Vocabulary Size Test. Analysis suggested two factors, which we interpreted as the possible existence of two types of difficult words: unknown words for general knowledge and unknown words for specialized knowledge. In Study 2, we attempted to develop a vocabulary test for each type of word, and these tests were then validated in Study 3. Since the test for general words proved too easy for the target population, we improved it in a fourth study by creating and testing more difficult items. Finally, a fifth study was conducted to validate the new test. Unexpectedly, Study 5 found a high correlation (r = .82) between the general knowledge vocabulary test and the specialized knowledge vocabulary test, suggesting that they measure the same latent factor, contrary to our initial assumption. Both tests have high reliability (r > .85) and correlate well (r > .4) with general knowledge, author recognition, and reading comprehension. In addition, a collection of other language tests was used and improved to verify the validity of the vocabulary tests. An exploratory factor analysis of all tests identified three factors (text comprehension, crystallized intelligence, and reading speed), with the vocabulary tests loading on the factor crystallized intelligence, which in turn correlates with reading comprehension. Structural equation modeling confirmed the interpretation.

Evaluating the reliability and validity of a questionnaire used to measure experiences of teamwork among student pharmacists in a quality improvement course

IntroductionThe psychometric properties of instruments used to capture student pharmacists' perspectives of teamwork have not been well assessed. This study measured the reliability and validity of an instrument designed to assess teamwork experiences among student pharmacists in a quality improvement (QI) class at one United States pharmacy school. MethodsThe psychometric properties of a previously conducted 17-item questionnaire (response options: “strongly agree,” “agree,” “disagree,” or “strongly disagree”) about second-year student pharmacists' teamworking experiences were assessed. A Rasch rating scale model was used to construct measures of teamwork experience. Principal component analysis (PCA) assessed unidimensionality. Item- and person-fit statistics were assessed. Construct and content validity and reliability were estimated utilizing student and item separation indices (SI) and reliability coefficients (RC). ResultsSixty student pharmacists were included. PCA conveyed a unidimensional construct. Four items with infit and outfit mean-squared values outside the suggested range were removed. Item responses “disagree” and “strongly disagree” were merged to improve scale functionality. The average person measure was 1.74 ± 2.03 logits. Student and item RC were 0.81 (SI = 2.04) and 0.97 (SI = 2.17), respectively. The easiest item endorsed was team's ability to reach consensus, while the most difficult item was interest to do collaborative work again. Mismatch of student experience and item difficulty level on the continuum scale suggested additional items are needed to match student teamwork experience. ConclusionThe instrument demonstrated evidence of reliability and validity to measure student pharmacists' teamwork experience in a QI class, but additional instrument modifications are recommended.

Assessing Patterns and Stability of ADL Hierarchical Scales for Functional Disability Assessment.

This study examined the stability over time of activities of daily living (ADL) items in 3 comparable longitudinal data sets and evaluated ADL loss sequences for older adults in the United States, South Korea, and Japan. Data from the U.S. Health and Retirement Study, and its 2 international sister surveys, were analyzed. Participants were community-dwelling adults aged 60 and older. For each data set, Rasch analysis was implemented to determine if the ordering of items remained stable across multiple waves (2006-2014), such that a single ADL hierarchy may be derived from multiwave data. Data fitted the Rasch model well. Item calibrations were sufficiently stable across measurement periods in each data set, reflecting a stable frame of reference. Results were also robust to sample variations. The derived ADL hierarchies based on scaled logit scores revealed that "dressing" and "bathing" were relatively more difficult items for older adults in all study populations. Scale stability is essential when exploiting longitudinal data to analyze patterns in ADL disabilities. The consistency in ADL scales across measurement periods supports their use as screening tools and identifying those at risk for transitions in care. Interventions to reduce dependency in bathing and dressing can help improve independent functioning for community-dwelling older adults.

Graph Mapping: A novel and simple test to validly assess fluid reasoning

We present Graph Mapping – a simple and effective computerized test of fluid intelligence (reasoning ability). The test requires structure mapping – a key component of the reasoning process. Participants are asked to map a pair of corresponding nodes across two mathematically isomorphic but visually different graphs. The test difficulty can be easily manipulated – the more complex structurally and dissimilar visually the graphs, the higher response error rate. Graph Mapping offers high flexibility in item generation, ranging from trivial to extremally difficult items, supporting progressive item sequences suitable for correlational studies. It also allows multiple item instances (clones) at a fixed difficulty level as well as full item randomization, both particularly suitable for within-subject experimental designs, longitudinal studies, and adaptive testing. The test has short administration times and is unfamiliar to participants, yielding practical advantages. Graph Mapping has excellent psychometric properties: Its convergent validity and reliability is comparable to the three leading traditional fluid reasoning tests. The convenient software allows a researcher to design the optimal test variant for a given study and sample. Graph Mapping can be downloaded from: https://osf.io/wh7zv/