Item Discrimination Research Articles

Development an Achievement Test on the Subject of “Basic Compounds in the Structure of Living Things”

The aim of this study is to develop a valid and reliable achievement test on the subject of ‘Basic Compounds in the Structure of Living Things’. During the preparation of the draft form of the test, a 32 item-question pool was created by the researchers in the light of the relevant literature. Then, these questions were presented to expert opinion in terms of language, intelligibility and content. This draft test with 32 multiple choice questions, was applied to 252 students studying at three different high schools. Item difficulty and item discrimination indices were calculated for the data obtained. After the item analysis, seven questions were removed from the test. Thus, the average item difficulty of the 25-question achievement test was calculated as 0.44, and the average item discrimination was 0.44. The KR-20 reliability coefficient of the test was calculated as 0.75. This result shows that the test is of medium difficulty and excellent discriminative reliability. In addition, according to the revised Bloom taxonomy, seven of the questions in the test are at the level of remembering, fifteen at the level of understanding, two at the level of application and one at the level of evaluation.

Boosting learning achievement in physics among Ugandan form-2 students: effect of problem-based learning

Enabling students to learn smoothly at an early stage of learning is a paramount effort that African education should takeoff. This study established the effect of problem-based learning (PBL) on students’ learning achievement. The participants were selected from form-2 of lower secondary schools in Sheema District, Western Uganda. Participants were randomly assigned to the treatment class with PBL instruction and the control class with content-based learning (CBL). A quantitative approach and a quasi-experimental design were used. Thus, the pretest-posttest non-equivalent quasi-experimental design was applied. A learning achievement test in simple machines was used as a data collection tool. The test was validated by experts and piloted with a split-half reliability (r = 0.87). Data was analysed in MS Excel and SPSS. A repeated measures analysis of variance revealed a very high statistically significant difference (p < .001) after learning in both classes. The same significance was found between PBL and CBL in favour of PBL. Classical test theory was used to present item difficulty and discrimination index. Teachers were recommended to adopt these methods with more practical hands-on activities to boost their skills and conceptual understanding.

Psychometric properties of Mental Health Scale: An Item Response Theory Approach

Indonesia Family Life Survey (IFLS) the fifth wave used the Centre for Epidemiological Studies Scale (CES-D10) to measure mental health state, which was depressive symptoms construct. It was primarily used in some studies regarding depressive symptoms. However, there was no specific validation research in the Indonesian context. Thus, this study aimed to (1) investigate the item quality of depressive symptom measurement from IFLS fifth wave, (2) measurement precision of this scale, and (3) the measurement invariance based on gender using the IRT approach. This study used data from ILFS-5 in the KP ("Keadaan Psikologis" or Mental Health) section, which was CES-D10 consisting of 10 items and conduct IRT analysis using a 2-PL model. The number of participants from the data was 31,447. CFA analysis resulted a unidimensional model fit for this scale. Moreover, finding showed that this scale had good psychometric properties, including item discrimination, item location, and item fit, except item 5 and item 8. Despite the poor quality for those items, the reliability coefficient, including the items, met a reliable measurement criterion. Also, this scale had much information for assessing medium and severe depressive symptoms. Moreover, the Differential Item Functioning (DIF) analysis indicated that there was no item exhibiting DIF.

Development and evaluation of short-form version of the Constitution in Chinese Medicine Questionnaire: study a new and best brief instrument of Chinese medicine for health management

BackgroundMore efficient instruments for body constitution identification are needed for clinical practice. We aimed to develop the short-form version of the Constitution in Chinese Medicine Questionnaire (CCMQ) and evaluate for health management.MethodsFirst, the short forms were developed through expert survey, classical test theory (CTT), and modern item response (IRT) based on the CCMQ. A combination of e-mail and manual methods was used in expert survey. Then, five indexes of CTT including criteria value-critical ratio, correlation coefficient, discrete tendency, internal consistency, and factor loading were used. And, IRT method was used through analyzing the discrimination and difficulty parameters of items. Second, the three top-ranked items of each constitution scale were selected for the simplified CCMQ, based on the three combined methods of different conditions and weights. Third, The psychometric properties such as completion time, validity (Construct, criterion, and divergent validity), and reliability (test–retest and internal consistency reliability) were evaluated. Finally, the diagnostic validity of the best short-form used receiver operating characteristic (ROC) curve.ResultsThree short-form editions were developed, and retained items 27, 23 and 27, which are named as WangQi nine body constitution questionnaire of Traditional Chinese Medicine (short-form) (SF-WQ9CCMQ)- A, B, and C, respectively. SF-WQ9CCMQ- A is showed the best psychometric property on Construct validity, Criterion validity, test–retest reliability and internal consistency reliability. The diagnostic validity indicated that the area under the ROC curve was 0.928 (95%CI: 0.924–0.932) for the Gentleness constitution scale, and were 0.895–0.969 and 0.911–0.981 for unbalance constitution scales using the cut-off value of the original CCMQ as 40 (“yes” standard) and 30 (“tendency” standard), respectively.ConclusionsOur study successfully developed a well short-form which has good psychometric property, and excellent diagnostic validity consistent with the original. New and simplified instrument and opportunity are provided for body constitution identification, health management and primary care implementation.

Validation of an Instrument to Assess Deductive Reasoning in Solving Types of Problems

The accelerated pace of knowledge generation requires engineering students to develop differenttypes of reasoning during their education. Deductive reasoning is essential to establishingself-regulated judgments based on reasoned argumentation. This paper aims to describeand illustrate the process by which a test of multiple-choice, open-response verbal mathematicalproblems was designed, applied, and validated to assess the deductive reasoning ofsecond-semester students of two engineering degrees from a university in a region of southernChile. The research used a non-experimental, cross-sectional approach focused on psychometricaspects. The evaluation instrument was developed on the basis of a typology ofmathematical problems and a model of deductive reasoning. The resolutions of types of problemsare classified according to their nature, routine and non-routine, and according to theircontext, real, realistic, fantasy, and purely mathematical, while the deductive argumentativemodel comprises the phases of data, claim, and warrant. The results guarantee sufficient contentand construct validity, item discrimination, and reliability; therefore, they represent auseful tool for measuring the level of deductive reasoning among first-year engineering studentsduring the process of solving a type of mathematical problem.

Evaluation of Psychometric Characteristics of A Logical Reasoning Tool With The Classic Test Theory Approach

Research on logical reasoning began to be carried out in Indonesia, it is necessary to conduct an evaluation of the psychometric characteristics possessed by logical reasoning measurement tools made by researchers. Logical reasoning measurement tools made by researchers need to be re-evaluated considering there are weaknesses in terms of using a relatively small number of subjects, so that it can affect the standardization of logical reasoning measuring devices. This study aims to obtain an overview of item difficulty index, item discrimination index, effectiveness of the distractors, validity, reliability, and error measurement from the logical reasoning measurement tool. Respondents in this study amounted to 7.730 students, consisting of 3.897 men and 3.833 women with an age range of 15-19 years. The research data were analyzed with the help of Microsoft Excel 2007, SPSS version 22.00, and ITEMAN version 3.6. The results of this study indicate that the item difficulty index moves from 0.35 to 0.80 which belongs to the easy and medium category. This study also shows that the item discrimination index moves from 0.10 to 0.71 with four items rejected and need to be aborted. This study also shows that the effectiveness of the distractors contained in each item in the logical reasoning measure is in the effective category. This study also shows that each item in the logical reasoning measure is considered valid in terms of factorial validity through exploratory factor analysis (EFA) procedures. This study also shows that the measurement of logical reasoning is relatively reliable with a Cronbach Alpha coefficient of 0.906. This study also shows that the measurement error obtained in this study is ± 3.92 of the total score obtained using a logical reasoning measure with a confidence level of 95%, with the actual score estimate obtained by respondents moving from 12.08 to 19.92, if the scores obtained by respondents sixteen.

Analysis of the psychometric properties of business studies achievement test for senior secondary schools in Nigeria

The study analyzed the psychometric properties of business studies achievement test for senior secondary schools in Nigeria. One research question guided the study. The study adopted an instrumentation design. The area of the study was Anambra State. The population of the study consists of all 23,077 students and 586 business studies teachers in two hundred and sixty-one senior secondary schools in Anambra State. Taro Yamane formula was used to determine the sample size of the study, which is 630 made up of 393 students in SS1 and 238 business studies teachers in Anambra State. A multi-stage sampling procedure involving two stages was adopted for this study. Proportionate stratified and purposive sampling techniques were employed at the first and second stages respectively. Items were subjected to item analysis to determine the difficulty and discriminatory indices. The study revealed that 125 items were found to be appropriate for inclusion in the business studies achievement test, these test's items met the criteria for item difficulty index and item discrimination. Based on the findings of the study it was recommended that teachers of business studies like commerce, accounting, and office practice among others should employ the created business studies Achievement Test. The BSAT items should be used as a model when creating other accomplishment exams for senior secondary schools in business studies.

An Investigation of the Validity and Reliability of Turkish Tampa Kinesiophobia Scale-Short Form in Individuals with Low Back Pain

The aim of this study was to investigate the validity and reliability of the Turkish Tampa Kinesiophobia Scale-Short Form in individuals with low back pain. A total of 111 volunteer patients aged between 30 and 65 years (mean 48±12 years) and diagnosed with mechanical low back pain were included in the study. Explanatory factor analysis (EFA) and confirmatory factor analysis (CFA) were applied to the data to test the construct validity. Item-total correlations were calculated to test item discrimination and Cronbach-α values were calculated to test reliability. To determine the criterion validity, the Turkish versions of the Visual Analog Scale (VAS), the SF-36, the Fear-Avoiding Beliefs Questionnaire (FABQ), and the Roland-Morris Disability Questionnaire (RMDQ) as well as the TSK-11, which was translated into Turkish, were administered. There was no difference between the TSK-11 test and retest measurement results (P&gt;0.05). Cronbach α levels for test and retest were calculated as 0.74 and 0.81. The variance of the first factor in the factor analysis was 35.372%, indicating that the Turkish version of the TSK-11 consisted of one dimension. As a result of DFA, the TSK-11 fit index criteria in other fit indices. In the test-retest analyses of the Turkish version of TSK-11, intraclass correlation coefficient (ICC) values ranged from 0.403 to 0.479, while TSK-11 Cronbach's alpha values were 0.727-0.748. There was a positive relationship between TSK-11 and FABQ, RMDQ, and VAS (r=436**, 589**, 589**, respectively), and a moderate negative relationship between TSK-11 and SF-36. In our study, the Turkish version of the TSK-11 was found to be valid and reliable. The TSK-11 can be used in clinical practice and research to evaluate fear of movement in individuals with low back pain in Turkish society.

Application of Item Response Theory as a Modern Statistical Tool to Test Item Development and Analysis

In the educational and psychological testing, there are two major theories through which tests can be developed, validated and ultimately used for assessing examinee’s performance. These are classical test theory (CTT) and item response theory (IRT) and their corresponding models. Item Response Theory (IRT) as a test theory came into existence to provide probabilistic approach to surmount some of the inherent limitations of the classical test theory and maximize objectivity in educational assessment. Item response theory (IRT) is a quantitative approach to testing the reliability and validity of an instrument based on its items. It has statistics for evaluating individual items from a quantitative perspective. The purpose of this paper is to describe in details, the application of item response theory in test item development and analysis. The reason for the application of IRT is to have test items that will yield a reasonable degree of reliability. The statistics used in this respect are – item difficulty parameter, which is a measure of the proportion of testees who responded to an item correctly; the item discrimination parameter, which is the measure of how well the items discriminate between examinees with high and low levels of knowledge or ability and pseudo-guessing parameter, which expresses the probability that an examinee with low ability can be able to get an item correctly. The acceptable range of values of the aforementioned parameters were also discussed.

Validity and Reliability of the Arabic Thyroidectomy-Related Voice and Symptom Questionnaire (A-TVSQ) for Assessing Post-Thyroidectomy Patients

Post-thyroidectomy patients commonly experience voice changes, swallowing symptoms, and neck discomfort, a condition known as post-thyroidectomy syndrome. The Thyroidectomy-Related Voice and Symptom Questionnaire (TVSQ) is a validated tool showing strong correlations with established tools with a potential predictive value for post-thyroidectomy voice outcomes. Our study aims to translate and validate the TVSQ into Arabic. This validation study was conducted at King Abdulaziz University Hospital and King Khalid University Hospital in Riyadh, Saudi Arabia. Forward and backward translations ensured the conceptual equivalence of the Arabic TVSQ (A-TVSQ) and were reviewed by an expert panel. A pilot test assessed the A-TVSQ's face validity and clarity, leading to necessary cultural adjustments. The final A-TVSQ was preoperatively and postoperatively administered to thyroidectomy patients to evaluate its test-retest reliability. Data were collected from 116 participants (53 cases [45.6%] and 63 controls [54.3%]). The internal consistency analysis showed strong intercorrelation among the A-TVSQ items, with Cronbach's alpha coefficients (0.951) indicating excellent estimated internal consistency within both domains. Test-retest reliability showed a moderate level of reliability with an intraclass correlation coefficient of 0.696. A-TVSQ item discrimination analysis showed significant associations between the total and domain scores pre- and post-thyroidectomy. External validity analysis showed significant changes in A-TVSQ voice change (P<0.010) and neck discomfort (P=0.011) domains and total scores (P<0.010) between before and 2 weeks after thyroidectomy. However, no significant changes were observed between 2 and 4 weeks after thyroidectomy. The translated and validated A-TVSQ showed excellent internal consistency and moderate test-retest reliability, comparable to the original version. Voice change significantly worsened after thyroidectomy, highlighting the need for continued monitoring and management strategies for this complication.

문항반응이론을 적용한 KCYPS 창의적 성격 문항의 타당화 연구

This study aimed to examine whether the creative personality items of KCYPS 2018, adapted from Gough &amp; Heilbrun’s (1983) Creative Personality Scale (CPS) reliably and validly measure the creative personality of middle school students. Applying item response theory to the creative personality items, it was found that five items, namely ‘snobbish, unconventional, individualistic, informal, reflective,’ demonstrated very high difficulty, while six items such as ‘unconventional, humorous, individualistic, informal, interests wide, reflective’ showed very low item discrimination. In addition, the analysis of DIF(Differential item function) of gender revealed that the item ‘sexy’ was more favorable for female students, while four items ‘capable, clever, insightful, inventive’ were more advantageous for male students. This indicated that, even with the same level of creative personality, it can be measured differently depending on gender. This study holds significance in rigorously validating the KCYPS 2018 creative personality items through item response theory. We finally highlighted the need for comprehensive development of creative personality scales considering factors such as age, sub-components of creativity, and response patterns.

Validity and Reliability of Balanced Nutrition Knowledge and Eating Behavior Questionnaires among Adolescent Girls

Background: Assessing balanced nutrition knowledge and eating behavior among adolescent girls is important because they affect health and nutritional status, thus it needs measurement instruments that have been tested for validity and reliability. Objective: To assess the validity and reliability of balanced nutrition knowledge and eating behavior questionnaires among adolescent girls. Method: This study involved 30 Senior High School girls aged 15-18 years. The validity of the data was tested using difficulty level analysis, differentiating power analysis, distractor analysis, Lawshe's Content Validity Ratio (CVR) analysis, and item discrimination test. The reliability of the questionnaire was tested using Cronbach's Alpha coefficient. Results: A total of 20 items regarding balanced nutrition knowledge were considered valid and 10 items about eating behavior were declared valid. The results of the reliability test of the questionnaire on balanced nutrition knowledge and eating behavior obtained Cronbach's Alpha values of 0.822 and 0.829, thus this questionnaire is declared reliable. Conclusion: The questionnaire on balanced nutrition knowledge and eating behavior has good validity and reliability values. The questionnaire can be used to obtain accurate and consistent data in order to assess the balanced nutrition knowledge and eating behavior among adolescent girls.

Reliability and validity of the Chinese version of the achievement emotions questionnaire for physical education in university students

BackgroundAchievement emotions have a significant impact on both the learning process and outcomes. However, there is currently no brief and effective questionnaire available to evaluate Chinese university students' achievement emotions in physical education courses. This study aimed to examine the reliability and validity of the Achievement Emotions for Physical Education Questionnaire (AEQ-PE) in a sample of Chinese university students, while also investigating its measurement invariance across gender and grade levels.MethodsA cluster randomization sampling method was used to select 694 first- and second-year university students in Shanghai, China for the survey. Descriptive statistics, item analysis, reliability testing, and measurement invariance testing were conducted on the full sample (n = 694). Subsequently, the full sample was randomly divided into two groups, with Sample 1 (n = 347) undergoing exploratory factor analysis (EFA), and Sample 2 (n = 347) undergoing confirmatory factor analysis (CFA) to test the structural validity, convergent validity, and discriminant validity of the Chinese version of the Achievement Emotions Questionnaire for Physical Education (AEQ-PE-C). Finally, Sample 3 (n = 45), which was retested one month later, was used to evaluate test–retest reliability.ResultsThe Chinese version of the Achievement Emotions Questionnaire for Physical Education consists of 6 dimensions and 24 items, with good item discrimination. The EFA supported a 6-factor structure model, while the CFA demonstrated good model fit indices (χ2/df = 3.086, CFI = 0.928, TLI = 0.916, RMSEA = 0.078) and good convergent and discriminant validity. The questionnaire exhibits high internal consistency reliability (0.794) and excellent test–retest reliability (0.792). Furthermore, the multi-group analysis confirms that the AEQ-PE-C questionnaire has measurement invariance across gender and grade levels.ConclusionThe Chinese version of the Achievement Emotions Questionnaire for Physical Education has good reliability and validity, as well as measurement invariance across gender and grade levels, making it an effective tool for measuring achievement emotions in physical education among Chinese university students.

Standard Assessment Model for Scientific Integration-Interconnection as Material for Evaluation of Vision and Mission Achievement

This study aims to develop a standardized assessment model of scientific integration-interconnection, which begins with conducting a literature review and needs assessment, setting goals, developing initial products, validating contents, testing readability, testing instruments, analyzing results, and ending with an interpretation of the results of the assessment. The method used is Research and Development. Testing of the assessment instruments was carried out for each element of the Tridharma of Higher Education, namely 76 lesson plans, 139 research and development works, and 41 community service works. The scientific integration-interconnection assessment model has characteristics according to the structure of the second order model, with the first order being 4 scientific integration-interconnection domains (philosophical domain, material domain, methodological domain, and strategic domain) and the second order is 15 aspects of scientific integration-interconnection. Data analysis techniques using Aiken-V, Exploratory Factor Analysis/EFA and Confirmatory Factor Analysis/CFA, Omega, Item Response Theory approach of polytomous scoring, and descriptive statistics. The assessment model is feasible, fulfilling the criteria of content validity, construct validity (p-value = 0.11949 and RMSEA = 0.027), estimated instrument reliability (α = 0.816), and construct reliability (ɷ =0.939). The scoring used the GRM approach with good quality in terms of the results of the 94% item fit test with an item discrimination index of 0.655 &lt; DISCi &lt; 1.153 and the average item difficulty level of -1.965 &lt; DIFFi &lt; -0.662.

Validation of two measures for assessing English vocabulary knowledge on web-based testing platforms: long-form assessments.

The goal of the current work was to develop and validate web-based measures for assessing English vocabulary knowledge. Two existing paper-and-pencil assessments, the Vocabulary Size Test (VST) and the Word Familiarity Test (WordFAM), were modified for web-based administration. In Experiment 1, participants (n=100) completed the web-based VST. In Experiment 2, participants (n=100) completed the web-based WordFAM. Results from these experiments confirmed that both tasks (1) could be completed online, (2) showed expected sensitivity to English frequency patterns, (3) exhibited high internal consistency, and (4) showed an expected range of item discrimination scores, with low frequency items exhibiting higher item discrimination scores compared to high frequency items. This work provides open-source English vocabulary knowledge assessments with normative data that researchers can use to foster high quality data collection in web-based environments.

Psychometric evaluation of the Chinese version of the Positive Health Behaviours Scale for clinical nurses: a cross-sectional translation

BackgroundOccupational health is essential for nurses in clinical nursing practice. However, there is no specific tool for measuring the health behaviour of clinical nurses in China. This study aimed to translate the Positive Health Behaviours Scale into Chinese and validate its psychometric properties among clinical nurses.DesignA cross-sectional design with repeated measures.MethodsA total of 633 clinical nurses were recruited by convenience sampling from hospitals in Liaoning Province, China. After obtaining the authorization of the original author, the PHBS was translated into Chinese by the Brislin back-translation method. Item analysis was completed to evaluate item discrimination, and the Delphi method was adopted to analyse content validity. Exploratory factor analysis and confirmatory factor analysis were conducted to explore and validate the underlying factor structure. Internal consistency and test-retest reliability were calculated to evaluate reliability.ResultsA total of 29 items were retained in the item analysis, and the content validity index of the translated scale was 0.956. In the EFA, four common factors were extracted (nutrition, physical activity, relaxation and behaviours related to mental health and preventive behaviours), explaining 60.81% of the total variance. The results of the CFA were as follows: χ2/df = 1.363, GFI = 0.902, NFI = 0.909, IFI = 0.974, TLI = 0.971, CFI = 0.974, RMSEA = 0.034, and RMR = 0.023. The results of the EFA and CFA showed that the translated scale had good structural validity. Cronbach’s α coefficient, the split-half reliability and the test-retest reliability of the Chinese version of the PHBS were 0.928, 0.953 and 0.891, respectively. At the same time, the translated scale had good reliability.ConclusionsThe Chinese version of the PHBS for clinical nurses had good psychometric properties. The results of the questionnaire survey effectively and comprehensively reflect the level of health behaviours in clinical nurses, which provides a scientific reference for determining the intervention target.

Patients’ expectancy scale of acupuncture: Development and clinical performance test

PurposeThis study aims to develop and validate a concise tool for evaluating acupuncture expectancy that is easy to understand and conforms to acupuncture characteristics. Materials and methodsA draft was created using the Delphi consensus method. Reliability, validity, discrimination, and feasibility tests were conducted at the item and scale levels. ResultsThe scale themes were defined as disease-related, treatment-related, process-related, and outcome-related. After two rounds of Delphi surveys with good experts' reliability (authority coefficients of experts were 0.86 and 0.87 in the two rounds) and agreement (Kendall's concordance coefficient of the participants were 0.33 and 0.15 in the two rounds, P < 0.05), 11 items (the mean score for item importance, full mark ratios, and coefficient of variation of items were ≥3.5, ≥25%, and ≤0.30, respectively) were included in the draft. A total of 145 individuals were recruited to test the draft. Reliability was assessed by Cronbach's α coefficient (0.90), split-half reliability coefficient (0.89), and test–retest reliability (Pearson's coefficient = 0.74, P < 0.05). Content validity was assessed by the content validity index (Item-CVI ≥ 0.78 and Scale-CVI/Ave = 0.92), and a confirmatory factor analysis was performed to assess the construct validity. The discrimination of scale items was evaluated by the critical ratio (CR > 3.00) and the homogeneity test (item-total correlations >0.40). Feasibility was assessed through the acceptance rate (recovery rate = 98.60%, response rate = 100%), completion rate (100%), and completion time (4.99 ± 6.80 min). ConclusionThe patients’ expectancy scale of acupuncture (PESA) consists of 11 items with four themes, disease-related, treatment-related, process-related, and outcome-related. It has great reliability, validity, discrimination, and feasibility and has the potential to evaluate acupuncture expectancy in clinical trials.

Difficulty of Reading Comprehension and the Proficiency of the Grade 10 Students of Aloran Trade High School, Philippines

In the 2018 Program for International Student Assessment (PISA), the Philippines ranked the lowest among the 79 countries participating in the assessment. This study aimed to determine the proficiency of the grade 10 students at Aloran Trade High School and the difficulty of reading comprehension. The instruments used in this study were adapted reading materials from PISA assessments with questions needing subjective answers, categorized according to the different comprehensions proposed in Barrett’s Taxonomy. The researchers collected data from 40 grade 10 students who were selected randomly using a random sampling technique. Item analysis (item difficulty and item discrimination) and descriptive analysis (mean and standard deviation) were used to analyze the data. After tabulating the data gathered, the researchers determined that the students have low proficiency in reading comprehension. Furthermore, no comprehension has an easy difficulty with the proficiency of grade 10 students. Literal and reorganizational comprehension have moderate difficulty, while inferential and evaluative comprehensions are difficult among the students. These imply that students need to be proficient in understanding a text’s explicit and implicit information. It is therefore recommended that interventions be made to address this issue.

Why multimedia might matter: The impact of animations and images on item performance and test‐taker behaviour

AbstractThe use of animations and images in technology‐based assessments (TBAs) represents a significant change in assessment design. To ensure that appropriate inferences can be drawn from assessments that use multimedia stimuli, their impact on test‐taker performance and behaviour must be investigated. To achieve this, an experiment was conducted with 251 Irish post‐primary students using an animated and text‐image version of the same TBA of scientific literacy. Eye movement (n = 33) and interview data (n = 12) were also collected as a measure of test‐taker attentional behaviour. Overall, there was no significant difference in test‐taker performance when identical items used animated or text‐image stimuli. However, items with dynamic stimuli often had higher discrimination indices indicating that these items were better at distinguishing between those with differing levels of knowledge. Eye movement data also revealed that dynamic item stimuli encouraged longer average fixation durations on the response area of an item. These findings indicate that multimedia stimuli may potentially affect how test‐takers interact with online assessments. This has implications for what claims can be made about a learner's performance on an assessment. Recommendations for policy, practice and future research are considered. Practitioner notesWhat is already known about this topic: The use of multimedia stimuli in the form of diagrams, high‐resolutions images, animations, and simulations are becoming more commonplace in technology‐based assessments (TBAs) for post‐primary aged learners. It is unclear what impact the use of multimedia stimuli can have on an individual's performance and behaviour in assessment/testing contexts. Eye movement data can be used to support our understanding of test‐takers' interactions with TBAs. What this paper adds: By comparing the use of different types of multimedia stimuli (animations vs images), this study has responded to calls for a more in‐depth examination of test items involving multimedia stimuli for TBAs. While there was no difference in test‐taker performance between those who saw test items with different forms of multimedia stimuli, key differences in attentional behaviour were noted. Test‐takers interacted with the assessment differently pending the multimedia stimuli used. The current study showed that dynamic stimuli may be a way to improve item discrimination, which is generally desirable in assessments. Implications for practice and/or policy: Certain design features appeared to add additional ‘assessment load’. This information could be leveraged to improve test item design and test specifications. It may also encourage test‐developers to reconsider what claims they make about individuals who complete assessments with these features. Eye‐tracking technology has huge potential to support research in online assessment environments.

Psychometric properties of the Coronavirus Anxiety Scale based on Classical Test Theory (CTT) and Item Response Theory (IRT) models among Chinese front-line healthcare workers

BackgroundSince March 2022, the COVID-19 epidemic has rebounded widely and frequently in China. Healthcare workers have faced grand challenges such as soaring COVID-19 patients, being busy with the nucleic acid screening of all the populations in the epidemic areas every day, and testing positive for COVID-19, all of which contributed to anxiety easily according to the Conservation of Resources theory. However, anxiety among healthcare workers is not only associated with personal health but also adversely affects the quality of health services. Therefore, it is crucial to search for suitable tools to monitor the anxiety related to COVID-19 among healthcare workers. The current study aimed to test the Coronavirus Anxiety Scale (CAS) in Chinese healthcare workers.MethodsThe current study employed a cross-sectional design. The CAS was translated into Chinese. Then, according to Classical Test Theory (CTT) and Item Response Theory (IRT) models, the psychometric properties of the Chinese version were measured among 811 healthcare workers.ResultsThe split‐half reliability was 0.855. The Cronbach’s α coefficient was 0.895. The retest coefficient was 0.901 with 10 days as the retest interval. The content validity index was 0.920. In exploratory factor analysis, one common factor was extracted and explained 72.559% of the total variance. All item load values on the common factor ranged from 0.790 to 0.885, and the communality of each item ranged from 0.625 to 0.784. With confirmatory factor analysis, the single factor model showed an excellent goodness-of-fit, chi-square/degree of freedom (χ2/df) = 3.339, goodness of fit index (GFI) = 0.992, adjusted goodness of fit index (AGFI) = 0.975, root-mean-square error of approximation (RMSEA) = 0.054, root mean square residual (RMR) = 0.005, incremental fit index (IFI) = 0.967, Tucker-Lewis index (TLI) = 0.932, and comparative fit index (CFI) = 0.966. The multiple-group confirmatory factor analysis revealed the invariance measuring anxiety of COVID-19 was in similar ways across ages, hospital degrees, and professional titles. With convergent validity, the CAS was positively correlated with post-traumatic stress disorder (r = 0.619, P < 0.001), fear of COVID (r = 0.550, P < 0.001), and depression (r = 0.367, P < 0.001). According to IRT models, the results showed that all item discrimination parameters were higher than 1.70 and difficulty parameters ranged from 1.13 to 2.83.ConclusionThe Chinese version of CAS has good psychometric properties in healthcare workers after China adjusted the COVID-19 management measures during the COVID-19 Omicron epidemic, and can be used for assessing the anxiety associated with COVID-19 in Chinese healthcare workers.