Item Difficulty Index Research Articles

Children of Parents With a Mental Illness-Stigma Questionnaire: Development and Piloting.

Children of parents with a mental illness are a particularly vulnerable group as they have a high risk to develop a mental disorder themselves and those are associated with high stigma. Moreover, just like primary recipients of stigma, they are affected by the social taboo surrounding mental illness: they do not receive enough information, are often left alone with their problems, and are thus considered “invisible children”. In previous research, family stigma has only been assessed through general questionnaires for all family members. What has not yet been adequately investigated is how stigma difficulties affect the children of parents with mental illness in particular. To address these limitations, we developed the Children of Parents with Mental Illness-Stigma-Questionnaire (COPMI-SQ), a self-report instrument for young people aged 12–19 years, designed to assess young people's stigma experiences in daily life. Based on a systematic review preceding the questionnaire, we identified relevant stigma dimensions for children of parents with a mental illness that resulted in 93 items that according to theory were assumed to load on four different scales: experienced stigma, anticipated stigma, self-stigma, and structural discrimination. An expert discussion, and a comprehensibility analysis with the target group followed. In this paper, we report on the development process and initial pilot data (N = 32) on the psychometric properties of the COPMI-SQ. Item analyses via an item difficulty index, discriminatory power, as well as internal consistency analysis resulted in a revised instrument reduced to 67 items. We observed very high internal consistencies (between α = 0.868 and α = 0.975) for the subscales. The approach taken to develop the COPMI-SQ followed scientifically accepted principles by ensuring different construction phases and is considered a solid basis for further reliability and validity studies. The study is ongoing and undergoing a further validation investigation; dimensionality and factor structure will also be examined.

Item Analysis of a Reading Test in a Sri Lankan Context using Classical Test Theory

This paper is based on a research study on a reading test that evaluates the different cognitive processes prescribed by Khalifa and Weir (2009). The 25-item test was designed based on a test specification targeted at the B2 level of the Common European Framework of Reference for Languages (CEFR). The responses of 50 students were used to check the validity and reliability of the test. The validity of the test was ascertained through item analysis involving item difficulty indices, item discrimination indices, and distractor analysis. Each item was studied to provide detailed information leading to the improvement of test construction. To achieve test reliability, the Kuder-Richardson Formula 20 (KR-20) was applied. The results were achieved by simply using Microsoft Excel. Findings revealed that the test met the standards for content validity, indicating acceptable item difficulty indices, with 17 items at the moderate level between the ranges of 0.30 and 0.79. Except for three items, all others functioned well to differentiate between high- and low-ability students, and only five items had malfunction distractors. Meanwhile, the reliability value of the test scores was 0.82, which is deemed a good value, proving the consistency of the test results. It signifies that more than half, that is 88%, of the test items were well functioning and that the test proved to be valid and reliable. The present research can contribute to students, teachers, and test-makers having an insightful understanding of item analysis and test development.

Validation of diabetes-related awareness instrument (DRAI) to measure diabetics awareness towards risk factors and prevention strategies

PurposeAccessibility to a precise tool for healthcare management and self-precaution among diabetic patients is an absolute necessity. This paper aims to develop and validate diabetes-related awareness instrument (DRAI) – an instrument that measures diabetics awareness about risk factors and prevention strategies.Design/methodology/approachThe reliability and validity of the DRAI were tested with a sample of 112 diabetics. The construct validity of the DRAI was measured using exploratory and confirmatory factor analysis. Item discrimination, reliability, usefulness and validity of the items were determined by performing Cronbach's alpha, item difficulty and discrimination index analysis.FindingsThe study finds DRAI – a reliable and valid instrument to assess diabetics awareness towards diabetes mellitus, its associated risk factors and prevention strategies. The value of Cronbach's alpha for all three constructs was above the threshold level of 0.70. Under exploratory factor analysis, “Kaiser–Meyer–Olkin” test value of 0.805 exhibits a meritorious sample adequacy and “Bartlet's test of Sphericity” was statistically significant with p = 0.032. Therefore, results of confirmatory factor analysis (CFA) revealed that all fitness indices of the model to be excellent fit.Practical implicationsThe present instrument can help to determine whether the individual is susceptible to diabetes, timely prevention and reduction in the incidence of diabetes mellitus.Originality/valueDRAI is the first of its kind tool to assess the awareness and knowledge about diabetes-related risk factors and prevention strategies in such a demographically diverse population of India.

Development and validation of the safe transfusion assessment tool

Given the prevalence and risks of blood transfusion, it is essential that trainees and practicing clinicians have a thorough understanding of relevant transfusion medicine competencies. The aim of this research was to develop and gather validity evidence for an instrument to assess knowledge of core transfusion-related competencies. We developed the safe transfusion assessment tool (STAT) using a multistep process. Initially, 20 core competencies in transfusion medicine were identified through a consensus-driven Delphi process. Learning objectives and assessment items pertinent to each competency were created. Next, a 13-item assessment tool was piloted with multidisciplinary experts and trainees. Multiple iterative revisions were made based on feedback. Finally, the 12-item STAT was administered to 100 participants of varying training level and specialty to establish validity, difficulty and item discrimination indices, and perceived utility. Analysis of instrument item difficulty and item discrimination indices demonstrated the ability of the STAT to assess essential knowledge in transfusion medicine relevant to trainees and clinicians in multiple programs and practice settings. Eight of twelve items discriminated between learners with varying degrees of expertise. Hundred percent of students and trainees rated the STAT as Extremely Helpful or Somewhat Helpful and the majority planned to utilize the answer guide as a study aid. The STAT is a concise, valid, and reliable knowledge assessment tool that may be used by researchers and educators to augment transfusion medicine curricula (www.safetransfusion.ucsf.edu). Scores can help inform departments on areas in which trainees require additional support and areas of potential educational interventions.

Primary validation of Children Spiritual Intelligence Scale in a sample of Latvian elementary school pupils

ABSTRACT The paper aims to describe the development of the Children Spiritual Intelligence Scale (CSIS), focusing on the stage of primary validation of the scale. Based on the five components of the theoretical model (Creation and awareness of personal meaning, Self-understanding, Mastery of self-control, Awareness of personal authenticity and uniqueness, and Social mastery), a 63-item scale with validated content was developed to measure children’s spiritual intelligence. The validation sample consisted of 200 pupils from grades 1–4 representing comprehensive education schools situated in different regions of Latvia. The exploration resulted with the psychometric properties of CSIS, showing the item difficulty and discrimination indices of items, interpretation of the factorial structure and internal reliability of the scale. The final set of 23 CSIS items, measuring children spiritual intelligence in a framework of a one-factor model with four included subcomponents, showed acceptable psychometric properties, enabling to pursue the next steps of scale development.

Construction of knowledge test to measure the knowledge of dairy farmers about improved dairy farming practices

Due to the dearth of a suitable scale for measuring the knowledge of dairy farmers about improved dairy farming practices in Jammu district of J&K, it was necessary to construct a knowledge test for the purpose. Therefore, an attempt was made to develop a test for measuring the knowledge of dairy farmers about improved dairy farming practices. Relevant statements covering five major aspects of animal husbandry viz., breeds and breeding, feeding, housing, milking and health care and hygiene were collected. After getting a jury opinion on the items, item difficulty index, item discrimination index, the reliability and validity were worked out. For administering the knowledge test, a respondent was given one mark for each correct answer and a zero mark for each wrong answer. Sixty one statements were finally selected out of the total seventy eight statements.

Development of Test and Measurement of Knowledge Level of Sunflower Farmers

A test was constructed to measure the farmers’ knowledge level on sunflower cultivation during 2021-22. Relevant items were collected from review of literature and discussed with scientists. For relevancy test, 39 items were sent to 320 experts for their judgement. Based on 45 experts judgement, 18 items (overall mean score > 2.57) were selected for item analysis and administrated to 60 farmers of Chinnakoduru village of Siddipet district. Item analysis was done by item difficulty index, item discrimination index and point biserial correlation and 15 items were finalized for knowledge test. Reliability (Cronbach alpha, Spearman brown coefficient and Guttman split-half) and validity (point biserial correlation) were estimated for standardization of the test and found to be highly significant. The test was administered to 140 respondents in Andhra Pradesh and Telangana states. In Andhra Pradesh, 78.8 per cent and in Telangana 85.0 per cent of respondents were in medium level of knowledge category. The item analysis indicated significant differences in knowledge level of farmers pertaining to most suitable period of sowing sunflower crop during Rabi, suitable sunflower hybrids, optimum seed rate, critical period of weed competition, application of fertilizers and bee keeping in sunflower in two states.

Knowledge Test for Rainfed Farmers on Natural Resource Management Practices

Natural resource management (NRM) aims on efficient and sustainable utilization of natural resources. One of the greatest challenges facing the world is meeting the rising demand for food with a sustainable natural resource base. Agriculture, a sector being dependent on natural resources, it is high time to make farmers aware on the importance of NRM which require measuring the knowledge level of farmers on NRM practices for the better dissemination of NRM technologies. The study attempted to construct a knowledge test to measure the knowledge level of rainfed farmers during 2021. Items about NRM were collected from relevant literature and through consultation with experts. One pilot study was conducted in Lingaipalle village of Mahabubnagar district of Telangana for item analysis during September 2021. Item analysis provided item difficulty index, item discrimination index and item validity index. Based on the criteria, a total of 18 items were selected in the final test. To administer the knowledge test, one mark for each correct answer and zero mark for each wrong answer was assigned. Finally, the reliability of the test using split-half method found to be 0.749 indicating that the knowledge test is highly reliable for the present study.

Development of Cognitive Learning Scale to Test the Knowledge Level of Veterinary Students on ICT

In the present study, to test the knowledge level of veterinary students on ICT, one hundred and seventy-one items were initially constructed on the basis of promoting thinking rather than rote memorization. It was designed in a manner that could differentiate the well-informed veterinary students from less informed ones. The scores of the respondents were subjects to item analysis to find the item difficulty index and item discrimination index. In the final selection, a total of 34 items with difficulty index between 30 and 80 and discrimination index ranging from 0.30 to 0.55 were selected. The reliability of the knowledge test developed was tested using split half technique. The coefficient of correlation value in split half test was 0.89, which was found to be significant at 1 per cent level of significance. It was found that, the developed knowledge test scale of Veterinary students on ICT was highly stable and can be used for measurement.

Development of a diagnostic instrument for the assessment of teaching competencies in medicine (FKM_L): First results of the test statistical verification.

This project report describes the development of the Questionnaire for the Assessment of Teaching Competencies in Medicine (FKM_L) and the analysis of some of its psychometric properties. The design of the FKM_L is based on the model of Core Competency for Teachers in Medicine (KLM) model of the GMA Committee on Personnel and Organizational Development in Teaching. Global questions and in-depth items were formulated for each of the sub-competencies of the six core competencies of the KLM model. Depending on the number of sub-competencies, there are 3-4 subscales for each core competency, comprising 69 items in total. Data from 90 participants of medical didactic courses were analysed. Item analyses supported the hypothesized scales. The internal consistencies (Cronbach's alpha: CR-α) of the 22 subscales ranged from CR-α=.70 to CR-α=.93, and the item difficulty indices of the subscales ranged from 18% to 89%. For 2 subscales, some items had a difficulty index of more than 80%, and for 3 subscales, the difficulty index of some items was less than 25%. The FKM_L was developed to assess individual and group profiles of teachers' competence in medicine. The results of this first psychometric analysis are promising: With the help of the FKM_L, teachers can learn about and reflect on aspects of their teaching competencies in the context of medical didactic courses. Based on their FKM_L profiles, they can decide whether they want to selectively optimize their competence characteristics. For providers, the FKM_L is suitable as a screening tool to identify, among other things, gaps in the training offered. Further analyses are necessary to check limitations identified in some scales and to improve individual items. In addition, research on the construct and criterion-related validity of the instrument is required.

The Palliative Care Knowledge Questionnaire-Basic (PCKQ-B): Development and Validation of a Tool to Measure Knowledge of Health Professionals about Palliative Care in India.

Objectives:A well-educated and aware staff are the needed to overcome barriers to palliative care provision in the face of ever-increasing needs of palliative care in developing countries like India. Lack of a standardised and locally validated need assessment tool is felt as one of the important requirements for any successful educational program. This project is to develop and validate a basic palliative care knowledge needs assessment questionnaire.Materials and Methods:Development of Palliative Care Knowledge Questionnaire-Basic (PCKQ-B) for Indian nurses working in a tertiary care hospital in Delhi-NCR, India was undertaken with four basic processes, that is, item generation, content validity, pilot testing and field testing. Literature search helped us to find appropriate domains and items; content validity obtained with a panel of seven experts, pilot testing and finally field testing was done on 102 nurses working in 6 branches of Max Super Specialty Hospital, Delhi-NCR, India. The field study data were used to do item analyses including item difficulty, item discrimination and calculate reliability indicators including test-retest correlation and KR-20.Results:Content Validity index (CVI) of all the items was >0.78 (except 1 item) and Scale-CVI was 0.98 (excellent content validity). Items were selected on the basis of item difficulty index of 0.1 to 0.9 and an item discrimination index greater than 0.2. A total of 25 questions were selected that elicited KR-20 value of 0.65 and intra class correlation (test-retest reliability) of 0.52.Conclusion:Overall, PCKQ-B is a 25 item questionnaire with a moderate reliability. Although validated for nurses in tertiary care hospitals, this tool might find utility in education of primary health workers, community health workers, social workers, or physiotherapists in palliative care teams.

4.SINIF GEOMETRİK ŞEKİLLER KONUSU BAŞARI TESTİ GEÇERLİK VE GÜVENİRLİK ÇALIŞMASI

The validity and reliability analysis of developing a test for the fourth-grade mathematics curriculum's geometric shapes was described in this study. For this purpose, the elementary school fourth-grade mathematics curriculum (2018) was reviewed, the gains related to "Geometric Shapes" was listed, and a table of indicators was prepared. Based on this table, 21 multiple-choice and 5 open-ended items were created. After getting the experts’ opinions, the items were placed in the test according to difficulty level and test's appearance validity. A period of 40 minutes has been set for the implementation of the test. The test was administered one-on-one to 8 students from the 4th grade who already studied the subject to demonstrate test items' power to measure the intended properties and understandability. The students were asked to answer the questions by thinking aloud, and the test was revised accordingly. For the test's actual implementation, 203 students attending primary schools (4th grade) in the Altıeylül district of Balıkesir province in the 2017-2018 academic year were selected by random sampling method. The results of the test were analyzed with the R program. Item difficulty, item discrimination index, reliability, and construct validity analysis were performed with tetrachoric factor analysis. Five items with an item difficulty index less than 0.19 and two items with an item discrimination index below zero were removed from the test. Besides, six items with item discrimination between 0 and 0.19 were revised and included in the test. As a result, the 19-question final test consisting of 16 multiple-choice and 3 open-ended questions, was created. While calculating the Kuder Richardson-20 (KR-20) statistics, a manual calculation was performed using the R program. The KR-20 value was found to be 0.68. The answering time of the test was set as 30 minutes. The validity and reliability analysis results showed that the test could be used as a valid and reliable test on the 4th Grade Mathematics course's geometric shapes subject.

SEMI STANDARDIZATION OF A TEST OF SECONDARY SCHOOL MATHEMATICS

Present study intended to semi standardize a test of secondary school Mathematics for the students of 9th class. Two Mathematics Achievement Tests (MATs) were prepared from 302 items of previous papers of BISEs of Punjab province, each having 50 MCQs. A sample of 280 students from tehsil Quaid Abad of Khushab District consisted of 142 females and 138 males’ respondents was drawn from the population. MATs were administered on the respondents. Reliability of the tests and traditional item analysis was computed through MS Excel. For reliability of parallel form MATs correlation coefficient was computed. The value of correlation coefficient was 0.64. Through item analysis, item discrimination index and item difficulty index of each item of the test was computed. There was a lot of repetition in the items of Boards of Intermediate and Secondary Education. Violation of standard rules was there for item construction. Coverage of curriculum was incomplete. As a result of item analysis total 34 items were rejected, 6 items were rejected on the basis of both Discrimination index and Difficulty index, 27 items were rejected due to discrimination index, 13 items were rejected on the basis of difficulty index, 2 items were difficult and 11 items were easy. For Rasch calibration PROX method was used for item difficulty and person ability. On the basis of Rasch calibration 2 items were rejected. Rasch latent continuum, Item characteristic curve and person characteristic curve were drawn according to Rasch model which made the data clearer and more understandable.

Psychometric Validation of the Chinese Version of the Atrial Fibrillation Knowledge Scale in Chinese Patients With Atrial Fibrillation.

Assessment of patients' knowledge on atrial fibrillation (AF) and its management is important for evaluating their learning needs. However, a validated and comprehensive instrument to be used among Chinese patients is yet to be developed. The aim of this study was to develop and validate the Chinese version of the Atrial Fibrillation Knowledge Scale (AFKS-C) in Chinese patients. The 11-item AFKS was translated, and then content and face validations were conducted by an expert panel and patients with AF. A sample of 255 patients with AF was recruited from a university-affiliated hospital to evaluate its psychometric properties. The internal consistency and test-retest reliability were evaluated using the Kuder-Richardson formula 20 and κ statistics. Item analysis determined the item difficulty index and item discrimination coefficients. Factorial and discriminant validity were evaluated using exploratory factor analysis and the known-groups method. The content validity index of the AFKS-C was 0.94, and the Kuder-Richardson formula 20 value was 0.60. The difficulty indices of the items ranged from 0.36 to 0.89, and the point-biserial coefficients of the items ranged from 0.122 to 0.255, indicating sufficient discriminatory ability. The test-retest reliability was acceptable, because the κ values ranged from 0.234 to 0.710. The principal axis factoring analysis indicated a 3-factor structure that explained 50.4% of the total variance. The AFKS-C also demonstrated satisfactory discriminant validity, having yielded significantly different scores between patients with newly diagnosed and established AF. The AFKS-C has acceptable psychometric properties and can be used to measure the knowledge of patients and evaluate the effects of patient education programs.

To Implement Computerized Adaptive Testing by Automatically Adjusting Item Difficulty Index on Adaptive English Learning Platform

To Implement Computerized Adaptive Testing by Automatically Adjusting Item Difficulty Index on Adaptive English Learning Platform

Reliability of physiology MCQs’ examinations at the Faculty of Medicine, University of Khartoum

-

Item difficulty & item discrimination as quality indicators of physiology MCQ examinations at the Faculty of Medicine Khartoum University

Background:
 Item analysis is an essential tool used in the evaluation of the quality of MCQ examinations.
 Objectives:
 This study aimed at assessing the quality of items of physiology MCQ examinations at the
 Faculty of Medicine, Khartoum University, using post-examination item analysis.
 Methods:
 A descriptive, cross-sectional study was carried-out on test statistics reports using Remark-
 software which provides analysis of students’ responses in the form of Excel file formats. Ten physiology
 MCQ exams held from September 2015 - September 2016 were included in the study. Each exam paper
 consisted of 60-80 (five-option) items. Item Difficulty index [DIF I] and Discrimination Index (DI) (using
 point biserial correlation coefficient/
 r
 pbis
 ) were measured as quality indicators.
 Results:
 The mean DIF I of 645 items was (56.01±19.97) & the mean
 r
 pbis
 was (0.37±0.13). The majority
 of items (63.3%) were of acceptable difficulty; (25.5%) were easy & (11.1%) were difficult. The highest
 proportion (43.5%) of items were excellent; followed by (30%) good; (16.6%) acceptable and (9.6%) poor
 discriminating items. There were only 5 negatively discriminating items that constituted 0.8% out of total
 items. The relationship between difficulty and discrimination indices was dome-shaped with maximal
 discrimination at acceptable difficulty level.
 Conclusion:
 Generation of high quality examinations with average difficulty and good discrimination
 power is the outcome of careful construction and meticulous review of exams’ items by our expert staff
 members. Maximizing exam discrimination is achieved by reducing the number of the very difficult and
 very easy items.

Validation of an instrument to determine oral health knowledge, attitudes, and practices during pregnancy

BackgroundOral health of the mother-infant dyad is important to preserve general health. However, there are few instruments in Spanish for the evaluation of knowledge, attitudes and practices that determine this construct. Therefore, this research aimed to develop and evaluate the psychometric properties of the Maternal Oral Health Knowledge, Attitudes and Practices Questionnaire (CAPSOM in Spanish).MethodsIn this instrument development study that carried out in 2018–2019, involving pregnant women between the ages of 18 and 45 in the city of Leon, Guanajuato, Mexico. The sample size was calculated based on 10 women per questionnaire item (n = 10 k). The study used Cronbach’s alpha, the modified Lawshe test of validity criteria, factor analysis, and the level of difficulty and discrimination of the items.Results207 women took part with their signed, informed consent (25 ± 6 years). The internal consistency of the instrument, both total and by dimension was α = 0.70, α = 0.66 knowledge, α = 0.74 attitudes, and α = 0.66 practices. Values of Content Validity Ratio’ ≥ 0.60 were obtained for the final 10 items and Content Validity Index’ = 0.90. The average difficulty index of items was 0.40, and there were significant differences (Kruskall–Wallis, p < 0.001) in the discrimination test. Factor analysis demonstrated three main components.ConclusionsA valid and reliable 10-item Spanish questionnaire was designed to measure pregnant women’s oral health knowledge, attitudes, and practices.

ESTABLISHING A MEASUREMENT AND EVALUATION LABORATORY IN A NURSING FACULTY: A DESCRIPTIVE STUDY

Aim of this study, to establish a measurement and evaluation laboratory of a nursing faculty and evaluate the quality of tests applied as part of the education curriculum as a Descriptive cross-sectional design. A total of 42 tests related to nursing vocational courses and 2,671 test items. The research was conducted in a state university nursing faculty between June 26, 2014 and June 28, 2018. In the first stage, a measurement and evaluation laboratory was established. In the second stage, trainers were trained. In the third stage, the item difficulty and item discrimination indexes of nursing professional tests were determined. Number and percentage distributions and averages were obtained in the analysis of the data. Of all the test items of vocational nursing courses, 40.49% in the 2015-2016 academic year and 38.90% in the 2016-2017 academic year were very easy according to the item difficulty index (0.81-1.00), and 46.83% and 43.56%, respectively had very low item discrimination indexes (0.19 and below). Most of the tests applied in nursing courses were easy, and the discrimination of all tests was very low. Further studies are recommended to investigate the item analysis of multiple-choice questions that measure nursing professional knowledge and item spelling errors.

Validity and reliability of Japanese version of the pressure ulcer knowledge assessment tool

AimTo develop Japanese version of the pressure ulcer knowledge assessment tool, a tool for measuring nurses’ pressure ulcer knowledge to effectively evaluate the efficacy of current educational programs, resulting in prevention and early treatment, and to verify its validity and reliability among Japanese nurses. Materials and methodsA total of 1716 nurses across three university hospitals participated in this study. All had been employed for at least one year. Managers, part-timers, and those in the operating room and the outpatient ward were excluded from the study as it is limited to bedside preventive care.The original tool was translated into Japanese using the Brislin's translation model. A cross-sectional study was used to examine the reliability and validity of the measure within a Japanese sample. Assessment of the multiple-choice test items included analysis of the validity (item difficulty and discriminating index), construct validity, internal consistency, and stability (test–retest reliability). ResultsThe item difficulty indices ranged from 0.17 to 0.95, whereas values for item discrimination ranged from 0.15 to 0.45. Known group validity of the scale was confirmed; therefore, the higher-expertize group consisting of wound, ostomy, and continence nurses significantly outperformed nurses certified in other fields. The overall internal consistency reliability was 0.86 with a two-week test–retest intraclass correlation of 0.60. ConclusionsThe instrument may be applied as a reliable and valid measure to assess nurses’ pressure ulcer knowledge in the fields of nursing education, research, and practice in Japan.