Outfit Mean Square Research Articles

Catquest-9SF questionnaire: Validation in a Greek-speaking population using Rasch analysis.

The Catquest-9SF questionnaire is a tool measuring visual disability and vision-related limitation in patients' daily activities. The primary objective of this study was the validation of Catquest-9SF in a Greek-speaking cataract population. The questionnaire was translated into Greek and translated back into English. A pre-final Greek version was formed and tested by 10 Greek-English bilingual participants and by the translation team, and the final version was produced. Patients scheduled for cataract surgery completed the questionnaire preoperatively and postoperatively. Rasch analysis was performed for the assessment of the Catquest-9SF psychometric properties, including response category ordering, item fit statistics, principal components analysis, precision, differential item functioning and targeting for preoperative and postoperative data collectively. A total of 100 (55 men, 45 women, mean age = 71.94±6.63) cataract patients completed the Greek version of Catquest-9SF questionnaire preoperatively and postoperatively. Rasch analysis showed a significant improvement in the median person Rasch score from -1.49 preoperatively to -4.71 logits postoperatively, while the effect size was 1.3. Unidimensionality was confirmed since infit and outfit mean square values varied between 0.66 and 1.37. Rasch analysis showed good precision and separation ability (Person Separation Index of 3.28, and Person Reliability of 0.92). Four response categories were found for all items. The item-person means difference was -1.83 logits. The difference between preoperative and postoperative Catquest-9SF logit score was positively correlated with preoperative Catquest-9SF logit score (coeff. = 0.798, p<0.0001) and negatively correlated with postoperative spherical equivalent (coeff. = -0.825, p = 0.011). The Greek version of Catquest-9SF proved to be reliable, valid, unidimensional and responsive to changes after cataract surgery presenting good psychometric properties for cataract patients. Some postoperative mistargeting was found indicating that the tasks were easily performed by respondents after cataract surgery. NCT05323526 -retrospectively registered.

Job Readiness Elements for Hospitality Graduates: A Pilot Study

Job readiness questionnaire was developed to measure the level and important element for hospitality graduate. A pilot study was conducted in order to determine the reliability and validity of the questionnaire. There are six constructs were identified such as communication, problem solving, ICT, organization and leadership teamwork, workplace attitude, self-management, and technical skills for this questionnaire. A total of 30 lecturers from Kolej Vokasional Setapak were selected to participated in this survey. Rasch Measurement Model (i.e., PTMEA CORR, item fit, item polarity) was used to analyse the responses using Winsteps. This study found that PTMEA CORR has a positive value, in which items are able to differentiate the capabilities of the respondents. Besides, the results of infit and outfit mean square are ranged between 0.60 and 1.4. The quality of items is high because the reliability value is also high. In addition, the separation of item and person is at the acceptable range. However, statistical data shows that 8 out of 122 items have been dropped.

Rater Reliability in Speaking Assessment in a Japanese Senior High School: Case of Classroom Group Discussion and Debate 日本の高校におけるスピーキング評価の採点者信頼性—教室内グループ型のディスカッションとディベートの場合

高校の授業内スピーキングテストにおいて、シンプルなルーブリックを用い、詳細な採点者トレーニングを行わない場合に採点者信頼性が十分確保できるかを、グループ型のディスカッションとディベートで検証した。227名の高校生の発話をそれぞれ教員2名で採点し、多相ラッシュ分析・一般化可能性理論等で分析した。その結果、採点者間一致度・一貫性と採点者内一貫性の観点で十分な信頼性が満たされていることが示された。グループ型タスクの場合に、ルーブリックにやり取りの適切さでなく言語面の観点を入れる、生徒が話す時間を長めに設定する、生徒の役割や発言する順番を決める、共通認識がある教員で採点を行う等の信頼性を高める方法とその問題点が示唆された。 Securing rater reliability for classroom speaking tests can be difficult because teacher-raters typically do not have much time to engage in rater training to understand and discuss rubrics and scores. Furthermore, a teacher typically faces difficulties asking colleagues to help double mark each student’s performance. Intensive rater training and double scoring are typical procedures to maintain high reliability (Knoch et al., 2021) but are not well practiced in the classroom. However, in some cases, extensive training or double scoring is not necessary when teachers use a rubric with a few criteria and levels, which is simpler than conventional detailed rubrics (Koizumi &amp; Watanabe, 2021). Thus, we use a group discussion and a debate to explore rater reliability when Japanese senior high school teachers use simple analytic rubrics without detailed rater training. We pose the following research questions (RQs): RQ1: To what degree are raters similar in terms of interrater consensus and consistency? RQ2: To what degree do raters score students’ responses consistently? RQ3: How many raters are required to maintain reliability? We analyzed ratings for two speaking tests administrated in September or November to 227 third-year students at a public senior high school. Each test, taken by a group of four students, included either a five-minute group discussion or a 21-minute group debate; the test administration and marking were conducted during the lesson time. An analytic rubric was developed for each task and consisted of three or four criteria with three levels (e.g., content, expression, and technique). Two of the three raters scored each student’s response during the test. Teachers had no time to discuss the rubrics in detail and engaged in only a 10-minute discussion about the rubrics before the tests. The ratings were analyzed separately for each test using weighted kappa statistics, Spearman’s rank-order correlations, many-facet Rasch measurement (MFRM), and multivariate generalizability theory (mG theory). The results indicated that the overall rater reliability was adequate, but some cases required careful training. For RQ1, the kappa statistics of two raters’ scores for each criterion ranged from poor to substantial agreement (-.06 to .84). Correlations between two raters’ scores ranged from negligible to strong (-.07 to .91) and there were not large differences in rater severity (i.e., differences in fair mean-based average values of 0.07 to 0.16 with full marks of 3). In addition, the overall agreement percentages from MFRM were higher than those predicted by MFRM (e.g., 72.9% &gt; 71.6%). The intrarater consistency examined for RQ2 using Infit and Outfit mean squares from MFRM was also adequate (e.g., 0.86 to 1.35). The number of raters needed to maintain sufficient reliability (Φ = .70) for RQ3 was one at the overall test levels and one to three at the criterion levels. Using simple rubrics, a group discussion task, and a debate task, the results showed that rater reliability can be maintained without extensive rater training. Although the current results may have been affected by study contexts, such as procedures and students’ and raters’ characteristics, they provide pedagogical and methodological implications for developing speaking assessment tasks and procedures and reporting rater reliability statistics from multiple perspectives.

Translation and validation of convergence insufficiency symptom survey to Italian: Psychometric results

PurposeThis study aimed to translate the Convergence Insufficiency Symptom Survey (CISS) into the Italian language and assess psychometric properties of the translated questionnaire (CISS_I). MethodsThe CISS_I was arranged according to guidelines for a comprehensive multistep methodologic process for translating, adapting, and validating psychometric instruments in health care research. The CISS_I questionnaire was administered to 103 volunteers (21.8 ± 2.2 years), students in higher education, at two different times. A complete optometric evaluation was performed including subjective refraction, best corrected visual acuity, near point of convergence, prism fusional ranges to blur, diplopia and recovery, TNO stereo test and prism cover test for measurement of heterophoria. ResultsThe performance of the CISS_I in terms of validity showed some points of weakness. Sensitivity was 42%, specificity was 74%, positive predictive value was 27% and negative predictive value was 85%. The area under the ROC curve was 0.672. On the contrary, the results showed good internal consistency of the CISS_I (Cronbach's alpha - α=0.89) and good test-retest reliability (ICC = 0.92). Rasch analysis showed good model fit (all items, except one, with infit and outfit mean square between 0.7 and 1.3), good measurement precision (person separation = 2.66) and good targeting –0,81 logits but also some evidence of multidimensionality. ConclusionsThe CISS_I showed some point of weakness in terms of validity but also good psychometric properties and has been shown to be applicable to an Italian speaking population to quantify the visual discomfort associated with near vision in higher education students. The results show that high CISS_I score is not necessarily linked to convergence insufficiency, while low scores can exclude the presence of this anomaly. The CISS_I can help in interpreting and monitoring convergence insufficiency symptoms in already identified subjects, but it is not suitable for screening a general population of young adults.

Development, validation and item reduction of a food literacy questionnaire (IFLQ-19) with Australian adults

BackgroundFood literacy is theorised to improve diet quality, nutrition behaviours, social connectedness and food security. The definition and conceptualisation by Vidgen & Gallegos, consisting of 11 theoretical components within the four domains of planning and managing, selecting, preparing and eating, is currently the most highly cited framework. However, a valid and reliable questionnaire is needed to comprehensively measure this conceptualisation. Therefore, this study draws on existing item pools to develop a comprehensive food literacy questionnaire using item response theory.MethodsFive hundred Australian adults were recruited in Study 1 to refine a food literacy item pool using principal component analysis (PCA) and item response theory (IRT) which involved detailed item analysis on targeting, responsiveness, validity and reliability. Another 500 participants were recruited in Study 2 to replicate item analysis on validity and reliability on the refined item pool, and 250 of these participants re-completed the food literacy questionnaire to determine its test–retest reliability.ResultsThe PCA saw the 171-item pool reduced to 100-items across 19 statistical components of food literacy. After the thresholds of 26 items were combined, responses to the food literacy questionnaire had ordered thresholds (targeting), acceptable item locations (< -0.01 to + 1.53) and appropriateness of the measurement model (n = 92% expected responses) (responsiveness), met outfit mean-squares MSQ (0.48—1.42) (validity) and had high person, item separation (> 0.99) and test–retest (ICC 2,1 0.55–0.88) scores (reliability).ConclusionsWe developed a 100-item food literacy questionnaire, the IFLQ-19 to comprehensively address the Vidgen & Gallegos theoretical domains and components with good targeting, responsiveness, reliability and validity in a diverse sample of Australian adults.

The Development and Validation of the Comprehensive Assessment of Acceptance and Commitment Therapy Processes (CompACT)-Malay Version.

Objectives: psychological flexibility is a crucial construct highly correlated with psychological wellness. There is a need for a tool to measure psychological flexibility in order to accurately ascertain the effects of treatment. The existing industry standard, the Acceptance and Action Questionnaire-II (AAQ-II), has issues with conflating psychological flexibility with distress; moreover, it does not cover the hexaflexes. The 23-item CompACT was designed to surmount these limitations. Methods: the classical test theory (CTT) and Rasch measurement theory (RMT) were used to check the validity and reliability of the Malay version of the CompACT Scale. Cronbach’s α, McDonald’s Ω, and greatest lower bound were used to measure internal consistency. A Pearson’s correlation test was used to measure test–retest reliability of the Malay versus the original English version. For validity, convergent validity was established by using the Malay AAQ-7 Scale. The dimensionality of the Malay version of the CompACT Scale was explored using exploratory factor analysis. For the RMT, weighted fit statistics (infit) and outlier sensitive fit statistic (outfit) mean square (MnSq) values were used at the item level, while item and person separation reliability values and item and person separation indices were applied at the scale level. Results: the internal consistency measures, including Cronbach α and McDonald’s Ω, passed the suggested cutoff points. Convergent validity with the AAQ-II was 0.693. The quality of the Malay version of the CompACT Scale was also satisfactory, as all item and person reliability values and indices exceeded the suggested cut-off points. Conclusions: the Malay CompACT is a psychometrically sound instrument to assess psychological flexibility in both clinical and research settings.

Psychometric properties of the 26-item eating attitudes test (EAT-26): an application of rasch analysis

BackgroundThe 26-item Eating Attitudes Test (EAT-26) is a commonly used tool to assess eating disorder risk. The purpose of this study was to examine the psychometric properties of the EAT-26 with a combined sample: (1) of adults with overweight and obesity enrolled in a behavioral weight loss program and (2) general adult sample (n = 469; age = 36.17 ± 17.83 years; female = 72.5%; white = 66.3%; obese BMI category = 58%).MethodsRasch modeling was used to assess model-data fit, create an item-person map to evaluate relative distribution items and persons, item difficulty, and person’s eating disorder (ED) risk level of the EAT-26. Differential item functioning (DIF) and rating scale functioning of the EAT-26 were also evaluated using Rasch analysis.ResultsA total of 7 misfit items were removed from the final analysis due to unacceptable Infit and Outfit mean square residual values. The item-person map showed that the items were biased toward participants with moderate to high levels of ED risk and did not cover those who had low risk for having an ED (< − 1 logits). The DIF analyses results showed that none of the items functioned differently across sex, but 5 items were flagged based on obesity status. The six-category Likert-type rating scale did not function well indicating a different response format may be needed.ConclusionSeveral concerns were identified with the psychometric evaluation of the EAT-26 that may question its utility in assessing ED risk in individuals at low risk for ED, within samples of people who have overweight and obesity seeking weight loss treatment.Plain English SummaryThe 26-item Eating Attitudes Test is a self-rated measure of eating attitudes that measures symptoms and concerns of eating disorders (ED). Very little is known about how this instrument performs differently based on individual factors like body mass index (BMI) and sex (male/female). We used an advanced measurement theory (i.e., Rasch analysis) to determine if the EAT-26 is an adequate measure to detect disordered eating in men and women of different BMIs. Results indicated that the EAT-26 was biased toward participants with moderate to high levels of disordered eating risk and did not adequately detect individuals at low risk for disordered eating. The EAT-26 did not function differently based on sex (male/female). However, five questions did function differently based on obesity status (those without obesity/ those with obesity). Finally, we observed the six-category rating scale did not function appropriately and that a new response format may be warranted. In sum, there were several issues (e.g., poor rating scale and different item functioning) with the EAT-26 and future work should develop screening tools that detect low risk of disordered eating as well as function well in adults with overweight and obesity.

Development and validation of the symptom burden questionnaire for long covid (SBQ-LC): Rasch analysis

ObjectiveTo describe the development and validation of a novel patient reported outcome measure for symptom burden from long covid, the symptom burden questionnaire for long covid (SBQ-LC).DesignMultiphase, prospective mixed methods...

Reliability and Validity of the Knee Injury and Osteoarthritis Outcome Score in Patients Undergoing Unicompartmental Knee Arthroplasty

BackgroundThe Knee Injury and Osteoarthritis Outcome Score (KOOS) was developed to document outcomes from knee injury, including the impact of osteoarthritis on knee function. The purpose of this study is to determine the reliability and validity of the KOOS subscales for evaluating outcomes following unicompartmental knee arthroplasty (UKA). MethodsKOOS Pain, Activities of Daily Living (ADL), Sport, Symptoms, and Quality of Life (QoL) scores collected from 172 patients who underwent UKA were used in the analysis. KOOS subscales were tested for reliability and validity of scores through a Rasch model analysis. ResultsKOOS Sport, KOOS ADL, and KOOS QoL had good evidence of reliability with acceptable person reliability, person separation, and item reliability. For overall scale functioning, KOOS Pain, Symptoms, and ADL all had 1 question that did not have an acceptable value for infit or outfit mean square value. Questions in KOOS Sport and QoL all had acceptable values. There was a positive, linear relationship between the Short-Form 12 Physical Component Summary and the KOOS subscales which indicated good evidence of convergent validity. These associations were also seen when the cohort was separated in medial and lateral UKA. ConclusionTwo of the 5 KOOS subscales (KOOS Sport and KOOS QoL) were considered adequate in measuring outcomes, as well as reliability. The KOOS ADL had borderline values; however, it had adequate infit and outfit values. The KOOS Pain and Symptom score performed poorly in this analysis. For documenting outcomes following UKA, this study supports the use of KOOS ADL, Sport, and QoL.

Rasch analysis of fatigue severity scale in patients with epilepsy

ObjectiveTo utilize the Rasch model to validate and assess the psychometric properties of the Fatigue Severity Scale (FSS) in patients with epilepsy. MethodsA total of 307 patients (age > 18 years) with a confirmed diagnosis of epilepsy were consented to participate. Exclusion criteria included patients with psychogenic nonepileptic events, cognitive disabilities, and patients who did not speak/understand English. The nine-step FSS was programmed into software administered to patients on electronic tablets, and patient responses were auto-scored. The Rasch rating scale model (RSM) was used to evaluate the unidimensionality, reliability, and targeting of the FSS. To assess unidimensionality, we examined infit and outfit mean squares. We also assessed unidimensionality of the FSS using a principal component analysis of Rasch residuals, where residuals are understood as the difference between observed and expected data values. We evaluated the internal consistency of person and item performance by examining separation reliability estimates and separation ratio. Differential Item Functioning (DIF) was calculated for gender. ResultsThere was mixed evidence regarding the extent to which the FSS fit the Rasch model. Outfit values ranged from 0.52 to 2.72 and infit values were 0.60 to 2.18, strongly suggesting the presence of misfitting items: Item 1 (“My motivation is lower when I am fatigued”) and Item 2 (“Exercise brings on my fatigue”). SignificanceThe nine-item FSS showed fair psychometric properties in this sample of patients with epilepsy. Our study provides unique, supportive information for the use of a modified version of the FSS, omitting the first two items, in patients with epilepsy. Given the prevalence of fatigue and other neuropsychiatric comorbidities of epilepsy, having a validated fatigue scale can aid healthcare providers to identify moderate-to-severe fatigue levels in patients with epilepsy and address the plausible risk factors.

Psychometric Validation of the Athens Insomnia Scale Among Nurses: A Robust Approach Using Both Classical Theory and Rating Scale Model Parameters.

PurposeFew studies have investigated the validity of the Athens insomnia scale (AIS) using a robust approach of both classical theory and the rating scale model. Therefore, in this study, we investigated psychometric validation of the AIS using both of these approaches in nurses.MethodsNurses (n= 563, age= 33.2±7.1 years) working in health facilities in Saudi Arabia participated in a cross-sectional study. Participants completed the AIS, socio-demographics tool, and sleep health-related questions.ResultsConfirmatory factor analysis (CFA) favored a 2-factor structure with both comparative fit index (CFI), and incremental fit index (IFI) having values above 0.95. The 2-factor model had the lowest values of Akaike information criterion (AIC), root mean square error of approximation (RMSEA), χ2, and χ2/df. This 2-factor structure showed configural invariance (CFI more than 0.95, RMSEA less than 0.08, and Χ2/df less than 3), and metric, scalar, and strict invariance (based on Δ CFI ≤-0.01, and Δ RMSEA ≥ 0.015 criteria). No ceiling/floor effects were seen for the AIS total scores. Infit and outfit mean square values for all the items were within the acceptable range (<1.4, >0.6). The threshold estimates for each item were ordered as expected. Cronbach’s α for the AIS tool, factor-1 score, factor-2 score was 0.86, 0.82, and 0.72, respectively. AIS factor scores-1/2 were significantly associated with a habitual feeling of tiredness after usual night sleep (p<0.001), Impairment of daytime socio-occupational functioning (p<0.05), and with a feeling of daytime fatigue, irritability, and restlessness (p<0.05).ConclusionThe findings favor the validity of a 2-factor structure of the AIS with adequate item properties, convergent validity, and reliability in nurses.

Determining the Optimal Virtual Reality Exergame Approach for Balance Therapy in Persons With Neurological Disorders Using a Rasch Analysis: Longitudinal Observational Study.

BackgroundVirtual reality (VR) exergames have gained popularity in the rehabilitation of persons with neurological disorders as an add-on therapy to increase intensity of training. Intensity is strongly dependent on the motivation of the patient. Motivation can be increased by delivering variation within training and challenging exercises. However, patients are often underchallenged, as exergame difficulty often does not match the patient’s ability. A Rasch analysis can establish hierarchy of exergame items in order to assist the delivery of patient-centered therapy.ObjectiveThe aim of this study was to apply the Rasch model to create a hierarchical order of existing VR balance exergames and to relate these exergames to the abilities of persons with neurological disorders, in order to deliver challenge and variation.MethodsA total of 30 persons with stroke and 51 persons with multiple sclerosis (MS) were included in the study. All participants performed a training program, lasting 3 weeks for persons with MS and 4 weeks for persons with stroke, in which they performed VR balance exergames with a movement recognition–based system (MindMotion GO; MindMaze SA). VR exercise scores, Berg Balance Scale scores, and clinical descriptive data were collected. Berg Balance Scale and device scores were analyzed with the Rasch model using a repeated-measures approach to examine whether the distribution of exercise scores fitted the Rasch model. Secondly, a person-item map was created to show the hierarchy of exercise difficulty and person ability.ResultsParticipants completed a selection of 56 balance exercises (ie, items), which consisted of a combination of various balance tasks and levels (ie, exercises). Using repeated measures, this resulted in a count of 785 observations. Analysis showed strong evidence for unidimensionality of the data. A total of 47 exercises (ie, items) had a sufficiently good fit to the Rasch model. Six items showed underfit, with outfit mean square values above 1.5. One item showed underfit but was kept in the analysis. Three items had negative point-biserial correlations. The final model consisted of 47 exercises, which were provided for persons with low to moderate balance ability.ConclusionsThe VR exercises sufficiently fitted the Rasch model and resulted in a hierarchical order of VR balance exercises for persons with stroke and MS with low to moderate balance ability. In combination with the Berg Balance Scale, the results can guide clinical decision-making in the selection of patient-focused VR balance exercises.Trial RegistrationClinicalTrials.gov NCT03993275; https://clinicaltrials.gov/ct2/show/NCT03993275

A Rasch analysis assessing the reliability and validity of the Arizona CoVHORT COVID-19 vaccine questionnaire

BackgroundDespite the widespread availability of COVID-19 vaccines in the United States, many that have chosen not to be vaccinated have done so because of vaccine hesitancy. This highlights the need for tools that accurately capture the knowledge, attitudes, and beliefs towards COVID-19 vaccines, and provide steps toward improving vaccine acceptance. MethodsParticipants of the Arizona CoVHORT (COVID-19 Cohort) received a one-time, electronic based cross-sectional questionnaire intended to capture underlying motivations regarding vaccination, as well as hesitations that may prevent people from getting vaccinated. Rasch analysis was conducted among 4703 CoVHORT participants who had completed the vaccine questionnaire to assess questionnaire reliability and validity. Response categories were grouped to optimize scale functioning and to ensure independent probabilities of participant endorsement. ResultsA total of 4703 CoVHORT participants completed the questionnaire, of whom 68% were female, and who had a mean age of 48 years. Participants were primarily White (90%), highly educated (63% with a college degree or above, with most respondents (45%) having an income of more than $75,000 per annum. The results indicated the questionnaire has good reliability and construct validity for assessing attitudes and beliefs about the COVID-19 vaccines. In-fit mean-squares for included items ranged from 0.61 to 1.72 and outfit mean-squares ranged from 0.56 to 1.75, and correlation coefficients ranged from 0.25 to 0.75. The person-item map indicated normal distribution of logit scores measuring perceptions about COVID-19 vaccinations. ConclusionsThe CoVHORT vaccine questionnaire demonstrated satisfactory reliability and construct validity in assessing attitudes and beliefs about COVID-19 vaccines. Overall results provide a starting point for a reliable and valid tool to assess knowledge and perceptions about COVID-19 vaccination, ultimately providing public health professionals with an instrument to assess the factors that are associated with vaccine acceptance or hesitancy.

Online Mental Health Survey for Addressing Psychosocial Condition During the COVID-19 Pandemic in Indonesia: Instrument Evaluation

BackgroundRegular monitoring of the pandemic’s psychosocial impact could be conducted among the community but is limited through online media. This study aims to evaluate the self-rating questionnaire commonly used for online monitoring of the psychosocial implications of the coronavirus disease 2019 (COVID-19) pandemic.MethodsThe data were taken from the online assessment results of two groups, with a total of 765 participants. The instruments studied were Self-Rating Questionnaire (SRQ-20), post-traumatic stress disorder (PTSD), and Center for Epidemiological Studies Depression Scale-10 (CESD-10), used in the online assessment. Data analysis used Rasch modeling and Winsteps applications. Validity and reliability were tested, and data were fit with the model, rating scale, and item fit analysis.ResultsAll the scales for outfit mean square (MnSq) were very close to the ideal value of 1.0, and the Chi-square test was significant. Item reliability was greater than 0.67, item separation was greater than 3, and Cronbach’s alpha was greater than 0.60; all the instruments were considered very good. The raw variance explained by measures for the SRQ-20, PTSD, and CESD-10 was 30.7%, 41.6%, and 47.6%, respectively. The unexplained eigenvalue variances in the first contrast were 2.3, 1.6, and 2.0 for the SRQ-20, PTSD, and CESD-10, respectively. All items had positive point-measure correlations.ConclusionThe internal consistency of all the instruments was reliable. Data were fit to the model as the items were productive for measurement and had a reasonable prediction. All the scales are functionally one-dimensional.

Examining the STEM-Science Achievement Test (SSAT) Using Rasch Dichotomous Measurement Model

The main purpose of this study is to develop a valid and reliable instrument to measure the STEM-Science achievement of primary school students in Malaysia. Six Year 4 Science topics (Scientific Skills, Life Processes of Human, Properties of Materials, Measurement, Solar System, Importance of Technologies in Life) and Six Year 5 Science topics (Rules and Regulation in Science Lab, Life Processes of Plants, Acids and alkali, Electricity, Earth and Space Science, Technology and Sustainable Life) have been included in the development of the STEM-Science Achievement Test (SSAT). 226 Year 4 and 226 Year 5 primary school students in Sabah responded to the developed instrument to test their STEM-Science knowledge. The Rasch dichotomous measurement model approach was used to evaluate the validity and reliability of the SSAT. The validity assessed the Point-Measure Correlation (PTMEA CORR), Principal Component Analysis of Residuals (PCAR), as well as Infit and Outfit Mean Squares (MNSQ). In terms of reliability, Cronbach’s alpha, item reliability, and item separation index were analysed. The analysis results revealed the presence of unidimensionality for both objective and subjective items. For objective items, the reported values for Cronbach’s Alpha are .81 and .83; item reliability are .95 and .95; item separation are 4.21 and 4.25 for Year 4 and Year 5 students, respectively. Standardised residual correlations for Year 4 and Year 5 subjective items also showed satisfactory values. The assessment using Rasch measurement model has proven that SSAT is a valid and reliable instrument to measure Malaysian primary students’ STEM-Science-related knowledge.

Developing and Validating the Narcissistic Personality Scale (NPS) among Older Thai Adults Using Rasch Analysis.

Background: Being older could be stressful, especially among people with narcissistic personality disorders. Nevertheless, the tool is yet to be available among older Thai individuals. The study aimed to develop a tool to detect symptoms of narcissistic personality, and to validate its psychometric properties among older Thai adults. Methods: The Narcissistic Personality Scale (NPS) was developed based on nine domain symptoms of narcissistic personality disorder from the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5), consisting of 80 items. The original scale was field-tested using Rasch analysis for item reduction, rendering a final 43 items. NPS was further investigated among 296 seniors aged 60 years old. Rasch analysis was used to assess its construct validity. Result: Of 43 items, 17 were further removed as infit or outfit mean square >1.5. The final 26-item NPS met all necessary criteria of unidimensionality and local independence without differential item functioning due to age and sex, and good targeting with subjects. Person and item reliability were 0.88 and 0.95, respectively. No disordered threshold or category was found. Conclusions: The NPS is a promising tool with a proven construct validity based on the Rasch measurement model among Thai seniors. This new questionnaire can be used as outcome measures in clinical practice.

Determining English Language Lecturers’ Quality of Marking in Continuous Assessment through Rasch Analysis

English language lecturers at matriculation colleges are generally equipped with assessment criteria for marking students’ written assessment. However, these criteria are normally susceptible to lecturers’ interpretation and understanding, which threatens quality marking. Therefore, this study aimed to determine the severity and consistency of English language lecturers’ marking of English academic writing (EAW) in continuous assessment. The participants were five English language lecturers and 50 matriculation students. Each lecturer selected ten EAWs randomly from 318 matriculation students. The five-part EAW was marked first by the class’s lecturer and later, it was marked by pre-assigned peer moderators who did not teach the students. The total data set collected was 250 (5 lecturers x 10 EAWs x 5 parts of EAW). The data were analyzed with Many-Facets Rasch Measurement (MFRM) application. Semi-structured interviews were conducted with both lecturers and students for triangulation purposes. Findings revealed that four out of five lecturers were lenient in marking but the marking was found to be internally consistent with infit and outfit mean squares for each lecturer ranged between 0.5 and 1.5. From interview responses analyzed, students perceived their lecturers as fair but strict in awarding marks. These responses were consistent with most lecturers’ responses on their strict adherence to assessment criteria. Discussion of findings is centered on the issue of severity and consistency of the assessors. This study could offer a practical solution in providing evidence for quality marking of written assessment and, consequently, aid in developing remedial measures for misfit assessors in educational institutions.

The Falls Efficacy Scale International is a valid measure to assess the concern about falling and its changes induced by treatments

Objective To test with the Rasch analysis the psychometric properties of the Falls Efficacy Scale International, a questionnaire for measuring concern about falling. Design Longitudinal observational study, before–after rehabilitation. Setting Inpatient rehabilitation. Subjects A total of 251 neurological patients with balance impairment. Interventions Physiotherapy and occupational therapy aimed at reducing the risk of falling. Main measures Participants (median age, first–third quartile: 74.0, 65.5–80.5 years; stroke and polyneuropathy: 43% and 21% of the sample, respectively) received a balance assessment (Falls Efficacy Scale International included) pre- and post-rehabilitation. Rasch analysis was used to evaluate the Falls Efficacy Scale International. Differential item functioning, which assesses the measures’ stability in different conditions (e.g. before vs. after treatment) and in different groups of individuals, was tested for several variables. Results Patients suffered a moderate balance impairment (Mini-BESTest median score; first–third quartile: 15; 11–19), mild–moderate concern about falling (Falls Efficacy Scale International: 28; 21–37) and motor disability (Functional Independence Measure, motor domain: 70.0; 57.0–76.5). Falls Efficacy Scale International items fitted the Rasch model (range of infit and outfit mean square statistics: 0.8–1.32 and 0.71–1.45, respectively) and the questionnaire's reliability was satisfactory (0.87). No differential item functioning was found for treatment, gender, age and balance impairment. Differential item functioning was found for diagnosis and disability severity, but it is shown that it is not such as to bias measures. Conclusions Falls Efficacy Scale International ordinal scores can be turned into interval measures, i.e. measures of the type of temperature. Being differential item functioning-free for treatment, these measures can be safely used to compare concern about falling before and after rehabilitation, such as when interested in assessing the rehabilitation effectiveness.

Adaptation and Validation of the Cognitive and Affective Mindfulness Scale-Revised (CAMS-R) in People Living with HIV in Myanmar

ObjectivesValid and reliable instruments for the measurement of mindfulness are crucial for people living with HIV. However, there was no Myanmar version of such an instrument.MethodsWe adapted the English version of the 12-item Cognitive and Affective Mindfulness Scale-Revised (CAMS-R) based on standard cross-cultural procedures. By randomly sampling methods, a sample of 248 eligible people living with HIV was contacted from a closed Myanmar Facebook group; 159 PLHIV completed the initial 12-item version of the adapted survey.ResultsThree items were removed due to low item-to-total correlations of the corrected item-total correlation as well as having infit and outfit mean squares outside the range of 0.6 to 1.4. After deleting the 3 items, the three-factor structure was confirmed by confirmatory factor analysis, which indicated good model fit. The resultant 9-item CAMS-R in Myanmar (CAMS-R-M-2) achieved good internal reliability (Cronbach’s α of 0.75 to 0.87, and the corrected item-total correlation ranged from 0.44 to 0.81). Construct validity of the scale was demonstrated by significant association with self-reported HIV stigma and social support levels (r = 0.63, and − 0.53). In Rasch analysis, the infit and outfit mean squares for each item ranged from 0.49 to 1.24, and the person reliability was 2.17 and the separation index was 0.83.ConclusionsThe 9-item CAMS-R-M-2 with a three-factor structure has good reliability and validity. Higher total scores and subscale score reflected greater mindfulness qualities in people living with HIV in Myanmar.

Development of the 8-Item Phlegm Pattern Questionnaire (PPQ-8) Using Rasch Analysis.

The 25-item Phlegm Pattern Questionnaire (PPQ) has been widely used to examine the relationship between the phlegm pattern (PP), quality of life, tongue colour, vocal qualities, and dysfunctional breathing. However, the concerns of response burden and differences in the respondent's abilities or item difficulty for the original version of the PPQ have not been sufficiently addressed. This study aimed to develop a short-form PPQ using Rasch analysis, an item response theory. Based on the retrospective data, the response order, differential item functioning (DIF), dimensionality, reliability, concurrent validity, and fitting errors were examined for 291 normal participants and 61 inpatients. The discriminative ability of the short-form PPQ was examined using receiver operating characteristic curve analysis. Along with Rasch analysis, another short-form PPQ was developed using equidiscriminative item-total correlation (EITC) analysis and the results between the two short-form PPQs were compared accordingly. Rasch analysis results suggested a 6-point response category for the PPQ, and finally, 8 items without fitting errors or DIF variability were selected for the PPQ (PPQ-8). The PPQ-8 had satisfactory reliability (person separation index = 2.23), unidimensionality (unexplained variance in the first contrast = 1.598), fitting levels (infit mean square, 0.80–1.39; outfit mean square, 0.79–1.34), sensitivity (70.5%), and specificity (76.5%). The PPQ-8 had a moderate discriminative ability of the PP (area under the curve = 0.759), and the cut-off point was 23. Although the 8-item PPQ developed using EITC analysis showed similar levels of reliability, validity, and discriminative ability of the PP to the PPQ-8, it could not present the information of item hierarchy and differences in the respondents' abilities. In conclusion, the PPQ-8 by Rasch analysis is recommended for future use to evaluate the clinical severity of PP.