Comprehension Items Research Articles

Head and Neck Cancer Survivors' Assessment of Mealtimes: Translation and Validation : Assessment and Rehabilitation of Dysphagia in Head and Neck Cancer Patients.

Dysphagia is a prevalent complication before, during and after treatment for head and neck cancer (HNC). Besides the medical and societal consequences, dysphagia has a negative impact on functioning, activity, participation and quality of life. These aspects are all affected by the environmental factors (EF). However, patient-reported outcome measures (PROMS) such as the Head-and-Neck Cancer Survivors' Assessment of Mealtimes (HNSAM), which thoroughly assess participation and EF in addition to function and activity, are rare. Therefore, this study aimed to translate and validate the HNSAM into Dutch (D-HNSAM). The HNSAM was translated according to the standardized procedure of translation & back-translation and according to the international cross-cultural adaptation process. A pilot study was then conducted with 10 HNC patients to assess the linguistic features and comprehensibility of the test items. Finally, the D-HNSAM was completed by 50 participants who were at least 6months post-treatment for HNC. The Performance Status Scale for Head and Neck cancer patients (PSS-HN)- subscales normalcy of diet and eating in public, the Dysphagia Handicap Index (DHI), the Functional Oral Intake scale (FOIS) and the Utrecht Scale for Evaluation of Rehabilitation-Participation (USER-P) were used to examine the psychometric properties of the D-HNSAM. High correlations with related assessment tools and low correlations with unrelated assessment tools were expected. Internal consistency was found to be weak to good. Test-retest reliability, convergent validity and divergent validity were demonstrated except for the EF subscale. The D-HNSAM can detect differences in impact of dysphagia on daily functioning and quality of life. The D-HNSAM is a reliable and clinically valuable PROM for assessing the impact of dysphagia on daily functioning and quality of life in patients with HNC. The unique aspect of this PROM, the subscale EF, has unfortunately weak psychometric properties and requires further refinement.

12481 Evaluating The Device Handling And Preference Assessment Questionnaire For Growth Hormone Deficiency: Results From Content Validation Interviews

Abstract Disclosure: J. Neergaard: Employee; Self; Novo Nordisk. S. Akhtar: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. B. Berg: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. S. Hamilton: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. E. Hildebrand: Consulting Fee; Self; Novo Nordisk. J. Medina: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. G. Ter-Borch: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. N. Rasmussen: Employee; Self; Novo Nordisk. Stock Owner; Self; Novo Nordisk. Background: Patient experiences with injection devices may affect treatment adherence, and hence it is important to ensure patient-reported outcomes (PRO) instruments used to assess experience are supported by evidence of validity. As part of a study that compared patient preference and ease of use between the devices used to administer Somapacitan (Sogroya®, Novo Nordisk A/S) and Somatrogon (Ngenla®, Pfizer), we assessed relevance, comprehensiveness and comprehension of a PRO instrument, the Device Handling and Preference Assessment Questionnaire (DHPAQ), in adolescent patients with growth hormone deficiency (GHD) and their caregivers. Methods: We recruited 33 adolescent patients with GHD and 37 caregivers across five sites in the US for an open label, randomized, crossover study in which participants simulated an injection with each device using an injection pad or mannequin. The DHPAQ was used to assess patients’ device preference. Two-sided statistical tests with 5% significance levels were performed using DHPAQ responses. After completing the DHPAQ, a random subgroup of 10 adolescents and 10 caregivers were offered additional compensation to participate in a cognitive interview (CI). The CIs were conducted in accordance with the US Food and Drug Administration 2022 Principles for PRO Instruments and 2009 PRO Guidance for Industry. Trained interviewers used a structured guide to evaluate questions, statements, tasks and response options in the DHPAQ. Participants’ interview responses for each question were coded and added to summary tables. These summary tables were evaluated by the study team to determine the relevance, comprehensiveness and participant comprehension of instructions, item stems and response options. Results: When asked if any questions in the DHPAQ did not seem relevant to their device experience, all 20 CI participants answered “no”. Questions on “learning how to use the device”, “selecting the correct dose” and “injecting the dose” were ranked as most relevant. Participants responses from the CIs supported perceived comprehensiveness of all DHPAQ items. All CI participants reported comprehension of all test items in the DHPAQ, and only three items did not receive a “no difficulty” response from all 20 participants. In the main study, the number of participants who preferred the somapacitan device was significantly greater than those who preferred the somatrogon device (84.3% vs 12.9%, p<0.0001). More participants found the somapacitan device easy or very easy to use compared with the somatrogon device (98.6% vs 74.3%). Conclusions: The results from the CIs indicated that the instructions, item stems and response options of the DHPAQ were relevant, comprehensive and fully comprehended by all participants. In the main study, patients with GHD and their caregivers were found to prefer the somapacitan device to the somatrogon device. Presentation: 6/2/2024

Can the different versions of the Shirom-Melamed Burnout Measure be used to measure burnout among healthcare professionals? A systematic review of psychometric properties

BackgroundThe prevalence of Occupational Burnout is high among healthcare professionals (HCP). Hence, it is crucial to have robust measures for ascertaining burnout in this population. The Shirom-Melamed Burnout Measure is a prevalent tool used in the diagnosis of burnout, and in the delivery planning of mental health services. The 14-item Shirom-Melamed Burnout Measure (SMBM) was developed after a methodological revision of the 22-item Shirom-Melamed Burnout Questionnaire (SMBQ). Studies on the psychometric properties of the SMBM and SMBQ exist, but there remains a need for thorough evaluation to assess the methodological quality of individual studies. To address this gap, this systematic review aimed to critically appraise the measurement properties of the different versions of the Shirom-Melamed Burnout Measure/Questionnaire (SMBM/Q) used among healthcare professionals.MethodologyFour databases (PubMed, CINAHL, PsychINFO, and Scopus) were searched for studies on the psychometric properties of all versions of the SMBM/Q among HCP. The methodological quality of the studies was evaluated using the COSMIN Risk of Bias checklist. Evidence supporting the measurement properties (EMP) of the SMBM was synthesized using the Grading of Recommendations Assessment, Development, and Evaluation (GRADE) approach.ResultsOur final pool consisted of five research articles. One study on the 12-item SMBM was included to estimate content validity, two studies reported on the 14-item SMBM, while the other two employed the 22-item version. Interestingly, each study used the SMBM in a different language, namely English/Hebrew, Chinese, French, Serbian, and Swedish. Structural validity, internal consistency, and construct validity are the SMBM’s most assessed measurement properties. The Hebrew and French versions demonstrated high levels of structural validity and internal consistency, and the remaining reports on validity demonstrated low levels due to methodological flaws.ConclusionsPer COSMIN guidelines, the SMBM should not be utilized for clinical purposes due to insufficient content validity, but has promising potential with ongoing research. Engaging critical stakeholders for concept elicitation will ensure the relevance, comprehensiveness, and comprehensibility of the PRO items. Likewise, establishing an MIC will allow capturing change over time, which will benefit longitudinal experimental studies.Plain English summaryOccupational burnout is a significant problem among healthcare professionals, and it is crucial to have a reliable tool to measure it. The Shirom-Melamed Burnout Measure (SMBM) is commonly used to diagnose burnout and plan mental health services. Studies on the psychometric properties of the SMBM exist, but there remains a need for thorough evaluation to assess the methodological quality of individual studies. To address this gap, this systematic review critically appraised the measurement properties of the different versions of the Shirom-Melamed Burnout Measure (SMBM) used among healthcare professionals. Our findings indicate that only a few studies have examined the SMBM, and they have used the tool in different languages. Structural validity, internal consistency, and construct validity are the SMBM’s most assessed measurement properties. We recommend that more research is needed to assess the content validity of the SMBM. We also suggest that critical stakeholders should be involved in the development of the SMBM to ensure that it is relevant, comprehensive, and understandable.

Measuring wellbeing in the Global Flourishing Study: insights from a cross-national analysis of cognitive interviews from 22 countries

AbstractPrior research suggests that cross-national cognitive interviewing can provide preliminarily insights into the extent to which survey items that will be employed in large-scale global research might be understood similarly or differently across countries. Against the backdrop of the recently launched Global Flourishing Study, we used multinational cognitive interview data from 116 individuals (M = 41.4 years, SD = 14.9, female = 50.9%) in 22 countries to explore similarities and differences in item difficulty and comprehension of five of the Global Flourishing Study survey items that are related to personal wellbeing. Interviewer observations indicated that most participants (≥ 90%) did not experience a lot of difficulty responding to each of the items. Focusing on a specific comprehension probe that was common across the five items (i.e., “In your own words, what is this question asking?”), we applied the constant comparative analytic method to generate an overarching theme for each item that captured the common core or essence of how participants across the countries interpreted the items. However, there was some variability—both between and within countries—in how responses to the probe in the different countries related to the overarching theme for each item. Given the richness of the Global Flourishing Study as a possible data resource, including its broad representativeness of the global population and availability as an open access dataset, these findings will be useful to researchers who are interested in using data from the Global Flourishing Study to study human wellbeing in different parts of the world.

Validation of the mental visualization effect on answer selection in closed questions in surveys

The selection of listed answers in surveys is influenced by numerous aspects. At the moment of decoding, the mental representation of the item can represent a bias if it is distorted by personal impressions. The present research aims to identify these influences at the moment of choice of items presented sequentially in a closed-ended question by a nominal scale. For this purpose, the answers of a survey were correlated with the results of psychometric experimental pretests on mental image formation by applying them to experimental groups. Although the item selection affects several aspects of linguistic elements, it seems that this no influences the choice. The comparison between the data of mental visualisation of item and item frequency showed a very high correlation unusually in social or psychosocial sciences. Wording can increase the item comprehensibility but not eliminate the visual imagery effects, therefore represents a permanent bias.

Examining the effect of augmented reality experience duration on reading comprehension and cognitive load

AbstractThe purpose of this study is to examine the effect of the duration of students’ augmented reality (AR) experience on reading comprehension and cognitive load in reading activities carried out with an augmented reality storybook. The participants are 134 secondary school students aged between 11 and 12 attending a school in Antalya. They were divided into three groups. The same texts were designed with different AR durations producing three different AR storybooks. At the pre-test all groups read the coursebook texts in a traditional manner and then answered the comprehension items. At the post-test groups read the texts in accordance with their specific AR design and answered the items. In addition, the students completed the cognitive load survey after reading AR storybooks. The results of the within-group evaluations showed that the use of AR content had a strong effect on their reading comprehension. Therefore, it can be stated that augmented reality storybooks increase the reading comprehension performance of the children aged 11–12. In the comparisons between the groups, it is found that the comprehension scores of Group-2 (24–27 s) and Group-3 (37–39 s) were significantly higher than Group-1 (11–13 s). In addition, the cognitive loads of Group-2 and Group-3 are significantly lower than Group-1. On the other hand, it is found that the group with the lowest mental load was Group-2. These results suggest that the duration of AR content while reading a storybook has an impact on the students’ comprehension and cognitive load.

(117) EVALUATING THE USE OF THE FEMALE SEXUAL HEALTH AFTER CANCER SCREENING CHECKLIST WITH YOUNG ADULT CANCER SURVIVORS

Abstract Introduction Sexual health, a key component of general well-being and quality of life, is often significantly disrupted by cancer treatment. In particular, when patients undergo cancer treatment early in life, disruptions to physical, sexual, and social development can have a considerable impact on sexual health by the time survivors reach young adulthood. However, extant sexual health measures are primarily intended for sexually active adults and do not accurately capture the range of sexual health concerns experienced by young adult cancer survivors (YACS). In order to provide comprehensive sexual health screening and care for this vulnerable population, measures must be developed specifically for young survivors. This study represents a critical next step in measure development for YACS: evaluating the utility and gaps of an existing sexual health screening checklist for cancer survivors. Objective To elicit feedback from female YACS regarding how the Female Sexual Health After Cancer Checklist captures their sexual health concerns as well as how the checklist may be updated for this population. Methods Cognitive interviews were conducted with 17 female YACS (ages 23-39). Participants were presented with the Female Sexual Health After Cancer Checklist and asked to provide feedback on each item. Specifically, Willis’s Concurrent Verbal Probing Technique was used to gather information on relevance of each item to YACS, item comprehension, item acceptability, decision processes, and response processes. For example, participants were asked whether the checklist items accurately captured their sexual health concerns (i.e., response processes). Results Overall, young adult participants reported that checklist items were relevant to their sexual health concerns. However, several participants noted wording for two items seemed to assess change in symptoms after cancer (e.g., “decreased vaginal lubrication”) which is difficult for YACS to answer if their cancer occurred prior to becoming sexually active. More broadly, participants reported that the checklist failed to capture many of their most pressing concerns (e.g., delays in sexual and romantic experience). Additionally, participants felt it would be important to assess barriers to sexual and romantic relationships such as negative body image and difficulty explaining cancer history to a prospective partner. Sexually inactive participants, in particular, noted adding items about barriers to sexual and romantic relationships would make the checklist more applicable to them. Conclusions Findings suggest that existing measures, such as the Female Sexual Health After Cancer Checklist, may be a starting point for a targeted YACS’ sexual health measure. However, revision and addition of items is necessary to fully capture the needs of this younger population. As a next step, we are currently amending items based on participant feedback. For example, items implying change after cancer (e.g., “decreased vaginal lubrication”) will be updated to focus on current symptoms (e.g., “vaginal dryness”) to better apply to YACS who were treated before sexual debut. Items capturing delays in sexual and romantic experience will also be added. Revised and new items will then be gathered into an item pool for further revision by YACS and sexual health experts as part of the process of creating and validating a YACS’ specific sexual health measure. Disclosure No.

Development of the Scale of Parental Playful Attitude During the Co-Occupation of Play

Abstract Importance: There is a need for tools to measure adults’ playfulness in reference to parent–child joint play. Objective: To (1) develop items for the Scale of Parental Playful Attitude (PaPA), (2) evaluate evidence for validity based on content, and (3) evaluate evidence for validity based on the response process. Design: A mixed-methods design with three phases: (1) development of the first draft of the PaPA (based on the literature, focus groups, and semistructured interviews), (2) review by experts and revision of the first draft, and (3) cognitive interviews with parents. Setting: Various (e.g., schools, a university). Participants: Phase 1 had 32 participants: 13 parents of children ages 2.5–6 yr without major diagnosed conditions; 8 parents of autistic children age 4–7 yr; and 11 preschoolers age 3–5 yr. Phase 2 involved 10 experts. Phase 3 had 5 participants: 3 mothers and 2 fathers of children ages 2.5–6 yr; 1 child was autistic. Outcomes and Measures: Interview protocols and experts’ review form. Results: Phase 1 involved the initial item development. Extrinsic motivation and barriers to play emerged as important areas to include. Experts’ review enhanced representation of the construct of playfulness. Content validity index values ranged from .78 to 1.00. Cognitive interviews provided insights into parents’ comprehension of PaPA items. Conclusions and Relevance: Validity related to test content and the response process was supported. Additional sources of validity testing are necessary. Plain-Language Summary: Adult playfulness is critical for both the child and the parent, especially for developing a deeper connection. The Scale of Parental Playful Attitude (PaPA) assesses parents’ self-perceptions of their playfulness when playing with their child. The study authors developed the PaPA (in both English and Spanish) to address a lack of tools for assessing how parents perceive their own playfulness relative to their child’s playfulness. Understanding how parents perceive their own playful attitudes can create opportunities for occupational therapists to guide parents to approach play with an attitude that is as attuned as possible to their child’s playfulness. Future validity studies are needed to support the clinical use of the PaPA.

Investigating the capability of ChatGPT for generating multiple-choice reading comprehension items

The development of multiple-choice reading comprehension items based on specific reading subskills is crucial in teaching, learning, and testing reading. But it remains a challenging task because this process is time-consuming and costly. This study aims to investigate the capability of ChatGPT (Chat Generative Pre-trained Transformer) for generating multiple-choice reading comprehension items. Psychometric models and human review were adopted to evaluate the item quality based on the benchmark of human-authored items. The results showed that ChatGPT-authored multiple-choice items were acceptable and comparable to human-authored items in terms of psychometric properties, and human review by questionnaire, expert judgment, and interview found that ChatGPT had the potential capability to serve as a test developer and assistant for teaching and learning reading. However, some shortcomings and potential pitfalls were also identified and room for improvement was discussed when ChatGPT is applied to generate items for educational purposes.

An evaluation of the psychometric properties of the Australian Collaborative Practice Assessment Tool

Objectives This study aimed to validate the Collaborative Practice Assessment Tool (CPAT) in the Australian setting and provide a quality instrument in terms of psychometric properties that can be used to measure interprofessional outcomes for both healthcare practitioners and students. The outcomes evaluated include the capacity to work in an interprofessional team, good interprofessional communication skills, leadership skills, ensuring clear division of tasks and roles in a team, effective conflict management, and being actively involved with patients and their families/communities in care. Methods The COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) taxonomy and standards were used as guides for evaluating the psychometric properties of the Australian CPAT, which include evaluations regarding instrument development requirements of sample target and size, content validity, internal structure (structural validity, internal consistency reliability and measurement invariance), and hypotheses testing. CPAT Australia was developed through two stages involving pilot studies and a validation study, both of which included healthcare practitioners and students as participants. A pilot study examined content validity regarding item relevance, item comprehensibility, and instrument comprehensiveness. The validation study was carried out to assess the internal structure of CPAT Australia for aspects of structural validity, internal consistency reliabilities, and configural, metric and scalar measurement invariance. The structural validity was explored using the following three steps: exploratory, confirmatory, and multi-group factor analysis. Construct validity was evaluated to confirm direct and indirect paths of assumptions based on a previously validated model. Data collected between August 2021 and May 2022. Results The content validity evaluation confirmed that all items were relevant, understandable and comprehensive for measuring interprofessional collaborative care in Australia. Three hundred ninety-nine participants contributed to the validation study (n=152 practitioners; n=247 students). The original instrument model of 8-Factor 56-Item was improved in the Australian CPAT. Two items, Item 27 (Physicians assume the ultimate responsibility) and Item 49 (Final decision rest with the physician), were consistently rejected and therefore discarded. The internal structure of the 7-Factor 54-Item solution was confirmed as a suitable model with fit indices meeting COSMIN standards for a good model in practitioner and student cohorts. Configural, metric and scalar invariances were confirmed, indicating the invariance of the instruments when used for the practitioner and student cohorts. The construct validity evaluation indicated that 81.3% of direct and indirect assumptions were accepted, fulfilling the COSMIN requirement of >75% of proposed assumptions being accepted. Conclusion The Australian CPAT with a 7-factor 54-item solution was confirmed as a quality measure for assessing interprofessional education and collaborative practice for both healthcare practitioners and students in Australia with robust psychometric properties.

Measuring persistent somatic symptom related stigmatisation: Development of the Persistent Somatic Symptom Stigma scale for Healthcare Professionals (PSSS-HCP)

ObjectivePersistent somatic symptoms (PSS) describe recurrent or continuously occurring symptoms such as fatigue, dizziness, or pain that have persisted for at least several months. These include single symptoms such as chronic pain, combinations of symptoms, or functional disorders such as fibromyalgia or irritable bowel syndrome. While stigmatisation by healthcare professionals is regularly reported, there are limited measurement instruments demonstrating content validity. This study develops a new instrument to measure stigmatisation by healthcare professionals, the Persistent Somatic Symptom Stigma scale for Healthcare Professionals (PSSS-HCP). MethodsDevelopment was an iterative process consisting of research team review, item generation and cognitive interviewing. We generated a longlist of 60 items from previous reviews and qualitative research. We conducted 18 cognitive interviews with healthcare professionals in the United Kingdom (UK). We analysed the relevance, comprehensibility and comprehensiveness of items, including the potential for social desirability bias. ResultsAfter research team consensus and initial feedback, we retained 40 items for cognitive interviewing. After our first round of interviews (n = 11), we removed 20 items, added three items and amended five items. After our second round of interviews (n = 7), we removed four items and amended three items. No major problems with relevance, comprehensibility, comprehensiveness or social desirability were found in remaining items. ConclusionsThe provisional version of the PSSS-HCP contains 19 items across three domains (stereotypes, prejudice, discrimination), demonstrating sufficient content validity. Our next step will be to perform a validation study to finalise item selection and explore the structure of the PSSS-HCP.

The revision and factor analytic evaluation of the German version of the depression literacy scale (D-Lit-R German)

BackgroundDepression is a common mental health disorder and the second leading cause of disability worldwide. In people with depression, low depression literacy, which could be characterized by a poor recognition of depressive symptoms and less knowledge about the availability of treatment options, can hinder adequate therapy for depression. Nevertheless, questionnaires measuring depression literacy in Germany are rare. Consequently, for the present study, the German Depression Literacy Scale (D-Lit) has been revised and evaluated.MethodsFirst, a team of clinical psychologists revised the D-Lit German scale. Next, cognitive interviews were conducted with patients with depression to improve the comprehensibility of the scale items. Our revision of the D-Lit-R German scale was then subjected to an anonymous online study. Finally, the data went through an exploratory factor analysis, and sociodemographic subgroup analyses were performed.ResultsN = 524 individuals (age 18–80) completed the D-Lit-R German scale and a questionnaire on their sociodemographic data. Cronbach´s alpha was estimated as α = .72, and McDonald's Omega (categorical) was estimated as ω = .77. The mean Item difficulty was M = .75 (SD = .15). An EFA was performed for a unidimensional model, a 5-factor-model and at last a 3-factor-model. The 5-factorial model showed a good model fit (χ2emp,WLSMV(131) = 92.424, p > .05; CFI = 1, RMSEA = 0, SRMR = .07) but was rejected since the content of the potential 5 factors could not be determined. The 3-factor model showed an arguable model fit. The Chi2 test was significant (χ2emp,WLSMV(168) = 199.912, p < .05), but the CFI and the RMSEA met an acceptable model fit (CFI = .990, RMSEA of .019, 90% CI[.003, .029]). Substantively, the three factors were defined as (1) Distractors and other symptoms, (2) Depressive symptoms, and (3) Pharmacological and psychotherapeutic depression treatment. Furthermore, there were significant differences in sum scores regarding the subgroup's gender, treatment for mental health problems, depression treatment, experience with depression, and different career fields.ConclusionsThe D-Lit-R German scale is a time-efficient scale to assess some aspects of the depression literacy construct that can be easily applied. Since there was no perfect model fit, it is recommended to continue to revise the scale. Further evaluation studies could ask for knowledge of the etiological factors of depression. Future studies could then use this instrument to convey depression literacy. This instrument could assess the growth of knowledge after psychoeducational interventions in different settings.Trial registrationThis trial was preregistered at the platform osf.io (https://osf.io/49xdh).Registration number: https://doi.org/10.17605/OSF.IO/49XDHDate of registration: 28 April 2022.

The questionnire on learning strategies “Mes Outils de Travail Intellectuel”: Adaptation and validation among Moroccan nursing students

IntroductionThis study aims to adapt and assess the validity and reliability of the “Mes Outils de Travail Intellectuel” (My Intellectual Work Tools) scale for measuring learning strategies among Moroccan nursing students. Subjects and methodsThe validation study was conducted in the Marrakesh-Safi region of Morocco, utilizing the “Mes Outils de Travail Intellectuel” scale developed by Wolfs in 1998. To accommodate the Moroccan linguistic context, the scale was maintained in French, undergoing revision by local experts and pre-testing with 30 nursing students. Subsequently, it was administered to a comprehensive sample of 465 first-year undergraduate students for psychometric validation. Acceptability was evaluated by examining refusal rates, missing data, and item comprehension. Factorial validity was assessed through principal component analysis for each questionnaire construct, while reliability was determined using Cronbach's alpha coefficient. Results and discussionThe questionnaire achieved a response rate of 98%, with no instances of misunderstanding or missing data. Principal component analysis facilitated the removal of irrelevant items and extraction of essential factors from the dataset. The reliability of the questionnaire constructs was satisfactory, with Cronbach's alpha exceeding 0.7. These findings affirm the efficacy of the adapted “Mes Outils de Travail Intellectuel” scale for evaluating learning strategies among Moroccan nursing students. Although the study is limited to first-year students, it establishes a framework for the instrument's application in the specific context of nursing education in Morocco.

Adolescents’ understanding of the Nepalese version of HLS-CHILD-Q15: qualitative pre-testing in ninth-graders

IntroductionResearch on health literacy is still at an early stage, lacking a dedicated measurement instrument for assessing children’s and adolescents’ health literacy. Such tools are necessary to generate the evidence required for informed intervention development. This study translated the validated German HLS-Child-Q15 into the Nepalese context, creating the HLS-Child-Q22-NEP.MethodsThe research team initially created the HLS-Child-Q22-NEP using an additional item pool. We conducted thirteen one-on-one cognitive interviews with adolescent students from community schools in three districts of Nepal during the pre-test. We employed verbal probing techniques and deductively analysed the interviews based on Tourneau’s model, uncovering four main themes: (1) comprehension (with the two sub-categories: a) item comprehension and b) word comprehension); (2) retrieval; (3) judgement; and (4) response.ResultsOverall, participants responded positively to the HLS-Child-Q22-NEP. However, this study revealed comprehension challenges such as unfamiliarity, misunderstandings, and translation issues. Additionally, the study identified retrieval challenges and poor judgement, indicating limitations in the assessment. Participants experienced varying levels of difficulty with some items, emphasising the need for revised instructions. Subsequent revisions, guided by pretest insights, led to the development of the HLS-Child-Q24-NEP.ConclusionThe development of the HLS-Child-Q22-NEP is a significant step in addressing Nepali adolescents’ lack of health literacy measurement. Despite its generally positive reception, this study encountered challenges in comprehending the scale, prompting enhancements, and developing the HLS-Child-Q24-NEP. Further research, both qualitative and quantitative, is necessary to evaluate the validity and reliability of the modified items.

Development of the What Matters 2 Adults (WM2A) wellbeing measure for Aboriginal and Torres Strait Islander adults

PurposeAs wellbeing is culturally bound, wellbeing measures for Aboriginal and Torres Strait Islander peoples must be culturally relevant and grounded in Aboriginal and Torres Strait Islander values and preferences. We describe the development of a nationally-relevant and culturally grounded wellbeing measure for Aboriginal and Torres Strait Islander adults: the What Matters to Adults (WM2A) measure. MethodsWe used a mixed methods approach to measure development, combining Indigenist methodologies and psychometric methods. Candidate items were derived through a large national qualitative study. Think-aloud interviews (n = 17) were conducted to assess comprehension, acceptability, and wording of candidate items. Two national surveys collected data on the item pool (n = 312, n = 354). Items were analysed using exploratory factor analysis (EFA), and item response theory (IRT) to test dimensionality, local dependence and item fit. A Collaborative Yarning approach ensured Aboriginal and Torres Strait Islander voices were privileged throughout. ResultsFifty candidate items were developed, refined, and tested. Using EFA, an eight factor model was developed. All items met pre-specified thresholds for maximum endorsement frequencies, and floor and ceiling effects; no item redundancy was identified. Ten items did not meet thresholds for aggregate adjacent endorsement frequencies. During Collaborative Yarning, six items were removed based on low factor loadings (<0.4) and twelve due to conceptual overlap, high correlations with other items, endorsement frequencies, and/or low IRT item level information. Several items were retained for content validity. The final measure includes 32 items across 10 domains (Balance & control; Hope & resilience; Caring for others; Culture & Country; Spirit & identity; Feeling valued; Connection with others; Access; Racism & worries; Pride & strength). ConclusionsThe unique combination of Indigenist and psychometric methodologies to develop WM2A ensures a culturally and psychometrically robust measure, relevant across a range of settings and applications.

Developing a Measure of Key Adult Outcomes in Adults with Developmental Disabilities: Conceptual Model and Item Generation of the REALS (Relationships, Employment, Autonomy, and Life Satisfaction).

Employment, social relationships, and autonomy are priorities topeople with intellectual and developmental disabilities (IDDs). However, few validated measures exist to systematically assess these key adult outcomes in this population. This research includes first steps to develop self- and proxy report measures of life outcomes for adults with IDDs-the Relationships, Employment, Autonomy, and Life Satisfaction (REALS). A literature search identified existing adult outcome measures, and comparison of their domains informed initial conceptual model development. External consultants revised the model, and items were generated. Autistic adults (n = 15), adults with other IDDs (n = 7), caregivers of autistic adults (n = 13), and caregivers of adults with other IDDs (n = 10) completed in-depth cognitive interviews to assess comprehension of items and response categories, factors influencing how participants respond to items, and the inclusiveness of the item pool. A final conceptual model was generated with three subdomains (social relationships, employment, and autonomy), including assessment of life satisfaction within each domain. Cognitive interviews revealed that response set restructuring and item-level revisions were needed to capture the complexity of adult life and make the measure more accessible across a range of abilities. This study developed a conceptual model of relationships, work, and autonomy specific to adults with IDDs. Future work will involve collecting data from 800 + self-reporters with IDDs and 800 + caregivers of adults with IDDs to conduct psychometric analyses. Improving measurement in this area is critical to better understanding the needs of adults with IDDs and improving services available to them.

Exploring women’s interpretations of survey questions on pregnancy and pregnancy outcomes: cognitive interviews in Iganga Mayuge, Uganda

BackgroundIn 2021, Uganda’s neonatal mortality rate was approximately 19 deaths per 1000 live births, with an estimated stillbirth rate of 15.1 per 1000 total births. Data are critical for indicating areas where deaths occur and why, hence driving improvements. Many countries rely on surveys like Demographic and Health Surveys (DHS), which face challenges with respondents’ misinterpretation of questions. However, little is documented about this in Uganda. Cognitive interviews aim to improve questionnaires and assess participants’ comprehension of items. Through cognitive interviews we explored women’s interpretations of questions on pregnancy and pregnancy outcomes.MethodsIn November 2021, we conducted cognitive interviews with 20 women in Iganga Mayuge health and demographic surveillance system site in eastern Uganda. We adapted the reproductive section of the DHS VIII women’s questionnaire, purposively selected questions and used concurrent verbal probing. Participants had secondary school education and were English speaking. Cognition was measured through comparing instructions in the DHS interviewers’ manual to participants’ responses and researcher’s knowledge. A qualitative descriptive approach to analysis was undertaken.ResultsWe report findings under the cognitive aspect of comprehension. Some questions were correctly understood, especially those with less technical terms or without multiple sections. Most participants struggled with questions asking whether the woman has her living biological children residing with her or not. Indeed, some thought it referred to how many living children they had. There were comprehension difficulties with long questions like 210 that asks about miscarriages, newborn deaths, and stillbirths together. Participants had varying meanings for miscarriages, while many misinterpreted stillbirth, not linking it to gestational age. Furthermore, even amongst educated women some survey questions were misunderstood.ConclusionsPopulation surveys may misclassify, over or under report events around pregnancy and pregnancy outcomes. Interviewers should begin with a standard definition of key terms and ensure respondents understand these. Questions can be simplified through breaking up long sentences, while interviewer training should be modified to ensure they thoroughly understand key terms. We recommend cognitive interviews while developing survey tools, beyond basic pre-testing. Improving respondents’ comprehension and thus response accuracy will increase reporting and data quality.

Identification of Suicide Risk in a Pediatric Psychiatric Emergency Setting: Comparing the Ask Suicide-Screening Questions and the Kiddie-Computerized Adaptive Test-Suicide Scale

ABSTRACT While the emergency department (ED) is an important setting for identifying youth with psychiatric symptoms and connecting them to services, the demands of the ED make efficient and accurate measurement essential in the implementation of mental health screening. The Kiddie-Computerized Adaptive Test (K-CAT), a new electronically administered measure that offers quick and comprehensive assessment across several mental health domains, may be particularly useful in this setting. Given current recommendations for youth suicide risk screening in EDs, this study compared the K-CAT-Suicide Scale (K-CAT-SS) and the Ask Suicide-screening Questions (ASQ), a widely used measure in EDs, in a sample of participants presenting to a pediatric, psychiatric emergency setting. The measures agreed on the presence of suicide risk in over 85% of cases (κ = 0.59), and the characteristics of youth who screened at risk on both were similar. Cases of disagreement were more often male and more often had educational accommodations. They had lower symptom levels of and were less often diagnosed with internalizing disorders and were less often identified as high risk by ED psychiatrists and psychologists. Examination of item endorsement patterns in cases of disagreement revealed important areas of future study, including the role of caregiver report in suicide risk screening, item comprehension concerns, and the validity of assessing youth with neurodevelopmental disabilities. While additional research would be beneficial into its psychometrics when deployed in real-world settings, the K-CAT-SS should be considered a viable alternative for suicide risk screening in EDs.

Using OpenAI GPT to Generate Reading Comprehension Items

AbstractThe purpose of this study is to introduce and evaluate a method for generating reading comprehension items using template‐based automatic item generation. To begin, we describe a new model for generating reading comprehension items called the text analysis cognitive model assessing inferential skills across different reading passages. Next, the text analysis cognitive model is used to generate reading comprehension items where examinees are required to read a passage and identify the irrelevant sentence. The sentences for the generated passages were created using OpenAI GPT‐3.5. Finally, the quality of the generated items was evaluated. The generated items were reviewed by three subject‐matter experts. The generated items were also administered to a sample of 1,607 Grade‐8 students. The correct options for the generated items produced a similar level of difficulty and yielded strong discrimination power while the incorrect options served as effective distractors. Implications of augmented intelligence for item development are discussed.

Occupational Therapy Observation Tool-Adjustment Support Details for autistic children: Investigation of content validity and clinical utility.

Standardised testing is crucial for autistic children to receive appropriate interventions, necessary services, and funding. Behaviours associated with autism can hinder children's test performance and participation. The 18-item, two-page Occupational Therapy Observation Tool-Adjustment Support Details (OTOT-ASD) is used in conjunction with standardised assessments. The OTOT-ASD enables the recording of autism-related behaviours and the reasonable assessment accommodations made to support participation in testing. The current study aims to investigate the content validity and clinical utility of the OTOT-ASD from occupational therapists' perspectives. A specifically designed online questionnaire was distributed Victoria-wide to occupational therapists with clinical experience with autistic children. The questionnaire was designed in line with the COnsensus-based Standards for selection of health Measurement INstruments (COSMIN), including relevance, comprehensiveness, and comprehensibility of items evaluated for content validity. Applicability and clinical usefulness were also investigated. Quantitative data were analysed using descriptive statistics. Seventy-one occupational therapists responded to the questionnaire and 46 provided a full dataset. Over 96% of the respondents considered documenting behaviours and testing accommodations as important, yet less than 45% reported having resources to record this information. Ninety-five percent of the respondents perceived the OTOT-ASD to be useful in clinical practice. Over 70% agreed that OTOT-ASD items were significantly relevant to practice and comprehensive. To improve comprehensibility, identified items were altered following feedback. The findings suggest that the OTOT-ASD is clinically useful and content validity is sound. Further research on other psychometric properties of the OTOT-ASD and users' qualitative experiences in utilising the tool is recommended.