Establish Content Validity Research Articles

542 - Validation of the skin pain numerical rating scale of the Pruritus and Symptoms Assessment for Atopic Dermatitis for adults and adolescents with moderate-to-severe atopic dermatitis

Abstract Background Skin pain is a common and burdensome symptom of atopic dermatitis (AD), and its severity increases with disease progression. The Pruritus and Symptoms Assessment for Atopic Dermatitis (PSAAD) is a validated 11-item patient-reported instrument developed in accordance with regulatory agency guidance for assessment of the daily symptoms of patients with AD, including skin pain. Objective To establish content validity and evaluate the psychometric properties of the skin pain numerical rating scale (NRS) item of the PSAAD. Methods The skin pain NRS is a single-item measure from the PSAAD and is used to assess skin pain severity over the past 24 hours with the question “How painful was your skin over the past 24 hours?” on a 10-point scale from 0 (not painful) to 10 (extremely painful). A qualitative validation analysis consisting of concept elicitation (CE) interviews; electronic completion of the PSAAD measure, including the skin pain NRS; and cognitive debriefing (CD) interviews was conducted for adults and adolescents with mild-to-severe AD. Evaluation of psychometric properties of the skin pain NRS—including test-retest reliability, estimation of meaningful within-patient change (MWPC), known-group validity, ability to detect change, construct validity, quality of completion, and ceiling and floor effects—was conducted based on a post hoc qualitative and quantitative analysis of pooled data, with skin pain NRS as part of the PSAAD in the abrocitinib monotherapy phase 3 trials JADE MONO-1 (NCT03349060) and JADE MONO-2 (NCT03575871). Data from patients (≥12 years of age) with moderate-to-severe AD were pooled across all treatment arms. Results Among 30 CE interviews (adolescents [12-17 years], n=15; adults [≥18 years], n=15) conducted over 2 rounds of testing, 20 of 30 (67%) patients reported skin pain. During CD interviews (n=30), 100% of patients interpreted the skin pain NRS item as intended, and 73% reported it as relevant. The psychometric validation analysis included 736 patients (JADE MONO-1, N=347; JADE MONO-2, N=389). The test-retest reliability of the skin pain NRS was evidenced by large intraclass correlation coefficients (ICC; 0.76 and 0.72 in JADE MONO-1 and MONO-2, respectively) for a single daily skin pain score, which increased to 0.93 and 0.91 for a measurement averaging at least 4 scores per week. Convergent validity was demonstrated by correlations between skin pain and other clinician- and patient-­reported AD measures (median correlation >0.4 starting at week 4). The skin pain NRS distinguished between “clear” and “severe” disease severity groups, assessed using the Patient Global Assessment (PtGA) and Investigator’s Global Assessment (IGA) as continuous and categorical anchors (P<0.0001); differences were also significant among other disease severity groups using the PtGA and IGA as continuous anchors (P<0.0001). Based on the PtGA and IGA anchor analysis, the estimated MWPC was 1.9 and ranged from 2.7 to 3.9 points using the empirical cumulative distribution function analysis. Ability to detect change was demonstrated by linear relationships between disease severity measures and skin pain scores (correlation of ≥0.5 between the skin pain NRS and the PtGA). More than 90% of patients provided at least 4 daily skin pain scores each week. No floor or ceiling effects were observed. Conclusions The qualitative and quantitative data from initial PSAAD development and psychometric evaluation of the skin pain NRS from the PSAAD support its content validity, reliability, construct validity, and ability to detect change. These findings corroborate the 4-point improvement in skin pain as the adequate and meaningful cut-off to define a skin pain responder in JADE MONO-1 and MONO-2.

Cross-Cultural Validation of the Thai Oral Health Impact Profile for Temporomandibular Disorders

ObjectivesThe Oral Health Impact Profile for Temporomandibular Disorders (OHIP-TMDs) is a validated condition-specific outcome measure to help guide decision-making in the management of the condition. There is no Thai version of OHIP-TMDs, and therefore the original English version needs cross-cultural adaptation translation, and validation with a Thai population to reduce the anomalies due to language and cultural differences. This study aimed to develop the Thai-language version of OHIP-TMDs, perform a cross-cultural adaptation to Thailand, and assess its content validity, internal consistency, reliability, and construct validity. MethodsThe original English version of OHIP-TMDs was forward and backward translated into Thai language using the International Network for Orofacial Pain and Related Disorders methodology (INfORM) protocol for cross-cultural adaptation. The Content Validity Index (CVI) was performed by 5 orofacial pain (OFP) specialists to establish content validity. The OHIP-TMDs-T was then tested in 2 groups of Thai dental patients including 110 TMD patients and 110 control participants. The internal reliability and test-retest reliability (n = 30) were investigated in the TMD group using Cronbach alpha coefficient and intraclass correlation coefficient (2-way mixed effect model), respectively. The difference in OHIP-TMDs-T score between the TMD group and control group was investigated for known group validity. ResultsCronbach alpha and intraclass correlation coefficients were 0.942 and 0.797, respectively. The CVI collected from the OFP specialists was 0.92. There was a statistical difference in the OHIP-TMDs-T overall score between the TMD group (95% CI, 40–46) and control group (95% CI, 2.0–3.4) (Z = 9.060, r = 1, P < .001). ConclusionsThe OHIP-TMDs-T is a valid and reliable tool for evaluating the quality of life and the impact on oral health in Thai patients with TMD.

Development of a New Tool for Writing Research Key Information in Plain Language.

The complexity of research informed consent forms makes it hard for potential study participants to make informed consent decisions. In response, new rules for human research protection require informed consent forms to begin with a key information section that potential study participants can read and understand. This research study builds on exiting guidance on how to write research key information using plain language. The aim of this study was to develop a valid and reliable tool to evaluate and improve the readability, understandability, and actionability of the key information section on research informed consent forms. We developed an initial list of measures to include on the tool through literature review; established face and content validity of measures with expert input; conducted four rounds of reliability testing with four groups of reviewers; and established construct validity with potential research participants. We identified 87 candidate measures via literature review. After expert review, we included 23 items on the initial tool. Twenty-four raters conducted 4 rounds of reliability testing on 10 informed consent forms. After each round, we revised or eliminated items to improve agreement. In the final round of testing, 18 items demonstrated substantial inter-rater agreement per Fleiss' Kappa (average = .73) and Gwet's AC1 (average = .77). Intra-rater agreement was substantial per Cohen's Kappa (average = .74) and almost perfect per Gwet's AC1 (average = 0.84). Focus group feedback (N = 16) provided evidence suggesting key information was easy to read when rated as such by the Readability, Understandability and Actionability of Key Information (RUAKI) Indicator. The RUAKI Indicator is an 18-item tool with evidence of validity and reliability investigators can use to write the key information section on their informed consent forms that potential study participants can read, understand, and act on to make informed decisions. [HLRP: Health Literacy Research and Practice. 2024;8(1):e29-e37.].

Assessing HIV knowledge in Mexican people living with HIV: development and validation of CC-VIH questionnaire.

The level of knowledge that people living with human immunodeficiency virus (HIV) have about their disease can impact their adherence to treatment. The aim of this study was to develop a tool to assess the knowledge about HIV among people receiving treatment at a specialized clinic in Mexico City. To establish content validity, expert judges were invited to conceptualize the tool and propose items for the defined dimensions. A total of 490 individuals living with HIV completed the 91-item questionnaire, with 82.2% being male and a mean age of 36.1 years. We conducted an exploratory factor analysis, resulting in a reduced questionnaire of 45 questions. A three-factor solution explained 36.2% of the variance in HIV knowledge. The total scale had a reliability coefficient of 0.937, and each subscale had reliabilities of 0.828, 0.856 and 0.859. Lower educational level (F(336) = 8.488, p < 0.001) and female gender (t(399) = 2.003, p = 0.046) were associated with lower scores on the HIV knowledge questionnaire. This tool appears suitable for measuring HIV knowledge in people living with HIV, although future studies are required to confirm its structure and reduce its extension.

Development of a Patient Reported Outcome Daily Diary to Assess Symptom Burden in Sickle Cell Disease

Introduction Sickle cell disease (SCD) is a genetic vascular disease that causes pain, fatigue and multiple complications, ultimately affecting health-related quality of life of patients (Osunkwo et al. Am J Hematol 2021). The purpose of this study was to conduct qualitative interviews to inform the development of a patient reported outcome (PRO) daily diary to capture key symptoms and impacts of vaso-occlusive crisis (VOCs) experienced at home, based on methods consistent with US Food and Drug Administration (FDA) guidance for incorporating patient input and clinical outcome assessments into clinical research (FDA Fed Reg 2009, FDA PFDD Draft Guidance 3 2022). Methods This was a cross-sectional, non-interventional, qualitative study involving 16 concept elicitation (CE) interviews with adults (18-65 years), 15 cognitive interviews with adults (two rounds of n=12 and n=3), and 8 hybrid CE/cognitive interviews with adolescents (12-17 years). Up to 50% of adult participants were able to participate in both CE and cognitive interviews. Adult participants were recruited by clinical sites in the United States (US) and adolescent participants were recruited by a recruitment vendor. Patients were eligible to participate if they had a confirmed diagnosis of SCD, at least two VOCs in the past 12 months, a hemoglobin level of ≥4 gm/dL in past six months, and did not have a blood transfusion in the past three months. All interviews were conducted in English according to the approved study protocol and interview guide. Key symptoms and impacts were identified during CE interviews as those that were reported by at least 75% of sample and experienced daily by at least 50% of sample. The daily diary items were refined based on cognitive interviews to ensure patient understanding, medical expert input (n=1) and medical expert advisory board input (n=4) to ensure clinical relevance. Results Adult CE and cognitive interview participants (n=25 unique participants, with 6 participating in both CE and cognitive interviews) were mostly female (n=18; 72.0%), with a mean age of 38.0 ± 9.4 years (range 20 to 56 years). Among adolescent participants, nearly all were female (n=7, 87.5%), with a mean age of 14.6 ± 2.1 years (range 12 to 17 years). All adult and adolescent participants self-identified as African American. Twenty percent (n=5) of adult participants had a history of taking crizanlizumab-tmca, 16.0% (n=4) had a history of taking voxelotor, 8.0% (n=2) had a history of taking both, and 56.0% (n=14) had never taken crizanlizumab-tmca or voxelotor. Twelve and a half percent (n=1) of adolescent participants had a history of taking crizanlizumab-tmca, 50.0% (n=4) had a history of taking voxelotor, and 37.5% (n=3) had never taken crizanlizumab-tmca or voxelotor. Findings from CE interviews identified the following key concepts: pain severity, pain crisis, tiredness, stiffness, impact on daily activities, impact on work or school, and impact on sleep. Four items were included based on medical expert/advisory board input to address swelling, ability to keep up with peers (for adolescents only), ability to control pain, and overall SCD severity. The current draft PRO daily diary includes four symptoms, four SCD-related impacts, and overall SCD severity, which will undergo further qualitative testing. Conclusion Results from this qualitative study provides further insight into the daily experience of patients with SCD, including their experience with pain crises. Findings from this study were used to develop a daily diary PRO tool for use in future clinical trials. Future work includes additional CE/cognitive interview with adults and adolescents to refine the PRO as needed and establish content validity and psychometric validation.

Conceptual Framework and Content Validity for the Creighton Competency Evaluation Instrument 2.0© (C-CEI 2.0©)

BackgroundNursing education is undergoing a transformational shift toward competency-based education, necessitating psychometrically sound instruments to assess student competence. For the past 15 years, the Creighton Simulation Evaluation Instrument (C-SEI©) and the Creighton Competency Evaluation Instrument (C-CEI©) have gained national and international attention and utilized as assessment methods to evaluate competency. MethodsThe purpose of this study was to revise the C-CEI© and establish content validity of the C-CEI 2.0©. Content validity was established through three content validity surveys, two focus groups, and an iterative process of data collection and analysis and tool revision. ResultsParticipants rated that each item was essential to be included (M=3.70, SD=0.53), was reflective of the category in which it was included (M=3.70, SD=0.57), and was easy to understand (M=3.57, SD=0.65). The overall mean of the three categories combined was positive (M=3.65, SD=0.59). Through an iterative process of conducting and analyzing findings from content validity surveys and focus groups, revisions were made, and the instrument was finalized. ConclusionInitial findings suggest that the C-CEI 2.0© demonstrates strong content validity.

Development and Validation of a Short Broad Job Crafting Scale

The dominant perspectives in the job crafting literature have different views on “what elements of their job employees craft” and “why employees craft their jobs”. To better grasp both perspectives, we developed and validated a short 4-item instrument (BJCS) to capture job crafting as the self-initiated changes made to one’s job with the intention to optimize one’s functioning. We validated the BJCS through five phases: (1) we developed the BJCS and conducted a qualitative study to establish content validity, (2) we establish its reliability and psychometric properties, (3) we demonstrated discriminant and convergent validity of the BJCS in relation to existing job crafting measures, and (4 &amp; 5) the BJCS showed to be predictive and incrementally valid over and above existing measures in relation to indicators of optimal and dysfunctional functioning. Taken together, we demonstrated the validity of a broad short measure for job crafting which is especially relevant for the use in repeated measurement research.

Establishing Reliability and Validity of the FACE-Q Craniofacial Module for Pediatric Head and Neck Cancer.

Purpose: We aimed to establish content validity and assess the psychometric properties of the FACE-Q Craniofacial Module, a patient-reported outcome measure, for use in pediatric and adolescent patients with head and neck cancer (HNC). Methods: To establish content validity (Part 1), between June 2017 and August 2019, cognitive interviews were conducted with survivors of pediatric HNC (n = 15), and input was obtained from clinical experts (n = 21). To examine item and scale performance (Part 2), Rasch Measurement Theory (RMT) analysis was performed using data from two international studies (n = 121). Results: Part 1: Qualitative data from 15 survivors and input from 21 experts provided evidence to support the use of the FACE-Q Craniofacial Module in pediatric HNC. Part 2: The field-test study sample included 121 survivors of pediatric HNC. RMT analysis provided evidence of reliability and validity for 10 FACE-Q scales. Data for each scale fit the RMT model. Scale reliability was high, with Person Separation Index and Cronbach's alpha values ≥0.82 for 9 scales. Mean scores on the Appearance, Psychological, and Social scales were higher for those who liked aspects of their face more. For participants with (vs. without) a facial difference, mean scores were lower for the Face, Jaws, Psychological, and Social scales. Conclusion: The FACE-Q Craniofacial Module evidenced reliability and validity for HNC survivors aged 8-29 years and can be used in research and clinical care to measure quality of life of pediatric survivors with HNC.

Content validation of a mental wellness measuring instrument for adolescents living with HIV: a modified delphi study

A growing body of evidence suggests that improving the mental wellness of adolescents living with HIV (ALHIV) will also result in improved adherence to antiretroviral therapy (ART), as well as improving their general health and wellbeing as they age into adulthood. However, to develop effective strategies and interventions aimed at improving mental wellness, we require age and culturally appropriate instruments to build an evidence base. Currently, there is a lack of mental wellness measures developed for ALHIV, especially in the African context. To address this gap, we developed a measure of mental wellness following modified guidelines set out by DeVellis [1] and Godfred et al. as a guiding framework [2]; (1) Identifying the gap, (2) Set the theoretical foundations and identify domains and (3); Instrument development and initial validation. For the first two steps, we conducted a systematic review, photovoice study and integrative review – which we briefly describe as the findings have been published. Following this we describe the processes to develop the instrument and to establish content validity through a modified Delphi Study. Through this process we were able to refine the instrument which will be subject to further testing.Implications and Contribution: This study aims to add to the body of knowledge on promoting mental health (mental wellness) among adolescents living with HIV in South Africa through developing an appropriate and valid measure of mental wellness for this population. This study reports on the results of a Delphi Study aimed at improving the content validity of the instrument Mental Wellness Measure for Adolescents Living with HIV (MWM-ALHIV).

CritCom: assessment of quality of interdisciplinary communication around deterioration in pediatric oncologic patients.

High-quality clinical care requires excellent interdisciplinary communication, especially during emergencies, and no tools exist to evaluate communication in critical care. We describe the development of a pragmatic tool focusing on interdisciplinary communication during patient deterioration (CritCom). The preliminary CritCom tool was developed after a literature review and consultation with a multidisciplinary panel of global experts in communication, pediatric oncology, and critical care to review the domains and establish content validity iteratively. Face and linguistic validity were established through cognitive interviews, translation, and linguistic synthesis. We conducted a pilot study among an international group of clinicians to establish reliability and usability. After reviewing 105 potential survey items, we identified 52 items across seven domains. These were refined through cognitive interviews with 36 clinicians from 15 countries. CritCom was piloted with 433 clinicians (58% nurses, 36% physicians, and 6% other) from 42 hospitals in 22 countries. Psychometric testing guided the refinement of the items for the final tool. CritCom comprised six domains with five items each (30 total). The final tool has excellent reliability (Cronbach's alpha 0.81-0.86), usability (93% agree or strongly agree that the tool is easy to use), and similar performance between English and Spanish tools. Confirmatory factor analysis was used to establish the final 6-domain structure. CritCom is a reliable and pragmatic bilingual tool to assess the quality of interdisciplinary communication around patient deterioration for children in diverse resource levels globally. Critcom results can be used to design and evaluate interventions to improve team communication.

Validation of Nursing Outcomes Classification: Knowledge and Self-management for Cardiac Disease.

The Nursing Outcomes Classification provides two outcomes, Knowledge: Cardiac Disease and Self-management: Cardiac Disease, to assess knowledge and self-management behaviors of adults with cardiac disease. The purpose of this study was to validate the two nursing-sensitive outcomes to establish content validity. A methodological design was used using the Delphi technique. A total of 13 nurse experts in two domains participated in this study: five in standardized nursing terminologies and eight in self-management. Descriptive statistics and the Nurse-Patient Outcome Content Validity method were used to validate four aspects: definition adequacy of each outcome, clinical usefulness of measurement scales, importance of outcome indicators, and content similarity between the two outcomes. The definition adequacy, clinical usefulness, and content similarity of both outcomes were acceptable. A total of 81 indicators from the two outcomes were validated, and 60 were designated as critical. Nurses can evaluate cardiac patient outcomes effectively and accurately using these validated outcomes. The validated Nursing Outcomes Classification outcomes will also support the clinical decision-making of nursing students when they learn about patients with cardiac disease.

TENDINopathy Severity Assessment - Achilles (TENDINS-A): Development and Content Validity Assessment of a New Patient-Reported Outcome Measure for Achilles Tendinopathy.

OBJECTIVE: To develop a new patient-reported outcome measure (PROM) assessing TENDINopathy Severity of the Achilles (TENDINS-Achilles) and evaluate its content validity. DESIGN: Mixed-methods, modified Delphi. METHODS: We performed 1 round of semistructured one-on-one interview responses with professionals and patients, for initial item generation. This was followed by 1 round of survey responses for professionals and a final round of semistructured one-on-one interviews with patients. The work culminated in a PROM to quantify Achilles tendinopathy severity under the core health domain of disability. Participants identified 3 subdomains contributing to the severity of disability of Achilles tendinopathy: pain, symptoms, and functional capacity. RESULTS: All 8 patient participants invited to participate were enrolled. Forty professional participants (50% women, six different continents) were invited to participate and 30 were enrolled (75% response rate). Therefore, a total of 30 professionals and 8 patients were included within this study. Following 3 rounds of qualitative or quantitative feedback, this study has established the content validity of TENDINS-A (good relevance, comprehensibility, and comprehensiveness) as a new PROM to assess the severity of Achilles tendinopathy, which assesses aspects of pain, symptoms, and functional capacity. CONCLUSION: TENDINS-A has established content validity and is appropriate for use with clinical and research populations. We recommend users interpret TENDINS-A results cautiously, until further testing evaluates the most appropriate scoring scale, reliability, construct validity, criterion validity, and responsiveness of TENDINS-A. Until these psychometric properties are established, we suggest using TENDINS-A alongside existing tools. J Orthop Sports Phys Ther 2023;53(11):1-16. Epub: 24 August 2023. doi:10.2519/jospt.2023.11964.

Exploring the Development of an Interprofessional Situational Judgment Test

INTRODUCTION Although the prevalence of interprofessional education (IPE) is increasing, research measuring competency in IPE is underdeveloped (MacDonald et al., 2010; Shrader et al., 2017). Interprofessional assessment tools are plentiful, but most rely on self-report rather than performance data. Thus, the purpose of this study was to develop a computer-based situational judgment test to evaluate health professional student preparedness for interprofessional practice. METHODS The investigators developed the Creighton Interprofessional Situational Judgment Test (C-IPSJT) which includes nine interprofessional clinical case scenarios, each with multiple questions reflecting IPEC competencies. Subsequently, focus groups representing both IPE and health education experts were convened to assess the C-IPSJT. RESULTS Qualitative analysis revealed several themes. Participants found that the C-IPSJT was a novel assessment of interprofessional skills and case scenarios presented were reflective of current clinical practice. Challenges were found in understanding the role of the test taker, discerning correct responses, and the need for further validity testing. Additional themes included the impact team dynamics have on learner responses and how student responses could be affected by their place on the learning continuum. DISCUSSION Findings from the initial testing of the C-IPSJT demonstrate positive content validity through two independent content expert reviews. Content experts identified that the C-IPSJT could be a viable mechanism to differentiate between novice and competent health professional students. Future research includes item revision, testing over a larger sample in which psychometric properties and item analysis can be studied, and establishing content validity.

Examining the validity, reliability and feasibility of capturing children's physical literacy through games-based assessment in physical education.

Observational tools can help refine practice design and guide the creation of effective learning environments. The intention of this study was to design and validate an observational instrument for assessing physical literacy that remains more faithful to the philosophically complex and holistic nature of the concept. Framed by concepts of ecological dynamics, the emergent games-based assessment tool enables capture of children's interactions with their environment, providing insight on the manifestation of physical literacy within physical education games. The design and validation of the instrument consisted of a multistage process: (1) design of the observational instrument and establishing face validity; (2) pilot observation study; (3) expert qualitative and quantitative review to establish content validity; (4) observation training; and (5), establishing observer reliability. Following expert qualitative and quantitative evaluation, Aiken's V coefficient was used to determine content validity. Results achieved demanding levels of validity (V ≥ 0.78) for all retained measurement variables. Cohen's κ values for inter- and intra-observer reliability ranged from 0.331 to 1.00 and 0.552 to 1.00, generally reporting "substantial" agreement during inter-observer analysis and "substantial" to "almost perfect" agreement during intra-observer analysis. The final model of the emergent games-based assessment tool, with 9 ecological conceptualisations of behaviour, 15 measurement variables, and 44 categorical observational items was found to be valid and reliable, providing both educators and researchers with a useful mechanism to assess physical literacy during gameplay.

Quality-Based Private Madarasah Aliyah Education Financing Management

The purpose of this study is to describe the management of Quality-Based Private Madarasah Aliyah Education Financing in Private Aliyah Madrasahs throughout the City of Bandung. Data was collected with a mixed-method approach, starting with the use of quantitative methods followed by qualitative methods. Quantitative data on the level of use of process skills by learners were collected using questionnaires. The questionnaire was developed by modifying the steps of Madaus et al., Claeys et al., and Wheeler, through the stages of literature review, constructing instruments, establishing content validity by means of expert validation, and field Other qualitative, obtained through observation, field note taking, field note categorization, and in-depth interviews. All data were analyzed by explanatory mixed-method analysis. The results showed that in general it can be concluded that the management of education financing has an impact or affects the quality of education in the Bandung City Private Aliyah Madrasah. This means that the quality of education is determined by good and systematic financing management.

(008) Suggestions for Modifications to the Female Sexual Function Index to be Inclusive of All Populations – Cognitive Interviews with Sexual and Gender Minority and Heterosexual Persons

Abstract Introduction Patient-reported outcome measures that were initially designed for research studies including heterosexual, partnered, educated cisgender women may have limited applicability for clinical use among Sexual and Gender Minority (SGM) patients and patients without a partner. Objective We aimed to conduct cognitive interviews with both SGM persons and heterosexual women to determine readability, comprehension, and applicability of questionnaire items to assess sexual function in populations with diverse sexual practices, focusing on items from the Female Sexual Function Index (FSFI), and to establish content validity of a modified measure. Methods We conducted four rounds of cognitive interviews with 52 participants (28 SGM, 24 heterosexual) who provided feedback on their comprehension and wording of FSFI questionnaire items and response scales. Participants made recommendations for changes to the questionnaire, which was iteratively revised between interview rounds. Two independent coders analyzed the transcripts using structural coding based on five predefined codes (i.e., satisfaction with item, specificity/language change needed, missing/suggested item, patient definitions of concepts, and confusion with item). Results After four rounds of cognitive interviews and revisions to the questionnaire, participants found the final version of the questionnaire acceptable and understandable, thereby reaching thematic saturation and establishing content validity of the modified FSFI. Modifications included: replacing all instances of “sexual stimulation” and “intercourse” with “sexual activity (alone or with a partner)”; broadening the definition of “vaginal penetration” beyond penile-vaginal penetration; adding skip logic to include the option “no sexual activity”. Participants identified missing concepts important to their sexual health, such as including the use of external lubricant. Heterosexual participants found the questionnaire initially refined by SGM participants to be acceptable, and offered targeted feedback including clarifying the clinical relevance of certain domains (e.g., questions about “emotional closeness”), clarifying terminology (e.g., sexually aroused (turned on) and naturally lubricated (wet)) and re-ordering domains (e.g., “pain should be asked about earlier”). Conclusions Feedback from direct interviews supports modifying FSFI items and further psychometric testing. This and other similar questionnaires need to be adapted to the broader clinical practice population such that all persons’ experiences are accurately reflected and assessed, ultimately assuring that sexual health needs can be met more inclusively. Further research in racially and educationally diverse populations is needed. Disclosure No

Content Validation of the Athletic Training Milestones: A Report from the Association for Athletic Training Education Research Network.

The Athletic Training Milestones were developed as a comprehensive framework to assess athletic trainers' knowledge, skill, and behavior acquisition across the continuum of athletic training practice. However, without established content validity, it is unclear whether the Athletic Training Milestones can be used effectively as a clinical evaluation and research tool to evaluate competence and performance across multiple users and sites. We conducted a highly conservative content validity index (CVI) with data from 12 content experts. Our findings revealed an extremely high overall scale CVI of 0.99, and CVI scores of the 28 individual subcompetency items assessed ranged from 0.83 to 1.00. For the athletic training profession to truly embrace competency-based evaluation and performance assessments, we need a highly valid and comprehensive instrument, such as the Athletic Training Milestones.

P035 Patient-reported outcome measures for rheumatoid arthritis disease activity: using Rasch measurement theory to achieve more meaningful measurement

Abstract Background/Aims Disease activity (DA) monitoring is a standard of care in rheumatoid arthritis (RA), and there is demand for achieving this through patient-reported outcome measures (PROMs). A systematic review of PROMs for RA DA following COSMIN guidelines demonstrated a lack of sufficient evidence for content validity for the 10 existing PROMS, thus none could be recommended for use. The aim of this study was to use Rasch measurement theory (RMT) to develop a valid item pool for measurement of DA in RA. Methods Paper questionnaires were sent to people aged 18 or over with RA from four South Wales University Health Boards between September 2020 and November 2021. The questionnaire included 268 individual RA DA items extracted from the 10 PROMs identified by the systematic review, another four PROMs and a non-measurement group of items. Further items suggested by patient feedback were incorporated, including a Pain Activity Scale, discomfort when walking, standing, and exercising, plus fear of falling when walking. Items were grouped into domains established by OMERACT: tenderness and swelling, patient global, pain, fatigue, physical functioning; and additionally stiffness, swelling, discomfort/fear and mood. Exploratory factor analyses (EFA) were used to indicate which items from which domains loaded onto factors together. For each separate domain, psychometric properties were assessed by RMT analyses. This provided results on targeting, model fit, internal consistency, local dependency, unidimensionality and item threshold ordering. Results A test dataset of n = 398 and a validation dataset of n = 293 were available. EFA of the test dataset showed that 30 items from across the tenderness and swelling, patient global, pain, fatigue, physical functioning and stiffness domains loaded together. RMT analyses of these items in the test dataset indicated that the patient global domain comprised two distinct domains, which followed a clear content difference of general health and disease activity items. In assessing the best items to measure the now 7 domains, 12 items were discarded. These 7 domains were confirmed in the validation data. Subtest analyses indicated that patient global general health and fatigue did not measure the same overall construct as the other 5 domains, and instead worked together to measure a separate construct. This structure was confirmed using bi-factor model confirmatory factor analyses in the validation data. Conclusion Patient global items relating to general health and disease activity were two separate domains. RA DA can be measured using tenderness and swelling, patient global disease activity, pain, physical functioning and stiffness items, whilst fatigue and patient global general health items measure a health-related quality of life construct. The next steps are to undertake cognitive interviews to establish content validity and to develop a computer adaptive test based on anchored locations calculated using these data. Disclosure T. Pickles: None. M. Horton: None. K.B. Christensen: None. R. Phillips: None. D. Gillespie: None. N. Mo: None. E. Choy: None.

Suggestions for modifications to the Female Sexual Function Index based on cognitive interviews with sexual and gender minority individuals and cisgender, heterosexual persons.

Patient-reported outcome measures for sexual health were often designed for research studies that included only heterosexual, partnered, and cisgender participants; as such, they may have limited applicability for clinical use among sexual and gender minority (SGM) individuals or those without a partner. We aimed to conduct cognitive interviews with SGM persons and heterosexual women to determine the readability, comprehension, and applicability of questionnaire items to assess sexual function among diverse sexual and gender identities. We conducted 4 rounds of cognitive interviews with 52 participants (28 SGM; 24 cisgender, heterosexual) who provided feedback on the comprehension and wording of questionnaire items and response scales. We used items from the Female Sexual Function Index (FSFI) and focused on establishing content validity of a modified measure. Participants made recommendations for changes to the questionnaire, which was iteratively revised between interview rounds. Two independent coders analyzed the transcripts using structural coding based on 5 predefined codes: satisfaction with item, specificity/language change needed, missing/suggested item, patient definitions of concepts, and confusion with item. Content validity. After 3 rounds of cognitive interviews and revisions to the questionnaire, participants found the final version acceptable and understandable, thereby reaching thematic saturation and establishing content validity of the modified FSFI. Modifications included the following: replacing all instances of "sexual stimulation" and "intercourse" with "sexual activity (alone or with a partner)," broadening the definition of "vaginal penetration" beyond penile-vaginal penetration, and adding skip logic to include the option "no sexual activity." Participants identified missing concepts important to their sexual health, such as use of an external lubricant. The FSFI and similar questionnaires need to be adapted to broader clinical practice populations such that all persons' experiences are accurately reflected and assessed, ensuring that sexual health needs can be met more inclusively. A strength of the study was using cognitive interviews engaging patient perspectives, which is considered the gold standard for establishing content validity. One limitation is that participants included predominantly White and highly educated women. Feedback from interviews supports modifying FSFI items and further psychometric testing, and future studies should evaluate the measure among racially and educationally diverse groups.

Development of an instrument to measure awareness and mitigation of bias in maternal healthcare.

Implicit bias among maternal healthcare professionals contributes to disrespectful care and racial and ethnic disparities in patient outcomes, and there is growing consensus that implicit bias training is a key component of birth equity initiatives. A requirement for implicit bias training for healthcare professionals has become more widespread, but the impact training has is largely unknown, in part, because of a lack of validated instruments. Therefore, there is an urgent need for a psychometrically valid instrument for use in the evaluation of implicit bias training. This study aimed to develop a valid and reliable instrument to assess implicit bias awareness and mitigation practices among maternal care professionals and that can be used to evaluate interventions aimed at mitigating such bias in clinical practice. We conducted an instrument development and validation study in 3 phases. In phase 1, item development, we generated a 43-item bank from literature and consultation and established content validity with subject matter experts. In phase 2, instrument development, we administered a revised set of 33 items to 307 nurses and midwives and conducted exploratory factor analysis to demonstrate construct validity and reliability. In phase 3, instrument evaluation, we confirmed the factor structure and compared the means of implicit bias training-exposed and -unexposed participants to further demonstrate construct validity with a representative state sample of 2096 maternal healthcare professionals (physicians, midwives, and nurses). Based on phase 2 results, we retained 23 items for the Bias in Maternal Health Care scale, which showed high internal consistency (Cronbach's alpha, 0.86). We identified 3 subscales, namely a 9-item Bias Awareness subscale (Cronbach's alpha, 0.86), a 7-item Bias Mitigation Practice subscale (Cronbach's alpha, 0.82), and a 7-item Bias Mitigation Self-Efficacy subscale (Cronbach's alpha, 0.81). Validation of the Bias Awareness and Bias Mitigation Practice subscales in phase 3 demonstrated the instrument's high reliability (Cronbach's alpha 0.86 and 0.83, respectively) and discriminating performance among maternal healthcare professionals. We developed a reliable and valid instrument for measuring awareness and mitigation of bias among maternal healthcare professionals. It can be used to evaluate implicit bias training and other bias mitigation interventions in maternal healthcare settings.