Cognitive Pretesting Research Articles

Translation and cross-cultural adaptation of the Spanish version of the King’s Parkinson’s Disease Pain Scale (KPPS)

Purpose To translate and cross-culturally adapt the King’s Parkinson’s Disease Pain Scale (KPPS) into Spanish. Methods The English KPPS was forward translated into Spanish, back translated, and revised by the original developers. Cross-cultural adaptation of relevant target groups was conducted following COSMIN standards. Cognitive pretesting in Spanish people with Parkinson’s Disease (PD) and pain was performed via three semi-structured focus group meetings. Changes were implemented based on qualitative and quantitative analyses. A Delphi study (two rounds) of individual assessments by Spanish experts was conducted. Changes were implemented based on the Content Validity Index (CVI) at item and scale levels (I-CVI/S-CVI). Results Five Spanish professionals performed the initial translation. Thirty-eight inter-translation inconsistencies were identified, and consensus was reached for a unified version. After back translation and confirmation with scale developers, the pre-final Spanish KPPS was obtained. Cognitive pretesting in 30 people with PD-related pain identified 5 items where terminology changes were made to assure comprehensibility. The Delphi study in 14 Spanish experts underlined the necessity to further modify two items. After the second round, all experts agreed on the modifications and I-CVI and S-CVI were satisfactory. Conclusions The KPPS was successfully translated and cross-culturally adapted into Spanish.

Validation of the Kazakh Version of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale

Background and PurposeThe International Movement Disorder Society revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) is widely used in the assessment of the severity of Parkinson’s disease (PD). This study aimed to validate the Kazakh version of the MDS-UPDRS, explore its dimensionality, and compare it to the original English version. MethodsThe validation was conducted in three phases: first, the English version of the MDS-UPDRS was translated into Kazakh and thereafter back-translated into English by two independent teams; second, the Kazakh version underwent a cognitive pretesting; third, the Kazakh version was tested in 360 native Kazakh-speaking PD patients. Both confirmatory and exploratory factor analyses were performed to validate the scale. We calculated the comparative fit index (CFI) for confirmatory factor analysis and used unweighted least squares for exploratory factor analysis. ResultsThe CFI was higher than 0.90 for all parts of the scale, thereby meeting the pre-set threshold for the official designation of a validated translation. Exploratory factor analysis also showed that the Kazakh MDS-UPDRS has the analogous factors structure in each part as the English version. ConclusionsThe Kazakh MDS-UPDRS had a consistent overall structure as the English MDS-UPDRS, and it was designated as the official Kazakh MDS-UPDRS, which can reliably be used in the Kazakh-speaking populations. Presently, Kazakhstan stands as the sole country in both Central Asia and Transcaucasia with an MDS-approved translated version of the MDS-UPDRS. We expect that other Central Asian and Transcaucasian countries will embark on the MDS Translation Program for MDS-UPDRS in the near future.

Development and validation of the Workplace Learning Inventory in Health Sciences Education: a multimethod study

Health sciences students face many challenges in regard to clinical practical learning. A better understanding of student learning is required to address student needs in this crucial phase. The theory of self-regulated learning provides a comprehensive view of learning and could serve as a basis for further research. There are instruments to assess self-regulated learning in preclinical academic learning. However, there are no such instruments for workplace learning. The aim of the present study is to provide a comprehensive inventory from which researchers can select those scales that are relevant to their research questions in the investigation of workplace learning. Hence, the aim is to develop and validate a set of scales to assess undergraduates’ workplace learning in health sciences education in four areas (cognition, motivation, emotion, and context) on two levels (the learning process level and the metalevel). Study 1 is a qualitative multimethod study to identify indicators and develop items. It integrates the perspectives of students, teachers, and researchers and includes six steps: literature review, interviews, synthesis, item development, expert review, and cognitive pretesting. This study yields a set of scales for each area on both levels. Study 2 is a quantitative study to assess the psychometric properties. The results show acceptable values in terms of unidimensionality, reliability and validity for each of the 31 scales. The newly developed Workplace Learning Inventory is comprehensive; the scales are relevant to workplace learning and short enough that their administration is feasible in the workplace setting. The rigorous process of questionnaire development contributes to the validity of scales. By providing the Workplace Learning Inventory, we hope to encourage research on workplace learning in health sciences education from an educational psychology perspective.

Curriculum implementation challenges: Development and validation of an integrated curriculum implementation challenges tool.

To develop an instrument to identify the challenges faced by faculty while implementing an integrated curriculum in an undergraduate dentistry program. The study was conducted between September 2020 and October 2021 at the University College of Medicine and Dentistry (UCMD), University of Lahore (UOL). A preliminary questionnaire, developed through literature review and faculty interviews was sent to 10 medical education experts for content validation via the Delphi technique. Content Validity Index (CVI) was calculated for individual items (I-CVI) as well as for the composite scale (S-CVI). A panel agreement of more than 75% was considered as the criterion for the inclusion of items in the questionnaire. Cognitive pretesting of five faculty members was conducted and pilot testing was subsequently done with 27 faculty members. The reliability of the tool was determined by Cronbach's alpha. After the Delphi process, the final Integrated Curriculum Implementation Challenges (ICIC) questionnaire had 42 items. S-CVI was 0.87 and the cut-off value for I-CVI was taken as 0.78 as the criterion for item deletion. Cognitive interviews and pretesting revealed good item interpretation. Cronbach's alpha for this tool was 0.87. ICIC is a useful instrument with good reliability and content validity. It can be used to identify the presence and extent of challenges faced by the faculty while implementing an integrated curriculum.

Validation Of An Assessment Tool For Professionalism In Medical Students: A Mixed Method Study.

To develop and validate a tool for the assessment of professionalism in medical students during clerkship. The mixed-method study was conducted from February to September 2018 at the Islamic International Medical College, Riphah University, Islamabad, Pakistan. A preliminary focus group discussion was followed by two rounds of Delphi technique to validate micro-scenarios for the relevance of each item of the instrument alongside its content validity. For content and construct validation of the tool, content validity, cognitive validity, cognitive pretesting, confirmatory factor analysis and reliability assessment was done. Piloting was done on medical students doing clerkship. Data was analysed using SPSS 21 and AMOS 21. A 35-item preliminary tool from micro-scenarios generated after focus group discussion was developed for the first Delphi round. For the second round, a 29-item tool was resent to the experts for validation. After the second round, the content validity scale was 0.94. Removal of 1 item produced a 28-item tool. Pre-cognitive testing of the 28 items resulted in rephrasing of 4, modification of 2, and deletion of 1 and a tool with 27 items. Piloting was carried out and after CFA resulted in a reduced 21 item tool. Cronbach's alpha value was 0.96. A good to excellent model fit was produced after performing confirmatory factor analysis. A tool comprising of 21 items scored on a 5-point Likert scale that had acceptable construct validity and good content validity was developed. It can be used to evaluate the level of professionalism of local medical students doing clerkship.

The Gut Dysmotility Questionnaire for Parkinson's disease: Insights into development and pretest studies.

A total of 48% of patients with Parkinson's disease (PD) present symptoms of gastrointestinal dysfunction, particularly constipation. Furthermore, gastrointestinal tract (GIT)-related non-motor symptoms (NMSs) appear at all stages of PD, can be prodromal by many years and have a relevant impact on the quality of life. There is a lack of GIT-focused validated tools specific to PD to assess their occurrence, progress, and response to treatment. The aim of this study was to develop and evaluate a novel, disease- and symptom-specific, self-completed questionnaire, titled Gut Dysmotility Questionnaire (GDQ), for screening and monitoring gastrointestinal dysmotility of the lower GIT in patients with PD. In phase 1, a systematic literature review and multidisciplinary expert discussions were conducted. In phase 2, cognitive pretest studies comprising standard pretests, interviews, and evaluation questionnaires were performed in patients with PD (n = 21), age- and sex-matched healthy controls (HC) (n = 30), and neurologists (n = 11). Incorporating these results, a second round of cognitive pretests was performed investigating further patients with PD (n = 10), age- and sex-matched HC (n = 10), and neurologists (n = 5). The questionnaire was adapted resulting in the final GDQ, which underwent cross-cultural adaptation to the English language. We report significantly higher GDQ total scores and higher scores in five out of eight domains indicating a higher prevalence of gastrointestinal dysmotility in patients with PD than in HC (p < 0.05). Cognitive pretesting improved the preliminary GDQ so that the final GDQ was rated as relevant (100/100%), comprehensive (100/90%), easy to understand concerning questions and answer options (100/90%), and of appropriate length (80/100%) by neurologists and patients with PD, respectively. The GDQ demonstrated excellent internal consistency (Cronbach's alpha value of 0.94). Evidence for good construct validity is given by moderate to high correlations of the GDQ total score and its domains by intercorrelations (r s = 0.67-0.91; p < 0.001) and with validated general NMS measures as well as with specific items that assess gastrointestinal symptoms. The GDQ is a novel, easy, and quick 18-item self-assessment questionnaire to screen for and monitor gastrointestinal dysmotility with a focus on constipation in patients with PD. It has shown high acceptance and efficacy as well as good construct validity in cognitive pretests.

Use of a focus group-based cognitive interview methodology to validate a cooking behavior survey among African-American adults.

Disparities in diet-related diseases persist among African-Americans despite advances in risk factor identification and evidence-based management strategies. Cooking is a dietary behavior linked to improved dietary quality and cardiometabolic health outcomes. However, epidemiologic studies suggest that African-American adults report a lower frequency of cooking at home when compared to other racial groups, despite reporting on average cooking time. To better understand cooking behavior among African-Americans and reported disparities in behavior, we sought to develop a survey instrument using focus group-based cognitive interviews, a pretesting method that provides insights into a survey respondent's interpretation and mental processing of survey questions. A comprised survey instrument was developed based on input from a community advisory board, a literature review, and a content review by cooking behavior experts. The cognitive interview pretesting of the instrument involved African-American adults (n = 11) at risk for cardiovascular disease who were recruited from a community-based participatory research study in Washington, D.C., to participate in a focus group-based cognitive interview. Cognitive interview methodologies included the verbal think-aloud protocol and the use of retrospective probes. Thematic analysis and evaluation of verbalized cognitive processes were conducted using verbatim transcripts. Five thematic themes related to the survey were generated: (1) Clarity and relevancy of question items; (2) influence of participants' perspectives and gender roles; (3) participant social desirability response to questions; (4) concern regarding question intent. Eleven survey items were determined as difficult by participants. Cooking topics for these items were: cooking practices, cooking skills, cooking perception (how one defines cooking), food shopping skills, and socialization around cooking. Question comprehension and interpreting response selections were the most common problems identified. Cognitive interviews are useful for cooking research as they can evaluate survey questions to determine if the meaning of the question as intended by the researcher is communicated to the respondents-specific implications from the results that apply to cooking research include revising questions on cooking practice and skills. Focus-group-based cognitive interviews may provide a feasible method to develop culturally grounded survey instruments to help understand disparities in behavior for culturally relevant diet behaviors such as cooking.

Getting the FACS: A Protocol for Developing a Survey Instrument to Measure Carer and Family Engagement with Mental Health Services

Government policies recommend, and all stakeholders benefit, when mental health services meaningfully engage with carers and family. However, health service engagement with carers is inadequate, and often non-existent with children whose parents are service users. There are seven fundamental ways that carers and families want to be integrated with and engaged by health services but current survey instruments do not capture these seven engagement practices. This protocol describes the development of two closely aligned Family and Carer Surveys (FACS) to measure engagement of service users in mental health services. The new measures are based on the seven engagement themes and a conceptual distinction between the carer and family, with particular focus on where the service user is a parent. The instruments will be developed in five stages; (1) item generation (2) Cognitive pretesting of survey (3) preliminary item content quantitative assessment (4) psychometric analysis of a large data collection and (5) selection of items for short form instruments. These steps will operationalise the seven fundamental ways that families and carers want to be engaged with mental health services, thereby providing valid and reliable measures for use in research and benchmarking of carer and family engagement.

Skills and Attitudes of Students which Affect Fostering of Critical Thinking in Undergraduate Medical Students

Objective: To identify students' skills and attitudes which affect fostering critical thinking in undergraduate medical students.&#x0D; Study Design: Modified Delphi Study.&#x0D; Place and Duration of Study: University of Lahore, Lahore Pakistan, from Oct 2018 to Apr 2019.&#x0D; Methodology: This study was carried out to develop consensus. Participants were selected through purposive sampling. A draft questionnaire was developed on Google Forms and sent to five experts for pilot study, construct validation and cognitive pre-testing. Wait Time, rating on the Likert Scale and Consensus criteria of 80% were defined before the start of the study. A new questionnaire was sent to participants in each round.&#x0D; Results: The study enrolled 27 experts, of which only 14 participated in all the three rounds. Experts initially identified 18 factors related to students' skills and attitudes. The consensus was achieved for 14 out of the final 16 factors. Pre-medical education through Matric/FSc. rote memorization, lack of logical and rational thinking, no active participation in interactive sessions and strategic learning by students hamper fostering critical thinking. On the other hand, students from O/A level possess better critical thinking skills because they were trained to criticize and question during class activities.&#x0D; Conclusion: Individual skills/attitudes of students affect fostering of critical thinking among undergraduate medical students. They should be addressed primarily in pre-medical schools for better critical thinking abilities in medical school.

Fragile X-associated tremor ataxia syndrome rating scale: Revision and content validity using a mixed method approach.

BackgroundThe original Fragile X-associated Tremor Ataxia Syndrome Rating Scale (FXTAS-RS) contained 61 items, some requiring modifications to better meet recommendations for patient-focused rating scale development.PurposeProvide initial validation of a revised version of the FXTAS-RS for motor signs.MethodWe conducted a two-phase mixed-method approach. In Phase 1, revision, we implemented a Delphi technique identifying pertinent domains/subdomains and developing items through expert consensus. In Phase 2, content validation, we conducted cognitive pretesting assessing comprehensibility, comprehensiveness, and relevance of items to FXTAS motor signs.ResultsAfter five rounds of Delphi panel and two rounds of cognitive pretesting, the revised version of the FXTAS-RS was established with 18 items covering five domains and 13 subdomains of motor signs. Cognitive pretesting revealed adequate content validity for the assessment of FXTAS motor signs.ConclusionThe revised FXTAS-RS has been successfully validated for content and it is now ready for large-scale field validation.

Faculty and its Impact on Critical Thinking Abilities of Undergraduate Medical Students – A Delphi Study

Aim: To identify individual skills/attitude of faculty which influence fostering of critical thinking in undergraduate medical students Study Design: Modified Delphi Study Place and Duration of study: This study was carried out at University of Lahore from October 2018 to April 2019. Methodology: Delphi Study was used to develop consensus. Experts were MME/MHPE/ MCPS HPE qualified faculty members with 03 years of teaching experience. A draft questionnaire was developed on Google Forms after thorough literature review and sent to 05 experts for Pilot Study, construct validation and cognitive pretesting. A Wait Time of 04 weeks, rating of questions on 5 point Likert Scale and consensus criteria of 80% were defined before the start of study. SPSS 25 was used for data analysis. During 1st Round, draft questionnaire was amended in the light of opinion of experts to open ended questions and was sent to all the participants for their response. During 2nd Round, a new questionnaire was developed after data analysis and duly amended questionnaire was sent to the participants of Round 1. Results: A total of 27 experts were enrolled in the study. 17 experts (62.96%) responded during the first round. Consensus was achieved for 13 items (65%) out of 20. In the 2nd round, 15(88.23%) out of 17 responded. Consensus was achieved for all 7 items (100%). The study revealed that majority of our faculty lacks proper training and does not possess knowledge about various educational theories / principles and their application. They do not know how to act as facilitator, conduct tutorials, ask high order questions, reflect and provide constructive feedback. They do not give enough wait time to students after asking an open ended / higher order question. They are over committed with administrative work besides teaching. Conclusion: Individual skills / attitude of faculty hamper fostering of CT in undergraduate medical students. They are mainly due to lack of formal faculty training which should be addressed not only in the medical institutions but in the pre-medical schools as well to inculcate higher order critical thinking skills in students. Keywords: Critical Thinking (CT), Undergraduate Medical Students, Faculty, Reflections, Feedback

Faculty and its Impact on Critical Thinking Abilities of Undergraduate Medical Students – A Delphi Study

Aim: To identify the individual skills / attitude of faculty which influence fostering of critical thinking in undergraduate medical students Study design: Modified Delphi Study Place and Duration of study: This study was carried out at University of Lahore from October 2018 to April 2019. Methodology: Delphi Study was used to develop consensus. Experts were MME/MHPE/ MCPS HPE qualified faculty members with 03 years of teaching experience. A draft questionnaire was developed on Google Forms after thorough literature review and sent to 05 experts for Pilot Study, construct validation and cognitive pretesting. A Wait Time of 04 weeks, rating of questions on 5 point Likert Scale and consensus criteria of 80% were defined before the start of study. SPSS 25 was used for data analysis. During 1st Round, draft questionnaire was amended in the light of opinion of experts to open ended questions and was sent to all the participants for their response. Results: A total of 27 experts were enrolled in the study. 17 experts (62.96%) responded during the first round. Consensus was achieved for 13 items (65%) out of 20. In the 2nd round, 15 (88.23%) out of 17 responded. Consensus was achieved for all 7 items (100%). The study revealed that majority of our faculty lacks proper training and does not possess knowledge about various educational theories / principles and their application. They do not know how to act as facilitator, conduct tutorials, ask high order questions, reflect and provide constructive feedback. Conclusion: Individual skills / attitude of faculty hamper fostering of CT in undergraduate medical students. They are mainly due to lack of formal faculty training which should be addressed not only in the medical institutions but in the pre-medical schools as well to inculcate higher order critical thinking skills in students. Keywords: Critical Thinking (CT), Undergraduate Medical Students, Faculty, Reflections, Feedback

Translation, cultural adaptation, and validation of the Integrated Palliative Outcome Scale-renal (IPOS-r) to Czech

BackgroundPatients with advanced kidney disease suffer from burdensome symptoms, which should be assessed by valid and reliable patient-reported outcome measures.This study aimed to provide a translation, cultural adaptation, and validation of the Czech version of the IPOS-r.MethodsThe IPOS-r was translated to Czech and culturally adapted using cognitive interviews. During the validation phase, patients and staff in dialysis centres and outpatient renal clinics completed the IPOS-r. Internal consistency was tested with Cronbach’s alpha, its reliability via intraclass correlation coefficient for total IPOS-r score, and weighted Kappa (for test-retest and interrater reliability of individual items). Convergent validity was tested with Spearman correlation to Kidney Disease Quality of Life Survey-Short Form 1.2 (KDQOL-SF 1.2). We assessed sensitivity to change using a distribution-based approach.ResultsTwo sets of translators independently performed forward and backward translations of the IPOS-r. Ten patients and ten health care professionals participated in cognitive pre-testing. The sample size for validation included 88 patients (mean age 66 ± SD13.8; 58% men) who were treated with haemodialysis (70.5%), home haemodialysis (5.5%), peritoneal dialysis (3%), and conservative management (21%). Cronbach’s alpha was 0.72, and the intraclass correlation was 0.84 for test-retest reliability and 0.73 for interrater reliability. The IPOS-r correlated with KDQOL-SF 1.2 had a rho between 0.4–0.8 for most of the IPOS-r items, showing good convergent validity. The IPOS-r measure is feasible and takes 9 minutes to complete. Patients who reported a change in health status after 1 month demonstrated a total IPOS-r score change of eight points in both positive and negative directions.ConclusionsThe process of translation and cultural adaptation of the IPOS-r was successful, and the Czech IPOS-r measure is a valid and reliable tool. The Czech IPOS-r can be used to assess symptoms in patients with advanced chronic kidney disease.Trial registrationGAUK [82121].

Measuring the implementation of media literacy statewide: a validation study

ABSTRACT Even though media literacy education has grown in the United States and around the world, policymakers still lack knowledge about the scope and depth of implementation of media literacy learning activities in the elementary and secondary grades. This study reports how the Media Literacy Implementation Index (MLI) was developed and validated using a quota sample of stakeholders from across all school districts and local communities in Rhode Island. Using an established theoretical framework for media literacy, we first constructed a scale to examine the likelihood that learners may encounter a set of media literacy learning activities. Multiple methods were then used to assess reliability and validity of the instrument including cognitive pretesting, think aloud protocols, a content validation of MLI learning activities in relation to national education standards documents, and statistical validation including principal components analysis to assess internal consistency, reliability, and validity. An easy-to-implement measure of the scope and depth of media literacy education in elementary and secondary schools will be useful to diverse stakeholders including educators, school leaders, librarians, parents, community members, and elected public officials.

Validation of the Arabic Version of the Movement Disorder Society-Unified Parkinson's Disease Rating Scale.

The Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS) has become the gold standard for evaluating different domains in Parkinson's disease (PD), and it is commonly used in clinical practice, research, and clinical trials. The objectives are to validate the Arabic-translated version of the MDS-UPDRS and to assess its factor structure compared with the English version. The study was carried out in three phases: first, the English version of the MDS-UPDRS was translated into Arabic and subsequently back-translated into English by independent translation team; second, cognitive pretesting of selected items was performed; third, the Arabic version was tested in over 400 native Arabic-speaking PD patients. The psychometric properties of the translated version were analyzed using confirmatory factor analysis (CFA) as well as exploratory factor analysis (EFA). The factor structure of the Arabic version was consistent with that of the English version based on the high CFIs for all four parts of the MDS-UPDRS in the CFA (CFI ≥0.90), confirming its suitability for use in Arabic. The Arabic version of the MDS-UPDRS has good construct validity in Arabic-speaking patients with PD and has been thereby designated as an official MDS-UPDRS version. The data collection methodology among Arabic-speaking countries across two continents of Asia and Africa provides a roadmap for validating additional MDS rating scale initiatives and is strong evidence that underserved regions can be energically mobilized to promote efforts that apply to better clinical care, education, and research for PD. © 2022 International Parkinson and Movement Disorder Society.

The 27-Item Multiple Sclerosis Quality of Life Questionnaire: A New Brief Measure Including Treatment Burden and Work Life.

Treatment- and work-related aspects have been neglected in health-related quality of life (HRQOL) measures in multiple sclerosis (MS). We aimed to develop a brief instrument covering all important impairment-, activity-, participation-, and treatment-related aspects for use in research and practice. The 27-item Multiple Sclerosis Quality of Life Questionnaire (MS-QLQ27) was developed using open item collection, a multidisciplinary expert panel, and cognitive pretesting. It was evaluated for reliability, construct validity, and responsiveness in 100 patients presenting with relapse (84 at follow-up ~14 days later). Construct validity was analyzed by correlating the MS-QLQ27 with the disease-specific Hamburg Quality of Life Questionnaire in MS (HAQUAMS) and generic HRQOL instruments. The Expanded Disability Status Scale (EDSS) was used to analyze known-groups validity. Responsiveness was determined as the correlation of changes in MS-QLQ27 scores with changes in validation criteria. Internal consistency was high (Cronbach α = 0.94 at baseline and 0.93 at follow-up). Convergent validity was supported by direction and magnitude of associations with disease-specific and generic instruments. Correlations with change in convergent criteria were strong, indicating responsiveness. The HAQUAMS showed the strongest associations with the MS-QLQ27. The MS-QLQ27 showed the highest effect size compared with other patient-reported outcomes and the EDSS. It successfully distinguished between levels of disease severity. These results indicate that the MS-QLQ27 is a reliable, valid, and highly responsive instrument for assessing HRQOL during relapse evolution in MS. Its advantages are that it is brief yet comprehensive, covering work- and treatment-related aspects not addressed in previous measures.

Impact of the COVID-19 Pandemic on Postgraduate Training in Radiation Oncology

Purpose/Objective(s)To report the degree to which post-graduate trainees in radiation oncology perceive their education has been impacted by COVID-19.Materials/MethodsA cross-sectional online survey was administered in June 2020 to trainee members of Canadian Association of Radiation Oncology (CARO). The 82-item survey was adapted from a similar survey administered during SARS and included the Stanford Acute Stress Reaction and Ways of Coping Questionnaires. The survey was developed using best practices including expert review and cognitive pre-testing. Frequency statistics are reported.ResultsThirty-four trainees (10 fellows, 24 residents) responded. Nearly half of participants indicated that the overall impact of COVID-19 on training was negative/very negative (n = 15; 46%) or neutral (n = 15; 46%) with a small number indicating a positive/very positive (n = 3; 9%). Majority of trainees agreed/strongly agreed with the following statements: “I had difficulty concentrating on tasks because of concerns about COVID-19” (n = 17; 52%), “I had fears about contracting COVID-19” (n = 17; 52%), “I had fears of family/loved ones contracting COVID-19” (n = 29; 88%), “I felt socially isolated from friends and family because of COVID-19” (n = 23; 70%), “I felt safe from COVID-19 in the hospital during my clinical duties“ (n = 15; 46%), and “I was concerned that my personal safety was at risk if/when I was redeployed from my planned clinical duties” (n = 20; 61%). The changes that had a negative/very negative impact on learning included “the impact of limited patient contact” (n = 19; 58%), “the impact of virtual patient contact” (n = 11; 33%), and “limitations to travel and networking” (n = 31; 91%). Most reported reduced teaching from staff (n = 22; 66%). Two-thirds of trainees (n = 22, 67%) reported severe (> 50%) reduction in ambulatory clinical activities, 16 (49%) reported a moderate (< 50%) reduction in new patient consultations, while virtual follow-ups (n = 25: 76%) and in-patient clinical care activities (n = 12; 36%) increased. Nearly half of respondents reported no impact on contouring (n = 16; 49%), on-treatment management (n = 17; 52%) and tumor boards (n = 14; 42%) with the majority of other respondents reporting a decrease in these activities. Electives were cancelled in province (n = 10/20; 50%), out-of-province (n = 16/20; 80%) and internationally (n = 15/18; 83%).ConclusionSignificant changes to radiation oncology training were wrought by the COVID-19 pandemic and roughly half of trainees perceive that these changes had a negative impact on their training. Safety concerns for self and family were significant and strategies to mitigate these concerns should be a priority.

Validation of the Finnish Version of the Unified Dyskinesia Rating Scale

Introduction: The Unified Dyskinesia Rating Scale (UDysRS) was developed to provide a comprehensive rating tool of dyskinesia in Parkinson’s disease (PD). Because dyskinesia therapy trials involve multicenter studies, having a scale that is validated in multiple non-English languages is pivotal to international efforts to treat dyskinesia. The aim of the present study was to organize and perform an independent validation of the UDysRS Finnish version. Methods: The UDysRS was translated into Finnish and then back-translated into English using 2 independent teams. Cognitive pretesting was conducted on the Finnish version and required modifications to the structure or wording of the translation. The final Finnish version was administered to 250 PD patients whose native language is Finnish. The data were analyzed to assess the confirmatory factor structure to the Spanish UDysRS (the reference standard). Secondary analyses included an exploratory factor analysis (EFA), independent of the reference standard. Results: The comparative fit index (CFI), in comparison with the reference standard factor structure, was 0.963 for Finnish. In the EFA, where variability from sample to sample is expected, isolated item differences of factor structure were found between the Finnish and Reference Standard versions of the UDysRS. These subtle differences may relate to differences in sample composition or variations in disease status. Conclusion: The overall factor structure of the Finnish version was consistent with that of the reference standard, and it can be designated as the official version of the UDysRS for Finnish speaking populations.

Know thyself: Development and validation of self-awareness of teaching skills (SATS) measuring instrument for medical teachers

Background Self-awareness is measured previously in the higher education however not specifically in relation to ‘effectiveness of teaching’ thus leaving a gap in the literature. Aim To develop and validate an instrument for medical teachers to measure the self-awareness of their teaching. Methods This was an instrument development multiphase mixed-method study design, based on Brookfield’s four lenses theory. The instrument was validated after administering content validity, cognitive pretesting, confirmatory factor analysis and reliability analysis. Contrasting groups’ method of standard setting was used to determine the cut-off scores for the levels of self-awareness of medical teachers. Results A 26-item preliminary draft instrument was reduced to a 19-item final instrument having four themes, that is; self-reflection, communication with students, student feedback, and peer review. Content Validity Scale was 0.92. Confirmatory factor analysis yielded a model with a good-to-excellent fit and an internal consistency of 0.85. The cut-off values were calculated to be 79%, 70%, 59%, and 53% for excellent, good, average and poor self-awareness of teaching skills respectively. Conclusion The Self-awareness of the teaching skills instrument has excellent validity and good reliability in measuring the self-awareness of teaching skills of medical teachers.

The Value of Cognitive Pretesting: Improving Validity and Revealing Blind Spots through the Development of a Newborn Screening Parent Experiences Survey.

Surveys are used to gather a range of data on newborn screening (NBS) processes. We describe the development of a survey about parents’ NBS experiences, in the United States, informed by cognitive pretest interviews among parents with varying NBS test results (true-positive, false-positive, normal). Cognitive pretest interviews were conducted following a semi-structured script and notes were taken to identify problematic survey items. The study team met weekly to discuss pretest feedback, draft changes, and generate revised items. Pretests indicated that parent experiences with NBS are varied and NBS screening procedures are not well understood. Substantial modifications were made to survey questions concerning NBS testing and result communication. Pretesters often associated NBS with other tests/exams/scales—APGAR scores, Ages and Stages questionnaires, and genetic testing during pregnancy. Some pretesters recalled receiving NBS blood spot results during their hospital admission, an uncommon practice, and few recalled knowing results would be provided to them or their pediatrician in the first few weeks of life. Thorough explanations regarding NBS procedures and expectations were embedded within the survey to enhance and improve interpretation of survey questions. Future NBS experience surveys should utilize cognitive pretesting to capture divergent experiences and improve response validity.