A Portable Hip Arthroscopy Simulator Demonstrates Good Face and Content Validity with Incomplete Construct Validity

Abstract
 Introduction & Objectives : Various simulation models are available for cataract surgery training. Farra Eye Model, a new cataract surgery simulator, was developed to provide the resident with more affordable options. This study aims to determine the validity and reliability of the Farra Eye Model as a surgical simulator for capsulorhexis training.
 Methods : A cross-sectional pilot study was performed among ophthalmology residents and consultants to assess face, content, and construct validity of a new surgical simulator. Subjects were divided into novice, intermediate, and expert groups according to their level of expertise. Face and content validity was assessed using a validated questionnaire with a 5-point Likert scale response. Construct validity was done by comparing capsulorhexis performance between the three groups. Two raters assessed performance using capsulorhexis indices in ICO-OSCAR: Phaco assessment tool, number of forceps grabs, and duration of capsulorhexis.
 Results : A total of 33 subjects were recruited and divided equally into three groups. The overall face validity score was favorable (3.67 ± 0.67). However, the capsule elasticity was rated poor (2.73 ± 1.1) among the intermediate group. The content validity was favorable regarding the overall score (4.15± 0.58) and each assessment component. In the construct validity test, intermediate and expert groups showed better capsulorhexis performance than the novice group on all parameters (p<0.001), with good inter-rater reliability (ICC>0.7).
 Conclusion : Farra Eye Model has a good face and content validity for capsulorhexis training and is able to differentiate the novice group from intermediate and expert groups. However, it remains a challenge to replicate human lens capsule elasticity.

Introduction Changing surgical training models as a result of reduced working hours, shorter schemes and the recent pandemic has resulted in reduced operative hours being logged by trainees. Novel and validated training methods are needed to rectify the deficit faced by surgical trainees, to maintain continued development and retention of skill throughout their training scheme. Method A comparative interventional study was carried out in a regional trauma unit hospital. Volunteers were stratified into novice, intermediate and expert groups based on self-reported levels of experience. Each carried out a simulated proximal femoral nail on a virtual platform following instruction on its use. Face and content validity was also assessed. Results The proximal femoral nail module demonstrated construct validity. Intermediate surgeons performed significantly better than novices (P=.03), with shorter procedural times (P=.013) Three of the intermediate group achieved the proficiency level set by the expert group. Time taken to completion for expert surgeons was less than intermediate group, although this did not reach significance (P=.16). Face and content validity was reported for the module. Conclusions The proximal femoral nail module on the Precision OS platform demonstrated good face, content, and construct validity under expert scrutiny. Further research evaluating use of virtual platforms to optimise surgical training is needed.

PD6-03 MULTI-INSTITUTIONAL VALIDATION OF FUNDAMENTAL INANIMATE ROBOTIC SKILLS TASKS (FIRST)

MP01-15 FACE, CONTENT, AND CONSTRUCT VALIDATION OF ENDOSCOPIC NEEDLE INJECTION (ENI) SIMULATOR FOR TRANSURETHRAL BULKING AGENT IN TREATMENT OF STRESS URINARY INCONTINENCE

IntroductionAs work-stress, is associated with Non Communicable Diseases, and decreased work productivity, health and economic benefits are expected from periodic work-stress screening among employees using valid and reliable tools. Tool to Assess and classify Work Stress (TAWS– 16) was developed to overcome limitations in existing work-stress assessment tools in India. This study aims to test face, content, criterion and construct validity of TAWS– 16 in a sample of managerial-supervisory employees.MethodsNine domain experts rated face and content validity of TAWS– 16. Content validity was measured by Content Validity Indices (I-CVI, S-CVI) and Modified Kappa statistics. Empirical validity was tested by analysing data reported from 356 Information Technology (IT) professionals wherein Exploratory Factor Analysis was conducted for the assessment of Construct Validity. Self-reported data was collected in an unlinked and anonymous manner using a web-link, which was emailed to the study subjects, after initial introductory telephone or personal conversation. Criterion Validity was tested against stress sub-scale of DASS– 21. This study was approved by NIMHANS ethics committee.ResultsFindings revealed that TAWS– 16 has good face validity. The content validity is acceptable (CVI = 0.829). Construct Validity is appropriate as 60.8% of the total variance was explained by the factors identified in our study. Criterion Validity was moderate (Kappa Value 0.208) due to inappropriate work-stress instrument for comparison with TAWS– 16.ConclusionsOverall, TAWS– 16 demonstrated good face, content and construct validity. It measures work-stressors, coping abilities and psycho-somatic symptoms associated with work-stress. We recommend use of TAWS– 16 for periodic screening and classification of work-stress among employees.

As work-stress, is associated with Non Communicable Diseases, and decreased work productivity, health and economic benefits are expected from periodic work-stress screening among employees using valid and reliable tools. Tool to Assess and classify Work Stress (TAWS- 16) was developed to overcome limitations in existing work-stress assessment tools in India. This study aims to test face, content, criterion and construct validity of TAWS- 16 in a sample of managerial-supervisory employees. Nine domain experts rated face and content validity of TAWS- 16. Content validity was measured by Content Validity Indices (I-CVI, S-CVI) and Modified Kappa statistics. Empirical validity was tested by analysing data reported from 356 Information Technology (IT) professionals wherein Exploratory Factor Analysis was conducted for the assessment of Construct Validity. Self-reported data was collected in an unlinked and anonymous manner using a web-link, which was emailed to the study subjects, after initial introductory telephone or personal conversation. Criterion Validity was tested against stress sub-scale of DASS- 21. This study was approved by NIMHANS ethics committee. Findings revealed that TAWS- 16 has good face validity. The content validity is acceptable (CVI = 0.829). Construct Validity is appropriate as 60.8% of the total variance was explained by the factors identified in our study. Criterion Validity was moderate (Kappa Value 0.208) due to inappropriate work-stress instrument for comparison with TAWS- 16. Overall, TAWS- 16 demonstrated good face, content and construct validity. It measures work-stressors, coping abilities and psycho-somatic symptoms associated with work-stress. We recommend use of TAWS- 16 for periodic screening and classification of work-stress among employees.

Face, Content, and Construct Validations of Endoscopic Needle Injection Simulator for Transurethral Bulking Agent in Treatment of Stress Urinary Incontinence

PurposeTo develop and test the validity of a surgical competency assessment tool for simulated small-incision cataract surgery (SICS).SettingParticipating ophthalmologists contributed from 8 countries.DesignQualitative and quantitative development and evaluation of face and content validity of an assessment rubric, and evaluation of construct validity and reliability.MethodsThe SICS Ophthalmic Simulated Surgical Competency Assessment Rubric (Sim-OSSCAR) was developed and assessed for face and content validity by an international group of experienced ophthalmologists. Groups of novice and competent surgeons from 4 countries were recorded performing surgery, and masked assessments were performed by 4 expert surgeons, to determine construct validity and reliability.ResultsThe Sim-OSSCAR for SICS was assessed by a panel of 12 international experts from 8 countries. In response to the question, “Do you think the OSSCAR represents the surgical techniques and skills upon which trainees should be assessed?,” all respondents either agreed or strongly agreed. Face validity was rated as 4.60 (out of 5.0). The content was iteratively agreed to by the panel of experts; final content validity was rated as 4.5. Interobserver reliability was assessed, and 17 of 20 items in the assessment matrix had a Krippendorff α correlation of more than 0.6. A Wilcoxon rank-sum test showed that competent surgeons perform better than novices (P = .02).ConclusionsThis newly developed and validated assessment tool for simulation SICS, based on the International Council of Ophthalmology's Ophthalmology Surgical Competency Assessment Rubric, has good face and content validity. It can play a role in ophthalmic surgical education.

Temporal bone drilling simulators are valuable tools for developing anatomical knowledge and drill handling motor skills. However, there are currently no simulators focusing on learning fine microsurgical gestures using micro-instruments. The aim of this study was to validate a training kit and simulation program dedicated to otologic microsurgery. A prospective single-center comparative study was conducted. The Micro-TRAIN kit comprised a mannequin and six interchangeable exercise modules. Two simulation sessions were conducted two months apart, with debriefing for two selected subgroups after the first session. Progression in performance, assessed by task completion time, execution time and technical skills, was evaluated in three groups: 10 novices, 10 intermediate level and 5 experts. Content validity, face validity and participant satisfaction were also assessed. In session 1, there were significant differences in performance between the three groups (P<0.005). Both the novice and intermediate groups showed improvement between sessions. Improvement in total score in the intermediate group was statistically significant (P=0.0019) and suggestive in the novice group (P=0.0059). Participants who received debriefing tended to improve more (novices: P=0.012; intermediate: P=0.036). Experts rated the simulator's realism and relevance as above 8/10. This study confirmed the content, face and construct validity of the Micro-TRAIN simulator. It is an effective tool for acquiring microsurgical skills in otologic surgery.

AimsThe validity of using the Canadian Medication Safety Self‐Assessment for Long‐Term Care (MSSA‐LTC) tool in the international context is unknown. This study aims to determine the face and content validity of the Canadian MSSA‐LTC tool for assessing medication safety‐related processes in the Australian long‐term care (LTC) setting.MethodsA modified two‐round RAND/UCLA Appropriateness Method was used to assess the face and content validity of the 129 items in the Canadian MSSA‐LTC tool for use in the Australian LTC setting. An expert panel of health professionals with expertise in medication management in the Australian LTC setting rated each item separately for relevance to the Australian LTC context (face validity) and importance for medication safety (content validity). Items, where there was panel agreement that the item was definitely or potentially relevant for the Australian context/important for medication safety, were considered valid for the relevant attribute.ResultsNine expert panel members participated in the study. The expert panel rated 118 (91.5%) of the 129 items of the MSSA‐LTC tool as acceptable relevant for the Australian LTC setting and, therefore, having face validity and all items (100%) as important for medication safety, thus having good content validity. The expert panel members were unable to agree about the relevance of nine items for use in the Australian LTC context and considered two items as not relevant for the Australian LTC context.ConclusionsThe Canadian MSSA‐LTC tool appears valid for measuring medication safety‐related processes in Australian LTC with good face and content validity.

BackgroundThe reliability and validity of instruments used to survey health-care providers' views about and experiences with research evidence have seldom been examined.MethodsCountry teams from ten low- and middle-income countries (China, Ghana, India, Iran, Kazakhstan, Laos, Mexico, Pakistan, Senegal and Tanzania) participated in the development, translation, pilot-testing and administration of a questionnaire designed to measure health-care providers' views and activities related to improving their clinical practice and their awareness of, access to and use of research evidence, as well as changes in their clinical practice that they attribute to particular sources of research evidence that they have used. We use internal consistency as a measure of the questionnaire's reliability and, whenever possible, we use explanatory factor analyses to assess the degree to which questions that pertain to a single domain actually address common themes. We assess the questionnaire's face validity and content validity and, to a lesser extent, we also explore its criterion validity.ResultsThe questionnaire has high internal consistency, with Cronbach's alphas between 0.7 and 0.9 for 16 of 20 domains and sub-domains (identified by factor analyses). Cronbach's alphas are greater than 0.9 for two domains, suggesting some item redundancy. Pre- and post-field work assessments indicate the questionnaire has good face validity and content validity. Our limited assessment of criterion validity shows weak but statistically significant associations between the general influence of research evidence among providers and more specific measures of providers' change in approach to preventing or treating a clinical condition.ConclusionOur analysis points to a number of strengths of the questionnaire - high internal consistency (reliability) and good face and content validity - but also to areas where it can be shortened without losing important conceptual domains.

PurposeTo investigate the concurrent, face and content validity of an evaluation tool for Myofascial Adhesions in Patients after Breast Cancer (MAP-BC evaluation tool).Methods1) Concurrent validity of the MAP-BC evaluation tool was investigated by exploring correlations (Spearman’s rank Correlation Coefficient) between the subjective scores (0 –no adhesions to 3 –very strong adhesions) of the skin level using the MAP-BC evaluation tool and objective elasticity parameters (maximal skin extension and gross elasticity) generated by the Cutometer Dual MPA 580. Nine different examination points on and around the mastectomy scar were evaluated. 2) Face and content validity were explored by questioning therapists experienced with myofascial therapy in breast cancer patients about the comprehensibility and comprehensiveness of the MAP-BC evaluation tool.Results1) Only three meaningful correlations were found on the mastectomy scar. For the most lateral examination point on the mastectomy scar a moderate negative correlation (-0.44, p = 0.01) with the maximal skin extension and a moderate positive correlation with the resistance versus ability of returning or ‘gross elasticity’ (0.42, p = 0.02) were found. For the middle point on the mastectomy scar an almost moderate positive correlation with gross elasticity was found as well (0.38, p = 0.04) 2) Content and face validity have been found to be good. Eighty-nine percent of the respondent found the instructions understandable and 98% found the scoring system obvious. Thirty-seven percent of the therapists suggested to add the possibility to evaluate additional anatomical locations in case of reconstructive and/or bilateral surgery.ConclusionsThe MAP-BC evaluation tool for myofascial adhesions in breast cancer patients has good face and content validity. Evidence for good concurrent validity of the skin level was found only on the mastectomy scar itself.

Holmium Laser Enucleation of the Prostate: Simulation-Based Training Curriculum and Validation

ObjectiveOur poster at the 2009 winter BTS meeting described the Chest drain training module for CMT1 and CMT2 trainees in West Midlands Deanery. We have proceeded further to formally validate...

Face Validity