Lawshe's Content Validity Ratio Research Articles

Cross-Cultural Adaptation, Reliability and Validity of the Persian Version of COVID-19 Preventive Measures Questionnaire in Radiation Therapy Center, Yazd, Iran

Background: Due to the overflow of people in radiation therapy centers and the importance of not interrupting the treatment process, preventive measures is one of the most important measures to deal with COVID-19; therefore, this study aims to evaluate intercultural compatibility, reliability, and validity of COVID-19 prevention measures in a radiation therapy center. Methods: This analytical cross-sectional study was performed with 20 personnel at radiation therapy center of Yazd, Iran using a census method in September 2021. A COVID-19 prevention scale with 29 items was used as measurement tool. First, cultural adaptation was assessed, and then, face validity was determined by calculating the impact score (IS). Content validity was evaluated by calculating the content validity ratio (CVR) and content validity index (CVI), and reliability was determined using the test-retest method, and Kuder–Richardson (KR20) alpha coefficient, and calculating intra-class correlation coefficient (ICC). All the analyses were performed in SPSS software version 24 with a significance level of 5%. Results: 30% (6 people) of the subjects were, 70% (40 people) of whom had less than 15 years of work experience. Four items of the questionnaire were removed based on the minimum Lawshe's CVR values, as their CVR was less than 0.62. Finally, the (KR20) coefficient, scale-level CVI, and ICC were estimated at 0.827, 0.98, and 0.52, respectively. Conclusion: The validity and reliability of the questionnaire were confirmed; therefore, this Persian version of 25-item scale is proposed as a suitable and effective tool for COVID-19 prevention measures in Persian language

Psychometric properties of the spatial skills self-efficacy scale: A validity and reliability study

The aim of this study was to develop a valid and reliable spatial skills self-efficacy scale for middle school students. To this end, the study sampled 5th-, 6th-, 7th-, and 8-grade students in the spring term of the 2021–2022 academic year. The data were collected from 1369 students for the pilot study and the validity and reliability analysis to investigate the psychometric properties of the scale. Twenty-five subject matter experts were asked for opinions and Lawshe's content validity ratio (1975) was used to measure the content validity of the scale. The results of the exploratory factor analysis conducted for the construct validity yielded a four-factor structure consisting of seventeen items, which accounted for 41.37 % of the total variance. The factor loadings of the scale items ranged from 0.41 to 0.75. When the fit indices of the model were examined as a result of Confirmatory Factor Analysis, it was determined that the firstorder multifactor structure were within the excellent or acceptable value ranges. The Cronbach's alpha coefficient was found to be 0.77 for the total scale. The results of the independent samples t-test for the upper and lower 27 % groups showed a significant difference between the groups for all scale items. The correlation coefficients of the subscales with one another and with the total scale showed significant positive moderate and high correlations. The test–retest reliability coefficients also showed significant positive moderate and high correlations. These results indicate that the Spatial Skills Self-Efficacy Scale is a measurement tool that produces valid and reliable measurements and can be used to determine the spatial skills self-efficacy of middle school students.

An In-depth Examination of Validity Assessment: Exploring Diverse Methodologies and Dimensions of Validity in Social Research Studies

Validity refers to the extent to which a test accurately measures what it claims to measure. Validity is a matter of degree. The study is done to know the various methods of validity. Validity is a fundamental aspect of research quality, which is why it is important to assess the various methods of validity. Descriptive research was used for the study. It was examined that face validity refers to an indicator that seems to measure its construct “on its face”. Content validity is a form of judgmental validity, that involves determining if a test adequately represents the content domain. Criterion-related validity methods include concurrent and predictive approaches. Construct validity assesses how well theoretical ideas are translated into measurable variables. Face validity involves examining whether a measurement appears suitable based on its operationalization on the face of it. Lawshe's content validity ratio quantifies expert agreement on the essentiality of individual items in a test or scale. In contrast, the content validity index considers overall expert agreement on the representativeness of items collectively within a test or scale. It considers the agreement among experts regarding the content validity of the entire set of items. Cohen’s Kappa is used to assess the level of agreement between two or more raters or observers when coding or categorizing data. The kappa coefficient is particularly useful when chance agreement among raters is possible. Item Objective Congruence evaluates the efficacy of items in measuring one or more objectives, accomplished by an unbiased expert panel provides evidence of content validity. The most significant assessment at this stage is determining if items and objectives are congruent. Concurrent and predictive validity are assessed using Pearson's correlation and Spearman rank correlation coefficients respectively. The Multi-trait Multimethod Matrix (MTMM), evaluates the construct validity of measures. Validity is essential for ensuring credible and meaningful research findings by enhancing the trustworthiness of the measuring instruments.

Validity and Reliability of Balanced Nutrition Knowledge and Eating Behavior Questionnaires among Adolescent Girls

Background: Assessing balanced nutrition knowledge and eating behavior among adolescent girls is important because they affect health and nutritional status, thus it needs measurement instruments that have been tested for validity and reliability. Objective: To assess the validity and reliability of balanced nutrition knowledge and eating behavior questionnaires among adolescent girls. Method: This study involved 30 Senior High School girls aged 15-18 years. The validity of the data was tested using difficulty level analysis, differentiating power analysis, distractor analysis, Lawshe's Content Validity Ratio (CVR) analysis, and item discrimination test. The reliability of the questionnaire was tested using Cronbach's Alpha coefficient. Results: A total of 20 items regarding balanced nutrition knowledge were considered valid and 10 items about eating behavior were declared valid. The results of the reliability test of the questionnaire on balanced nutrition knowledge and eating behavior obtained Cronbach's Alpha values of 0.822 and 0.829, thus this questionnaire is declared reliable. Conclusion: The questionnaire on balanced nutrition knowledge and eating behavior has good validity and reliability values. The questionnaire can be used to obtain accurate and consistent data in order to assess the balanced nutrition knowledge and eating behavior among adolescent girls.

The Eligibility of the Encyclopedia of Circulatory System Diseases and Disorders Based on Traditional Medicinal Plants for Hypertension as Learning Media

Learning media is an essential element to support the success of learning activities. It is used stimulate the thoughts, feelings, attention, and interest of students to learn. Learning media can be developed by adding concepts and photos from the results of a research. The purpose of this study was to determine the feasibility of the encyclopedia of circulatory system diseases and disorders as a learning media. The content of encyclopedia contains the result from the inventory of medicinal plants for hypertension at Babane Village, Bengkayang, Indonesia. This study applied a research and development method. This development procedure refers to the Sugiyono development model, which consists of five stages namely potential and problems, data collection, product design, design validation, and design revision. The validity of the encyclopedia was obtained through content validation executed by five experts in the field of both biology and learning media. Encyclopedia validation is reviewed from four aspects: content, presentation, language, and graphics. Validation analysis applied the Lawshe's content validity ratio (CVR) and content validity index (CVI) approach. The results showed that the average of CVR was 1.00 which result in the CVI value of 1.00 in the "valid" category. Based on the result, can be concluded that the encyclopedia was feasible as a learning media

디자인씽킹 교과목 학습 설문의 내용타당도(CVR) 분석연구

The rapid spread and development of the virtual educational environment occured with the onset of the COVID-19 pandemic in 2019. Interest and demand for effective educational methods for virtual learning from educational institutions have been increasing due to this phenomenon. As a result, there has been a shift from the origianly design thinking educaiton model that existed solely in person, has now had to transition into a virtual format. For this reason, research related to design thinking education in the online environment is insufficient. This study proposes an educational model for online design thinking based on flipped-learning. Students can actively form a team by learning common theories and sharing individual topics. The model was created based on the hypothesis that it would positively affect online team activity. The design thinking subject questionnaire was designed to cover four areas: class content, concept application stage satisfaction, instructor intervention, team activity. These four fields were tested originally by Lawshe's CVR(Content Validity Ratio) analysis in order to analyze the descriptive statistics for the mean and the standard deviation. Additionally, the KS(Kolmogorov-Smirnov) test was utilized in order to analzye the difference between these items. The validity of all the questions presented in the four areas: class content, concept application stage satisfaction, instructor intervention, and team activity method, were all proven to be valid. The items whose validity has been proven through this study can be used as primary data for systematizing the ideal design thinking class model of flipped-learning for instructors in the university class setting.

Development and Validation of Self-Paced Learning Digital Module in Mathematics 10

This study used the design and development analysis methodology to provide an empirical foundation for instructional design and to systematically analyze the data about the development and validation of Self – Paced Learning Digital Module (SPLDM) in Mathematics 10 based on the K-12 curriculum. It specifically addressed the following questions: 1) how can the Self–Paced Learning Digital Module in Mathematics 10 be developed? and 2) how can the Self–Paced Learning Digital Module in Mathematics 10 be validated by experts in terms of 2.1) Content Quality; 2.2) Design/Format; 2.3) Technical/Technological Quality, and; 2.4) Pedagogical Quality?
 The module underwent three phases: Preparation, Development, and Validation Phase. Using the Evaluation Instrument for Topics to be Included in the DE/ODel Module, ten (10) subject matter experts in mathematics 10 selected the essential and non-essential learning competencies. The Lawshe's content validity ratio (CVR) found 38 of the subject's 50 learning competencies as essential. Nine (9) experts utilized Evaluation Rating Sheet for DE and ODeL Modules to assess the content, format, technical/ technology, and pedagogy elements of the created SPLDM. The results of the validation of the developed SPLDM was acceptable in terms of content, design/format, technical/technological, and pedagogical after undergoing evaluation and full approval of all evaluators. As a result of this research, the generated SPLDM can be used as a learning material in the classroom when teaching Mathematics 10.

Questionnaire Validity and Reliability of Intuitive Eating Behaviour towards Obesity: Behavioural perspective

Introduction: Intuitive eating involves consuming food based on internal hunger and satiety cues. This study aims to establish a validated and reliable questionnaire. Methods: The questionnaire items' difficulty, uncertainty, and inappropriateness were conducted. The validity is analysed using the Content Validity Index and Lawshe's Content Validity Ratio, estimated by inter-item reliability (Cronbach's Alpha) and test-retest reliability (Intraclass Correlation Coefficient). Results: The CVIs correlation coefficient is 0.89 – 0.99. The Cronbach's Alpha coefficient value is 0.66 – 0.68. Conclusion: This study showed that the questionnaire has good validity and reliability towards obesity from an intuitive eating behaviour perspective. Keywords: intuitive eating, obesity, validity, reliability eISSN: 2398-4287 © 2022. The Authors. Published for AMER ABRA cE-Bs by e-International Publishing House, Ltd., UK. This is an open-access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/). Peer–review under the responsibility of AMER (Association of Malaysian Environment-Behaviour Researchers), ABRA (Association of Behavioural Researchers on Asians/Africans/Arabians), and cE-Bs (Centre for Environment-Behaviour Studies), Faculty of Architecture, Planning & Surveying, Universiti Teknologi MARA, Malaysia. DOI: https://doi.org/10.21834/ebpj.v7i21.3678

Content validation of school readiness module and school readiness scale for assessing school readiness in children with autism spectrum disorder.

There is a paucity of research on interventions targeting preschool children with autism spectrum disorders (ASD) for school readiness. The objectives of this study are to develop and validate a school readiness module for making children with ASD ready for inclusive education and a scale to assess school readiness in them. Based on literature review, principles of learning, and techniques of behavioral intervention, a module was developed and reviewed by independent experts regarding the utility of the contents. A scale to assess school readiness was also developed to measure the impact of administering the module on children with ASD which was also validated by the same set of experts. Lawshe's content validity ratio was used to assess the appropriateness of each item for inclusion in the module and scale. Experts (n=6) gave their opinion on the usefulness of the School Readiness module for children with ASD. The experts agreed that most of the content under each component were valid with the exception of identification of objects by function, identification of environmental sounds and answering social questions. Similarly, in the school readiness scale there was good agreement for all items except for 1 item under domain 2 and 2 items under domain 5. A school readiness module and a scale to assess school readiness based on interventions provided as per the school readiness module were developed and validated. Further studies are needed to assess the utility of the module and scale in children with ASD.

Designing, validation, and feasibility of a yoga module for patients with ankylosing spondylitis

BackgroundAnkylosing spondylitis (AS) is a chronic inflammatory disease that causes significant disability and reduced quality of life. Scientific studies on yoga have revealed its various health benefits in chronic conditions, including autoimmune diseases. However, whether yoga is feasible for AS patients or not is not studied. Further, no validated yoga module is available for AS patients. Objective(s)This study intended to develop a yoga module for AS patients and investigated its feasibility of use. Materials and methodsThe study was completed in three stages. In Stage I, six yoga experts prepared a list of 64 yoga practices based on the classical and contemporary yogic literature review. Of these practices, 41 were included in the designed yoga module. In Stage II, 41 experts with a minimum of five years of experience in yoga therapy were invited for yoga module validation. The usefulness of the practices was rated by experts on a 3-point scale (1: not at all useful, 2: moderately useful, and 3: very much useful). The Lawshe content validity ratio (CVR) method was used for the content validity of the yoga module. Practices with a CVR score of > 0.3 were retained in the final yoga module. In Stage III, a certified yoga instructor administered the validated yoga module to 19 AS patients (average age: 35.5 ± 10.7 years) thrice weekly for a month. Feasibility was assessed on the basis of the attrition rate, retention rate, attendance of the participants, and the subjective response on practical sessions using a structured checklist. ResultsOf the 41 practices in the module, 31 had a CVR score of > 0.3 and were included in the final yoga module. Of the 25 participants, 19 (76%) completed the study while six dropped out (24%). Nineteen patients reported greater improvement in pain and flexibility. They found yoga relaxing and easy to practice. Most participants (65%) were able to practice a minimum of 30 min/day. ConclusionThe present study offers a validated yoga module consisting of 31 practices for AS patients. The results of the pilot suggested that the module is feasible, acceptable, and easy to practice for AS patients. We recommend that AS patients should practice this yoga module for a minimum of 30 min every day under the supervision of a yoga expert.

The Effect of Flipped Classroom on Iranian Adolescents: Elementary EFL Learners’ Vocabulary Recall and Retention

Vocabulary as an inseparable part of language learning has a vital role in communication, which requires to be taught in new approaches. A quasi-experimental method was used to achieve the aims of the current study. In doing this, 48 Iranian female adolescent elementary learners (13–17) were assigned based on the Oxford Quick Placement Test from 70 female students to take part in 4 sessions of treatment. The participants were randomly divided into two groups: an experimental group (n = 24) and a control group (n = 24). The experimental group was taught using the flipped classroom, while the control group was exposed to the conventional methods. Research instruments involving an English language placement test, vocabulary pretest, immediate posttest, and delayed test (the study lasted for 9 weeks). A list of 28 target words was given to both groups. Words were selected by the Lawshe content validity ratio (CVR) and content validity index (CVI) to have validity. The findings of ANCOVA indicated that conducting the flipped classrooms has significant impacts on Iranian adolescent elementary students’ vocabulary recall and retention. Hence, it is an applicable approach that carries important implications for teachers, foreign language syllabus designers, and curriculum planners.

Face validity of standardized assessments for wheeled mobility & seating evaluations

ABSTRACT A problem in the Complex Rehabilitation Technology industry is the lack of standardization in the assessment for wheeled mobility and seating (WMS). The aim of this paper was to identify assessment tools commonly used by clinicians during WMS evaluations. After the tools were identified by a panel of 12 subject matter experts, a presentation at the 2018 International Seating Symposium in Vancouver, Canada and the 2018 European Seating Symposium in Dublin, Ireland polled attendees via the Sli.do polling application to determine professional opinions of each tool, resulting in face validity for use in wheelchair evaluations. The Lawshe Content Validity Ratio was used to convert this anecdotal data into numerical data, indicating which tools were most and least used by attendees. Finally, a literature search was conducted to determine the reliability, validity, and International Classification of Functioning, Disability, & Health domain for each measure. The findings indicate that while there are many standardized and reliable assessment tools available for wheeled mobility and seating evaluations, most clinicians use only a few standardized assessment tools during WMS evaluations.

E-customer loyalty in gamified trusted store platforms: a case study analysis in Iran

Customer satisfaction, trust, and loyalty are the three most fundamental elements of e-marketing. Previous researchers have noted that satisfaction is a key factor in commanding loyalty. However, the relationship between satisfaction, trust, and loyalty is strongly dependent on the type of platform provided by digital stores. On the other hand, gamification in E-businesses has grown rapidly in recent years. In this context, it is necessary to explore the effects of gamification on e-customer satisfaction and loyalty. In this paper, it is argued that customer satisfaction alone cannot inspire loyalty. Simply speaking, customers’ satisfaction with gamified services can lead to developing trust and, in turn, loyalty. This research also presents a thorough review of the effect of store-related motivational factors, such as gamification, on trust. These factors include moderator and mediator variables. The hypotheses of this study are considered in the context of one of the largest online retail stores in Iran which enjoys a large market share in the Middle East. To this end, Lawshe content validity ratio is utilized and expert opinions are applied to the proposed model. Evaluation results, obtained through the smartPLS, established the robustness of our modeling in terms of reliability analysis, significance level analysis, discriminant validity analysis, coefficient of determination, model fitting, and cross-validation.

Adaptación y validación en Chile del instrumento de identificación de pacientes con necesidad de atención paliativa: NECPAL-CCOMS-ICO 3.1©

ObjectiveAdapt and validate the NECPAL instrument in Chile. DesignProspective, longitudinal, analytical study for validation of the instrument in 5 stages: cultural adaptation, content validation, pilot test, application, and statistical analysis. PlaceFour primary care centers of the South East Metropolitan Health Service, in Santiago, Chile. ParticipantsPrimary health care physicians and nurses for cultural adaptation and application, and palliative care experts for content validation. Main measurementsCultural adaptation was carried out through cognitive interviews. Content validity was measured using Delphi method and the Lawshe content validity ratio (CVR) was obtained. In the pilot test, we measured stability (test–retest), inter judge harmony and application time in 14 chronic advanced patients (CAP). The test was applied to this same group, calculating the sample according to Nunally's recommendation. ResultsA sample of 118 CAP was obtained. The CVR was 0.75 and the average testing time was 6.7 min (SD = 4.01). The test–retest obtained a Kappa test concordance index between 0.632 and 1.0; and the interjudge harmony agreement between 0.192 and 0.692. The surprise question (PS) was positive in 20.3% of the sample. The main conditions associated with the disease-specific severity item, were fragility (23.7%), chronic heart disease (21.2%) and chronic lung disease (12.7%). The demand group and specific severity indicators obtained a greater predictive capacity of PS+, with an area under the curve of 0.808 (95% CI: 0.697–0.918). ConclusionsNECPAL is feasible to be used in Chile, has adequate psychometric properties and will allow early detection of patients in need of palliative care.

Polish adaptation of emotional Stroop test in assessment of pedophilia - a pilot study.

A pilot study was conducted in order to construct a Polish adaptation of emotional Stroop test in assessment of pedophilia. The study consisted of two stages. The first stage involved creating test material by ranking words in adequate lists by competent experts. The second stage consisted of empirical verification of the principle of emotional Stroop test in a non-clinical population. Based on the assessment of five competent experts, words were ordered from the most to the least sexually arousing (Kendall's W from 0.368 to 0.693). Six ranked lists were obtained, and the competent experts were subsequently asked to assess whether these lists were suitable for the study (Lawshe's Content Validity Ratio from 0.6 to 1.0). Two categories of words were merged. Five ranked lists were obtained, and the competent experts were subsequently asked again to assess whether these lists were suitable for the study (Lawshe's Content Validity Ratio 1.0). The created lists of words were approved by allcompetent experts. Based on the experimental study conducted on a non-clinical population, it was shown that, in accordance with the principle of the test, the mean response time for sexually related words was longer that for neutral words. The mean response time for children-related words did not differ significantly from response time for neutral words. Based on the study with competent experts and conducted experiments, an initial Polish adaptation of the emotional Stroop test for diagnosis of pedophilia has been created. Further studies with persons with pedophilia are needed to implement the test in clinical setting.

Development, validation, and feasibility of a school-based short duration integrated classroom yoga module: A pilot study design.

BACKGROUND:The practice of yoga is proven to have physical, cognitive and emotional benefits for school children. Despite this many schools do not include yoga in their daily schedule. The reasons cited are lack of time and resources. To overcome these problems the present study aimed to develop and validate a short duration Integrated classroom yoga module. The design guidelines were that it should be possible to practice in the classroom environment and that it could be led by the class teacher. In this way the module would overcome the problem of both time and resource.MATERIALS AND METHODS:The study had two main phases. In the first phase, selected ICYM practices based on the literature review were validated by 21 subject matter experts using Lawhse's content validity ratio (CVR) formula. In the second phase, a pilot study using a paired sample prepost measurement design was carried out on 49 high school children. The study was conducted in June 2019. The intervention period was 1 month, and the test variables were physical fitness, cognitive performance, self-esteem, emotional well-being, and personality characteristic. Paired sample t-test was the analysis tool and the software used was the Statistical Package for the Social Science version 26.RESULTS:In the Lawshe's CVR analysis, 17 out of the 24 practices tested were rated by experts as essential as was the overall module (CVR score ≥0.429). In the pilot study, there were significant differences in the postmean scores compared to premean scores, for all the 4 EUROFIT physical fitness testing battery tests (P < 0.02), all the three scores of the Stroop color-word naming task (P < 0.001) and the Rosenberg self-esteem scale (P < 0.008).CONCLUSION:ICYM was validated and found feasible by the present study. It was found to have a statistically significant impact on physical fitness, cognitive performance, and self-esteem variables. However, a randomized control trial with a longer intervention period is needed to strengthen the present study.

Sustainability Indicators to Assess Infrastructure Projects: Sector Disclosure to Interlock with the Global Reporting Initiative

Infrastructure projects have great potential to impact the sustainability of cities due to typically being large-sized projects and having a high level of intervention. Thus, evaluating the sustainability of these projects through sustainability reports is highly relevant, mainly regarding their impacts on the environment, public health, and the local economy. The Global Reporting Initiative (GRI) is the most widespread and internationally accepted Sustainability report tool. However, the GRI does not have an infrastructure sector disclosure. This research addresses this gap by providing a sustainability assessment instrument for infrastructure projects that interlocks with the Global Reporting Initiative (GRI). An extensive and detailed literature review was conducted, identifying 97 potential indicators to measure the sustainability of infrastructure projects. These indicators were evaluated following a top-down approach, conducting a survey of professionals experienced in the relevant field using Lawshe's content validity ratio. The results showed that 42 indicators were validated as essential, with 21 of them, not specifically related to infrastructure projects, already covered by the standard disclosure of the GRI. This assessment enabled the proposal of a sector disclosure formed by 21 new indicators related to the environmental, economic, and social dimensions. This study closes a gap in the evaluation of the sustainability of infrastructure projects and contributes to the discussion about sustainability indicators in infrastructure projects.

Development and Validation of an Information Leaflet on Oral Care for Irradiated Patients.

PurposeThe aim of the study was to develop an optimally designed and comprehensibly formulated patient information leaflet (PIL) to improve patients’ memory of information provided by physicians during a radiotherapy (head and neck area) consultation. This PIL was tested on unaffected probands for its usefulness in clinical practice.Patients and MethodsA panel of experts compiled the main topics using Lawshe’s content validity ratio. Flesch’s Reading Ease Score (FRE) and the Baker Able Leaflet Design (BALD) index were adapted to appropriate values to determine text comprehensibility and graphic design. The evaluation involving unaffected participants (231 men, 380 women, 21 not specified; mean age = 32 ± 13.63 years, range = 18–79 years) was conducted based on three questionnaires for four groups of respondents with varying prior knowledge of the subject. When answering the questionnaires, only half the participants had access to the PIL.ResultsThe expert panel included 59 out of 75 proposed topics. After reformulations, the FRE was adjusted from 38.5 to 51.4. The BALD index ranged from 24 to 26, depending on the printout edition. The evaluation of 632 unaffected participants indicated a difference in the correctly answered items that ranged from 2.86% to 30.76% between participants with and without access to the PIL.ConclusionThe general guidelines for the design of written patient information material were met. The evaluation of unaffected volunteers resulted in an advantage by answering the questionnaires after receiving the PIL. This study supports health practitioners in the development, design, and evaluation of written information material using scientific methods. An evaluation should be performed on affected patients.

Design and content validation of a survey questionnaire assessing the determinants of human papillomavirus (HPV) vaccine hesitancy in France: A reactive Delphi study

IntroductionThis study aimed to develop and undertake a preliminary validation of a French Survey Questionnaire for the Determinants of HPV Vaccine Hesitancy (FSQD-HPVH). MethodsWe undertook an electronic-based Delphi consultation among a panel of Francophone experts in two rounds. Round 1 consisted of the assessment of a structured questionnaire comprising of three parts ((i) Contextual influences, (ii) Individual and group influences, and (iii) Vaccine/vaccination-specific issues), in line with the WHO Strategic Advisory Group of Experts (SAGE) Vaccine Hesitancy (VH) Model of Determinants. Items included in this questionnaire were based on a literature review. Definitions of the factors included in the SAGE model were provided in the questionnaire. The panel of experts was asked to score each item using a 3-point Likert scale, in which 1 meant “Essential”, 2 “Useful but not essential”, and 3 “Not necessary”. The panel was also invited to comment on the clarity/comprehension of the questions and suggest reformulations/additional items. Lawshe's Content Validity Ratio (CVR) was computed to assess the level of consensus for each statement. Only items upon which agreement was not reached in Round 1 (CVR < 0.6) and newly proposed items were submitted for evaluation in Round 2, using the same procedure. ResultsFifteen experts completed the two rounds. Of 83 items evaluated in Round 1, 35 (42%) had a CVR ≥ 0.6 and were accepted without modification. In Round 2, 66 items were submitted to the same panel and consensus was reached for 22 (33%) items using the threshold of 0.6. The final FSQD-HPVH version includes 57 items. ConclusionThis study developed a survey instrument for the evaluation of HPV VH in France with good content validity. It will be used to assess the determinants of HPV VH, the first step towards an evidence-based approach to improving HPV vaccination rates in France.

A Framework for the Systematic Evaluation of Anatomy Education Apps

BackgroundMobile‐based applications (Apps) are one of the fastest growing trends in educational technology. A search on the iTunes App Store, for example, results in 142 Apps in response to the search term “anatomy”. However, most educational Apps are not peer‐reviewed. The quality and accuracy are suspect, and few have been evaluated in the classroom. A comprehensive resource that provides guidance as to the educational usefulness, quality, and content accuracy of Apps in the anatomical sciences is needed.Aims To triangulate existing educational technology frameworks into a framework for evaluating anatomy education Apps With a national sample of academic technology leads, faculty and students, test and validate the framework MethodsPhase 1 triangulated peer‐reviewed articles on Apps, with known educational technology frameworks from the literature (Hanrahan et al., 2014; Liu et al., 2010; Romrell et al., 2014; Ruiz et al., 2006; Van Nuland et al., 2017). Phase 2 validated the framework in two steps. Phase 2A used chain referral sampling (i.e. n = 6 with at least one educational technology lead from each of the four AAMC regions: East, West, South, and Central) to obtain feedback. In Phase 2B, participants were recruited from Anatomy Connected Open Forum and through chain referral sampling (n = 58 agreed to participate). Participants applied the revised framework to two anatomy apps (Complete Anatomy and Heart Pro III) and completed a content validity survey. All methods were approved by the Colorado Multiple Institutional Review Board.ResultsPhase 1 resulted in the formation of a framework that was revised in Phase 2a and is hosted here https://docs.google.com/forms/d/1s5VWA1OfE‐HGNLz_LezMb3ngofPztRaOb7PFCXl_jjc/viewform?edit_requested=true. A total of 25 participants (14 faculty, 4 tech leads and 7 students) fulfilled all parts of Phase 2b to completion. Content validity was analyzed using Lawshe's content validity ratio; critical values (&gt;0.39 as determined by Wilson et al., 2012) with 0.05 significance for two‐tailed test were achieved with 11 of the 26 criteria, including usability, organization, reliability, navigation, self‐assessment, content breadth, content depth, content accuracy, multimedia accuracy, recommendation for others, and overall value.Discussion and ConclusionAnatomy Education App framework validation presents a major asset for students, faculty and administrators based on feedback from a national panel; the framework is freely available. The validation is limited by the low number of respondents, but will be enhanced as more investigators apply the framework to their research.Support or Funding InformationRymer Small Grants Program, Academy of Medical Educators, University of Colorado School of MedicineThis abstract is from the Experimental Biology 2019 Meeting. There is no full text article associated with this abstract published in The FASEB Journal.