Assessment Tool For Students Research Articles

Improving Patient-Provider Communication: Evolution of a Tool to Assess Physical Therapist Students’ Spanish-Language Proficiency

ABSTRACTCommunication is at the foundation of safe and effective health care. When patients with limited English proficiency (LEP) have healthcare providers who do not speak their language, the communication barrier can lead to poor outcomes. Responding to the high number of Spanish-speaking individuals with LEP, a Doctor of Physical Therapy (DPT) program has integrated Spanish-language training in the curriculum. This paper describes their development and pilot testing of the Physical Therapy Spanish Proficiency Measure (PT-SPM). The PT-SPM was developed for English-Spanish bilingual faculty or clinicians to assess DPT students’ clinically relevant Spanish communication. The PT-SPM has 11 total items with four subscales: verbal proficiency, aural proficiency, written proficiency, and cultural competence. Twenty-three pairs of DPT students and their bilingual clinical instructors were recruited for an interrater agreement study; each used the PT-SPM to independently rate the student’s Spanish communication with Spanish-speaking patients with LEP in clinical settings. Cohen’s weighted kappa was calculated to evaluate interrater agreement between the students and clinicians on item level, using linear incremental weights. The weighted kappa coefficients ranged from 0.35 (fair agreement) to 0.80 (substantial agreement). All weighted kappa coefficients were statistically significant. Two items had fair agreement; four items had moderate agreement, and five items had substantial agreement. These results support that the PT-SPM may have value as an assessment tool for DPT students in clinical settings. Educators in other health professions may consider adapting the PT-SPM. This paper contributes to the broader interprofessional dialogue about how to assess and improve patient-provider communication.

Group OSCE (GOSCE) as a formative clinical assessment tool forpre-clerkship medical students at the University of Sharjah.

ObjectivesContinuous formative assessment with appropriate feedback is the pillar of effective clinical teaching and learning. Group Objective Structured Clinical Examination (GOSCE) has been reported as a resource-effective method of formative assessment. The present study aims to describe the development and evaluation of GOSCE as a formative assessment for pre-clerkship medical students.MethodsAt the University of Sharjah, GOSCE was introduced to medical students in Years 1, 2, and 3. The GOSCE was conducted as a formative assessment in which groups of 4–5 students were observed while they performed various clinical skills, followed by structured feedback from clinical tutors and peers. GOSCE was evaluated both quantitatively and qualitatively and appropriate statistical analysis was applied to evaluate their responses.ResultsA total of 232 students who attended the GOSCE responded to the questionnaires. Most of the students and clinical tutors preferred formative GOSCE over individual feedback. Both students and clinical tutors valued the experience as it helped students to identify gaps and to share knowledge and skills among group members.ConclusionThis study found that formative GOSCE provided a valuable and feasible educational opportunity for students to receive feedback about their clinical skills.

A Student Assessment Tool for Standardized Patient Simulations (SAT-SPS): Psychometric analysis

BackgroundThe evaluation of the level of clinical competence acquired by the student is a complex process that must meet various requirements to ensure its quality. The psychometric analysis of the data collected by the assessment tools used is a fundamental aspect to guarantee the student's competence level. AimTo conduct a psychometric analysis of an instrument which assesses clinical competence in nursing students at simulation stations with standardized patients in OSCE-format tests. MethodThe construct of clinical competence was operationalized as a set of observable and measurable behaviors, measured by the newly-created Student Assessment Tool for Standardized Patient Simulations (SAT-SPS), which was comprised of 27 items. The categories assigned to the items were ‘incorrect or not performed’ (0), ‘acceptable’ (1), and ‘correct’ (2). Participants499 nursing students. Data were collected by two independent observers during the assessment of the students' performance at a four-station OSCE with standardized patients.Descriptive statistics were used to summarize the variables. The difficulty levels and floor and ceiling effects were determined for each item. Reliability was analyzed using internal consistency and inter-observer reliability. The validity analysis was performed considering face validity, content and construct validity (through exploratory factor analysis), and criterion validity. ResultsInternal reliability and inter-observer reliability were higher than 0.80.The construct validity analysis suggested a three-factor model accounting for 37.1% of the variance. These three factors were named ‘Nursing process’, ‘Communication skills’, and ‘Safe practice’. A significant correlation was found between the scores obtained and the students' grades in general, as well as with the grades obtained in subjects with clinical content. ConclusionsThe assessment tool has proven to be sufficiently reliable and valid for the assessment of the clinical competence of nursing students using standardized patients. This tool has three main components: the nursing process, communication skills, and safety management.

FmCASES National Examination as a Pretest in a Family Medicine Clerkship.

Pretests have been shown to contribute to improved performance on standardized tests by serving to facilitate development of individualized study plans. fmCASES is an existing validated examination used widely in family medicine clerkships throughout the country. Our study aimed to determine if implementation of the fmCASES National Examination as a pretest decreased overall failure rates on the end-of-clerkship National Board of Medical Examiners (NBME) subject examination, and to assess if fmCASES pretest scores correlate with student NBME scores. One hundred seventy-one and 160 clerkship medical students in different class years at a single institution served as the control and intervention groups, respectively. The intervention group took the fmCASES National Examination as a pretest at the beginning of the clerkship and received educational prescriptions based on the results. Chi-square analysis, Pearson correlation, and receiver operating curve analysis were used to evaluate the effectiveness and correlations for the intervention. Students completing an fmCASES National Examination as a pretest failed the end-of-clerkship NBME exam at significantly lower rates than those students not taking the pretest. The overall failure rate for the intervention group was 8.1% compared to 17.5% for the control group (P=0.01). Higher pretest scores correlated with higher NBME examination scores (r=0.55, P<0.001). fmCASES National Examination is helpful as a formative assessment tool for students beginning their family medicine clerkship. This tool introduces students to course learning objectives, assists them in identifying content areas most in need of study, and can be used to help students design individualized study plans.

Development of a Health Literacy Assessment for Young Adult High School Students: A Pilot Study

Several conceptual models have been developed to measure health literacy. However, health literacy of the community, particularly the young is still under reported and under-explored. The purpose of this study was to develop and validate health literacy assessment tool for high school students. Participants were 150 students. The concept of health literacy evolved from defining, redefining, outlining and quantifying health literacy needs of a young population. Seven health education experts reviewed the initial item pool and helped select 85 questions for testing. The reviewer also provided confidential feedback via an evaluation questionnaire. Seven distinct health literacy domains meanings emerged viz health promotion, health care, disease prevention, health behaviour, health attitude, health wellbeing and health culture. The instrument has 85 questions that look promising for measuring health literacy in high school students, but needs additional and continuous testing with larger population to see how these questions continue to perform. The findings of present study might be a useful policy maker and health organizations to reflect and focus their promotion efforts on the more practical aspects of day-to-day in health promotion and are concerned with empowering young through enhancing their knowledge and improving their ability to make choices about their health wellbeing.

The National Clinical Assessment Tool for Medical Students in the Emergency Department (NCAT-EM).

IntroductionClinical assessment of medical students in emergency medicine (EM) clerkships is a highly variable process that presents unique challenges and opportunities. Currently, clerkship directors use institution-specific tools with unproven validity and reliability that may or may not address competencies valued most highly in the EM setting. Standardization of assessment practices and development of a common, valid, specialty-specific tool would benefit EM educators and students.MethodsA two-day national consensus conference was held in March 2016 in the Clerkship Directors in Emergency Medicine (CDEM) track at the Council of Residency Directors in Emergency Medicine (CORD) Academic Assembly in Nashville, TN. The goal of this conference was to standardize assessment practices and to create a national clinical assessment tool for use in EM clerkships across the country. Conference leaders synthesized the literature, articulated major themes and questions pertinent to clinical assessment of students in EM, clarified the issues, and outlined the consensus-building process prior to consensus-building activities.ResultsThe first day of the conference was dedicated to developing consensus on these key themes in clinical assessment. The second day of the conference was dedicated to discussing and voting on proposed domains to be included in the national clinical assessment tool. A modified Delphi process was initiated after the conference to reconcile questions and items that did not reach an a priori level of consensus.ConclusionThe final tool, the National Clinical Assessment Tool for Medical Students in Emergency Medicine (NCAT-EM) is presented here.

The Student Assessment Tools: A Reflection on the Pedagogical Practices of the Teacher

The Student Assessment Tools: A Reflection on the Pedagogical Practices of the Teacher

Role of cognitive assessment for high school graduates prior to choosing their college major.

[Purpose] Academic performance of college students can be impacted by the efficacy of students’ ability and teaching methods. It is important to assess the progression of college students’ cognitive abilities among different college majors and as they move from junior to senior levels. However, dearth of studies have been examined the role of cognitive ability tests as a tool to determine the aptitude of the perspective students. Therefore, this study assessed cognitive abilities of computer science and ART students. [Subjects and Methods] Participants were 130 college students (70 computer and 60 art students) in their first and final years of study at King Saud University. Cognitive ability was assessed using the Test of Nonverbal Intelligence, Third Edition. [Results] The cognitive ability of computer science students were statistically better than that of art students and were shown improvement from junior to senior levels, while the cognitive ability of art students did not. [Conclusion] The cognitive ability of computer science college students was superior compared to those in art, indicating the importance of cognitive ability assessment for high school graduates prior to choosing a college major. Cognitive scales should be included as an aptitude assessment tool for decision-makers and prospective students to determine an appropriate career, which might reduce the rate of university drop out.

Using Facebook's Open Source Capture the Flag Platform as a Hands-on Learning and Assessment Tool for Cybersecurity Education

There is a high demand for skilled cybersecurity professionals, however, entry-level employees and college graduates often lack the hands-on and real-world experience they need to be successful in the cybersecurity industry. This high demand has led educators to pursue innovative and smart application technologies as a way for students to gain hands-on skills and to practice those skills in a safe and fun environment. Cybergames are now being used by educational institutions and in private and public industry sectors. One popular game in the cybersecurity space is called Capture the Flag (CTF). This study investigates Facebook's CTF platform as a cost-effective learning and assessment tool for undergraduate and graduate cybersecurity adult students for an online university.

Development and evaluation of simulation based neurosurgery curriculum. Pilot study at the Poznan University of Medical Sciences.

Introduction. Neurosurgical emergencies are complex tasks. The current learning environment limits students’ ability to manage acute neurosurgical emergencies due to legal and safety concerns. Simulation provides an opportunity to participate in the care of neurosurgical emergencies and develop clinical decision making skills. Aim. We aim to determine whether neuroscience simulation curriculum improves student ability to: manage a critically ill patient, recognize neurosurgical emergencies, to assess how stress tolerance affects experience during simulations and effectiveness of students performance. The third objective is to develop a tool for student assessment.Material and Methods. The simulation was performed on SimMan 3G Human Patient Simulator (Laerdal Medical). Scenarios included common neurosurgical emergencies. Students were assessed before and after the course by completing a Likert type questionnaire. Response data was analysed using Cronbach’s reliability for Likert-type response data and Spearman's monotonic correlation. Results. 60 students of fifth and sixth year of medical studies attended the course. 39 students of them replied to the questionnaire. The simulated clinical experience was positive and it improved their knowledge about neurosurgical emergencies. There was an improvement in their confidence. Improvement in individual and team performance was also observed.Conclusions. Neurosurgical simulations improve students` ability to recognize neurosurgical emergencies. The level of stress related to simulation is important factor of the education process and should be reduced to improve students’ development. Our questionnaire is an effective tool for assessment of students experience during clinical simulations.

Development and validation of a new assessment tool for suturing skills in medical students

BackgroundIn recent years, emphasis has been put on that medical student should demonstrate pre-practice/pre-registration core procedural skills to ensure patient safety. Nonetheless, the formal teaching and training of basic suturing skills to medical students have received relatively little attention and there is no standard for what should be tested and how. The aim of this study was to develop and validate, using scientific methods, a tool for assessment of medical students’ suturing skills, measuring both micro- and macrosurgical qualities.MethodsA tool was constructed and content, construct, concurrent validity, and inter-rater, inter-item, inter-test reliability were tested. Three groups were included: students with no training in suturing skills, students who have had training, plastic surgery.ResultsThe results show promising reliability and validity when assessing novice medical students’ suturing skills.ConclusionsFurther studies are needed on implementation of the instrument. Moreover, how the instrument can be used to give formative feedback, evaluate if a required standard is met and for curriculum development needs further investigation.Level of Evidence: Not ratable.

Development and Validation of the Microbiology for Health Sciences Concept Inventory.

Identifying misconceptions in student learning is a valuable practice for evaluating student learning gains and directing educational interventions. By accurately identifying students’ knowledge and misconceptions about microbiology concepts, instructors can design effective classroom practices centered on student understanding. Following the development of ASM’s Curriculum Guidelines in 2012, we developed a concept inventory, the Microbiology for Health Sciences Concept Inventory (MHSCI), that measures learning gains and identifies student misconceptions in health sciences microbiology classrooms. The 23-question MHSCI was delivered to a wide variety of students at multiple institution types. Psychometric analysis identified that the MHSCI instrument is both discriminatory and reliable in measuring student learning gains. The MHSCI results correlated with course outcomes, showing the value of using the instrument alongside course level assessments to measure student learning. The MHSCI is a reliable and efficient way to measure student learning in microbiology and can be used both as a faculty development tool and an effective student assessment tool.

Global Leadership Education: Upping the Game

While scholars recognize that digital technology is a major tool employed by contemporary learners to access knowledge, its full capacity is yet to be utilized. This paper investigates opportunities to increase individual learning engagement and knowledge retention in higher education and corporate environments through integrating educational content with innovative digital technology. Currently, within commonly adopted e-learning platforms, the most utilized online content delivery and student progress assessment tools in leadership education appear to be discussion-based instruction, along with written assignments, and research projects (Jenkins, 2016). To inform leadership educators’ choice in innovative tools for increased effectiveness of instruction, this paper shares an experience of delivering integrated global leadership education pedagogical content (i.e., cross-cultural competency instruction for graduate management class at a U.S.-based public university) within a gamified real-time multiuser learning platform—MyAltis—and discusses implications for leadership education.

Using Patient Feedback as a Feedback and Assessment Tool for Medical Students

OBJECTIVE: To improve the quality of feedback and assessment of medical students through a patient satisfaction questionnaire about student performance. METHODS: To measure student performance from the patient’s perspective, five items were adopted from the validated Patient Satisfaction Questionnaire from the American Board of Internal Medicine. This was used to assess medical student skills in communication and professionalism, which were rated on a five-point Likert scale. All 3rd-year medical students rotating through the OBGYN clerkship participated in these unsupervised patient feedback encounters. Students then were asked to anonymously respond to a two-item questionnaire on usefulness of the patients’ feedback encounter and whether this process improved feedback that faculty provided to students. EVALUATION: This is a survey study; 65% (53 out of 81) of students found the patient feedback encounters helpful and 51% (34 out of 67) thought that the exercise encouraged preceptors to provide useful feedback. DISSEMINATION: The goal is to implement this feedback tool locally in the OBGYN Clerkship as well as other clerkships in our institution. The intention is to publish the results in peer-reviewed journal for wider dissemination. REFLECTIVE CRITIQUE: Providing feedback and assessment is an essential element of medical education. Feedback to medical students regarding their performance is usually provided by preceptors and standardized patients in simulated encounters. Little attention has been given to feedback provided directly by actual patients. Comments provided by patients have more positive impact compared to numerical rating. Empowering patients to participate in feedback improves the quality of feedback to students.

Inter-rater reliability of a reflective rubric to assess pharmacy students’ reflective thinking

IntroductionMany health education programs aspire to foster reflective practices to enhance the reflective thinking of students during their study years and throughout their professional careers. Given the increasing attention paid to incorporating reflective practice activities into health education and evidence attributed to rater variability, it is important for educators to carefully consider the assessment criteria and reliability of assessment tools. A reflective rubric was developed to assess pharmacy students’ levels of reflection from a reflective writing task. This study investigates the inter-rater reliability (IRR) of a rubric as a tool to assess the levels of reflective thinking apparent in students’ written statements. MethodsThe research involved four raters from different disciplines utilizing a rubric to assess a random sample of the same forty-three reflective statements from a cohort of two hundred and sixty-four students. The IRR was measured using the intra-class correlation coefficient (ICC), using a two-way random effects model (ANOVA) with absolute agreement, to determine reliability of the assessment tool among the raters. ResultsResults showed measures between the raters for (i) overall reflective statement scores, and (ii) average scores for stages of reflection with an “almost perfect” agreement, ICC = 0.81 (95% CI 0.61–0.90), (F(42, 126) = 7.83, p < 0.01); and ICC = 0.89 (95% CI 0.83–0.93), (F(42, 840) = 12.49, p < 0.01) respectively. Discussion and conclusionsThe proposed rubric utilized by four raters showed high agreement with each other's scores, and is a tool for academic assessment of pharmacy students’ reflective thinking processes.

Student Achievement in an Introductory Biology Course: Assessing Grade Motivation and Study Log Metacognition

As college educators, our main teaching goals are to increase student learning and improve student retention. This study describes how 54 science majors at a public, minority serv- ing institution reflected on their study time, study activities, and overall grade motivation in a core-curriculum introduc- tory biology course. Since 2005, less than 50% of students enrolled in this course at our university earned a final grade of “C” or better. In the Fall semester of 2009, we adminis- tered a pre and post assessment using the Science Motivation Questionnaire (SMQ) a 30-item Likert-type instrument devel- oped by Glynn and Koballa (2006) to better understand and address the student attrition from the introductory biology course. All 30 items from the SMQ were analyzed, but only&#x0D; 5 items relevant to grade motivation are presented in this paper. We also designed and implemented a weekly study log assessment tool for students to document their study time and study activities, wherein students submitted their study logs on a weekly basis during the course of this research study. Based on the number of study logs submitted and study time by each student, students were classified into ei- ther a high-metacognition or low-metacognition group. For our purposes, we defined metacognition as the awareness of one’s own thinking process (Merriam-Webster, 2012). The high-metacognition group submitted 75% of their study logs and earned a grade in the top 25% of the class. The low- metacognition group submitted only 25% of their study logs and earned a grade in the bottom 25% of the class. Thus both groups formed reasonable expectations for their overall class performance.

Avaliação por portfólio na Pedagogia: desenvolvendo o processo reflexivo nos futuros professores

O uso de portfólio tem sido frequentemente incentivado como um instrumento para estimular o pensamento reflexivo, embora poucos estudos tenham examinado o uso de portfólios como forma de avaliação em cursos superiores de formação de professores. Este trabalho discute a experiência de usar o portfólio como forma de avaliação em dois grupos de estudantes em um curso de Pedagogia, em uma instituição de Ensino Superior em São Paulo, durante o primeiro semestre de 2015, na disciplina Alfabetização e Letramento. O uso de portfólio como forma de avaliação de alunos de curso de Pedagogia foi motivado pela necessidade de desenvolver instrumentos de avaliação reflexiva, que permitam ao estudante uma autoavaliação reflexiva sobre suas habilidades práticas, mais importante ainda no caso de professores alfabetizadores. Os resultados demonstraram que é possível e desejável o desenvolvimento de um instrumento de avaliação reflexiva para estudantes universitários, que precisam desenvolver um processo reflexivo de sua prática profissional e permita aos alunos se tornarem conscientes do seu processo de aprendizagem e uma forma concreta de valorizar seu próprio trabalho.

Effect of Teachers Training Workshop Outcomes on Real Classroom Situations of Undergraduate Medical Students.

Faculty development by conducting regular training, workshops and research related to medical education has been a key feature to upgrade quality of medical education. The aim of this study was to explore responses of the health science teachers, students and peers after the workshop after providing training on student assessment tools and teaching-learning methods. Two teacher-training workshops were conducted to the faculty members of B.P. Koirala Institute of Health Sciences from the departments of basic, clinical and allied sciences in Oct. 2010 and Jan. 2011. Qualitative questionnaire based study was conducted, and the questions were validated before the study by expert peer review process. The effect of the training workshop in real classroom outcomes was assessed incorporating student's feedback, evaluation by peers and the self-evaluation by the teacher trainees. Pre-test and post-test scores of the participating teachers, before and after the workshop were 62.53 and 71.17 respectively. Among the participants 90.3% teachers expressed enhanced in their role as a teacher for medical undergraduates after the workshop. In present study, the faculty members showed accrued interest to participate in teacher's training workshops. The peer evaluation of teacher's performance in their real classroom situations were rated higher than evaluation by the students. Therefore, such training workshops will have a greater impact on the ability of teachers in effective teaching in real classroom situations.

Development of oral health literacy assessment tool for fifth and sixth grade elementary school students

Development of oral health literacy assessment tool for fifth and sixth grade elementary school students

Pilot Testing the Chinese Version of the ETS® Proficiency Profile Critical Thinking Test

AbstractChinese higher education is experiencing rapid development and growth. With tremendous resources invested in higher education, policy makers have requested more direct evidence of student learning. However, assessment tools that can be used to measure college‐level learning are scarce in China. To mitigate this situation, we translated the critical thinking test from the ETS® Proficiency Profile (EPP) into Chinese. EPP has been widely used in the United States to assess general college learning outcomes. We pilot tested the EPP–Chinese test with students from a university in China. Results suggest that (a) the test is unidimensional and therefore is sufficient to report a total score from a practical standpoint; (b) the total score reliability is satisfactory; (c) most items showed moderate correlations with the total score, but the translation of one item needs additional revision; (d) the test is correlated with related constructs (e.g., the Chinese college entrance examination and a national English test); and (e) no item showed differential item functioning or was found to be biased toward any subgroup. In summary, the Chinese version of the critical thinking test showed potential as a suitable assessment tool for Chinese college students.