Effectiveness Of Course Research Articles

Collaborative Development and Assessment of Student Learning Outcomes for LIS Electives

In higher education’s environment of accountability, the development and assessment of student learning outcomes (SLOs) are driven by both external stakeholder requirements for accreditation and internal institutional pressures to demonstrate student learning as the core function of universities and colleges. This paper presents a framework to reduce faculty workload and increase standardization of SLOs for LIS electives across multiple schools. The framework includes a value-added assessment with results that show a significant increase in the overall scores and specifically in areas of focus for the elective. This approach provides a framework for other popular LIS electives to collaborate across schools and standardize SLOs to determine the overall effectiveness of courses and programs without reinventing course objectives, SLOs, and evaluation techniques at each institution.

Developing Moral Maturity

The purpose of this study was to add to the limited body of knowledge on the effect of the media ethics course, specifically to investigate the effect of the course on the growth in moral judgment reasoning of students through a quasi-experimental pre-test, post-test analysis using the Defining Issues Test 2 (DIT-2). Results demonstrated significant advances in such reasoning among students in one section of the media ethics course but not in the other. It is difficult to determine the exact cause that led to significant increases in moral judgment reasoning in one section of a course and not the other, but the effect of different educational interventions seems clear. Although we cannot know which techniques or combination of techniques were successful, it would seem that interactive and experiential activities that require demonstrated application of principles in the classroom are more successful than traditional Socratic lecture and reflection essay methods.

Assessing the Development of Educational Research Literacy: The Effect of Courses on Research Methods in Studies of Educational Science

The ability to purposefully access, reflect, and use evidence from educational research (Educational Research Literacy) is expected of future professionals in educational practice. Based on the presented conceptual framework, a test instrument was developed to assess the different competency aspects: Information Literacy, Statistical Literacy, and Evidence-based Reasoning. This paper addresses the question of whether this test allows monitoring students’ development during courses on research methods in educational science. In winter semester 2013/14, two courses in Early Education and Teacher Training degree programs were accompanied. Both cross-sectional (multidimensional latent regression models) and longitudinal (paired t tests of plausible values) comparisons are reported. The results indicate no significant differences in competencies between the degree programs at start of the semester. For the end of semester, only data of the Early Education course was available. These course participants showed higher proficiency in all aspects of educational research literacy. In conclusion, the results are discussed with regard to the goals of the assessment of competencies in higher education, particularly the use of test feedback for course development, and desiderata for future research (e.g., ability to transfer research into practice) are stated.

Effect of PLT-Based Physics Enhancement Course to the Content Matter Competence of Grade 7 Science Teachers

The adoption of the spiraling curriculum in Philippines’ K-12 program necessitates enhancing the competence of Science teachers in fields that they did not specialize in during their undergraduate and graduate degrees. This study sought to determine the effect of a PLT- based (Progressive Learning Theory) Physics enhancement course to the Physics competence of Grade 7 Science teachers who are not Physics majors. Using a one-tail paired T-test, the Physics Achievement Test (PAT, Cronbach’s α=0.874) pretest and post-test scores of the participants were compared. The study revealed that there was a significant increase in the Physics competence of the participants. Further analyses have shown that the PAT scores (pretest, post-test, and post-pretest) of teachers having different fields of specialization do not vary significantly. Moreover, although the PAT pretest scores of new teachers were shown to be significantly higher than the more experienced teachers, the post-test scores do not vary significantly implying that the course had a leveling effect on the competence of the participants. Two-way ANOVA have shown that the length of teaching experience and the field of specialization do not interact to affect the PAT scores of the participants. Given the effectiveness of the course, DepEd (Department of Education) administrators and teacher education institutions may adopt the course model in this study in providing training to more teachers from various fields of expertise and grade levels. Further studies may also be conducted in their trainings to verify the findings in this current study as well as to evaluate its effect on other teacher qualities. Finally, follow-up studies may investigate the impact of the teacher trainings delivered using the training model to student performance.

Effect of PLT-Based Physics Enhancement Course to the Content Matter Competence of Grade 7 Science Teachers

The adoption of the spiraling curriculum in Philippines’ K-12 program necessitates enhancing the competence of Science teachers in fields that they did not specialize in during their undergraduate and graduate degrees. This study sought to determine the effect of a PLT- based (Progressive Learning Theory) Physics enhancement course to the Physics competence of Grade 7 Science teachers who are not Physics majors. Using a one-tail paired T-test, the Physics Achievement Test (PAT, Cronbach’s α=0.874) pretest and post-test scores of the participants were compared. The study revealed that there was a significant increase in the Physics competence of the participants. Further analyses have shown that the PAT scores (pretest, post-test, and post-pretest) of teachers having different fields of specialization do not vary significantly. Moreover, although the PAT pretest scores of new teachers were shown to be significantly higher than the more experienced teachers, the post-test scores do not vary significantly implying that the course had a leveling effect on the competence of the participants. Two-way ANOVA have shown that the length of teaching experience and the field of specialization do not interact to affect the PAT scores of the participants. Given the effectiveness of the course, DepEd (Department of Education) administrators and teacher education institutions may adopt the course model in this study in providing training to more teachers from various fields of expertise and grade levels. Further studies may also be conducted in their trainings to verify the findings in this current study as well as to evaluate its effect on other teacher qualities. Finally, follow-up studies may investigate the impact of the teacher trainings delivered using the training model to student performance.

Orthopaedic Trainees Retain Knowledge After a Partner Abuse Course: An Education Study.

Intimate partner violence (IPV) is a serious global issue that results in a large number of injuries and deaths among women. Educating clinicians about IPV can help providers identify, prevent, and treat victims, and, ultimately, improve care for victims of abuse. We sought to determine the effect of a half-day educational course on IPV for orthopaedic surgical trainees on knowledge and attitudes. We asked (1) whether a half-day educational course on IPV can improve orthopaedic surgical trainees' knowledge and (2) attitudes regarding IPV; and (3) whether a course on IPV can be accepted and viewed as valuable by trainees? Using published research on IPV in patients with musculoskeletal injuries, we developed a half-day educational course. The curriculum included lectures and discussion regarding the basics of IPV, the current state of IPV research, what to do when a patient is a victim or perpetrator, and the orthopaedic surgeon's role in recognizing, preventing, and assisting with IPV. All 33 course participants (30 men and three women), all orthopaedic surgical trainees, completed a questionnaire that included general true or false or agree or disagree statements regarding their knowledge, attitudes, and practices of IPV in the musculoskeletal setting; the questionnaire also included a knowledge test of 25 true or false statements. The questionnaire was administered immediately before, immediately after, and 3 months after the course; 76% (25 of 33) took the test immediately after the course and 82% (27 of 33) completed the test at 3 months. Participant knowledge scores were compared across the three different times to determine the effect of the course. Participants increased their knowledge after the course, and the increased knowledge was retained at retesting at 3 months; the mean percentage of correct answers before the course was 57%, which increased to 73% after the course, and was 68% 3 months later (F = 9.505; p = 0.001). Before the course, most of the course participants (30 of 32; 94%) agreed that IPV is an important issue; agreement increased to 100% immediately after the course. The largest change in attitude was in response to the statement: "I am skeptical that the health care system has the resources to screen for IPV." Before the course, 53% (17 of 32) of trainees endorsed this statement, but the percent decreased to 36% (nine of 25) after the course and remained low at 33% (nine of 27), at the 3-month test. Our findings show that a short course on IPV in patients with musculoskeletal injuries led to an improvement and retention of knowledge 3 months after the course. Based on our findings, we recommend that IPV education be integrated in training programs for orthopaedic surgeons. Future projects should focus on developing and implementing a sustainable education program that can affect practice for healthcare professionals and trainees in multiple clinical settings.

A Multiyear Curriculum-Integrated Information Literacy Program Increases the Confidence and Research Skills of Nursing Students, Although Not as Much as Expected

A Review of:
 Farrell, A., Goosney, J., & Hutchens, K. (2013). Evaluation of the effectiveness of course integrated library instruction in an undergraduate nursing program. Journal of the Canadian Health Libraries Association, 34(3), 164-175. doi: 10.5596/c13-061
 
 
 Objective – To evaluate nursing students’ information literacy (IL) confidence and competence after a four year implementation of a curriculum-integrated information literacy program.
 
 Design – Web-based and paper-based information literacy questionnaire.
 
 Setting – Two Schools of Nursing in Newfoundland, Canada. 
 
 Subjects – 422 undergraduate nursing students.
 
 Methods – A 20-item questionnaire was distributed to nursing students at two School of Nursing programs. Questions were designed to test one or more of the nine learning outcomes set forth for the IL program. At one School of Nursing, web-based questionnaires were sent via email, while paper-based questionnaires were distributed in class at the second School of Nursing.
 
 Main Results – Because response rates were low at one of the nursing schools, these questionnaire responses were not examined, nor were they included in the data analysis. The 422 completed questionnaires were divided into groups based on the number of IL sessions attended. The class of 2011 attended two out of three IL sessions, the class of 2012 attended all three IL sessions, and the classes of 2014 and 2015 did not attend any IL classes.
 
 There were 109 responses from class of 2011 students, 98 from class of 2012 students, and 215 from the classes of 2014 and 2015 (98 and 117, respectively). Results were reported according to the two main goals of the study – evaluating confidence levels and IL skills. Regarding confidence level, the class of 2012 felt more prepared than the other classes to begin evidence-informed practice (EIP). When asked about their confidence in the ability to perform research tasks, confidence levels rose between first year students and the class of 2011, but decreased between the class of 2011 and class of 2012.
 
 In regard to information literacy skills, when asked to identify appropriate tools for locating scholarly research articles, only 2.8% of the first year students correctly identified two resources. Within the class of 2011, 33.9% could correctly identify tools, while 57.1% of the class of 2012 correctly identified appropriate tools. For a series of questions that asked if students could make appropriate decisions about the use of scholarly and popular literature, there was overall improvement between the first year students and fourth year students. 
 
 Several questions asked students about their understanding of different study types; students in all classes had difficulty with these questions. Additionally, there were questions related to identifying components of PICO (problem/population, intervention, comparison, outcome), and none of the students in the first year classes or in the 2011 class were able to correctly identify all four parts although 11.2 % of the class of 2012 identified all components correctly. Students were asked to create an answerable question using the PICO format, and 11.9% of the class of 2011, along with 11.2% from the class of 2012 and 3.7% from the first year class were able to do so.
 
 When asked to identify criteria for evaluating websites, 28.4% of students in the class of 2011 and 30.6% of the class of 2012 were able to list three correct criteria. Only 8.3% of the first year students answered this question correctly. Finally, students were given a research question and asked to select the best search statement; those who answered correctly included 28.4% of the class of 2011, 27.6% from the class of 2012, and 20.8% of first year students.
 
 Conclusions – The authors concluded that overall, IL skills improved from first to fourth year, although not as much as the anecdotal information from nursing faculty would indicate. This ongoing, four year study has prompted discussion on ways to improve the current IL component of the undergraduate nursing degree curriculum, and changes will be implemented in a future version of this curriculum.

The Role of In-Service Training Courses for Librarians' Efficiencies at the Institute for the Intellectual Development of Children and Young Adults (IIDCYA)

Abstract. Objective: Training the librarians may be effective in their gaining experience and enhancing their performance. And by having a strategic development plan, it can also guarantee the survival and perfection of the organization. The present study aims to investigate the role of in-service training courses for librarians in their efficiency, in the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). Method: The current study is an applied research which was conducted using the descriptive survey research method. A self-made questionnaire was used for gathering information. This questionnaire included 32 questions on Likert Spectrum Scale. The reliability of the questionnaire was confirmed based on Cronbach's alpha coefficient (0.981), in preliminary studies. The statistical population consists of 117 librarians of the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). The analysis of data was conducted using Lisrel 8.5 and SPSS 22 depending on the kind of variables. Findings: The findings indicated that the mean of variables including coordination of courses with training needs, the magnitude of meeting the needs by the courses, being applicative of the course content and the effectiveness of courses on librarians' performances were 3.23, 3.42, 3.14 and 2.82, respectively, which were evaluated to be at moderate levels. The mean of other variables like the increase in skills acquired after the courses, the efficiency of courses for librarians, using electronic technology by librarians were 3.37, 3.26 and 3.67, respectively, which were evaluated to be at high levels. Also, there is a significant relationship between in-service training courses and the increase in efficiency according to the Pearson's correlation coefficient ( ρ = 0.770). Furthermore, there is a significant relationship between evaluating the in-service training courses and the increase in the knowledge and information of librarians ( ρ = 0.849). So the hypotheses of the study are confirmed. Conclusion: The in-service training courses are effective on the efficiency of the librarians of the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). Meanwhile by increasing the applied content of the training courses and more exact evaluation of effectiveness, their efficiency increases more. And at the end of this thesis, some recommendations are presented based on findings, in order to enhance the performance at the present and in the future.

Service learning in business education: What perceptions and expectations do undergraduate and graduate students have of service-learning courses?

Service Learning is a modern teaching-learning concept, which combines curricular content with supporting charitable organizations. The effects of courses using this concept on the students’ competences development has already been described in empirical studies. Within these studies the aspects regarding the instructional design were less emphasized unlike in this paper in which they are focused on. In addition to a conceptual foundation of service-learning and a literature review on the impact of service-learning, two didactic prototypes will be presented in the form of empirical case studies. The focus of this research is to describe the attitudes and perceptions of undergraduate and graduate students attending service-learning courses and which conclusions can be drawn on the instructional design. In this article, the concept of service-learning is understood as a course- and credit-based learning and teaching method (in contrast to community-service), which require a specific instructional design to evolve purposeful and curricular learning connections between the course and service projects outside the course.

The Role of In-Service Training Courses for Librarians in Their Efficiency, in the Institute for the Intellectual Development of Children and Young Adults (IIDCYA)

Abstract. Objective: Training the librarians may be effective in their gaining experience and enhancing their performance. And by having a strategic development plan, it can also guarantee the survival and perfection of the organization. The present study aims to investigate the role of in-service training courses for librarians in their efficiency, in the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). Method: The current study is an applied research which was conducted using the descriptive survey research method.A self-made questionnaire was used for gathering information. This questionnaire included 32 questions on Likert Spectrum Scale. The reliability of the questionnaire was confirmed based on Cronbach's alpha coefficient (0.981), in preliminary studies. The statistical population consists of 117 librarians of the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). The analysis of data was conducted using Lisrel 8.5 and SPSS 22 depending on the kind of variables. Findings: The findings indicated that the mean of variables including coordination of courses with training needs, the magnitude of meeting the needs by the courses, being applicative of the course content and the effectiveness of courses on librarians' performances were 23.3, 42.3, 14.3 and 82.2, respectively, which were evaluated to be at moderate levels. The mean of other variables like the increase in skills after the courses, the efficiency of courses for librarians, using electronic technology by librarians were 37.3, 26.3 and 67.3, respectively, which were evaluated to be at high levels. Also, there is a significant relationship between in-service training courses and the increase in efficiency according to the Pearson's correlation coefficient ( ρ = 0.770). Furthermore, there is a significant relationship between evaluating the in-service training courses and the increase in the knowledge and information of librarians ( ρ = 0.849). So the hypotheses of the study are confirmed. Conclusion: The in-service training courses are effective on the efficiency of the librarians of the Institute for the Intellectual Development of Children and Young Adults (IIDCYA). Meanwhile by increasing the applied content of the training courses and more exact evaluation of effectiveness, their efficiency increases more. And at the end of this thesis, some recommendations are presented based on findings, in order to enhance the performance at the present and in the future.

Effectiveness of a Business Communication Course

This study aims to show the effectiveness of a business communication course offered in a business school in India. All students who enrolled for the Master of Business Administration program in the school were considered for the study. The study adapted a pretest and posttest approach to find the effectiveness of the course. It also describes the curriculum offered and methodology followed in the school. The results show that the course was effective in improving the oral communication skills of the students. The study also discusses the implications for business schools, faculty, and students.

Examining the Impact of In-Service Training on Art’s Staff Efficiency based on Kirk-Patrick Model

Evaluation of the course effectiveness is one of the most important priorities & responsibilities of training management at all levels and different expertise courses in each organization. Human resource development cannot be made only by holding expertise courses, but what is leading to skilled human resources development is effectiveness of courses. The main purpose of this research is evaluation of in-service training efficiency on University of Art’s staff based on Kirk-Patrick Model which have been studied in seven components: staff’s knowledge, skills and abilities, order, the ability to use learning tool, productivity and efficiency principles and rules of the workplace and ultimately facilitate the tasks. This research project is descriptive and it is a survey. The statistic community is all staffs of the University of Art (400 seats) and 195 of them have been selected as sufficient sample size using stage –random sample method. The research tools is a questionnaire of 32 questions, grouped into reaction, learning and behavior (behavioral change) which is measured on a Multiple-choice Likert and the reliability coefficient is 95%. In order to analyze the data from the questionnaire, one sample t-test model were used. Finally, data analysis has shown that the impact of the in-service training on staffs’ efficiency based on Kirk-Patrick model below average and hasn’t high effect.

Verbal episodic memory along the course of schizophrenia and bipolar disorder: A new perspective

Impairment on episodic memory (EM) has been strongly correlated with psychiatric disorders, including schizophrenia (SZ) and bipolar disorder (BD). Morevover, the effects of course and progression of the illness on cognitive functioning have not been well established. The aim of the present study is to assess performance of episodic memory in BD and SZ according to their clinical stages.Subjects who met DSM-IV criteria for bipolar disorder (n=43) and schizophrenia (31), on euthymia or clinical remission, were recruited from the outpatients facilities at Hospital de Clínicas de Porto Alegre (Brazil). They were classified into two clinical stages (early or late for BD, and recent onset or chronic for SZ) and compared to 54 healthy controls. Episodic memory performance was assessed by means the Hopkins Verbal Learning Test-Revised (HVLT-R) that measures verbal learning and episodic memory in both disorders. Our results showed that patients in early stage of BD (EBD) performed better performance on the total immediate free recall (p<0.0001, F=12.060) as well as in delayed free recall (p<0.0001, F=13.914) compared to late stage (LBD) and SZ groups. In the ability to retain words learned, LBD and chronic (CSZ) were more impaired than other groups. Furthermore, the variation of learning (i.e, learning effects) along the 3 trials of immediate free recall was similar between groups.In conclusion, we found a cognitive decline alongside with the progression of BD whereas such impairment was evident in the early of SZ. Despite this, both groups (BD and SZ) seem to maintain the ability to learn. It emphasizes the relevance of studying new therapeutic strategies, in particular, cognitive rehabilitation/remediation techniques as promissory treatment for psychiatric patients, even in those with moderate disabilities.

Assessing sustainability knowledge of a student population

Purpose– The purpose of this article is to present a tool for assessing the sustainability knowledge of an undergraduate population.Design/methodology/approach– Multiple-choice questions were developed through soliciting expert input, focus groups, pilot testing, distribution via a large-scale online survey and analysis using item response theory.Findings– The final assessment consists of 16 questions from the environmental, economic and social domains, covering foundational concepts within the topic of sustainability.Research limitations/implications– This assessment represents an initial effort to quantify knowledge of the broad and abstract concept of sustainability. The authors plan to continue refining these questions to better differentiate between students with higher levels of knowledge and to replace those with answers that may change over time.Practical implications– With knowledge of sustainability concepts becoming increasingly included in institution-wide learning objectives, there is a growing demand for a way to measure progress in this area. Our assessment tool can easily be used (via a campus-wide survey or distributed at the classroom level) by institutions to gauge current levels of knowledge and track changes over time, as well as assess the effectiveness of courses and curricula at meeting sustainability knowledge goals.Originality/value– This assessment of sustainability knowledge is the first of its kind to include all three separate domains of sustainability, and we expect it to be useful across a variety of college and university contexts.

Simulation-based learning in management education

Purpose– Proponents have argued that simulation-based learning (SBL) offers capabilities that respond to persisting critiques of management education. This research intended to provide additional empirical evidence for the instructional effectiveness of SBL. This paper aims to discuss these issues.Design/methodology/approach– This research adopted a quasi-experimental, multiple time series design to examine the instructional effectiveness of courses that incorporated computer simulations in a Master of Management program at a business school in Thailand. It compared student perceptions of three SBL courses with courses that used a variety of other instructional approaches over a period of seven years.Findings– Results revealed that students rated the SBL courses significantly higher on overall perceived instructional effectiveness, as manifested by action-directed learning, student engagement, quality of assessment and feedback, and instructor effectiveness.Research limitations/implications– The consistency of significant results for a large number of course sections over a substantial period of time suggests that the SBL courses created a more active, productive environment in which to learn management theory and practice.Practical implications– The results support assertions that simulations offer potential for enhancing the quality of university-based management education.Originality/value– First, the research provides empirical insights into the implementation of SBL in management education; second, many instructors remain skeptical as to whether active learning methods imported from western contexts are suitable for Asian learners. The study addresses this issue in the light of data that describe one institution's sustained attempt to employ computer simulations in its graduate management education program.

The effect of movement intervention for women attending courses in weight reduction

Background: Accelerated pace of life and consumerism leads to an increase in obesity. The key element in programs to reduce overweight and obesity is regular physical activity (PA) by means of a healthy lifestyle. Walking, which can be simply assessed by pedometers is appropriate PA that is accessible to everyone, regardless of age and gender. Objective: The aim of our study was to analyse the effect of courses in weight reduction to the selected body composition (BC) parameters in the obese and overweight women of various age and PA level. Methods: 124 women with existing sendenary lifestyles participated in the study, they were differentiated by age (&lt;&#8239;40 years: aged 31.3&#8239;&#177;&#8239;5.4 years, body weight of 88.2&#8239;&#177;&#8239;18.5&#8239;kg, body height of 166.5&#8239;&#177;&#8239;9.5&#8239;cm and &#8805;&#8239;40 years: aged 51.6&#8239;&#177; 7.5 years, body weight of 88.9&#8239;&#177;&#8239;13.3&#8239;kg, body height of 163.8&#8239;&#177;&#8239;8.2&#8239;cm). We divided the monitored sample into sub-groups according to the level of their PA. To determine the average daily number of footsteps during the PA the Yamax pedometer was used. InBody 720 device using the Direct Multi-frequency Bioelectrical Impedance Analysis Method (DSM-BIA Method) was used to measure and analyse the BC. Results: Based on monitoring of PA and BC health risk indicators the effect of the exercise program, which included cognitive behavioural therapy, showed a decline in body mass index (BMI) and body fat expressed in percentages (PBF) and in the decrease of the visceral fat area (VFA). The amount of fat-free mass (FFM) remained the same. Preceding the therapy the BMI average was within the obesity group (BMI&#8239;&gt;&#8239;30&#8239;kg/m2). Following the three month therapy we observed in women with a higher level of PA a shift to the overweight group. Therapy contributed to a reduction in PBF from 2.4% to 3.9% even though the women's groups were still classified as obese (&gt;&#8239;35%). During the repeated measurements significant changes (p&#8239;&lt;&#8239;.05) occurred only in the VFA in relation to different ages. Conclusions: Research studies have shown positive relationships between the increase in PA and changes in health risk indicators. Observance of the recommended number of 10,000 footsteps per day primarily prevents the increase in body fat mass (BFM) and maintains FFM. Taking up a weight reduction program is essential when commencing the weight loss process in the course of lifestyle change.

A Short Preparatory Calculus Course: Is it Effective?

In the UTSC Math & Statistics Learning Centre, we designed a 2-week summer preparatory course to help incoming students review background materials that are essential for the success in their first-year calculus courses. This paper discusses the design, implementation, and effectiveness of the course. We examined the effectiveness of the preparatory course through assessing students’ mathematics skills and their confidence of their mathematics skills before and after the course. We also examined the students’ perception and satisfaction with the course. The results show that students demonstrate a higher level of mastery of the fundamental mathematics skills after completing the course; they also tend to be more confident about their mathematics skills. Additionally, students rated the course as very helpful, providing them with a more positive experience than their past experience with mathematics. The course also helped the students getting used to the university learning environment.

Mindfulness online: an evaluation of the feasibility of a web-based mindfulness course for stress, anxiety and depression

ObjectivesFace-to-face mindfulness interventions have been shown to significantly decrease perceived stress, anxiety and depression and research is beginning to show similar benefits for such courses delivered via the internet. We...

PTU-007 Evaluating Endoscopy Trainers; how Reliable Are Peer Evaluators?

Introduction The training of future endoscopists is important to ensure the ongoing provision of a safe endoscopy service within the UK; however endoscopy training is of a variable standard. Peer evaluation can be used to improve teaching but this does not currently routinely occur in local endoscopy units. We therefore wanted to assess the reliability of peer evaluations using an evaluation tool currently being developed to gain both trainee and peer evaluations. Methods The DOTS tool has been developed using the list of attributes described by Wells1. In order to gain an assessment of reliability the tool was trialled on JAG approved Training the Trainer courses. Courses from November to March 2012 were contacted and asked to participate. Each course attendant was then sent an information letter and consent form. On day two of the course participants were asked to complete a copy of the DOTS for each of the training episodes they observed. Data was analysed using SPSS 14; mean score and Cronbach alpha were calculated. Reliability was calculated using Generalisability theory; an initial analysis was performed using only trainers, peers and trainer: peer interaction as facets. A further analysis was then conducted including all possible sources of variance. Results Eight of the ten courses contacted agreed to participate; all course participants consented to the study. 189 evaluations were collected; these were completed by 58 different peers; 45 trainers were evaluated receiving from one to ten evaluations each. Mean total evaluation score was 63.3 (out of 85); standard deviation 8.6. The tool showed a high level of internal consistency with a Cronbach alpha of 0.895. In the initial analysis 44% of the variance of scores was explained by the difference in trainers’ ability to teach, 35% due to peer variance and 21% by peer:trainer interaction. The G-coefficient for one rater was 0.44 and three raters were required for a G-coefficient of 0.7. When the analysis was repeated the effect of course accounted for 20% of the variance in scores. Reliability was much lower with a G coefficient of 0.28 for one rater. Conclusion The DOTS tool showed a high level of internal consistency. On initial analysis only three peer reviewers were required to gain acceptable levels of reliability. However on reanalysis the effect of course was responsible for the 20% of the variance and if results were generalised across course then the tool showed poor reliability. The effect of course was unexpected and needs to be investigated further; the tool also needs to be trialled within local units. Disclosure of Interest None Declared Reference Wells, C., The characteristics of an excellent endoscopy trainer. Frontline Gastroenterology, 2010. 1 : p. 13–18.

Economic Understanding in US High School Courses

The effects of courses on student achievement are studied using 2006 data from the National Assessment of Educational Progress (NAEP) in economics. A regression analysis showed expected and significant achievement differences by course, with the highest scores in advanced economics, followed by general economics. Courses in business and personal finance were not substitutes for advanced or general economics courses. A probit analysis showed that students taking economics courses relative to personal finance courses are significantly more likely to think their courses helped them understand the US economy, the international economy, and current events--but not how to manage personal finances.