National Exit Research Articles

Improving the quality of clinical training in the workplace: implementing formative assessment visits

Family physicians have a key role to play in strengthening district health services in South Africa. There are a number of barriers to the supply of these specialists in family medicine, one of which is the quality of workplace-based training and low pass rate in the national exit examination. The South African Academy of Family Physicians in collaboration with the Royal College of General Practitioners has adopted a short course to train clinical trainers and a process of formative assessment visits (FAVs) for clinical trainers in the workplace. Training programmes have struggled to implement the FAVs and this article reports on the experience at Stellenbosch University and the issues identified. Clinical trainers who participated in FAVs mostly set developmental goals for themselves that focused on improving the learning environment and consolidating personal skills in training and assessment. The FAVs were beneficial for the family physician trainers, their managers and the academic family physicians at the university. The tools and process for conducting the FAVs may be of value to other programmes.

Career Paths of Public Health Medicine Specialists in South Africa.

Public health (PH) skills are core to building responsive and appropriate health systems, and PH personnel including medical specialists are embedded in many countries' health systems. In South Africa, the medical specialty in PH, Public Health Medicine (PHM), has existed for over 40 years. Four years of accredited training plus success in a single national exit exam allows specialist registration with the Health Professions Council of South Africa (HPCSA). However, there are few posts designated specifically for PHM specialists in SA's health system. In view of uncertain roles, this research was designed to determine specialists' career paths, their work, job satisfaction, and perspectives on the future of the specialty. We combined three databases to generate the study population and invited all specialists to participate in an online or hard-copy survey. We found that in 2010, PHM was a small specialty of less 200 physicians. Of the 151 contactable, eligible physicians, 55.6% completed the questionnaire. Participants represented an aging group (median age = 49) of specialists and recent graduates were increasingly women. They largely worked in academic institutions (as managers, teachers, and researchers) and in the public sector health system; were motivated by a sense of social justice and their training was formative, exposing them to work settings which they later entered; were largely highly satisfied at work, but many worked in non-specialist positions. Indeed, one fifth had not registered with the HPCSA as specialists. They were concerned about the specialty's poor visibility and identity, but did not see other PH professionals as a threat. They believed that the specialty should refine its competencies, demonstrate its value and advocate for service positions at all levels of the public sector health service. PHM has a contribution to make—reorienting services to protect communities, preventing ill health, analyzing disease burdens locally, identifying innovations in a resource-constrained health service, largely preoccupied with curative care services.

Women in STEM: does college boost their performance?

This article uses an added value measure to assess the gender-specific impact of attending a STEM (Science, Technology, Engineering and Math) program. Using the results of the mandatory Colombian national exit exams, we compare the math and reading scores at the end of high school and at the end of college by gender. A difference-in-differences technique combined with propensity score matching is used to address selection bias. We find that the gender-related achievement gap in math and reading scores increases after college affecting women. The gap is larger for those individuals studying a STEM major in comparison with a non-STEM major. Moreover, we find that the gender gap is higher in public and accredited universities.

Implementing and evaluating an e-portfolio for postgraduate family medicine training in the Western Cape, South Africa

BackgroundIn South Africa it is compulsory to submit a satisfactory portfolio of learning to gain entrance to the national exit examination of the College of Family Physicians and to qualify as a family physician. A paper-based portfolio has been implemented thus far and the need for an electronic portfolio (e-portfolio) was identified. The aim of the study was to describe and evaluate the implementation of an e-portfolio for the training of family medicine registrars in the Western Cape province of South Africa.MethodsMixed methods were used. A quasi-experimental study evaluated paper- and e-portfolios from the same 28 registrars in 2015 compared to 2016. Semi-structured interviews were conducted with 11 registrars or supervisors to explore their experiences of using the e-portfolio. Quantitative data was analysed in the Statistical Package for Social Sciences and qualitative data in Atlas.ti.ResultsMost respondents found the e-portfolio easier to use and more accessible. It made progress easier to monitor and provided sufficient evidence of learning. Feedback was made easier and more explicit. There were concerns regarding face-to-face feedback being negatively affected. It was suggested to have a feedback template to further improve feedback. Several aspects were significantly better in the e-portfolio such as feedback on the registrar’s general behaviour, alignment with learning outcomes, less feedback based on hearsay and acknowledgement of the feedback by the registrar. Although not statistically significant, there was an increase in the usage of the e-portfolio, compared to the paper portfolio.ConclusionIn general, the e-portfolio is an improvement on the paper-based portfolio. It is easier to access, more user-friendly and less cumbersome. It makes feedback and monitoring of progress and development of registrars easier and more visible and provides sufficient evidence of learning. Its implementation throughout South Africa is recommended.

Cambodia\u2019s health professionals and the ASEAN Mutual Recognition Arrangements: registration, education and mobility

BackgroundFrom 2006, the Association of South East Asian Nations (ASEAN) has been developing Mutual Recognition Arrangements (MRAs) across key professions, including medicine, dentistry and nursing, that would facilitate the development of an ASEAN Economic Community, with shared regional standards and easier mobility of the workforce. This paper examines the interface between those agreements and the registration, professional education and mobility of health personnel in Cambodia.MethodsThis qualitative health policy analysis combined documentary and policy review with key informant interviews with 16 representatives of agencies relevant to the development and implementation of the MRAs in health. Thematic analysis identified three themes: registration, education and mobility.ResultsCambodia is an active participant in the ASEAN MRA processes for doctors, dentists and nurses reporting progress annually. Education of health professionals has been increasingly formalised in the past 25 years, with nursing moving towards a 4-year bachelor degree. The private university sector has substantially increased, with English increasingly used as a language of instruction. Recent legislation provides for enforcement through fines and/or imprisonment to ensure all practising health professionals hold initial registration as a health professional and a renewable licence to practise as a health practitioner. Continuing Professional Development is a mandatory requirement for licence renewal. This is consistent with the MRA guidelines, though the capacity for enforcement appears limited. The Medical Council of Cambodia (MCC), and more recently, the Dental and Nursing Councils, have introduced continuing professional development initiatives, using them strategically as a positive reinforcer of registration. Midwifery education and registration in Cambodia does not conform with ASEAN guidelines. In education, course durations in medicine and dentistry are longer than regional counterparts, though anxiety around maintaining clinical standards has resulted in the introduction of a National Exit Examination and reluctance to abbreviate courses. The introduction of reforms appears to reference regional standards, though parity is still some way off. Mobility at present is infrequent and more likely to result from informal mechanisms than through the MRA mechanisms.ConclusionThe Royal Government of Cambodia is committed to the ASEAN MRA process. Developments in registration appear to use regional standards as benchmarks, as do reforms in the education of health professionals, though domestic factors appear to more directly impact on developments. Informal mechanisms facilitate the limited mobility currently occurring, with little formal application of the MRA provisions evident at this point.

National eligibility and entrance test and national exit test

National eligibility and entrance test and national exit test

The Newest Southerners: Generational Differences in Electoral Behavior in the Contemporary South

Young voters contributed disproportionately to Barack Obama’s presidential victory in 2012. In fact, if the electorate had been limited to those over 30 years old, Mitt Romney might be in the White House today. Obama captured 60 percent of the vote of those under 30, compared to 49 percent of those over 30, according to the national exit polls (Schier and Box- Steffensmeier 2013, 86). A similar pattern characterized the 2008 presidential election: Obama won 66 percent of the vote among those aged 29 or less, but under one-half of voters older than 45 (Pomper 2010, 53). The tendency for younger voters to be disproportionately Democratic emerged in the 2004 presidential election. Prior to that, Democratic presidential candidates did not consistently do better among younger voters. In 2000, for example, Al Gore did as well among older voters as he did among younger voters, and in 1992, Bill Clinton did his best among older voters, as did Walter Mondale in 1984 and Michael Dukakis in 1988 (Pomper 2001, 138; Pomper 1989, 133).

Reaching national consensus on the core clinical skill outcomes for family medicine postgraduate training programmes in South Africa.

BackgroundFamily physicians play a significant role in the district health system and need to be equipped with a broad range of clinical skills in order to meet the needs and expectations of the communities they serve. A previous study in 2007 reached national consensus on the clinical skills that should be taught in postgraduate family medicine training prior to the introduction of the new speciality. Since then, family physicians have been trained, employed and have gained experience of working in the district health services. The national Education and Training Committee of the South African Academy of Family Physicians, therefore, requested a review of the national consensus on clinical skills for family medicine training.MethodsA Delphi technique was used to reach national consensus in a panel of 17 experts: family physicians responsible for training, experienced family physicians in practice and managers responsible for employing family physicians.ResultsConsensus was reached on 242 skills from which the panel decided on 211 core skills, 28 elective skills and 3 skills to be deleted from the previous list. The panel was unable to reach consensus on 11 skills.ConclusionThe findings will guide training programmes on the skills to be addressed and ensure consistency across training programmes nationally. The consensus will also guide formative assessment as documented in the national portfolio of learning and summative assessment in the national exit examination. The consensus will be of interest to other countries in the region where training programmes in family medicine are developing.

Effects of scaffolded intensive reading on students’ reading comprehension performance

This research sought to explore the connection between a small-group intensive reading comprehension project and students’ performance in two sample English national exit exams (ENEEs) developed by the Ministry of Public Education, Costa Rica. The data were gathered from an intervention plan that combined the theoretical principles of schema theory, scaffolded reading comprehension, and intensive reading. The study adopts an action-research approach and uses a mixed design that combines quantitative and qualitative data in the analysis and interpretation of results. Participants included twelve students from a public high school in the Western Area of Costa Rica who needed special preparation for the ENEE, which narrows the research scope down to this population only. The data collection techniques included two sample ENEEs, field notes, and research artifacts. Findings reveal positive effects of scaffolded reading comprehension on student ENEE performance, but also it is warning that the generalizations to larger populations is not possible. The study sets implications at theoretical and practical level, and it calls for further investigation as a way to tackle the limitations identified.

Do personality traits assessed on medical school admission predict exit performance? A UK-wide longitudinal cohort study

Traditional methods of assessing personality traits in medical school selection have been heavily criticised. To address this at the point of selection, “non-cognitive” tests were included in the UK Clinical Aptitude Test, the most widely-used aptitude test in UK medical education (UKCAT: http://www.ukcat.ac.uk/). We examined the predictive validity of these non-cognitive traits with performance during and on exit from medical school. We sampled all students graduating in 2013 from the 30 UKCAT consortium medical schools. Analysis included: candidate demographics, UKCAT non-cognitive scores, medical school performance data—the Educational Performance Measure (EPM) and national exit situational judgement test (SJT) outcomes. We examined the relationships between these variables and SJT and EPM scores. Multilevel modelling was used to assess the relationships adjusting for confounders. The 3343 students who had taken the UKCAT non-cognitive tests and had both EPM and SJT data were entered into the analysis. There were four types of non-cognitive test: (1) libertariancommunitarian, (2) NACE—narcissism, aloofness, confidence and empathy, (3) MEARS—self-esteem, optimism, control, self-discipline, emotional-nondefensiveness (END) and faking, (4) an abridged version of 1 and 2 combined. Multilevel regression showed that, after correcting for demographic factors, END predicted SJT and EPM decile. Aloofness and empathy in NACE were predictive of SJT score. This is the first large-scale study examining the relationship between performance on non-cognitive selection tests and medical school exit assessments. The predictive validity of these tests was limited, and the relationships revealed do not fit neatly with theoretical expectations. This study does not support their use in selection.

Does the UKCAT predict performance on exit from medical school? A national cohort study.

ObjectivesMost UK medical programmes use aptitude tests during student selection, but large-scale studies of predictive validity are rare. This study assesses the UK Clinical Aptitude Test (UKCAT: http://www.ukcat.ac.uk), and 4 of its subscales, along with individual and contextual socioeconomic background factors, as predictors of performance during, and on exit from, medical school.MethodsThis was an observational study of 6294 medical students from 30 UK medical programmes who took the UKCAT from 2006 to 2008, for whom selection data from the UK Foundation Programme (UKFPO), the next stage of UK medical education training, were available in 2013. We included candidate demographics, UKCAT (cognitive domains; total scores), UKFPO Educational Performance Measure (EPM) and national exit situational judgement test (SJT). Multilevel modelling was used to assess relationships between variables, adjusting for confounders.ResultsThe UKCAT—as a total score and in terms of the subtest scores—has significant predictive validity for performance on the UKFPO EPM and SJT. UKFPO performance was also affected positively by female gender, maturity, white ethnicity and coming from a higher social class area at the time of application to medical school An inverse pattern was seen for a contextual measure of school, with those attending fee-paying schools performing significantly more weakly on the EPM decile, the EPM total and the total UKFPO score, but not the SJT, than those attending other types of school.ConclusionsThis large-scale study, the first to link 2 national databases—UKCAT and UKFPO, has shown that UKCAT is a predictor of medical school outcome. The data provide modest supportive evidence for the UKCAT's role in student selection. The conflicting relationships of socioeconomic contextual measures (area and school) with outcome adds to wider debates about the limitations of these measures, and indicates the need for further research.

THE EFFECTIVENESS OF PUBLIC VS PRIVATE SCHOOLS IN INDONESIA

This paper examines the effectiveness of public versus private schools using the national exit exam of Junior Secondary Schools in Indonesia. Using Indonesian Family Live Survey (IFLS) (1997), IFLS (2000) and IFLS (2007), this study find evidence that graduates of the public school have higher scores on the national exit exam than those of the private school controlling a wide variety of students' characteristics and family background. This finding is robust to several estimates, including Ordinary Least Square (OLS), Fixed Effects (FE), and Instrumental Variable (IV). Using the score of the national exit exam (Ebtanas) as a standard score, public school students are higher 0.30, 0.247 and 0.572 of standard deviations for OLS, FE and IV estimates respectively.Â

Patient satisfaction and perceived quality of care: evidence from a cross-sectional national exit survey of HIV and non-HIV service users in Zambia

ObjectiveTo examine the associations between perceived quality of care and patient satisfaction among HIV and non-HIV patients in Zambia.SettingPatient exit survey conducted at 104 primary, secondary and tertiary health clinics...

The Role of Information on Studentss Career Choice and School Effort: Experimental Evidence from Bogott, Colombia

This paper studies how providing information on the returns and financing of college affects higher education decisions. We conducted a randomized controlled trial in Bogota, Colombia, on a representative sample of 120 urban public high schools, 60 of which received a 35-minute informational talk delivered by local college graduates. We find no effects of the intervention on post-secondary enrollment rates. However, students in treated schools who do enroll chose more selective degrees. Students in treated schools also perform slightly better on the national exit exam. Since most individuals in our sample are from low-income families, and positive effects derive mostly from changes in behavior of students from better socioeconomic backgrounds, we conclude that informational policies to motivate the demand for higher education are less effective in contexts where credit constraints are sizable.

Patient satisfaction and perceived quality of care: evidence from a cross-sectional national exit survey of HIV and non-HIV service users in Zambia

Patient satisfaction and perceived quality of care: evidence from a cross-sectional national exit survey of HIV and non-HIV service users in Zambia

Factors Affecting the Effectiveness and Success of International Road Freight Transit from Iran

International freight transit involves the entrance of goods through one national border and exit through the other. Because of Iran's role in the economy of the region and the long record of international freight transit via this route, international freight can be a significant source of foreign currency. A review of the history of this sector provides the basis to model the factors affecting transit, and thereby to classify the questions and hypotheses to identify and prioritize those factors influencing success. Using questionnaires with managers, those involved in the member companies of the International Transportation Union, and those with a minimum of five years of experience in the industry, the authors developed a statistical sample. Using a random sample method, and the confirmatory factor analysis, results were obtained. These results indicate the effect of the model constructs-Infrastructure, Policies/Rules, Technology, Operation Management and Capacity-building-on the effectiveness of the sector (international road freight transit from Iran). Five hypotheses were confirmed, and we discovered that among various factors, the one that most significantly affects success is to revise the policies/rules. After this most significant factor, other factors-technology, infrastructure, operation management and capacity-building consecutively-affect success. In conclusion, we provide suggestions to make international road freight transit more successful.

Reliability testing of a portfolio assessment tool for postgraduate family medicine training in South Africa

BackgroundCompetency-based education and the validity and reliability of workplace-based assessment of postgraduate trainees have received increasing attention worldwide. Family medicine was recognised as a speciality in South Africa six years ago and a satisfactory portfolio of learning is a prerequisite to sit the national exit exam. A massive scaling up of the number of family physicians is needed in order to meet the health needs of the country.AimThe aim of this study was to develop a reliable, robust and feasible portfolio assessment tool (PAT) for South Africa.MethodsSix raters each rated nine portfolios from the Stellenbosch University programme, using the PAT, to test for inter-rater reliability. This rating was repeated three months later to determine test–retest reliability. Following initial analysis and feedback the PAT was modified and the inter-rater reliability again assessed on nine new portfolios. An acceptable intra-class correlation was considered to be > 0.80.ResultsThe total score was found to be reliable, with a coefficient of 0.92. For test–retest reliability, the difference in mean total score was 1.7%, which was not statistically significant. Amongst the subsections, only assessment of the educational meetings and the logbook showed reliability coefficients > 0.80.ConclusionThis was the first attempt to develop a reliable, robust and feasible national portfolio assessment tool to assess postgraduate family medicine training in the South African context. The tool was reliable for the total score, but the low reliability of several sections in the PAT helped us to develop 12 recommendations regarding the use of the portfolio, the design of the PAT and the training of raters.

Implications of the 2012 Election for Health Care — The Voters' Perspective

In October, we reported on voters' views of health care and how those views might influence their choices in the 2012 election.1 Now that the election is over, we have analyzed a range of pre-election and post-election polls as part of a Robert Wood Johnson Foundation project. The data are derived from three types of polls. The first is a 2012 national exit poll, comprising the responses of 26,565 voters as they exited voting places and those from 4408 telephone interviews (landline and cell phone) with early and absentee voters. The second are 11 pre-election polls conducted by telephone (landline . . .

Reforming Medical Curriculum While Ignoring Examinations

Successive national initiatives in India to improve the medical curriculum are plagued by the lack of objective data that can help grade the existing quality of medical education. Without such quantitative data, which can be conveniently obtained through national exit exams, it is infructuous to embark upon curricular reforms. The results of the national exit exams must serve as a starting point for all subsequent reforms.

Curricular Battles

This paper is a response to the two commentaries (Sircar, 2012; Sarbadhikari, 2012) written for the article “Reforming Medical Curriculum in India in Recent Years: Conflicts of Political, Regulator, Educationist and Professional Nature and Strategies for their Resolution” (Sood & Ananthakrishnan, 2012). Considering the enormity of the process of bringing about a change, often compromises have to be made for a step wise approach to make the proposed changes more acceptable across the board. A ‘one size fits all’ approach is unlikely to work in India because of numerous conflicting pressures and the solution has to be tailored to the specific situations in the country. While a national exit examination as a measure of quality control looks admirable, a number of issues have to be addressed before it can become a reality. The authors conclude that a much higher role of educationists and professionals, consensus building, and political support are essential to achieve successful curricular reforms. A strong regulatory agency could play a definitive role in maintaining standards.