Predicting student performance using artificial neural network analysis

Abstract

Context and setting The selection of medical students into South African medical schools is largely based on academic performance in high school and relies upon the marks achieved in the final examination of the penultimate school year. Our medical school uses the aggregate mark for that year and the marks achieved in mathematics and science. We are currently actively aligning the demographic profile of our student body with that of our country. Selection of Black students is a priority, but selection using these traditional criteria is problematic. Why the idea was necessary As a legacy of apartheid, many Black students are still exposed to a disadvantaged schooling system and academic performance at school does not necessarily reflect the potential to succeed at tertiary level. In addition, changes in the curriculum and assessment methodology at high-school level will commence this year, and these criteria will no longer be available to use in the selection process. Therefore, alternative predictors for successful study in our medical curriculum need to be identified. Using predictors that are more appropriate and addressing possible obstacles to student success in our curriculum should assist our school in its mission to select and retain more students from disadvantaged communities. What was done Artificial neural network analysis (ANNA) was applied to identify the predictors of success using input and output data for students admitted into our undergraduate medical programme since 1999. A total of 99 input variables in 3 categories were used: demographic (3); quantitative (16), and qualitative (80). Student demographics included gender, ethnic group and home language. Quantitative variables were academic performance in high school and the results of the national health sciences placement tests (HSPTs), which assess potential to succeed at tertiary education level. A set of 80 qualitative variables, related to generic skills, life views and attitudes, were obtained using a questionnaire completed by the students on admission to the programme. Success was measured by academic performance during the first study year according to whether a student passed, passed with distinction or failed. Evaluation of results and impact The study population comprised 171 students for whom complete datasets were available. Ten (6%) of these students failed the first year. ANNA showed that, when using all the input variables, student performance could be predicted with close to 100% accuracy. The most powerful predictors were the results of the HSPTs. Using only these, the 10 failures could be predicted with 90% accuracy. A similar result was found using the 80 qualitative variables in isolation. The results also showed that the average mark achieved during the first year was significantly related to the student’s home language and ethnicity. Although future work will need to validate these findings against more data, especially for more unsuccessful students, they already have an impact on our selection process, particularly in helping to identify high-risk students. The study will be expanded to include all years of training, other university programmes, and other health sciences programmes in the country.