The effectiveness of tools used to evaluate successful critical decision making skills for applicants to healthcare graduate educational programs: a systematic review

Abstract

Background Students leave healthcare academic programs for a variety of reasons. When they attrite, it is disappointing for the student as well as their faculty. Advanced practice nursing and other healthcare professions require not only extensive academic preparation, but also the ability to critically evaluate patient care situations. The ability to critically evaluate a situation is not innate. Critical decision making skills are high level skills that are difficult to assess. For the purpose of this review, critical decision making and critical thinking skills refer to the same constructs and will be referred to globally as critical decision making skills. Objective The objective of this review was to identify the effectiveness of tools used to evaluate critical decision making skills for applicants to healthcare graduate educational programs. Inclusion criteria Types of participants Adult (18 years of age or older) applicants, students enrolled and/or recent graduates (within one year from completion) of healthcare graduate educational programs. Types of interventions This review considered studies that evaluated the utilization of unique tools as well as standard tools, such as the Graduate Record Exam or grade point average, to evaluate critical decision making skills in graduate healthcare program applicants. Types of studies Experimental and non-experimental studies were considered for inclusion. Types of outcomes Successful quantitative evaluations based on specific field of study standards. Search strategy The search strategy aimed to find both published and unpublished studies. Studies published in English after 1969 were considered for inclusion in this review. Databases that included both published and unpublished (grey) literature were searched. Additionally, reference lists from all articles retrieved were examined for articles for inclusion. Methodological quality Selected papers were assessed by two independent reviewers using standardized critical appraisal instruments from Joanna Briggs Institute. Any disagreement between reviewers was resolved through discussion or with a third reviewer. Data collection Data was extracted independently by each reviewer from papers included in the review using a Microsoft Excel spreadsheet. Included data included study type, r values, number of subjects and reported p values. These were indexed by author, year and study title. Data synthesis The meta-analysis was performed using the method for effect size analysis from Hunter and Schmidt. The syntax for equations was transposed into a Microsoft Excel spreadsheet for data entry, analysis and graph creation. Results No articles or paper addressing unique tools for ascertaining critical decision making skills met the inclusion criteria. Standard tools, which were represented in the literature, assess critical decision making skills via prediction of academic and clinical success, which indicates the presence of critical decision making skills in graduate healthcare students. A total of 16 studies addressing standard tools were included in this review. All were retrospective case series studies. The date range for the included studies was 1970 to 2009. The strongest relationship was undergraduate grade point average's correlation to graduate grade point average (small effect size with an r value of 0.27, credibility interval [CrI] of 0.18–0.37). The second strongest relationship was between Graduate Record Examination's verbal section and graduate grade point average (small effect size with an r value of 0.24, CrI of 0.11–0.37). Conclusions An applicant's undergraduate GPA has the strongest correlation with graduate healthcare program success of the indicators analyzed (r = 0.27, small effect size). The next best predictor of graduate healthcare program success was the GRE Verbal score (r = 0.24, small effect size). However, all of the variables carried positive correlations with graduate success, just of lesser effect size strength. With this information in mind, the review reinforces that traditional predictors of graduate school success and CDM skills are valid. Implications for practice This review supports the continued use of traditional indicators of graduate school potential in the undergraduate grade point average and the various sections of the Graduate Record Examination for the selection of graduate healthcare applicants. Implications for research Primary studies should be funded and performed to assess the use of unique tools in assessing critical thinking in graduate healthcare students.