The role of technology in assessment, documentation, and tracking of clinical competencies in radiography education

Abstract

Competency-based education (CBE) has gained momentum dramatically across the landscape of higher education [[1]Cunningham J. Key E. Capron R. An evaluation of competency-based education programs: a study of the development process of competency-based programs.J Competency Based Educ. 2016; 1: 130-139Crossref Google Scholar]. However, current challenges facing healthcare education include assessing clinical competency [[2]Ferenchick G.S. Solomon D. Using cloud-based mobile technology for assessment of competencies among medical students.Peer J. 2013; 1 (e): e164Crossref PubMed Scopus (13) Google Scholar]. Documenting students’ experiences and performance during clinical rotations has become even more important. As clinical education is paramount to providing future health professionals with authentic experiences [[3]Romig B.D. Maillet J.O.S. Chute P.M. McLaughlin R.J. Clinical education from a specialized accreditation viewpoint a report of the ASAHP clinical education task force.J Allied Health. 2013; 42: 151-156PubMed Google Scholar], clinical competencies must be tracked and recorded adequately and efficiently for the assessment at different levels of learning across the entire learning journey of students [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. In radiography, regulatory boards have set standards for students to achieve to ensure students’ preparedness for future employment [[5]M. Nicole, M. Fairbrother, S.V. Nagarajan, J. Blackford, L. Mcallister Student-led clinics: building placement capacity and filling service gaps. Australian Collaborative Education Network (ACEN) Limited Available at: http://acen edu au/2014Conference/wp-content/uploads/2014/10/Student-Led-Clinics-Building-Placement-Capacity-and-Filling-Service-Gaps pdf (accessed 30 November 2016). 2014.Google Scholar]. For example, in Australia, the Medical Radiation Practice Board of Australia (MRPBA) uses professional capabilities to identify the minimum knowledge, skills and professional attributes necessary for safe practice [[6]MRPBAProfessional Capabilities of Medical Radiation Practice. Australian Health Practitioner Regulation Agency (AHPRA), 2020http://www.medicalradiationpracticeboard.gov.au/Registration/Professional-Capabilities.aspxGoogle Scholar], whereas, in the United Kingdom (UK), the Heath and Care Professions Council (HCPC) outlines the standards of proficiencies (SoPs). Despite the differences, in the definition of competencies, they must also find a way to make these capabilities accessible and meaningful [[1]Cunningham J. Key E. Capron R. An evaluation of competency-based education programs: a study of the development process of competency-based programs.J Competency Based Educ. 2016; 1: 130-139Crossref Google Scholar]. Since many radiography programs have adopted competency-based education, there is a need to align students learning to the required professional capabilities which must be validated as a requirement for graduate professional registration. Students must also be supported during this process which requires the provision of comprehensive feedback, real-time monitoring and feed-forward control [[7]Young T. Newton M. Hay S. Moving Mid-Wifery Online. PebblePad Case Studies. Retrieved from www.pepplepad.com, 2014Google Scholar]. Clinical educators assess students during clinical placements to ensure students have achieved relevant knowledge, skills and attitudes for safe and competent practice. For many years this process involved the use of paper-based templates or logbooks [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. Students also provide self-evaluation reports, case studies, journals, verified clinical rosters and reflections to program faculty. For many institutions, these are still paper-based, hand-written documents which must be scanned, sent or uploaded to the university platforms for assessment by the unit coordinators. In the era of technological advancements, this process can be enhanced with a more efficient way to document competencies [[8]Hancock A.B. Brundage S.B. Formative feedback, rubrics, and assessment of professional competency through a speech-language pathology graduate program.J Allied Health. 2010; 39: 110-119PubMed Google Scholar] and facilitate the timely provision of feedback to the students. However, in radiography education, there are many considerations, including the need to improve the process of tracking competencies [[9]Hobbs D.L. Tracking clinical competencies on the web.Radiol Technol. 2005; 76: 345-353PubMed Google Scholar]. This commentary aims to discuss the challenges associated with the current paper-based approaches. As these challenges can be overcome by some of the emerging technologies, this article will provide specific examples and highlight some of the features that could be beneficial if incorporated in contemporary radiography education. The challenges associated with paper-based approaches to competency tracking are not unique to radiography. An article by Young et al [[7]Young T. Newton M. Hay S. Moving Mid-Wifery Online. PebblePad Case Studies. Retrieved from www.pepplepad.com, 2014Google Scholar] described how the La Trobe University's midwifery and nursing program used to distribute, collect, fill-in and validate many paper-based forms and document activities in a student-held bound workbook. It is reported that records of each placement were then transcribed into a paper-based form that was stored in the student file. With the paper-based system, limited information at hand was available to provide effective interventions for students who were struggling during clinical placement [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. In another study, Lounsbery et al [[10]Lounsbery J.L. Pereira C.R. Harris I.M. Moon J.Y. Westberg S.M. Kolar C. Tracking patient encounters and clinical skills to determine competency in ambulatory care advanced pharmacy practice experiences.Am J Pharm Educ. 2016; 80: 14Crossref PubMed Scopus (9) Google Scholar] reported challenges with the tracking data using paper-based methods at the University of Minnesota's College of Pharmacy program, which developed a self-reporting student tracking tool on patient encounters. In this study, students were asked to track their patient encounters using a paper format tool. The students were then required to enter the data on a link provided on to a website. The challenges faced by this method of tracking encounters were lack of participation, loss of paper tool and was time-consuming [[13]Cordier R. McAuliffe T. Wilson N.J. Totino R. Dender A. Smith C. et al.The appropriateness and feasibility of an online e-portfolio for assessment of undergraduate allied health students.Aust Occup Ther J. 2016; 63: 154-163Crossref PubMed Scopus (12) Google Scholar]. The struggle with data management in assessment becomes apparent once paper data needs to be sorted into meaningful results. Transferring notes taken during clinical placement on pieces of paper to notebooks or institutional websites can be time-consuming, requires the organization of material, and there are risks of losing the notes or misplacing it [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. One of the challenges in radiography clinical education is the amount of information that educators must collect in the three (3) to four (4) years that students have enrolled in the program. This information is collected in various forms from both clinical and didactic settings. This data must be an accurate assessment of the students’ progress and success [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. It allows radiography programs to intervene and provide support to students and offer important evidence to accreditation agencies. Therefore, it is important that the radiography profession also identifies tools or digital platforms that will suit the needs of the profession in view of contemporary practice. When radiography students are on placement, their clinical experiences can vary. Depending on the technology and resources available in the department [[11]Sá dos Reis C. Pires-Jorge J.A. York H. Flaction L. Johansen S. Maehle S. Curricula, attributes and clinical experiences of radiography programs in four European educational institutions.Radiography. 2018; 24: e61-e68https://doi.org/10.1016/j.radi.2018.03.002Abstract Full Text Full Text PDF PubMed Scopus (8) Google Scholar], some students can experience all speciality imaging modalities and techniques while other students may not have the same access. In such situations, it is a challenging task to rely on traditional paper-based logbooks. The emerging modern tools could assist in assessing, documenting and tracking competencies in the radiography profession. The challenges highlighted in these studies have been encountered at the authors’ institution, which relies on the paper-based reporting system. While many institutions use learning management software (LMS) to manage clinical placement, the use of the paper for documentation is not easy to eliminate unless a dedicated software with more reliable features is identified. When students enrolled in a four-year Bachelor of Medical Imaging program are on clinical placement, their attendance is recorded on paper by students, verified and signed by the clinical supervisors. Students then upload the verified clinical roster to Blackboard, a web based LMS as an assessment item. Once assessed by the lecturers at the university, the verified clinical rosters are saved on a secure drive. The clinical supervisors’ clinical reports are also submitted on paper to the Work Integrated Learning Support (WILS) team for record-keeping. The challenge with such paper-based process is that the reports and rosters can easily be lost, damaged or misplaced. While there is no published evidence of this occurring, it is in the interest of the students, and the university to ensure there is an efficient and secure approach to managing how documentation of students’ competencies is handled. A review of the literature identifies several tools (Fig. 1). These tools are discussed in the literature and key features are highlighted to demonstrate how they might be useful for integration in radiography education.(a)E-portfolios An electronic portfolio (ePortfolios) can be classified as a collection of electronic evidence assembled and managed by the user [[12]Salyers V.L. Carter L. Antoniazzi C. Johnson S. Evaluating the effectivenessof aclinical tracking systemforundergraduate nursing students.Nurs Educ Perspect. 2013; 34: 37-42PubMed Google Scholar]. It can be shared with other users to enable the management and tracking of learning [[13]Cordier R. McAuliffe T. Wilson N.J. Totino R. Dender A. Smith C. et al.The appropriateness and feasibility of an online e-portfolio for assessment of undergraduate allied health students.Aust Occup Ther J. 2016; 63: 154-163Crossref PubMed Scopus (12) Google Scholar]. ePortfolio is a digital platform which offers the potential to transform portfolio learning by providing students with a self-managed, independent learning space, which is accessible in clinical settings [[14]Pincombe J. McKellar L. Weise M. Grinter E. Beresford G. ePortfolio in midwifery practice:“the way of the future.Women Birth. 2010; 23: 94-102Abstract Full Text Full Text PDF PubMed Scopus (20) Google Scholar]. Students can interact with their teachers, preceptors, and peers using ePortfolios and can showcase their work by collecting digital artefacts as evidence. School of Nursing and Midwifery at the University of Queensland introduced the clinical practice performance electronic portfolio (CPPeP) into the final year of its Bachelor of Nursing program [[15]Bogossian F.E. Kellett S.E. Mason B. The use of tablet PCs to access an electronic portfolio in the clinical setting: a pilot study using undergraduate nursing students.Nurse Educ Today. 2009; 29: 246-253Crossref PubMed Scopus (22) Google Scholar]. This step was a move from a traditional paper-based model to a modern digital method. The feedback was positive as it was highly valued by the students, faculty and the clinical preceptors. The disadvantage was the availability of a limited number of desktops at the clinical sites for students to use it effectively which led to further use of tablet PCs [[15]Bogossian F.E. Kellett S.E. Mason B. The use of tablet PCs to access an electronic portfolio in the clinical setting: a pilot study using undergraduate nursing students.Nurse Educ Today. 2009; 29: 246-253Crossref PubMed Scopus (22) Google Scholar]. Royal Melbourne Institute of Technology (RMIT) started a pilot project on using e-portfolios for Technical and Further Education (TAFE) and Higher Education Sector [[16]Botterill M. Allan G. Brooks S. Building Community: Introducing ePortfolios in University Education. Hello! Where are you in the landscape of educational technology, 2008Google Scholar] to grow the university's flexible assessment practices and to support student career development. The trial of ePortfolio at RMIT was done on the PebblePad platform which is commercially available software that supports learning [[16]Botterill M. Allan G. Brooks S. Building Community: Introducing ePortfolios in University Education. Hello! Where are you in the landscape of educational technology, 2008Google Scholar]. Some education institutions are using e-portfolios through Learning Management Systems (LMS) such as Blackboard while others prefer commercially available software. Students on clinical placements can simply use this platform if their associated educational institutions use Blackboard. Other LMS such as Moodle and Desire-to-Learn (D2L) also offer e-portfolio platforms that can be used by students to document their learning, reflections while on clinical placement.(b)Pebblepad The latest version of PebblePad facilitates the creation of a personalized online learning environment [[17]Gallagher H. Thompson L. Hughes M. Getting the most out of electronic portfolios: pedagogy and benefits.Adv Soc W Welf Educ. 2015; 17: 39Google Scholar]. Its ePortfolio component facilitates a wide range of reflective learning practices which is key for health students. PebblePad can also be used as an effective online clinical assessment platform and to provide real-time monitoring of students’ progress. A feature within PebblePad called the clinical placement workbook [[7]Young T. Newton M. Hay S. Moving Mid-Wifery Online. PebblePad Case Studies. Retrieved from www.pepplepad.com, 2014Google Scholar] was piloted by La Trobe University's Bachelor of Nursing/Bachelor of Midwifery first-year students. Feedback from students, clinical preceptors and academics were positive for this user-friendly tool [[7]Young T. Newton M. Hay S. Moving Mid-Wifery Online. PebblePad Case Studies. Retrieved from www.pepplepad.com, 2014Google Scholar]. Another essential feature of PebblePad is Active Teaching Learning and Assessment Space (ATLAS) [[18]Kirke A. Ball A. Edwards S. A New Approach to Confirmation of Skill Acquisition. Retrieved from www.pebblepad.com, 2016Google Scholar]. The clinical placement workbook can be submitted via ATLAS for assessment. This makes it easy for educators to embed Objective Structured Clinical Examination (OSCE) and use the PebblePad where assessments are easily submitted through ATLAS [[18]Kirke A. Ball A. Edwards S. A New Approach to Confirmation of Skill Acquisition. Retrieved from www.pebblepad.com, 2016Google Scholar].(c)Mobile electronic devices (MED) MEDs are small devices, portable and provide mobile computing tools [[15]Bogossian F.E. Kellett S.E. Mason B. The use of tablet PCs to access an electronic portfolio in the clinical setting: a pilot study using undergraduate nursing students.Nurse Educ Today. 2009; 29: 246-253Crossref PubMed Scopus (22) Google Scholar]. The literature on the use of MEDs in health professions has been growing in recent years, however, this is predominantly in nursing and medical education [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar]. While there are many different types of MEDs, Personal Digital Assistants (PDAs), smartphones, laptops, and tablet PCs have commonly been reported in clinical education. PDAs have the capability of changing how health care is taught and delivered in clinical settings. The primary benefit highlighted in the use of PDAs in clinical education is in providing timely access to data [[19]Farrell M.J. Rose L. Use of mobile handheld computers in clinical nursing education.J Nurs Educ. 2008; 47: 13-19Crossref PubMed Scopus (67) Google Scholar]. Students can generate a log of clinical experiences which facilitates easy access to information. Students on clinical placement can consult on drugs for treatment, to perform basic calculations or simply to make notes easily using MEDs [[20]Ventola C.L. Mobile devices and apps for health care professionals: uses and benefits.Pharm Ther. 2014; 39: 356Google Scholar]. While there is potential to use MEDs in radiography programs, it is important to carefully assess how the user could optimize learning. Radiography students can benefit from using MEDs during clinical placements to access positioning protocols or to access lecture notes. These tools can also save time to manage patient logs, document case studies for assignments, and record completed clinical competencies [[4]Garner RS. A Comparative Analysis of Data Collection Systems Used in Radiography Educational Programs and the Role Mobile Electronic Devices Play. Electronic Theses and Dissertations, 2015https://dc.etsu.edu/etd/2592Google Scholar].(d)Digital logbooks The use of digital logbooks is offered on platforms such as InPlace [[21]InPlaceInPlace: Smart Placement Solutions. QuantumIT, 2017https://www.inplacesoftware.com/features/Google Scholar] and CompTracker [[22]CompTracker. CompTrackerCompTracker: Solutions for Modern Learning. Great Big Solutions Ltd, 2017http://www.studentlogbook.com/Google Scholar]. Digital logbooks require students to log details of their day-to-day activities during clinical placement [[23]Crowther R. Sealey R. Harris R. A snapshot of placement activities and learning outcomes assessed through logbook submission for clinical exercise physiology students. focus on health professional education: a multi-disciplinary.Journal. 2017; 18: 67Google Scholar]. This includes a description of the knowledge and skills applied in clinical settings. Clinical logs have been used for many years as part of undergraduate nursing education and other health programs [[23]Crowther R. Sealey R. Harris R. A snapshot of placement activities and learning outcomes assessed through logbook submission for clinical exercise physiology students. focus on health professional education: a multi-disciplinary.Journal. 2017; 18: 67Google Scholar] to provide students, clinical supervisors, and faculty with valuable opportunities to address gaps in learning.(e)Inplace InPlace is a web-based platform. It has been primarily used as clinical placement software by linking radiography and other health students to hospitals and medical clinics via an educational body. According to Quantum IT [[24]IT Q. InPlace Smart Placement Solutions. QuantumIT, 2017https://www.inplacesoftware.com/Google Scholar], this platform facilitates complete management of Work Integrated Learning (WIL) placements primarily in countries such as Australia and the United Kingdom (UK). One of the modules available on InPlace is the student logbook which can integrate with student timesheets to reconcile tasks completed as well as competencies. This will enable to easily access information which has been one of the major drivers to integrate digital tools in the tracking of competencies [[21]InPlaceInPlace: Smart Placement Solutions. QuantumIT, 2017https://www.inplacesoftware.com/features/Google Scholar]. It is compatible with iPad or tablets and can be customized to host pre-filled information and default values. This makes it relatively easy to streamline tracking of competencies in radiography when students are on clinical placement. Daily recording of task assignments, student feedback, supervisory commentary and, industry codes for government accreditation can also be easily incorporated [[24]IT Q. InPlace Smart Placement Solutions. QuantumIT, 2017https://www.inplacesoftware.com/Google Scholar].(f)Comptracker CompTracker software has the capability of tracking clinical competencies using electronic devices [[25]Kostiuk W. The Efficacy of Mobile Competency Tracking. Retrieved from www.greatbigsolutions.com. Online submission, 2011Google Scholar]. While there is limited literature on its use, the software capabilities can enable students, instructors, and administrators to track clinical competencies using mobile device replacing the old paper-based methods. This allows clinical sites and educators to track students’ progress and manage their time more effectively. CompTracker provides opportunities to provide feedback, to collaborate through the application, and grade online submissions [[25]Kostiuk W. The Efficacy of Mobile Competency Tracking. Retrieved from www.greatbigsolutions.com. Online submission, 2011Google Scholar]. When students submit their clinical work, it can be reviewed, graded and feedback provided in real-time. Students can understand the program expectations better and would be happy not to drag a paper binder with them everywhere they go [[26]Great Big Solutions Ltd; 2019 [cited 2019 7/3/2019]. Available from: http://www.studentlogbook.com/.Google Scholar]. Any outstanding or incomplete competencies can be actioned right away through CompTracker. Clinical coordinators can initiate an action or learning plan through CompTracker to address the incomplete work [[26]Great Big Solutions Ltd; 2019 [cited 2019 7/3/2019]. Available from: http://www.studentlogbook.com/.Google Scholar]. This tool allows issues to be identified earlier, and appropriate measures can be put in place before it is too late. The Medical Radiation Sciences department at Mohawk-McMaster Institute of Applied Health Sciences piloted CompTracker in 2012 and fully implemented the system in 2013. It is used in the program for logbook entries, skills assessments, competencies, weekly and monthly evaluations learning contracts and attendance. It is also used to sign off forms such as agreement to policies. Many more Canadian schools such as Cambrian College, Lambton College and Confederation College have adopted CompTracker [[26]Great Big Solutions Ltd; 2019 [cited 2019 7/3/2019]. Available from: http://www.studentlogbook.com/.Google Scholar]. It is up to the educational institutions to consider what elements of clinical education can be supported by technology. Methods currently used in assessing competencies in clinical placements include case studies, reflective journals, completion and self-assessment of core competencies [[8]Hancock A.B. Brundage S.B. Formative feedback, rubrics, and assessment of professional competency through a speech-language pathology graduate program.J Allied Health. 2010; 39: 110-119PubMed Google Scholar]. Timely and clear formative and summative assessments are an integral part of clinical placements for many allied professions to allow students to learn and improve performance before the next assessment [[8]Hancock A.B. Brundage S.B. Formative feedback, rubrics, and assessment of professional competency through a speech-language pathology graduate program.J Allied Health. 2010; 39: 110-119PubMed Google Scholar]. Ensuring timely feedback urges development of more advanced tools for collecting information. Castillo et al. [[27]Castillo J. Caruana C.J. Wainwright D. The changing concept of competence and categorisation of learning outcomes in Europe: implications for the design of higher education radiography curricula at the European level.Radiography. 2011; 17: 230-234Abstract Full Text Full Text PDF Scopus (15) Google Scholar] argue that a higher qualification framework needs to be developed in order to keep up with the rapid advancement in technology and demands of the changing professional needs of the radiography programs. While the assessment is normally done by clinical supervisors and lecturers, self-assessment of competency is an area that also deserves attention from the allied health practitioners. Di Tondo et al. [[28]Di Tondo S. Ferretti F. Bielli S. Assessment of core competencies of physical therapists among students and professionals in Italy a cross-sectional study.J Allied Health. 2018; 47: 133-143PubMed Google Scholar] highlights the value of self-assessment of the competencies in physiotherapy. Therefore, accurate documentation of self-assessed competencies and including the use of guiding frameworks has become essential in allied health programs across the world. There is a range of tools that are used by different health programs in the assessment, documentation and tracking of competencies. Educational institutions are structuring and aligning their strategic plans and securing their future as leading partners in the digital delivery of education. Students are seen today as stakeholders or equal partners by educational institutions. Therefore, there is a huge drive to improve how students learn and creative ways of streamlining the teaching process [[29]McNamara J. Green D. A digital future.Imaging Ther Pract. 2017; Apr 1: 12Google Scholar]. Educational institutions are harnessing digital technologies to provide education that is equitable for all students regardless of their circumstances [[13]Cordier R. McAuliffe T. Wilson N.J. Totino R. Dender A. Smith C. et al.The appropriateness and feasibility of an online e-portfolio for assessment of undergraduate allied health students.Aust Occup Ther J. 2016; 63: 154-163Crossref PubMed Scopus (12) Google Scholar]. Employers are looking for new graduates who have developed a sound level of generic skills to cope with the working environment. Therefore, with the help of digital technologies universities are trying to be creative to engage students in group works or projects and their assessment of generic skills. From the patients perspective, use of digital technologies facilitates better decision making process and improved patient care outcomes [[20]Ventola C.L. Mobile devices and apps for health care professionals: uses and benefits.Pharm Ther. 2014; 39: 356Google Scholar]. Adoption of these technologies into educational and clinical settings not only require an analysis of the features of the technology or software but other considerations as well. Long term considerations include streamlining processes, improving efficiency and working effectively in assessing, documenting and tracking of clinical competencies. Barriers to adoption of digital platforms in education may include funding, reluctance to use, proper use and integration [[20]Ventola C.L. Mobile devices and apps for health care professionals: uses and benefits.Pharm Ther. 2014; 39: 356Google Scholar]. Other concerns might be the need for secure internet connections, the versatility of digital platforms and if it is going to improve the efficiency aspect of the process. With the ongoing developments in big data and machine learning methods, it is expected there will be some benefits to tracking clinical competencies in the future. Adoption of technology does require collaboration and research so that it can be beneficial to all stake holders in radiography education. Health programs are using a variety of methods and platforms to assess, document and track clinical competencies. Organizations require professionals who are flexible and adaptable and who can perform under pressure in a resource-strained environment. Hospitals and medical clinics are moving towards the digitalized and paper-free environment. The use of electronic digital technology in education and clinical settings can yield data that is more secure, efficient, and versatile than old paper-based systems [[29]McNamara J. Green D. A digital future.Imaging Ther Pract. 2017; Apr 1: 12Google Scholar]. Therefore, digital technologies such as MEDs, ePortfolios, and digital log books have been trialed and implemented by different health programs. Digital tracking, assessment and documentation of clinical competencies provides opportunities to further explore this area with formal studies.