Influencing the national training agenda

Abstract

The Journal of Bone and Joint Surgery. British volumeVol. 87-B, No. 9 AnnotationFree AccessInfluencing the national training agenda THE UK & IRELAND ORTHOPAEDIC ELOGBOOKJ. L. Sher, M. R. Reed, P. Calvert, W. A. Wallace, A. LambJ. L. SherConsultant Orthopaedic SurgeonDepartment of Orthopaedics and Trauma Surgery, Wansbeck Hospital, Woodhorn Lane, Ashington, Northumberland NE63 9JJ, UK.Search for more papers by this author, M. R. ReedConsultant Orthopaedic SurgeonDepartment of Orthopaedics and Trauma Surgery, Wansbeck Hospital, Woodhorn Lane, Ashington, Northumberland NE63 9JJ, UK.Search for more papers by this author, P. Calvertdeceased, Past SAC Chairman and Consultant Orthopaedic SurgeonSt George’s Hospital, London, UK.Search for more papers by this author, W. A. WallaceProfessor, Dean of Faculty of Health Informatics Royal College of Surgeons of Edinburgh, Nicholson Street, Edinburgh EH8 9DW, Scotland.Search for more papers by this author, A. LambDirector of Web Services Royal College of Surgeons of Edinburgh, Nicholson Street, Edinburgh EH8 9DW, Scotland.Search for more papers by this authorPublished Online:1 Sep 2005https://doi.org/10.1302/0301-620X.87B9.16433AboutSectionsView articleSupplemental MaterialPDF/EPUB ToolsDownload CitationsTrack CitationsPermissionsAdd to Favourites ShareShare onFacebookTwitterLinked InRedditEmail View articleA record of operative experience has always been a prerequisite for basic and higher surgical trainees. Although such records are usually examined during trainee assessments and hospital inspections, there has not hitherto been a systematic attempt to interrogate this data, which importantly reflects the day to day, “coalface” experience of trainees.There is no published data on the operative experience of a UK or Irish orthopaedic trainee before specialist registration. There is a widespread belief that the number of operations performed during specialist training is decreasing. Factors such as reduction in junior doctors hours of work and diversion of work away from training centres have major effects on patterns of work.The development of an elogbook has provided hard data on trainees’ experience and permitted insights into the “in training operative experience” of trainees in trauma and orthopaedic practice in the United Kingdom and Ireland and allows detailed analysis of the performance of trainee, trainer, hospital and training programme at the click of a button.BackgroundThe project began in the Northern deanery in 1998 and was adopted as a collaborative venture in 2000 by the British Orthopaedic Association (BOA) Education Committee, the Specialist Advisory Committee (SAC) in Trauma and Orthopaedics, the British Orthopaedic Trainees Association (BOTA) and the Royal College of Surgeons of Edinburgh (RCSEd). Funds were raised from the BOA, the Editorial Board of the Journal of Bone and Joint Surgery, the Charnley Trust, the Wishbone Trust, Smith & Nephew, Johnson & Johnson and Biomet.Over several years a committed group of trainees and trainers tested several versions of the logbook leading to the current product. Responsibility for the project has passed to the BOA eLogbook Validation & Authorisation Committee (eVAC) and the current software was produced and is maintained by the Faculty of Health Informatics at the RCSEd.Details of the elogbookCurrent options for elogbook entry (Fig. 1).Surgeons’ eLogbooks1 can be designed as ‘thin’ or ‘thick’ client applications. ‘Thin’ clients rely on a browser to surf the Internet (e.g. Internet Explorer or Netscape Navigator) and its most important advantage is that no software need be downloaded onto the user’s computer. This module avoids software conflicts but relies on a live connection to the Internet with the information held on a remote computer, in this instance at the Royal College of Surgeons, Edinburgh.A ‘thick client’ module is software loaded onto the user’s computer by CD or Internet. It is faster because it uses only one computer and does not require a constant Internet connection. The RCSEd ‘thick client’ eLogbook is uniquely designed to automatically synchronise with the ‘thin client’ module thus keeping both versions up to date.As more people get broadband connections at hospital and home, the ‘thin’ clients will be more advantageous but, at present, their use in isolation could exclude many users.The suite of modules has eLogbook programmes for the Palm and Pocket PC versions of personal digital assistants (PDA). All the modules are compatible and any or all versions can be chosen.Procedure classification.After much debate, a system was devised to encompass the information needed by the United Kingdom and Irish SAC. Users can submit suggestions for unlisted procedures, which once ratified by the eVAC committee, appear seamlessly as the users’ ‘Synchronisation’ button is next pressed. The great majority of users’ suggestions have been incorporated already.Data protection.Because data which is defined as ‘sensitive’ or ‘confidential’ by the UK Data Protection Act2 is collected in the logbook, each user must register with the data protection authorities as a ‘data controller’. Although all logbooks are password-protected, users are responsible for the safekeeping of their data. The RCSEd server uses the same level of encryption security as bank web sites and the data is stored simultaneously on two servers which are regularly backed up off-site. Each user owns their data and collated information is administered by the eVAC committee.Information provided.The eLogbook gives information on levels of supervision (Fig. 2) and training opportunities provided by specific trainers, hospitals and training programmes. This is stratified according to a trainee’s seniority (Fig. 3).In order to compare training posts more accurately, a number of reference operations or operation groups have been established and trainees can thereby be compared with their peers. The example in Figures 2 and 3 shows total hip replacement (THR) and the numbers of training operations under each supervision code, thereby defining the use of available opportunities.Such analysis allows comparison of a trainee’s experience in a given time period or collection of procedures (e.g. hand, foot and ankle, spine) with the national average. Training opportunities offered by training programmes, hospitals or trainers can also be compared with national figures. Such comparisons display not only total numbers of procedures but also identify unused potential learning experiences.Access to the reports is restricted to defined users. Trainees have access to their own and pooled national comparative data. Training programme directors can examine a local individual’s performance and individual trainers and hospitals. The SAC chairman has access to all regions and all training departments.ProgressSince 1 October 2003, it has been compulsory for all specialist registrars in the United Kingdom and Ireland to submit data electronically to the Trauma & Orthopaedic Logbook.1 Although uptake rapidly increased during 2003, compliance is now 92%. Trainees are also encouraged to upload voluntarily their historical operative data.Data collection and analysis.Trainees were categorised by year-in-training (YIT) based on the time since appointment to a training scheme. If the trainee reported their date of completion of specialist training, this date was used preferentially to calculate their YIT group.Only operations performed during 2004 and uploaded prior to April 2005 were included for analysis. This paper represents a snapshot of training activity in 2004. A list of eligible specialist registrars (SpR) and YIT for each operation is created.Each trainee’s work is analysed by YIT. Any trainee year with fewer than 150 operations is excluded on the basis of poor compliance. The process forms a ‘pot’ of operations for analysis divided by category such as region, hospital, or trainer.With regard to the level of supervision, a trainee must be present at the documented operations. Five levels of involvement are available (Table I). The level of involvement is categorised as assisted or performed, with or without the trainer’s presence. A trainee must have carried out at least 70% of the operation procedure in order to enter ‘performed’. This ensures that a trainee performing a THR (implanting both the components) or TKR (implanting both the femoral and tibial components) must carry out the majority of the operation.Regions.Five regions were defined in order to assess the location of a training scheme with regards to operative experience (Table II).ResultsBy April 2005, 1509 users were registered on the website of whom 999 stated they were specialist registrars. A total of 906 of these were confirmed by their programme director as specialist registrars and this cohort forms the basis of the analysis. Within trauma and orthopaedic surgery in the United Kingdom and Ireland there are approximately 1700 consultants and 760 staff and associate specialist (SAS) doctors, who can also use the software if they wish. Each user is recognised within the system through a unique General Medical Council number (both UK and Irish). Although the database now includes over 500 000 operations, the 2004 data represents 157 492 uploaded operations.Table III shows a snap-shot of the current operative experience which United Kingdom and Irish trainees currently have. This shows the experience in each training year for the average trainee and represents operations performed by the trainee. Missed opportunities where the trainee was present but did not perform the surgery are also shown.Figures 2 and 3 represent the National Data and show the evolving level of supervision as trainees pass from years 1 to 6. Figure 2 demonstrates how a trainee compares with the mean national data with regard to operative experience of THR. A similar histogram could be produced comparing an individual trainee with peers in a hospital or within a deanery. Figure 3 shows the percentage of operations where the trainee performs the surgery.Table IV shows the number of procedures which a trainee can expect to perform and assist with during six years of training across the different regions.DiscussionAlthough there is only a small difference in the number of trauma versus elective procedures during six years of Higher Surgical Training there is a striking difference in the percentage of procedures performed, 84% vs 57%. This could imply that trauma operations are easier, or that there is a different attitude to the hands-on supervision of trauma surgery.Even with common elective operations, such as THR and TKR, up to half the potential training opportunities are missed and the trainee appears to be an assistant rather than a surgeon-in-training. It is questionable whether good training involves trainees repeatedly assisting their trainer doing the same operation, such as THR or TKR, 45 times, the current mean for all trainees in this study.It is alarming that each trainee on average is performing less than ten THRs per year during their training. While the authors accept that a procedure classified as assisting could involve the trainee performing up to 70% of the operation, this is hardly enough practical exposure to the most common elective operations to reassure the public that the trainee will be competent at the end of their training.The results in Figure 3 suggest a subtle evolving pattern of increasing trainee performance with regard to THR and TKR but not in hand or paediatric practice. Trauma surgery from the outset provides intensive experience which changes very little with seniority.An impression of the effect of shortening higher surgical training which has been suggested by the Department of Health is gained from Table III. If training were to be shortened to four years at current training intensity, it would be to reduce the number of THRs and TKRs which were performed from 37 and 44 to 22 and 24, respectively, for the whole training programme. It would also reduce elective exposure by 39% and trauma by 28%.There is no clear view of the minimum number of procedures a surgeon should perform before accreditation, nor how many operations a trainee should assist before they perform one. This is likely to vary according to the complexity of the procedure and the aptitude of the trainee. However, this is unlikely to explain the whole picture as trainees about to become consultants still perform less than half the THRs they attend. Our data suggest that, were thresholds to be set, a significantly greater number of trainee-performed operations could be achieved within current arrangements.With shorter training it may be the time to stipulate the number of procedures a trainee should perform before completion of training. We have polled European orthopaedic associations and, of respondents, found Germany, Hungary, Norway, Spain and Switzerland already do this. Croatia, Ireland, the Netherlands, Sweden and the United Kingdom do not.Examination of individual deanery performances indicates more variation than pooled regional data. With time, an ‘operative profile’ of each training post will be obtained, allowing programme directors to build equity into their programmes and address trainer weakness proactively. This more confidential aspect of the logbook is restricted to the SAC and individual programme directors.The eLogbook can be a highly sensitive barometer of the training experience of UK and Irish trainees. This was highlighted by the recent identification of a 20% fall in trainee exposure to THR,3 believed to be related to the European Working Time Directive.4With the United Kingdom Government set on re-organising postgraduate medical education through the Postgraduate Medical Education Training Board and shortening training times, it is imperative that missed training opportunities are minimised. Trainees towards the end of training should be ideally performing rather than assisting at most routine surgery and, where possible, offering supervised training to those at the very beginning of their surgical career. We believe this is not the case at present in the United Kingdom and that the present Department of Health initiatives are having a disturbing effort on training.Table I. Categories of operative supervision in the orthopaedic eLogbook. For this analysis ‘performed’ = S-TS+S-TU+P (*the trainee must have performed more than 70% of the operation’)CategoryLevel of supervisionAAssisted onlyS-TSSurgeon supervised by trainer scrubbed*S-TUSurgeon supervised by trainer unscrubbedPPerformed with no in-theatre supervisionTTrained a colleague (usually a more junior surgeon)Table II. The five regional groups devised to assess location on training programmes on operative experienceRegionTraining programmeLondon/ThamesNorth East Thames (Middlesex/University College Hospital, Percival Potts, Royal London, Stanmore), South East Thames, North West ThamesNorthern EnglandLeicester/Trent, Mersey, North Western, Northern, Nottingham/Trent, Sheffield/Trent, West Midlands (Birmingham), West Midlands (Oswestry), YorkshireScotlandEast of Scotland, North of Scotland, South East Scotland, West of ScotlandSouthern EnglandEastern, Oxford, South Western Region (Bristol), Devon and Cornwall, WessexOthersWales, Northern Ireland, Ireland and Armed ForcesTable III. Operative experience of the average UK trainee in 2004; mean number of operations performed by the trainee in each training yearNumber of operationsOperation groupYear 1Year 2Year 3Year 4Year 5Year 6Total years 1 to 6Missed opportunity* (%)* where the trainee was present but did not perform a significant part of the operationHip replacement4.86.45.95.277.937.255Knee replacement4.67.56.25.6712.543.548External fixation0.71.31.11.41.31.16.830All hands18.119.715.622.216.711.1103.626All paediatrics4.64.45.88.34.26.73432All trauma76.486.371.480.46555.6435.116All elective54.865.959.565.868.290.840543Table IV. Levels of performance of operations based on region classification. Each column is the mean number of operations attended by all trainees analysed for that region during six years of 2004 trainingLondonNorth EnglandScotlandSouth EnglandOtherNationalHip replacement Performed253449405637 Assisted484450415746Knee replacement Performed394244455443 Assisted483931403440External fixation Performed47.756.29.66.8 Assisted1.53.7223.12.9All hands Performed1011108710091104 Assisted423932303236All paediatrics Performed243622363834 Assisted161510181316All trauma Performed372436339462504435 Assisted698664867980.5All elective Performed418430334395406405 Assisted342298255318248303Fig. 1 A diagram showing how data synchronisation is achieved. The computers ‘talk’ to each other to check that their data is identical. If not, data is transferred by the main server at the Royal College of Surgeons of Edinburgh (RCSEd).Fig. 2 How training in total hip replacements (THRs) was supervised, by seniority of trainee from years 1 to 6 - Trainee X is compared with the national mean for trainees in the same year in training (A, assisted only; S-TS, surgeon supervised by trainer scrubbed; S-TU, surgeon supervised by trainer unscrubbed; P, performed with no in-theatre supervision; T, trained by a colleague).Fig. 3 Does trainer practice evolve with maturing trainees? The percentage of cases where the trainee performed the surgery, for each year in training.References1 Website for the Electronic Logbook. http://www.rcsed.ac.uk/logbooks (accessed 06/06/05). Google Scholar2 Data Protection Act 1998. http://www.opsi.gov.uk/acts/1998/19980029.htm (accessed 20/06/05). Google Scholar3 Newman M. Fears mount for juniors’ training. Hospital Doctor 27 January 2005:1. Google Scholar4 European working time directive. http://www.dh.gov.uk/policy and guidance/ human resources and training/working differently/European working time directive (accessed 20/06/05). Google ScholarFiguresReferencesRelatedDetailsCited byAssessing Operative Skill in the Competency-based Education Era8 October 2021 | Annals of Surgery, Vol. 275, No. 4Differences in progression by surgical specialty: a national cohort study9 February 2022 | BMJ Open, Vol. 12, No. 2An international comparison of competency-based orthopaedic curricula and minimum operative experience - Review articleInternational Journal of Surgery, Vol. 94Changing Autonomy in Operative Experience Through UK General Surgery TrainingAnnals of Surgery, Vol. 269, No. 3Does Orthopaedic Training Compromise the Outcome in Knee Joint Arthroplasty?Journal of Surgical Education, Vol. 75, No. 5Randomised trials of total hip arthroplasty for fracture is our failure to deliver symptomatic of a wider scrutiny?M. Reed, F. S. Haddad1 November 2016 | The Bone & Joint Journal, Vol. 98-B, No. 11Learning Curves in Arthroplasty in Orthopedic TraineesJournal of Surgical Education, Vol. 73, No. 4Procedure-based assessments in trauma and orthopaedic training – The trainees’ perspective4 September 2014 | Medical Teacher, Vol. 37, No. 5Expert practical operative skills teaching in Trauma and Orthopaedics at a nominal costThe Surgeon, Vol. 10, No. 6The United Kingdom and Ireland Trauma & Orthopaedic eLogbook—An evidence base for enhancing trainingThe Surgeon, Vol. 10, No. 5Handwritten undergraduate surgical logbooks12 July 2012 | The Clinical Teacher, Vol. 9, No. 4Analysing the operative experience of basic surgical trainees in Ireland using a web-based logbook25 September 2011 | BMC Medical Education, Vol. 11, No. 1Skill sets and competencies for the modern military surgeon: Lessons from UK military operations in Southern AfghanistanInjury, Vol. 41, No. 5Assessing the Surgeonʼs Technical Skills: Analysis of the Available ToolsAcademic Medicine, Vol. 85, No. 5Training in the practical application of damage control and early total care operative philosophy – perceptions of UK orthopaedic specialist traineesThe Annals of The Royal College of Surgeons of England, Vol. 92, No. 2Orthopaedic training experience in the UK and Ireland: An analysis using the elogbookThe Surgeon, Vol. 7, No. 4Orthopaedic training worldwide: how they compare? A pilot study2 July 2008 | Archives of Orthopaedic and Trauma Surgery, Vol. 129, No. 5Do Independent Sector Treatment Centres (ISTC) Impact on Specialist Registrar Training in Primary Hip and Knee Arthroplasty?The Annals of The Royal College of Surgeons of England, Vol. 90, No. 6Trauma experience in the UK and Ireland: Analysis of orthopaedic training using the FHI eLogbookInjury, Vol. 39, No. 8Current neurosurgical trainees' perception of the European Working Time Directive and shift work6 July 2009 | British Journal of Neurosurgery, Vol. 22, No. 1Assessment of performance in orthopaedic trainingD. Pitts, D. I. Rowley, J. L. Sher1 September 2005 | The Journal of Bone and Joint Surgery. British volume, Vol. 87-B, No. 9Head, hand and heart MEASURING OUR ABILITIESD. H. A. Jones1 September 2005 | The Journal of Bone and Joint Surgery. British volume, Vol. 87-B, No. 9 Vol. 87-B, No. 9 Supplemental MaterialsMetrics History Published online 1 September 2005 Published in print 1 September 2005 InformationCopyright © 2005, The British Editorial Society of Bone and Joint Surgery: All rights reservedPDF download