Validation of a Visual-Spatial Secondary Task to Assess Automaticity in Laparoscopic Skills

Abstract

Our objective was to assess reliability and validity of a visual-spatial secondary task (VSST) as a method to measure automaticity on a basic simulated laparoscopic skill model. In motor skill acquisition, expertise is defined by automaticity. The highest level of performance with less cognitive and attentional resources characterizes this stage, allowing experts to perform multiple tasks. Conventional validated parameters as operative time, objective assessment skills scales (OSATS), and movement economy, are insufficient to distinguish if an individual has reached the more advanced learning phases, such as automaticity. There is literature about using a VSST as an attention indicator that correlates with the automaticity level. Novices with completed and approved Fundamentals of Laparoscopic Surgery course, and laparoscopy experts were enrolled for an experimental study and measured under dual tasks conditions. Each participant performed the test giving priority to the primary task while at the same time they responded to a VSST. The primary task consisted of 4 interrupted laparoscopic stitches (ILS) on a bench-model. The VSST was a screen that showed different patterns that the surgeon had to recognize and press a pedal while doing the stitches (PsychoPsy software, Python, MacOS). Novices were overtrained on ILS until they reach at least 100 repetitions and then were retested. Participants were video recorded and then assessed by 2 blinded evaluators who measured operative time and OSATS. These scores were considered indicators of quality for the primary task. The VSST performance was measured by the detectability index (DI), which is a ratio between correct and wrong detections. A reliable evaluation was defined as two measures of DI with less than 10% of difference, maintaining the cutoff scores for performance on the primary task (operative time <110 seg and OSATS >17 points). Novices (n = 11) achieved reliable measure of the test after 2 (2-5) repetitions on the preassessment and 3.75 (2-5) on the postassessment (p = 0.04); whereas laparoscopy experts (n = 4) did it after 3.5 (3-4) repetitions. Proficiency cutoff scores for the primary task were achieved on every measure for novices (prepost overtraining) and experts. Expert performance on VSST was DI 0.78 (0.69-0.87). Novice performance was significantly better on postassessment (DI-pre 0.48 [0.06-0.71] vs DI-post 0.78 [0.48-0.95], p = 0.003). Overtraining consisted in 140 (100-210) repetitions of ILS for all novices, made in 8 hours (3-15). By categorizing DI based on expert performance, novices with DI-post >0.65 achieved better OSATS score and less operative time than novices with DI-post<0.65 (p = 0.007 y, p = 0.089, respectively). Measuring automaticity is feasible using a VSST. This instrument is reliable and has a face, content and construct validity. A DI over 0.65 may be a cutoff point correlated with high standard performance on the primary task. This instrument measures performance on laparoscopic skills, and along with conventional indicators, would better define advance levels of expertise. More studies are required applying this VSST to achieve external validity by reproducing our results.