Abstract
Scoring reflex responsiveness and injury of aquatic organisms has gained popularity as predictors of discard survival. Given this method relies upon the individual interpretation of scoring criteria, an evaluation of its robustness is done here to test whether protocol-instructed, multiple raters with diverse backgrounds (research scientist, technician, and student) are able to produce similar or the same reflex and injury score for one of the same flatfish (European plaice, Pleuronectes platessa) after experiencing commercial fishing stressors. Inter-rater reliability for three raters was assessed by using a 3-point categorical scale (‘absent’, ‘weak’, ‘strong’) and a tagged visual analogue continuous scale (tVAS, a 10 cm bar split in three labelled sections: 0 for ‘absent’, ‘weak’, ‘moderate’, and ‘strong’) for six reflex responses, and a 4-point scale for four injury types. Plaice (n = 304) were sampled from 17 research beam-trawl deployments during four trips. Fleiss kappa (categorical scores) and intra-class correlation coefficients (ICC, continuous scores) indicated variable inter-rater agreement by reflex type (ranging between 0.55 and 0.88, and 67% and 91% for Fleiss kappa and ICC, respectively), with least agreement among raters on extent of injury (Fleiss kappa between 0.08 and 0.27). Despite differences among raters, which did not significantly influence the relationship between impairment and predicted survival, combining categorical reflex and injury scores always produced a close relationship of such vitality indices and observed delayed mortality. The use of the continuous scale did not improve fit of these models compared with using the reflex impairment index based on categorical scores. Given these findings, we recommend using a 3-point categorical over a continuous scale. We also determined that training rather than experience of raters minimised inter-rater differences. Our results suggest that cost-efficient reflex impairment and injury scoring may be considered a robust technique to evaluate lethal stress and damage of this flatfish species on-board commercial beam-trawl vessels.
Highlights
Observer rating of an animal’s condition is an established tool in animal welfare research to integrate multi-modal information and describe complex behaviours, which may be difficult to measure otherwise [1]
Our results indicate that low cost reflex impairment and injury scoring can be considered a robust technique to evaluate lethal stress and damage of a fish intended for discard given that rater-based reflex and reflex & injury indices were always significantly associated with mortality probability, despite some inter-rater differences, especially among injury scores
Our results further corroborate widespread evidence that reflex impairment is a strong predictor of survival probability amongst discarded organisms [6,7, 24], and that the fit of impairment indices with survival can be improved, by incorporating injury scores to fine-tune predictions [6, 10]
Summary
Observer rating of an animal’s condition (e.g. visually evaluating physical, physiological, and behavioural states) is an established tool in animal welfare research to integrate multi-modal information and describe complex behaviours, which may be difficult to measure otherwise [1]. To evaluate vitality and extent of injury of whole animals, the condition status of captured organisms can be assessed by observers (here ‘raters’) Their ratings involve abstraction and interpretation of criteria, which might be influenced by experience, cognitive ability, and other personal traits, and might be biased [1, 2]. The effect of such interrater variability on the application of the ratings has to be quantified, before any index can be recommended [1, 3,4]. Despite differences among rater’s scores, inter-rater reliability might still be high, if, for example, scoring differences are consistent [2]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Free) 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
