Strategies to alleviate flickering: Bayesian and smoothing methods for deep learning classification in video

Abstract

PurposeIncreasing reliance on autonomous systems requires confidence in the accuracies produced from computer vision classification algorithms. Computer vision (CV) for video classification provides phenomenal abilities, but it often suffers from “flickering” of results. Flickering occurs when the CV algorithm switches between declared classes over successive frames. Such behavior causes a loss of trust and confidence in their operations.Design/methodology/approachThis “flickering” behavior often results from CV algorithms treating successive observations as independent, which ignores the dependence inherent in most videos. Bayesian neural networks are a potential remedy to this issue using Bayesian priors. This research compares a traditional video classification neural network to its Bayesian equivalent based on performance and capabilities. Additionally, this work introduces the concept of smoothing to reduce the opportunities for “flickering.”FindingsThe augmentation of Bayesian layers to CNNs matched with an exponentially decaying weighted average for classifications demonstrates promising benefits in reducing flickering. In the best case the proposed Bayesian CNN model reduces flickering by 67% while maintaining both overall accuracy and class level accuracy.Research limitations/implicationsThe training of the Bayesian CNN is more computationally demanding and the requirement to classify frames multiple times reduces resulting framerate. However, for some high surety mission applications this is a tradeoff the decision analyst may be willing to make.Originality/valueOur research expands on previous efforts by first using a variable number of frames to produce the moving average as well as by using an exponentially decaying moving average in conjunction with Bayesian augmentation.