Automated animal behavior detection in videos

Abstract

Event Abstract Back to Event Automated animal behavior detection in videos Mayank Kabra1*, Alice Robie1, Marta Riveira-Alba1, Steve Branson2 and Kristin Branson1* 1 Howard Hughes Medical Institute, Janelia Farms Research Campus, United States 2 University of California San Diego, Computer Science, United States Video-based animal behavior quantification allows for a broader, more complete characterization of the animals' ethome, while still achieving high throughput. To this end, we have developed software that automatically detects behavior in tracked video using machine learning. In a machine learning framework, the biologist characterizes the animals' behavior by labeling a subset of frames of videos in which the animals are and are not performing the given behavior. Machine learning methods are then used to learn a classifier that can replicate this classification. Our tool has a unique user interface that allows the biologist to encode their understanding of behavior efficiently, accurately, and painlessly, without the necessity of understanding the underlying machine learning algorithm. This is achieved by combining a fast learning algorithm, an easy-to-use labeling interface, and visualizations of the learned classifier. The user labels the frames he chooses, trains the classifier, examines visualizations of the classifier's performance, chooses more frames to label, and repeats. This iterative framework has allowed us to overcome the somewhat continuous nature of behavior (particularly at behavior transitions), allowing the biologist to label only those frames that are easy to categorize. In addition, through the visualizations provided, the user can choose frames to label that will be most informative to the classfier: those that are currently classified incorrectly or with low confidence. Finally, it allows the user to easily find mislabeled examples, which can severely impact the generalization performance of the classifier. Using our software, we were able to develop behavior detectors that worked across thousands of Drosophila genotypes from the Rubin GAL4 collection in our neural activation screen. We plan to extend the tool to detect behaviors in Drosophila larvae and mice. References Kristin Branson, Alice A Robie, John Bender, Pietro Perona & Michael H Dickinson (2009), High-throughput ethomics in large groups of Drosophila, Nature Methods, 6, 451 - 457. Paul Viola and Michael Jones (2004), Robust real-time face detection, International journal of computer vision, Springer. Yoav Freund and Rob Schapire (1995), A desicion-theoretic generalization of on-line learning and an application to boosting, Computational learning theory, Springer. Keywords: Automation, Behavior Detection, machine learning, Video Analysis Conference: Tenth International Congress of Neuroethology, College Park. Maryland USA, United States, 5 Aug - 10 Aug, 2012. Presentation Type: Poster Presentation (see alternatives below as well) Topic: Computation Citation: Kabra M, Robie A, Riveira-Alba M, Branson S and Branson K (2012). Automated animal behavior detection in videos. Conference Abstract: Tenth International Congress of Neuroethology. doi: 10.3389/conf.fnbeh.2012.27.00262 Copyright: The abstracts in this collection have not been subject to any Frontiers peer review or checks, and are not endorsed by Frontiers. They are made available through the Frontiers publishing platform as a service to conference organizers and presenters. The copyright in the individual abstracts is owned by the author of each abstract or his/her employer unless otherwise stated. Each abstract, as well as the collection of abstracts, are published under a Creative Commons CC-BY 4.0 (attribution) licence (https://creativecommons.org/licenses/by/4.0/) and may thus be reproduced, translated, adapted and be the subject of derivative works provided the authors and Frontiers are attributed. For Frontiers’ terms and conditions please see https://www.frontiersin.org/legal/terms-and-conditions. Received: 30 Apr 2012; Published Online: 07 Jul 2012. * Correspondence: Dr. Mayank Kabra, Howard Hughes Medical Institute, Janelia Farms Research Campus, Ashburn, Virginia, 20147, United States, kabram@janelia.hhmi.org Dr. Kristin Branson, Howard Hughes Medical Institute, Janelia Farms Research Campus, Ashburn, Virginia, 20147, United States, bransonk@janelia.hhmi.org Login Required This action requires you to be registered with Frontiers and logged in. To register or login click here. Abstract Info Abstract The Authors in Frontiers Mayank Kabra Alice Robie Marta Riveira-Alba Steve Branson Kristin Branson Google Mayank Kabra Alice Robie Marta Riveira-Alba Steve Branson Kristin Branson Google Scholar Mayank Kabra Alice Robie Marta Riveira-Alba Steve Branson Kristin Branson PubMed Mayank Kabra Alice Robie Marta Riveira-Alba Steve Branson Kristin Branson Related Article in Frontiers Google Scholar PubMed Abstract Close Back to top Javascript is disabled. Please enable Javascript in your browser settings in order to see all the content on this page.