Wildlife Video Research Articles

Bio-Inspired Intelligent Swarm Confrontation Algorithm for a Complex Urban Scenario

This paper considers the confrontation problem for two tank swarms of equal size and capability in a complex urban scenario. Based on the Unity platform (2022.3.20f1c1), the confrontation scenario is constructed featuring multiple crossing roads. Through the analysis of a substantial amount of biological data and wildlife videos regarding animal behavioral strategies during confrontations for hunting or food competition, two strategies are been utilized to design a novel bio-inspired intelligent swarm confrontation algorithm. The first one is the “fire concentration” strategy, which assigns a target for each tank in a way that the isolated opponent will be preferentially attacked with concentrated firepower. The second one is the “back and forth maneuver” strategy, which makes the tank tactically retreat after firing in order to avoid being hit when the shell is reloading. Two state-of-the-art swarm confrontation algorithms, namely the reinforcement learning algorithm and the assign nearest algorithm, are chosen as the opponents for the bio-inspired swarm confrontation algorithm proposed in this paper. Data of comprehensive confrontation tests show that the bio-inspired swarm confrontation algorithm has significant advantages over its opponents from the aspects of both win rate and efficiency. Moreover, we discuss how vital algorithm parameters would influence the performance indices.

Action Detection for Wildlife Monitoring with Camera Traps Based on Segmentation with Filtering of Tracklets (SWIFT) and Mask-Guided Action Recognition (MAROON)

Behavioral analysis of animals in the wild plays an important role for ecological research and conservation and has been mostly performed by researchers. We introduce an action detection approach that automates this process by detecting animals and performing action recognition on the detected animals in camera trap videos. Our action detection approach is based on SWIFT (segmentation with filtering of tracklets), which we have already shown to successfully detect and track animals in wildlife videos, and MAROON (mask-guided action recognition), an action recognition network that we are introducing here. The basic ideas of MAROON are the exploitation of the instance masks detected by SWIFT and a triple-stream network. The instance masks enable more accurate action recognition, especially if multiple animals appear in a video at the same time. The triple-stream approach extracts features for the motion and appearance of the animal. We evaluate the quality of our action recognition on two self-generated datasets, from an animal enclosure and from the wild. These datasets contain videos of red deer, fallow deer and roe deer, recorded both during the day and night. MAROON improves the action recognition accuracy compared to other state-of-the-art approaches by an average of 10 percentage points on all analyzed datasets and achieves an accuracy of 69.16% on the Rolandseck Daylight dataset, in which 11 different action classes occur. Our action detection system makes it possible todrasticallyreduce the manual work of ecologists and at the same time gain new insights through standardized results.

Towards Automated Ethogramming: Cognitively-Inspired Event Segmentation for Streaming Wildlife Video Monitoring

Advances in visual perceptual tasks have been mainly driven by the amount, and types, of annotations of large-scale datasets. Researchers have focused on fully-supervised settings to train models using offline epoch-based schemes. Despite the evident advancements, limitations and cost of manually annotated datasets have hindered further development for event perceptual tasks, such as detection and localization of objects and events in videos. The problem is more apparent in zoological applications due to the scarcity of annotations and length of videos-most videos are at most ten minutes long. Inspired by cognitive theories, we present a self-supervised perceptual prediction framework to tackle the problem of temporal event segmentation by building a stable representation of event-related objects. The approach is simple but effective. We rely on LSTM predictions of high-level features computed by a standard deep learning backbone. For spatial segmentation, the stable representation of the object is used by an attention mechanism to filter the input features before the prediction step. The self-learned attention maps effectively localize the object as a side effect of perceptual prediction. We demonstrate our approach on long videos from continuous wildlife video monitoring, spanning multiple days at 25 FPS. We aim to facilitate automated ethogramming by detecting and localizing events without the need for labels. Our approach is trained in an online manner on streaming input and requires only a single pass through the video, with no separate training set. Given the lack of long and realistic (includes real-world challenges) datasets, we introduce a new wildlife video dataset–nest monitoring of the Kagu (a flightless bird from New Caledonia)–to benchmark our approach. Our dataset features a video from 10 days (over 23 million frames) of continuous monitoring of the Kagu in its natural habitat. We annotate every frame with bounding boxes and event labels. Additionally, each frame is annotated with time-of-day and illumination conditions. We will make the dataset, which is the first of its kind, and the code available to the research community. We find that the approach significantly outperforms other self-supervised, traditional (e.g., Optical Flow, Background Subtraction) and NN-based (e.g., PA-DPC, DINO, iBOT), baselines and performs on par with supervised boundary detection approaches (i.e., PC). At a recall rate of 80%, our best performing model detects one false positive activity every 50 min of training. On average, we at least double the performance of self-supervised approaches for spatial segmentation. Additionally, we show that our approach is robust to various environmental conditions (e.g., moving shadows). We also benchmark the framework on other datasets (i.e., Kinetics-GEBD, TAPOS) from different domains to demonstrate its generalizability. The data and code are available on our project page: https://aix.eng.usf.edu/research_automated_ethogramming.html

Instance segmentation and tracking of animals in wildlife videos: SWIFT - segmentation with filtering of tracklets

Reliably detecting and tracking animals in wildlife videos is an essential basis for researchers to be able to analyse animal behavior or recognize animal individuals. In order to correctly distinguish individual animals that are standing close to each other, bounding boxes around the animals are not sufficient. Instead, an exact contour of the animal, an instance mask, which is the result of an instance segmentation, is needed. In this paper, we present SWIFT, a novel multi-object tracking and segmentation (MOTS) pipeline that solves this task. We evaluate the functionality of our approach on a self-created wildlife video dataset containing red deer and fallow deer. Our dataset is one of the very few datasets in wildlife monitoring that is annotated with instance masks and tracking IDs. SWIFT significantly improves the quality of the instance masks compared to using a state-of-the-art instance segmentation approach from 0.432 average precision to 0.495 average precision. Our tracking algorithm uses multiple filtering steps to either delete tracks that are found incorrectly or to merge tracks that are not yet connected. This results in an increased multi-object tracking accuracy score in comparison to a state-of-the-art tracking approach from 57.2% to 63.8%, which means that our detected tracking results are less erroneous.

Drone Observations of Marine Life and Human–Wildlife Interactions off Sydney, Australia

Drones have become popular with the general public for viewing and filming marine life. One amateur enthusiast platform, DroneSharkApp, films marine life in the waters off Sydney, Australia year-round and posts their observations on social media. The drone observations include the behaviours of a variety of coastal marine wildlife species, including sharks, rays, fur seals, dolphins and fish, as well as migratory species such as migrating humpback whales. Given the extensive effort and multiple recordings of the presence, behaviour and interactions of various species with humans provided by DroneSharkApp, we explored its utility for providing biologically meaningful observations of marine wildlife. Using social media posts from the DroneSharkApp Instagram page, a total of 678 wildlife videos were assessed from 432 days of observation collected by a single observer. This included 94 feeding behaviours or events for fur seals (n = 58) and dolphins (n = 33), two feeding events for white sharks and one feeding event for a humpback whale. DroneSharkApp documented 101 interactions with sharks and humans (swimmers and surfers), demonstrating the frequent, mainly innocuous human–shark overlap off some of Australia’s busiest beaches. Finally, DroneSharkApp provided multiple observations of humpback and dwarf minke whales with calves travelling north, indicating calving occurring well south of traditional northern Queensland breeding waters. Collaboration between scientists and citizen scientists such as those involved with DroneSharkApp can greatly and quantitatively increase the biological understanding of marine wildlife data.

Saving costs for video data annotation in wildlife monitoring

In wildlife monitoring, large amounts of video data are generated by recordings from camera traps. Training of deep learning methods demands for annotated video data, i.e. video data where each frame is annotated with the correct number and species designation of the observed animals. But manual annotation of video clips is extremely time-consuming and laborious. In this proof of concept we compare three different state-of-the-art approaches to the annotation of video data: Manual annotation using the VGG Image Annotator, interactive annotation using the MiVOS video annotator and automated annotation utilizing an adapted and customized Tracktor approach that propagates annotations from frame to frame through complete video clips. An experimental proof of concept on wildlife video clips captured by camera traps show extreme time savings from hours down to minutes (i.e. in order of a magnitude) thereby not only maintaining the detection scores of animals in each frame but also improving detection scores from 54.7% to 58.5% compared to the employment of perfect but costly manual annotations in training.

Identification of animals and recognition of their actions in wildlife videos using deep learning techniques

Biodiversity crisis has continued to accelerate. Studying animal distribution, movement and behaviour is of critical importance to address environmental challenges such as spreading of diseases, invasive species, climate and land-use change. Camera traps are an appropriate technique for continuous animal monitoring in an automated 24/7/52 documentation. This study shows a proof-of-concept for an end-to-end pipeline to detect and classify animals and their behaviour in video clips. Video clips are captured with 8 frames per second by camera traps using infrared cameras and infrared flash-lights. The clips show deer, boars, foxes and hares - mostly at night time. Our approach shows an average precision of 63.8% for animal detection and identification. For action recognition the achieved accuracies range between 88.4% and 94.1%.

Automatic Identification of Individual Primates with Deep Learning Techniques.

SummaryThe difficulty of obtaining reliable individual identification of animals has limited researcher's ability to obtain quantitative data to address important ecological, behavioral, and conservation questions. Traditional marking methods placed animals at undue risk. Machine learning approaches for identifying species through analysis of animal images has been proved to be successful. But for many questions, there needs a tool to identify not only species but also individuals. Here, we introduce a system developed specifically for automated face detection and individual identification with deep learning methods using both videos and still-framed images that can be reliably used for multiple species. The system was trained and tested with a dataset containing 102,399 images of 1,040 individuals across 41 primate species whose individual identity was known and 6,562 images of 91 individuals across four carnivore species. For primates, the system correctly identified individuals 94.1% of the time and could process 31 facial images per second.

Neural Patterns are More Similar across Individuals during Successful Memory Encoding than during Failed Memory Encoding

After experiencing the same episode, some people can recall certain details about it, whereas others cannot. We investigate how common (intersubject) neural patterns during memory encoding influence whether an episode will be subsequently remembered, and how divergence from a common organization is associated with encoding failure. Using functional magnetic resonance imaging with intersubject multivariate analyses, we measured brain activity as people viewed episodes within wildlife videos and then assessed their memory for these episodes. During encoding, greater neural similarity was observed between the people who later remembered an episode (compared with those who did not) within the regions of the declarative memory network (hippocampus, posterior medial cortex [PMC], and dorsal Default Mode Network [dDMN]). The intersubject similarity of the PMC and dDMN was episode-specific. Hippocampal encoding patterns were also more similar between subjects for memory success that was defined after one day, compared with immediately after retrieval. The neural encoding patterns were sufficiently robust and generalizable to train machine learning classifiers to predict future recall success in held-out subjects, and a subset of decodable regions formed a network of shared classifier predictions of subsequent memory success. This work suggests that common neural patterns reflect successful, rather than unsuccessful, encoding across individuals.

Reliable Multi-Object Tracking Model Using Deep Learning and Energy Efficient Wireless Multimedia Sensor Networks

Presently, sensor-cloud based environment becomes highly beneficial due to its applicability in several domains. Wireless multimedia sensor network (WMSN) is one among them, which involves a set of multimedia sensors to collect data about the deployed region. Compared to traditional object tracking models, animal tracking in WMSN is a tedious process owing to the harsh, dynamic, and energy limited sensors. This article introduces a new Reliable Multi-Object Tracking Model using Deep Learning (DL) and Energy Efficient WMSN. Initially, the fuzzy logic technique is employed to determine the cluster heads (CHs) to attain energy efficiency. Next, in the second stage, a novel tracking algorithm by the use of Recurrent Neural Network (RNN) with a tumbling effect called RNN-T is developed. The proposed RNN-T model gets executed by every sensor node and the CHs execute the tracking algorithm to track the animals. Finally, the tracking results are transmitted to the cloud server for investigation purposes. In order to assess the performance of the presented model, an extensive experimental analysis is carried out by the use of a real-time wildlife video. The obtained results ensured that the RNN-T model has achieved better performance over the compared methods in different aspects.

A camera‐based method for estimating absolute density in animals displaying home range behaviour

The measurement of animal density may take advantage of recent technological achievements in wildlife video recording. Fostering the theoretical links between the patterns depicted by cameras and absolute density is required to exploit this potential. We explore the applicability of the Hutchinson-Waser's postulate (i.e. when animal density is stationary at a given temporal and spatial scale, the absolute density is given by the average number of animals counted per frame), which is a counter-intuitive statement for most ecologists and managers who are concerned with counting the same individual more than once. We aimed to reconcile such scepticism for animals displaying home range behaviour. The specific objectives of this paper are to generalize the Hutchinson-Waser's postulate for animals displaying home range behaviour and to propose a Bayesian implementation to estimate density from counts per frame using video cameras. Accuracy and precision of the method was evaluated by means of computer simulation experiments. Specifically, six animal archetypes displaying well-contrasted movement features were considered. The simulation results demonstrate that density could be accurately estimated after an affordable sampling effort (i.e. number of cameras and deployment time) for a great number of animals across taxa. The proposed method may complement other conventional methods for estimating animal density. The major advantages are that identifying an animal at the individual level and precise knowledge on how animals move are not needed, and that density can be estimated in a single survey. The method can accommodate conventional camera trapping data. The major limitations are related to some implicit assumptions of the underlying model: the home range centres should be homogeneously distributed, the detection probability within the area surveyed by the camera should be known, and animals should move independently to one another. Further improvements for circumventing these limitations are discussed.

Establishing the fundamentals for an elephant early warning and monitoring system.

BackgroundThe decline of habitat for elephants due to expanding human activity is a serious conservation problem. This has continuously escalated the human–elephant conflict in Africa and Asia. Elephants make extensive use of powerful infrasonic calls (rumbles) that travel distances of up to several kilometers. This makes elephants well-suited for acoustic monitoring because it enables detecting elephants even if they are out of sight. In sight, their distinct visual appearance makes them a good candidate for visual monitoring. We provide an integrated overview of our interdisciplinary project that established the scientific fundamentals for a future early warning and monitoring system for humans who regularly experience serious conflict with elephants. We first draw the big picture of an early warning and monitoring system, then review the developed solutions for automatic acoustic and visual detection, discuss specific challenges and present open future work necessary to build a robust and reliable early warning and monitoring system that is able to operate in situ.FindingsWe present a method for the automated detection of elephant rumbles that is robust to the diverse noise sources present in situ. We evaluated the method on an extensive set of audio data recorded under natural field conditions. Results show that the proposed method outperforms existing approaches and accurately detects elephant rumbles. Our visual detection method shows that tracking elephants in wildlife videos (of different sizes and postures) is feasible and particularly robust at near distances.DiscussionFrom our project results we draw a number of conclusions that are discussed and summarized. We clearly identified the most critical challenges and necessary improvements of the proposed detection methods and conclude that our findings have the potential to form the basis for a future automated early warning system for elephants. We discuss challenges that need to be solved and summarize open topics in the context of a future early warning and monitoring system. We conclude that a long-term evaluation of the presented methods in situ using real-time prototypes is the most important next step to transfer the developed methods into practical implementation.

Automated detection of elephants in wildlife video

Biologists often have to investigate large amounts of video in behavioral studies of animals. These videos are usually not sufficiently indexed which makes the finding of objects of interest a time-consuming task. We propose a fully automated method for the detection and tracking of elephants in wildlife video which has been collected by biologists in the field. The method dynamically learns a color model of elephants from a few training images. Based on the color model, we localize elephants in video sequences with different backgrounds and lighting conditions. We exploit temporal clues from the video to improve the robustness of the approach and to obtain spatial and temporal consistent detections. The proposed method detects elephants (and groups of elephants) of different sizes and poses performing different activities. The method is robust to occlusions (e.g., by vegetation) and correctly handles camera motion and different lighting conditions. Experiments show that both near- and far-distant elephants can be detected and tracked reliably. The proposed method enables biologists efficient and direct access to their video collections which facilitates further behavioral and ecological studies. The method does not make hard constraints on the species of elephants themselves and is thus easily adaptable to other animal species.

Wildlife video key-frame extraction based on novelty detection in semantic context

There is a growing evidence that visual saliency can be better modeled using top-down mechanisms that incorporate object semantics. This suggests a new direction for image and video analysis, where semantics extraction can be effectively utilized to improve video summarization, indexing and retrieval. This paper presents a framework that models semantic contexts for key-frame extraction. Semantic context of video frames is extracted and its sequential changes are monitored so that significant novelties are located using a one-class classifier. Working with wildlife video frames, the framework undergoes image segmentation, feature extraction and matching of image blocks, and then a co-occurrence matrix of semantic labels is constructed to represent the semantic context within the scene. Experiments show that our approach using high-level semantic modeling achieves better key-frame extraction as compared with its counterparts using low-level features.

WebWatch

BioTechniquesVol. 50, No. 1 WebWatchOpen AccessWebWatchKevin AhernKevin AhernSearch for more papers by this authorPublished Online:28 Jun 2018https://doi.org/10.2144/000113583AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinkedInReddit Directed CurrentsStaying current with scientific literature is a must for research scientists, but there is more to it than just reading journals. Helping both scientists and nonscientists see the “forest” of biology for its individual “trees” is biologynews.net, an up-to-the-minute news service focusing on fast-breaking coverage of the biological sciences. The opening page of the site has teaser headlines with a sentence or two of introduction and links to full articles. A lot of information is jammed into a relatively small space using this format, leaving the visitor with a lot of options. The site averages 2000 article views per day. Information is available in RSS feeds and the code is provided for streaming headlines across visitors’ own web sites.[www.biologynews.net]The Soul of SolanaceaeEconomically it's the third most important plant taxon—containing over 3000 species that evolved in the Andean/Amazonian areas of South America—and it contains the most variety of crops in terms of agricultural utility. Some grow above ground, others below. Some are delicious. Others are deadly poisonous. There are reports they may also be linked to cancer. The group is the Solanaceae family, which includes tomatoes, potatoes, eggplants, peppers, and even tobacco. Gathering up and reporting information about this diverse collection of species is the Sol Genomics Network, which keeps visitors abreast of genomic developments, phenoltypes, relevant genes, pathways. It even has a section for plant breeders.[http://solgenomics.net]Population BombForget nuclear bombs, the ozone hole, bioterrorism, or an economic meltdown. The greatest danger to a healthy Earth is the population bomb. At least that's what a growing number of endorsers of the message at Global Population Speak Out are poised to say. The web site reveals a ticking count of humans on Earth in real time and it's a bit unsettling. The designers of the site suggest the size of the human population is the root of environmental problems, and they have a plan for making the public more aware of the problem. During February 2011, a planet wide Speak Out about the problem is being organized. Professionals and concerned visitors can sign up to participate.[www.populationspeakout.org]The Nature of Mother NatureJolle Jolles is a young scientist who brings his love of “our weird and fascinating natural world” to everyone in the form of an interesting set of pages known as Mudfooted.com. He is also a prime example of how a diverse upbringing can lead to great things. Jolles's self-described journey through life, with world travels that include stops in the Netherlands, New Zealand, and Australia, field work in Borneo and South Africa, and time at Cambridge University is mirrored in his delightful web site. The highlight of the site just might be its collection of the most jaw-dropping wildlife videos to be found anywhere on the Web. Here visitors will discover the Jesus Christ lizard (walks on water), Bob (the fish that loves to be petted), and the intelligent use of tools by birds. Thirteen pages of videos and an interesting article archive await curiosity seekers.[http://mudfooted.com]A proboscis monkey jumping over a river.© 2010 by J. Jolles, www.mudfooted.com.PhosphotargetingProtein phosphorylation is a fairly simple phenomenon. The product of action of a kinase, the process adds phosphate to a protein's amino acids with hydroxyl side chains. These, of course, include serine, threonine, and tyrosine. The problem, though, is that there are many such residues in any given polypeptide and not every one of them is acted on. What, then, is the target of the kinases? That's the $64,000 question that Scansite attempts to answer. Tools to determine kinase targets include Motif scans of input or database sequences, database searches using a given motif, sequence pattern matches, and simple protein info determination (molecular weight, pI, and amino acid composition around known phosphorylation sites).[http://scansite.mit.edu]Tech TextingDavid Bradley is a self-described “science communicator since 1989,” who aims with his web site to (i) improve science literacy and numeracy in the media and (ii) encourage streaming, blogging and browsing for other science bloggers by revealing computing tricks and hacks he has uncovered in the process of running his site. In furtherance of his goals, David's insightful views on technology, science, and other topics abound. Indeed, the amount of writing, editing, and collecting he does on a daily basis to stay on top of his site is impressive. If you want a refreshing perspective focused on techno-biology, this is the place to find it.[www.sciencetext.com]FiguresReferencesRelatedDetails Vol. 50, No. 1 Follow us on social media for the latest updates Metrics History Published online 28 June 2018 Published in print January 2011 Information© 2011 Author(s)PDF download

Analysing animal behaviour in wildlife videos using face detection and tracking

An algorithm that categorises animal locomotive behaviour by combining detection and tracking of animal faces in wildlife videos is presented. As an example, the algorithm is applied to lion faces. The detection algorithm is based on a human face detection method, utilising Haar-like features and AdaBoost classifiers. The face tracking is implemented by applying a specific interest model that combines low-level feature tracking with the detection algorithm. By combining the two methods in a specific tracking model, reliable and temporally coherent detection/tracking of animal faces is achieved. The information generated by the tracker is used to automatically annotate the animal's locomotive behaviour. The annotation classes of locomotive processes for a given animal species are predefined by a large semantic taxonomy on wildlife domain. The experimental results are presented.

A semantic event-detection approach and its application to detecting hunts in wildlife video

We propose a three-level video-event detection methodology and apply it to animal-hunt detection in wildlife documentaries. The first level extracts color, texture, and motion features, and detects shot boundaries and moving object blobs. The mid-level employs a neural network to determine the object class of the moving object blobs. This level also generates shot descriptors that combine features from the first level and inferences from the mid-level. The shot descriptors are then used by the domain-specific inference process at the third level to detect video segments that match the user defined event model. The proposed approach has been applied to the detection of hunts in wildlife documentaries. Our method can be applied to different events by adapting the classifier at the intermediate level and by specifying a new event model at the highest level. Event-based video indexing, summarization, and browsing are among the applications of the proposed approach.

METHODOLOGICAL CONSIDERATIONS FOR MONITORING WILD BIRD NESTS USING VIDEO TECHNOLOGY

While testing a commercially manufactured wildlife video system, we developed a method for using this technology to monitor continuously the reproductive behaviors of wild birds during the nesting period. Two passerine bird nests, Northern Cardinal (Cardinalis cardinalis) and Red-winged Blackbird (Agelaius phoeniceus), were monitored using miniature video cameras at the nests. Cameras were connected to a 24-h time-lapse video recorder located up to 18 m from each nest. System installation required approximately 20 min and nests were monitored for 20 and 15 d, respectively. Significant events recorded on videotape included: egg laying, incubation, egg hatching, nestling rearing, fledging, mammalian depredation of inviable eggs after fledging, and reptilian depredation of nestlings prior to fledging. This technology reliably provided a highly detailed, non-intrusive means of observing nesting behavior and predation events equally well during day and night operations and under extreme environmental conditions. Each commercially manufactured video system and associated equipment used in this study cost approximately US $4000.