Aerial Video Footage Research Articles

Short Communication: Lateralised lungefeeding behaviour in Southern Hemisphere fin whales at Antarctic feeding grounds

Balaenopterids use dynamic feeding lunges to capture prey, often performing roll manoeuvres which may demonstrate a side preference or laterality. In this study, we use aerial video footage of fin whales (Balaenoptera physalus) feeding at Elephant Island, Antarctica, to investigate laterality in the Southern Hemisphere fin whale subspecies B. p. quoyi. Right‐biased lateralisation was observed in all 63 recorded side lunges, 57 of which displayed visible lunge directions, the majority being clockwise (98.3%). These results complement existing information on lateralised feeding behaviour in Northern Hemisphere fin whales, confirming a strong right‐sided bias across the different fin whale subspecies.

Using YOLO Object Detection to Identify Hare and Roe Deer in Thermal Aerial Video Footage—Possible Future Applications in Real-Time Automatic Drone Surveillance and Wildlife Monitoring

Wildlife monitoring can be time-consuming and expensive, but the fast-developing technologies of uncrewed aerial vehicles, sensors, and machine learning pave the way for automated monitoring. In this study, we trained YOLOv5 neural networks to detect points of interest, hare (Lepus europaeus), and roe deer (Capreolus capreolus) in thermal aerial footage and proposed a method to manually assess the parameter mean average precision (mAP) compared to the number of actual false positive and false negative detections in a subsample. This showed that a mAP close to 1 for a trained model does not necessarily mean perfect detection and provided a method to gain insights into the parameters affecting the trained models’ precision. Furthermore, we provided a basic, conceptual algorithm for implementing real-time object detection in uncrewed aircraft systems equipped with thermal sensors, high zoom capabilities, and a laser rangefinder. Real-time object detection is becoming an invaluable complementary tool for the monitoring of cryptic and nocturnal animals with the use of thermal sensors.

Preliminary Evaluation of a System with On-Body and Aerial Sensors for Monitoring Working Dogs.

This paper presents a system for behavioral, environmental, and physiological monitoring of working dogs using on-body and aerial sensors. The proof of concept study presented here includes two trained dogs performing nine scent detection tasks in an uncontrolled environment encompassing approximately two acres. The dogs were outfitted with a custom designed wearable harness to monitor their heart rate, activity levels and skin temperature. We utilized a commercially available micro-air vehicle to perform aerial sensing by tracking the terrain and movement of the dog in the outdoor space. The dogs were free to explore the space working at maximal speeds to complete a scent-based search-and-retrieval task. Throughout the experiment, the harness data was transferred to a base station via Wi-Fi in real-time. In this work, we also focused on testing the performance of a custom 3D electrode with application specific ergonomic improvements and adaptive filter processing techniques to recover as much electrocardiography data as possible during high intensity motion activity. We were able to recover and use 84% of the collected data where we observed a trend of heart rate generally increasing immediately after successful target localization. For tracking the dogs in the aerial video footage, we applied a state-of-the-art deep learning algorithm designed for online object tracking. Both qualitative and quantitative tracking results are very promising. This study presents an initial effort towards deployment of on-body and aerial sensors to monitor the working dogs and their environments during scent detection and search and rescue tasks in order to ensure their welfare, enable novel dog-machine interfaces, and allow for higher success rate of remote and automated task performance.

Predation in the Anthropocene: Harbour Seal (Phoca vitulina) Utilising Aquaculture Infrastructure as Refuge to Evade Foraging Killer Whales (Orcinus orca)

The rapid emergence of new marine developments (e.g., marine renewables, port infrastructure) alongside the substantial growth of existing industries has ultimately resulted in an unprecedented increase in anthropogenic structures within the marine environment over the previous century. Knowledge of whether marine species interact with, avoid, or accommodate and adapt to such structures is essential to ensure that further development of marine environments do not compromise conservation objectives of marine species. This article documents one such interaction. Herein, we describe the observation of a harbour seal (Phoca vitulina) seeking refuge from a group of foraging killer whales (Orcinus orca) within a blue (aka common) mussel (Mytilus edulis) farm. Aerial video footage (38 min 27 s) was collected using an unmanned aerial system during an encounter at an aquaculture site in Gulberwick Bay, Shetland, UK. Analysis of the footage showed the killer whale group spent 73.7% of the total encounter time exhibiting predatory associated behaviours and that they were observed interacting with the mussel farm infrastructure only during “predation activity” for a total of 26 min 52 s (72.8%). The harbour seal interacted with the mussel farm infrastructure during re- and proactive anti-predator behaviour and when exhibiting fatigue for 27 min 59 s, 94.4% of the total time the seal was observed. It is clear that both marine and terrestrial predator–prey interactions are increasingly occurring in settings that are in some way defined by the Anthropocene. The implications of this are discussed, including potential entanglement risk and human-altered “landscapes of fear.” As comprehension of the potential effects of human-altered risk grows, such knowledge should be taken into consideration prior to further modification of marine habitats.

Putting eagle rays on the map by coupling aerial video-surveys and deep learning

Reliable and efficient techniques are urgently needed to monitor elasmobranch populations that face increasing threats worldwide. Aerial video-surveys provide precise and verifiable observations for the rapid assessment of species distribution and abundance in coral reefs, but the manual processing of videos is a major bottleneck for timely conservation applications. In this study, we applied deep learning for the automated detection and mapping of vulnerable eagle rays from aerial videos. A light aircraft dedicated to touristic flights allowed us to collect 42 h of aerial video footage over a shallow coral lagoon in New Caledonia (Southwest Pacific). We extracted the videos at a rate of one image per second before annotating them, yielding 314 images with eagle rays. We then trained a convolutional neural network with 80% of the eagle ray images and evaluated its accuracy on the remaining 20% (independent data sets). Our deep learning model detected 92% of the annotated eagle rays in a diversity of habitats and acquisition conditions across the studied coral lagoon. Our study offers a potential breakthrough for the monitoring of ray populations in coral reef ecosystems by providing a fast and accurate alternative to the manual processing of aerial videos. Our deep learning approach can be extended to the detection of other elasmobranchs and applied to systematic aerial surveys to not only detect individuals but also estimate species density in coral reef habitats.

A Rapid Technique in Evaluating Tree Health Using Lidar Sensors

Crop management is one of the key aspects in precision agriculture. On-site crop management includes a scheduled prediction and prescription of fertilization and pesticide application on specific areas of the field. A prediction of the tree health is much needed in order to decide a suitable prescription for the plant. An autonomous vehicle, equipped with at least one LiDAR (light detection and ranging) sensor, could be used not only for detecting and mapping its surrounding, but to also help evaluate tree health for early distress detection. Currently, farmers have to rely on eyesight to identify trees or plants in distress. Big-scale plantations depend on costly scheduled airborne monitoring routines, which also relies on human vision by scouring through hours of aerial video footage. Both techniques have similar weaknesses in terms of the time it takes to detect a tree in distress and the accuracy of the detection using human vision. The objective of this research is to propose a technique in evaluating tree health using a simple LiDAR sensor that is commonly used in autonomous vehicles. In order to achieve this objective, an evaluation of the different intensity characteristics of tree leaves versus fruits was carried out, both in the lab and in the field. This study has chosen oil palm trees as its subject, as the problem of health monitoring in oil palm plantations is evident. A LiDAR with a 905 nm near-infrared (NIR) laser is used to scan both individual healthy leaves in the lab and different oil palm trees in the field. Since a LiDAR sensor is normally used for ranging, a systematic process was proposed to capture the reflected intensity value of the laser beam that was transmitted to the object. This whole system can be realized using a LiDAR sensor, servo motors, and an Arduino board. Processing software was used to test and store the captured information from the sensor. Later, MATLAB was used to plot the intensity map of the leaves and oil palm tree, classify the range intensity into histograms, and calculate the leaf area index (LAI) for the oil palm trees. From the experimental results, it is found that the reflectance intensity of the leaves shows consistent range values between 155-160 magnitude both in the lab and in the field. From there, three different trees with different number of leaves were scanned and evaluated based on their LAI values. The health of the tree is then predicted, where a healthy tree is estimated to have a higher LAI value. The resulting LAI value is found to correlate with the evaluation using eyesight. This proves that although using a single-wavelength NIR laser beam provided by the LiDAR sensor, as compared to multiple wavelengths of a spectrometer, the difference between the oil palm leaves, fruits and the background noise could be determined. In the future, where multi-wavelength laser LiDAR sensors could be possible, more materials could be characterized. In conclusion, detection, mapping, and materials characterization could be done by an autonomous vehicle utilizing a LiDAR sensor, where tree health could be predicted for crop care management.

Usability of aerial video footage for 3-D scene reconstruction and structural damage assessment

Abstract. Remote sensing has evolved into the most efficient approach to assess post-disaster structural damage, in extensively affected areas through the use of spaceborne data. For smaller, and in particular, complex urban disaster scenes, multi-perspective aerial imagery obtained with unmanned aerial vehicles and derived dense color 3-D models are increasingly being used. These type of data allow the direct and automated recognition of damage-related features, supporting an effective post-disaster structural damage assessment. However, the rapid collection and sharing of multi-perspective aerial imagery is still limited due to tight or lacking regulations and legal frameworks. A potential alternative is aerial video footage, which is typically acquired and shared by civil protection institutions or news media and which tends to be the first type of airborne data available. Nevertheless, inherent artifacts and the lack of suitable processing means have long limited its potential use in structural damage assessment and other post-disaster activities. In this research the usability of modern aerial video data was evaluated based on a comparative quality and application analysis of video data and multi-perspective imagery (photos), and their derivative 3-D point clouds created using current photogrammetric techniques. Additionally, the effects of external factors, such as topography and the presence of smoke and moving objects, were determined by analyzing two different earthquake-affected sites: Tainan (Taiwan) and Pescara del Tronto (Italy). Results demonstrated similar usabilities for video and photos. This is shown by the short 2 cm of difference between the accuracies of video- and photo-based 3-D point clouds. Despite the low video resolution, the usability of these data was compensated for by a small ground sampling distance. Instead of video characteristics, low quality and application resulted from non-data-related factors, such as changes in the scene, lack of texture, or moving objects. We conclude that not only are current video data more rapidly available than photos, but they also have a comparable ability to assist in image-based structural damage assessment and other post-disaster activities.

How accurate are small drones for measuring microscopic traffic parameters?

Small Unmanned Aerial Vehicles (sUAV or drones) have been one of the latest tools for monitoring transportation infrastructure and operations. Their lower cost compared to current fixed location camera systems or Manned Aerial Vehicles (MAV) and their ability to read just their view area depending on the situation they face, make them a promising tool of collecting both macroscopic and microscopic data. However, although drone technology and computer vision techniques are advancing fast, there is little information on how accurate and reliable they are for collecting microscopic traffic data. In this paper, we examine the potential of using sUAV as part of the ITS infrastructure as a way of extracting naturalistic trajectory data from aerial video footage from a low volume four-way intersection and a pedestrian passage. Moreover, the accuracy of speed data collected from a drone compared to data collected from an On-Board Diagnostics II (OBD-II) device is examined. For this, a controlled experiment where the vehicle was driven in various speeds and the drone flew in ranging altitudes was conducted. Results show that accuracy is highly dependent on the stabilization of the video and the geo-reference procedure. Moreover, the capabilities of such systems are examined in traffic applications and the way they can be part of future transportation infrastructure is discussed.

3D Reconstructions Using Unstabilized Video Footage from an Unmanned Aerial Vehicle

Structure from motion (SFM) is a methodology for automatically reconstructing three-dimensional (3D) models from a series of two-dimensional (2D) images when there is no a priori knowledge of the camera location and direction. Modern unmanned aerial vehicles (UAV) now provide a low-cost means of obtaining aerial video footage of a point of interest. Unfortunately, raw video lacks the required information for SFM software, as it does not record exchangeable image file (EXIF) information for the frames. In this work, a solution is presented to modify aerial video so that it can be used for photogrammetry. The paper then examines how the field of view effects the quality of the reconstruction. The input is unstabilized, and distorted video footage obtained from a low-cost UAV which is then combined with an open-source SFM system to reconstruct a 3D model. This approach creates a high quality reconstruction by reducing the amount of unknown variables, such as focal length and sensor size, while increasing the data density. The experiments conducted examine the optical field of view settings to provide sufficient overlap without sacrificing image quality or exacerbating distortion. The system costs less than e1000, and the results show the ability to reproduce 3D models that are of centimeter-level accuracy. For verification, the results were compared against millimeter-level accurate models derived from laser scanning.

Adapting astronomical source detection software to help detect animals in thermal images obtained by unmanned aerial systems

ABSTRACTIn this article, we describe an unmanned aerial system equipped with a thermal-infrared camera and software pipeline that we have developed to monitor animal populations for conservation purposes. Taking a multi-disciplinary approach to tackle this problem, we use freely available astronomical source detection software and the associated expertise of astronomers, to efficiently and reliably detect humans and animals in aerial thermal-infrared footage. Combining this astronomical detection software with existing machine learning algorithms into a single, automated, end-to-end pipeline, we test the software using aerial video footage taken in a controlled, field-like environment. We demonstrate that the pipeline works reliably and describe how it can be used to estimate the completeness of different observational datasets to objects of a given type as a function of height, observing conditions, etc. – a crucial step in converting video footage to scientifically useful information such as the spatial distribution and density of different animal species. Finally, having demonstrated the potential utility of the system, we describe the steps we are taking to adapt the system for work in the field, in particular systematic monitoring of endangered species at National Parks around the world.

Numerical Simulations of Large-Scale Sediment Transport Caused by the 2011 Tohoku Earthquake Tsunami in Hirota Bay, Southern Sanriku Coast

A numerical sediment transport model (STM) was used to investigate coastal geomorphic changes that resulted from the 2011 Tohoku earthquake tsunami in Rikuzentakata City and Hirota Bay on the southern Sanriku Coast of Japan. The simulation was verified using observed inundation processes and heights, measured topographic changes and sediment deposition. Aerial video footage recorded by the Iwate Prefectural Police was also used. The results show that the numerical model was able to predict the spatial distribution and volume of erosion and deposition in Hirota Bay, as well as sediment transport processes. The effects of sediment transport on tsuneimi inundation were also investigated. Numerical results revealed that the majority of the sand dunes were eroded by the first wave, especially during the strong return flow of the receding wave. Large flows and sand dune erosions can occur elsewhere if tsunamis inundate a plain with a limited shore-normal width. These events could cause large-scale morphological changes comparable to those that occurred in Rikuzentakata City.

Sustaining persistent lava lakes: Observations from high-resolution gas measurements at Villarrica volcano, Chile

Active lava lakes – as the exposed upper part of magmatic columns – are prime locations to investigate the conduit flow processes operating at active, degassing volcanoes. Persistent lava lakes require a constant influx of heat to sustain a molten state at the Earth's surface. Several mechanisms have been proposed to explain how such heat transfer can operate efficiently. These models make contrasting predictions with respect to the flow dynamics in volcanic conduits and should result in dissimilar volatile emissions at the surface. Here we look at high-frequency SO2 fluxes, plume composition, thermal emissions and aerial video footage from the Villarrica lava lake in order to determine the mechanism sustaining its activity. We found that while fluctuations are apparent in all datasets, none shows a stable periodic behaviour. These observations suggest a continuous influx of volatiles and magma to the Villarrica lava lake. We suggest that ascending volatile-rich and descending degassed magmas are efficiently mixed within the volcanic conduit, resulting in no clear periodic oscillations in the plume composition and flux. We compare our findings to those of other lava lakes where equivalent gas emission time-series have been acquired, and suggest that gas flux, magma viscosity and conduit geometry are key parameters determining which flow mechanism operates in a given volcanic conduit. The range of conduit flow regimes inferred from the few studied lava lakes gives a glimpse of the potentially wide spectrum of conduit flow dynamics operating at active volcanoes.