Whale Upcalls Research Articles

Adapting deep learning models to new acoustic environments - A case study on the North Atlantic right whale upcall

Passive acoustic monitoring is increasingly being used for studying marine mammals, leading to the accumulation of large acoustic datasets. Analyzing these datasets becomes impractical without automated detection and classification software. Detectors and classifiers based on deep neural networks have shown great potential, but their performance is often limited by the availability of sufficient quantities of annotated training samples, and their application restricted to the specific acoustic environment(s) from which their training data were collected. We address these limitations by employing transfer learning, a deep learning concept whereby knowledge from a source domain is transferred to a target domain. Specifically, we considered two different underwater acoustic environments as the source and target domains. The objective was to use a deep neural network that had been trained in one environment with abundant annotated training samples, and optimize its performance in the other environment where the annotated training samples were limited. Training and testing were conducted using three acoustic datasets containing North Atlantic right whale (Eubalaena glacialis) upcalls. Experiments show that adapting a trained model to the new environment led to a substantial improvement in recall from 70% to 85%, while maintaining a low false-positive rate of less than 5 per hour. The methodology is implemented as an open-source Python tool to facilitate the creation of more tailored deep learning-based acoustic detectors and classifiers for North Atlantic right whale vocalizations and other stereotyped marine mammal calls.

Exploring and verifying the acoustic presence of southern right whales (Eubalaena australis) off Elephant Island, Antarctica.

Passive acoustic monitoring (PAM) can be used to monitor acoustic presence and behaviour of cetaceans, providing continuous, long-term, and seasonally unbiased data. The efficiency of PAM methods, however, depends on the ability to detect and correctly interpret acoustic signals. The upcall is the most prevalent vocalization of the southern right whale (Eubalaena australis) and is commonly used as a basis for PAM studies on this species. However, previous studies report difficulties to distinguish between southern right whale upcalls and similar humpback whale (Megaptera novaeangliae) vocalizations with certainty. Recently, vocalizations comparable to southern right whale upcalls were detected off Elephant Island, Antarctica. In this study, these vocalizations were structurally analyzed, and call characteristics were compared to (a) confirmed southern right whale vocalizations recorded off Argentina and (b) confirmed humpback whale vocalizations recorded in the Atlantic Sector of the Southern Ocean. Based on call features, detected upcalls off Elephant Island could be successfully attributed to southern right whales. Measurements describing slope and bandwidth were identified as the main differences in call characteristics between species. With the newly gained knowledge from this study, additional data can be analyzed providing further insight into temporal occurrence and migratory behaviour of southern right whales in Antarctic waters.

Directional detection of simulated whale calls and ambient noise in a busy harbour

Timely information on marine mammal presence in, or near, sonar training areas can be used to minimize the risk of harming these animals. Defence Research and Development Canada (DRDC) has recently been investigating a variety of acoustic technologies to facilitate real-time marine mammal monitoring in high noise (including ship noise and sonar signals) environments. DRDC assessed the directional detection and localization capability of a network of sub-surface volumetric and vertical line arrays against the noisy backdrop of the busy harbour during an experiment conducted in Bedford Basin, NS. Playbacks of Southern Resident killer whale calls and North Atlantic right whale upcalls, as well as simulated calls, were transmitted. To assess variability in bearing estimates for whale calls in different frequency bands, an acoustic source was fixed at a single location for half of the experiment and dipped over the side of a boat at seven locations around the recorders during the other half. During this talk, ambient noise and environmental measurements will be presented along with acoustic localization results.

Evaluation of a coastal acoustic buoy for cetacean detections, bearing accuracy and exclusion zone monitoring

Abstract There is strong socio‐political support for offshore wind development in US territorial waters and construction is planned off several east coast states. Some of the planned development sites coincide with important habitat for critically endangered North Atlantic right whales. Both exclusion zones and passive acoustic monitoring are important tools for managing interactions between marine mammals and human activities. Understanding where animals are with respect to exclusion zones is important to avoid costly construction delays while minimizing the potential for negative impacts. Impact piling from construction of hundreds of offshore wind turbines likely require exclusion zones as large as 10 km. We have developed a three‐hydrophone passive acoustic monitoring system that provides bearing information along with marine mammal detections to allow for informed management decisions in real‐time. Multiple units form a monitoring system designed to determine whether marine mammal calls originate from inside or outside of an exclusion zone. In October 2021, we undertook a full system validation, with a focus on evaluating the detection range and bearing accuracy of the system with respect to right whale upcalls. Five units were deployed in Mid‐Atlantic waters and we played more than 3500 simulated right whale upcalls at known locations to characterize the detection function and bearing accuracy of each unit. The modelled results of the detection function error were then used to compare the effectiveness of a bearing‐based system to a single sensor that can only detect a signal but not ascertain directivity. Field trials indicated maximum detection ranges from 4–7.3 km depending on source and ambient noise levels. Simulations showed that incorporating bearing detections provide a substantial improvement in false alarm rates (6 to 12 times depending on number of units, placement and signal to noise conditions) for a small increase in the risk of missed detections inside of an exclusion zone (1%–3%). We show that the system can be used for monitoring exclusion zones and clearly highlight the value of including bearing estimation into exclusion zone monitoring plans while noting that placement and configuration of units should reflect anticipated ambient noise conditions.

Acoustic detection range of right whale upcalls identified in near-real time from a moored buoy and a Slocum glider.

The goal of this study was to characterize the detection range of a near real-time baleen whale detection system, the digital acoustic monitoring instrument/low-frequency detection and classification system (DMON/LFDCS), equipped on a Slocum glider and a moored buoy. As a reference, a hydrophone array was deployed alongside the glider and buoy at a shallow-water site southwest of Martha's Vineyard (Massachusetts, USA) over a four-week period in spring 2017. A call-by-call comparison between North Atlantic right whale upcalls localized with the array (n = 541) and those detected by the glider or buoy was used to estimate the detection function for each DMON/LFDCS platform. The probability of detection was influenced by range, ambient noise level, platform depth, detection process, review protocol, and calling rate. The conservative analysis of near real-time pitch tracks suggested that, under typical conditions, a 0.33 probability of detection of a single call occurred at 6.2 km for the buoy and 8.6-13.4 km for the glider (depending on glider depth), while a 0.10 probability of detection of a single call occurred at 14.4 m for the buoy and 22.6-27.5 km for the glider. Probability of detection is predicted to increase substantially at all ranges if more than one call is available for detection.

Evaluation of the Coastal Acoustic Buoy for offshore wind for real time mitigation of North Atlantic right whales

A recent Incidental Harassment Authorization requires a 10 km zone to be monitored with passive acoustics before and during pile driving. SMRU Consulting has built the Coastal Acoustic Buoy for Offshore Wind (CABOW) to address this need. Each CABOW consists of: a bottom lander with three hydrophones; data acquisition system; and onboard computer running PAMGuard. The CABOW detects potential calls and relays clips to a base station for classification, bearing estimation, and validation. Bench testing characterized detector performance and measured bearing accuracy as a function of SNR. Field trials used playbacks of real and simulated right whale upcalls to evaluate the detection probability with respect to the range and signal excess of the call; estimate bearing accuracy; and test the range and stability of the radio communications. Field trials indicated a maximum detection range of 4–12 km under high and very low noise levels, respectively. Finally, we used a simulation to compare true and false alarm rates of the CABOW to an equivalent single sensor system for monitoring a fixed area. Simulations show that incorporating bearings to calling animals could provide a substantial improvement in false alarm rates relative to single sensors if properly placed with respect to potential noise sources.

A multimodel deep learning algorithm to detect North Atlantic right whale up-calls.

We present a new method of detecting North Atlantic Right Whale (NARW) upcalls using a Multimodel Deep Learning (MMDL) algorithm. A MMDL detector is a classifier that embodies Convolutional Neural Networks (CNNs) and Stacked Auto Encoders (SAEs) and a fusion classifier to evaluate their output for a final decision. The MMDL detector aims for diversity in the choice of the classifier so that its architecture is learned to fit the data. Spectrograms and scalograms of signals from passive acoustic sensors are used to train the MMDL detector. Guided by previous applications, we trained CNNs with spectrograms and SAEs with scalograms. Outputs from individual models were evaluated by the fusion classifier. The results obtained from the MMDL algorithm were compared to those obtained from conventional machine learning algorithms trained with handcrafted features. It showed the superiority of the MMDL algorithm in terms of the upcall detection rate, non-upcall detection rate, and false alarm rate. The autonomy of the MMDL detector has immediate application to the effective monitoring and protection of one of the most endangered species in the world where accurate call detection of a low-density species is critical, especially in environments of high acoustic-masking.

Data augmentation for the classification of North Atlantic right whales upcalls.

Passive acoustic monitoring (PAM) is a useful technique for monitoring marine mammals. However, the quantity of data collected through PAM systems makes automated algorithms for detecting and classifying sounds essential. Deep learning algorithms have shown great promise in recent years, but their performance is limited by the lack of sufficient amounts of annotated data for training the algorithms. This work investigates the benefit of augmenting training datasets with synthetically generated samples when training a deep neural network for the classification of North Atlantic right whale (Eubalaena glacialis) upcalls. We apply two recently proposed augmentation techniques, SpecAugment and Mixup, and show that they improve the performance of our model considerably. The precision is increased from 86% to 90%, while the recall is increased from 88% to 93%. Finally, we demonstrate that these two methods yield a significant improvement in performance in a scenario of data scarcity, where few training samples are available. This demonstrates that data augmentation can reduce the annotation effort required to achieve a desirable performance threshold.

Optimizing passive acoustic systems for marine mammal detection and localization: Application to real-time monitoring north Atlantic right whales in Gulf of St. Lawrence

This study addresses the problem of determining optimal design of passive acoustic monitoring (PAM) systems for detecting and localizing whale calls in real-time in variable-noise environments. The performance of various PAM system is assessed using the detection theory and simulation modeling applied to the context of North Atlantic right whale (NARW) upcalls in feeding grounds and noisy shipping corridor of the Gulf of St. Lawrence. Realistic simulations are performed using an estimated NARW upcall source level (SL), the actual shipping traffic, measured local fleet ship SLs, and transmission loss (TL) from a regional 2.5-D propagation model accounting for the bathymetric and environmental structures. The comparisons consider single-hydrophone and hydrophone-array PAM systems, mounted on buoys, gliders, or cabled to shore and three families of NARW upcall detectors. The targeted performance is a low false-alarm rate of 1 per day and a detection probability > 0.5. The time-frequency-based detector offers the best trade-off between detection performance and robustness against NARW upcall variability. The effective detection ranges are ∼ 15 times lower with single-hydrophone systems compared to hydrophone-arrays, whose beamforming enhances the signal in the upcall direction while damping interfering discrete noise from nearby transiting ships in other directions. Detecting and localizing NARWs in the large target areas (>10000-km2 scale) is possible with a few well-located arrays of 10–20 hydrophones, which appears as the optimal cost/performance trade-off.

Performance of a deep neural network at detecting North Atlantic right whale upcalls.

Passive acoustics provides a powerful tool for monitoring the endangered North Atlantic right whale (Eubalaena glacialis), but robust detection algorithms are needed to handle diverse and variable acoustic conditions and differences in recording techniques and equipment. This paper investigates the potential of deep neural networks (DNNs) for addressing this need. ResNet, an architecture commonly used for image recognition, was trained to recognize the time-frequency representation of the characteristic North Atlantic right whale upcall. The network was trained on several thousand examples recorded at various locations in the Gulf of St. Lawrence in 2018 and 2019, using different equipment and deployment techniques. Used as a detection algorithm on fifty 30-min recordings from the years 2015-2017 containing over one thousand upcalls, the network achieved recalls up to 80% while maintaining a precision of 90%. Importantly, the performance of the network improved as more variance was introduced into the training dataset, whereas the opposite trend was observed using a conventional linear discriminant analysis approach. This study demonstrates that DNNs can be trained to identify North Atlantic right whale upcalls under diverse and variable conditions with a performance that compares favorably to that of existing algorithms.

The use of context in machine learning for bioacoustics

Biological acoustic signals are often produced in context. Contextual information includes things such as timing between calls, conspecific or interspecies cues, and physical environmental cues such as sunrise or sunset. We show how some forms of contextual information can be used to improve the results of detection and classification tasks for biological acoustic signals. We examine how context can be used to improve labeled data, resulting in more-accurate classification results, as well as how learners can exploit context directly. We demonstrate these improvements via two bioacoustic detection/classification tasks. The first algorithm detects odontocete echolocation clicks. We used a decision support system that allowed analysts to label echolocation clicks using between call timing cues as well as other measurements and found that deep learners trained with these high quality data are able to detect clicks in adverse environments. The second algorithm applies contextual information surrounding North Atlantic right whale upcalls to improve precision and recall. [This work was supported by ONR Grant Nos. N00014-17-1-2867 and N00014-15-1-2299.]

Two-stage detection of north Atlantic right whale upcalls using local binary patterns and machine learning algorithms

In this paper, we investigate the effectiveness of two-stage classification strategies in detecting north Atlantic right whale upcalls. Time-frequency measurements of data from passive acoustic monitoring devices are evaluated as images. Vocalization spectrograms are preprocessed for noise reduction and tone removal. First stage of the algorithm eliminates non-upcalls by an energy detection algorithm. In the second stage, two sets of features are extracted from the remaining signals using contour-based and texture based methods. The former is based on extraction of time–frequency features from upcall contours, and the latter employs a Local Binary Pattern operator to extract distinguishing texture features of the upcalls. Subsequently evaluation phase is carried out by using several classifiers to assess the effectiveness of both the contour-based and texture-based features for upcall detection. Comparing ROC curves of machine learning algorithms obtained from Cornell University’s dataset reveals that LBP features improved performance accuracy up to 43% over time–frequency features. Classifiers such as the Linear Discriminant Analysis, Support Vector Machine, and TreeBagger achieve highest upcall detection rates with LBP features.

North Atlantic right whale Eubalaena glacialis vocalization patterns in the central Gulf of Maine from October 2009 through October 2010

The central Gulf of Maine was recently identified as a persistent wintering ground and potential mating ground for non-calving North Atlantic right whales Eubalaena glacialis based on aerial survey data. However, these surveys were limited by bad weather and light. Here, we use passive acoustic monitoring to examine the long-term persistence of right whales in this area throughout a nearly continuous period from October 2009 through October 2010. Three archival marine acoustic recording units were deployed in the Outer Fall/central Gulf of Maine. The data were manually reviewed for right whale up-calls and gunshots to investigate seasonal and diel patterns. Up-calls and gunshots occurred seasonally, with the most calls recorded from October through January and fewer calls detected from February through July, increasing again in August through October. Up-calls were most frequent in November, and gunshots in December. There was a clear bimodal diel pattern in up-calls, with the majority of calls occurring between 04:00 through 08:00 h and 13:00 through 22:00 h. There was a clear peak in diel distribution of gunshots, with the majority of calls occurring between 16:00 and 22:00 h. Our data demonstrate the continuous presence of right whales in the central Gulf of Maine during the winter months. The rate of gunshots during winter months in Outer Fall supports the hypothesis that male adver- tisement and/or right whale mating behavior may be taking place in this region at that time.

A new method for detection of North Atlantic right whale up-calls

A study of detecting North Atlantic Right Whale (NARW) up-calls has been conducted with measurements from passive acoustic monitoring devices. Denoising and normalization algorithms are applied to remove local variance and narrowband noise in order to isolate the NARW up-calls in spectrograms. The resulting spectrograms, after binarization, are treated with a region detection procedure called the Moor-Neighbor algorithm to find continuous objects that are candidates of up-call contours. After selected properties of each detected object are computed, they are compared with a pair of low and high empirical thresholds to estimate the probability of the detected object being an up-call; therefore, those objects that are determined with certainty to be non up-calls are discarded. The final stage in the proposed call detection method is to separate true up-calls from the rest of potential up-calls with classifiers such as linear discriminate analysis (LDA), Naïve Bayes, and decision tree. Experimental results using the data set obtained by Cornell University show that the proposed method can achieve accuracy to 96%.

Variation in the vocal behavior of southern right whales (Eubalaena australis) in coastal Brazilian waters

Currently, there are three recognized species of right whales. The largest population is the southern right whale (Eubalaena australis), with circumpolar distribution in the southern hemisphere. One calving area for this population is in Brazilian waters, where increasing numbers of right whales have been sighted over the past decade along with an increase in anthropogenic activities including shipping traffic and fishing. The goals of this study were to describe the vocal behavior of southern right whales in Brazilian waters, assess the difference in vocalizations between areas with low and high human activity, and compare these results to studies conducted with North Atlantic right whales (Eubalaena glacialis) in the Western North Atlantic. Bottom-mounted archival acoustic recorders were deployed in October and November 2011 in two coastal locations in central Santa Catarina State, southern Brazil. One recorder was placed off Gamboa (27056'S and 48039'W, low traffic) and a second off Ribanceira (28011'S and 48037'W, high traffic). Automated detectors and noise statistic analysis tools developed for North Atlantic right whale upcalls were utilized to analyze the dataset. Calls produced by Brazilian right whales were significantly lower in fundamental frequency than North Atlantic right whale calls and the implications for these results will be discussed.

Seasonal and diel variation in North Atlantic right whale up-calls: implications for management and conservation in the northwestern Atlantic Ocean

Ship strikes are a major cause of anthropogenic mortality for the endangered North Atlantic right whale Eubalaena glacialis. Year-round data on animal presence are critical to man- aging ship strike mortality. Marine autonomous recording units were deployed throughout the Stellwagen Bank National Marine Sanctuary (SBNMS, Massachusetts Bay, USA) for 13 mo from January 2006 to February 2007 and on Jeffreys Ledge (JL, Gulf of Maine, USA) for 7 mo from November 2004 to May 2005 to determine whether passive acoustic monitoring (PAM) can improve information on right whale occurrence. Automated detection and manual review were used to determine presence and absence of right whale up-calls. In SBNMS, up-calls were detected year round, except during July and August, and calling rates were highest from January through May, peaking in April. In JL, up-calls occurred throughout all recording months, with the highest numbers from November through February. Up-calls were heard extensively in the win- tertime throughout SBNMS and JL, suggesting that these areas are important overwintering grounds for right whales. Additionally, up-calls showed a strong diel trend in both areas, with sig- nificantly more calls occurring during twilight than dark and light periods. These data indicate that right whales are present more often and over longer time periods in the western Gulf of Maine than previously thought using conventional visual techniques. Finally, this study demon- strates the utility of PAM in providing a detailed and long-term picture of right whale presence in an area that poses a significant risk of anthropogenic mortality.

Automatic Detection and Characterization of Dispersive North Atlantic Right Whale Upcalls Recorded in a Shallow-Water Environment Using a Region-Based Active Contour Model

Various automatic detection and characterization techniques have been proposed to implement mitigation measures to protect the endangered North Atlantic Right Whales in their habitats. These species prefer shallow waters during their seasonal migration. Such shallow-water acoustical environments act as a waveguide and cause the monocomponent upcall emitted by the vocalizing whale to become a dispersive multicomponent signal with different time arrivals and relative energy. In this paper, the discrimination between the Right Whale upcalls and background noise has been investigated using the support vector machine classifier. To perform this task, a region-based active contour segmentation method is proposed. In this work, both synthesized data based on typical Right Whale vocalizations and real data recorded in Cape Cod Bay are used to evaluate the proposed method. We show how the shallow-water dispersion effects which cause higher order mode generation affect the parameters used for classification and descriptive statistics. We compare the descriptive statistics of the call duration using both single-mode and multimode approaches. The single-mode analysis was performed by extracting the frequency contour of the first mode.

North Pacific right whale up-call source levels and propagation distance on the southeastern Bering Sea shelf

Call source levels, transmission loss, and ambient noise levels were estimated for North Pacific right whale (Eubalaena japonica) up-calls recorded in the southeastern Bering Sea in autumn of 2000 and 2001. Distances to calling animals, needed to estimate source levels, were based on two independent techniques: (1) arrival-time differences on three or more hydrophones and (2) shallow-water dispersion of normal modes on a single receiver. Average root-mean-square (rms) call source levels estimated by the two techniques were 178 and 176 dB re 1 μPa at 1 m, respectively, over the up-call frequency band, which was determined per call and averaged 90 to 170 Hz. Peak-to-peak source levels were 14 to 22 dB greater than rms levels. Transmission loss was approximately 15∗log(10)(range), intermediate between cylindrical and spherical spreading. Ambient ocean noise within the up-call band varied from 72 to 91 dB re 1 μPa(2)/Hz. Under average noise conditions, call spectrograms were detectable for whales at distances up to 100 km, but propagation and detection distance may vary depending on environmental parameters and anthropogenic noise. Obtaining distances to animals and acoustic detection range is a step toward using long-term passive acoustic recordings to estimate abundance for this critically endangered whale population.

A generalized baleen whale call detection and classification system

Passive acoustic monitoring allows the assessment of marine mammal occurrence and distribution at greater temporal and spatial scales than is now possible with traditional visual surveys. However, the large volume of acoustic data and the lengthy and laborious task of manually analyzing these data have hindered broad application of this technique. To overcome these limitations, a generalized automated detection and classification system (DCS) was developed to efficiently and accurately identify low-frequency baleen whale calls. The DCS (1) accounts for persistent narrowband and transient broadband noise, (2) characterizes temporal variation of dominant call frequencies via pitch-tracking, and (3) classifies calls based on attributes of the resulting pitch tracks using quadratic discriminant function analysis (QDFA). Automated detections of sei whale (Balaenoptera borealis) downsweep calls and North Atlantic right whale (Eubalaena glacialis) upcalls were evaluated using recordings collected in the southwestern Gulf of Maine during the spring seasons of 2006 and 2007. The accuracy of the DCS was similar to that of a human analyst: variability in differences between the DCS and an analyst was similar to that between independent analysts, and temporal variability in call rates was similar among the DCS and several analysts.