Acoustic Surveillance Research Articles

Innovative Hybrid UAV Design, Development, and Manufacture for Forest Preservation and Acoustic Surveillance

The research described in this paper focuses on the development of an innovative unmanned aerial vehicle (UAV) tailored for a specific mission: detecting the acoustic signature emitted by chainsaws, identifying deforestation, and reporting its location for legality assessment. Various calculations were conducted to determine the optimal solution, resulting in the choice of a fixed-wing UAV. A comparative analysis between tri-rotor and quadcopter systems was performed, leading to the selection of the tri-rotor configuration. The primary objective of this study is to design an innovative hybrid UAV concept with key features including a fixed-wing design and integrated VTOL (vertical takeoff and landing) capability in the experimental model. The aircraft has been constructed using advanced materials such as fiber-reinforced polymer composites, manufactured using both conventional and advanced techniques like continuous fiber additive manufacturing and the use of a polymer matrix. Additionally, the aerodynamic configuration is optimized to achieve a cruise speed of approximately 50 km/h and a flight autonomy exceeding 3 h. The UAV has been equipped with payloads for mounting sensors to collect meteorological data, and crucially, the VTOL system has been optimized to vectorize thrust for improved performance during the transition from hover to cruise flight. This paper details the entire manufacturing and assembly process of the drone, covering both the structural framework and associated electrical installations. A dedicated sound detection system is incorporated into the drone to identify chainsaw noise, with the aim of preventing deforestation.

A Survey on Deep Learning Based Forest Environment Sound Classification at the Edge

Forest ecosystems are of paramount importance to the sustainable existence of life on earth. Unique natural and artificial phenomena pose severe threats to the perseverance of such ecosystems. With the advancement of artificial intelligence technologies, the effectiveness of implementing forest monitoring systems based on acoustic surveillance has been established due to the practicality of the approach. It can be identified that with the support of transfer learning, deep learning algorithms outperform conventional machine learning algorithms for forest acoustic classification. Further, a clear requirement to move the conventional cloud-based sound classification to the edge is raised among the research community to ensure real-time identification of acoustic incidents. This article presents a comprehensive survey on the state-of-the-art forest sound classification approaches, publicly available datasets for forest acoustics, and the associated infrastructure. Further, we discuss the open challenges and future research aspects that govern forest acoustic classification.

Performance evaluation of the smartphone-based AI cough monitoring app - Hyfe Cough Tracker against solicited respiratory sounds

Background: Emerging technologies to remotely monitor patients’ cough show promise for various clinical applications. Currently available cough detection systems all represent a trade-off between convenience and performance. The accuracy of such technologies is highly contingent on the clinical settings in which they are intended to be used. Moreover, establishing gold standards to measure this accuracy is challenging. Objectives: We present the first performance evaluation study of the Hyfe Cough Tracker app, a passive cough monitoring smartphone application. We evaluate performance for cough detection using continuous audio recordings obtained within a controlled environment and cough counting by trained individuals as the gold standard. We propose standard procedures to use multi-observer cough sound annotation from continuous audio recordings as the gold standard for evaluating automated cough detection devices. Methods: This study was embedded in a larger digital acoustic surveillance study (clinicaltrial.gov NCT04762693). Forty-nine participants were included and instructed to produce a diverse series of solicited sounds in 10-minute sessions. Simultaneously, continuous audio recording was performed using a MP3 recorder and two smartphones running Hyfe Cough Tracker app monitored and identified cough events. All continuous audio recordings were independently labeled by three medically-trained researchers. Results: Hyfe Cough Tracker app showed sensitivity of 91% and specificity of 98% with a very high correlation between the cough rate measured by Hyfe and that of human annotators (Pearson correlation of 0.968). A standardized approach to establish an acoustic gold standard for identifying cough sounds with multiple observers is presented. Conclusion: This is the first performance evaluation of a new smartphone-based cough monitoring system. Hyfe Cough Tracker can detect, record and count coughs from solicited cough-like explosive sounds in controlled acoustic environments with very high accuracy. Additional steps are required to validate the system in clinical and community settings.

Protected habitats support bats in Mediterranean dry grasslands

The replacement of natural habitats by urbanization and agricultural land reclamation is one of the main drivers of biodiversity loss. Among European habitat types, natural grasslands are particularly prone to anthropogenic pressures, being also recognized as conservation priorities within the Habitats Directive. Nonetheless, little is known on the relationship between grasslands, their conservation quality, and most animals' taxa that may rely upon them. Here we focus on the role of Mediterranean dry grasslands protected by the EU legislation in sustaining bat populations, setting our study in the biodiversity hotspot of Mediterranean Italy. By conducting acoustic surveillance at 48 sites within a protected area devoted to conserve natural and semi-natural grasslands, we found that all the bat species found in the area are regular exploiters of such open environments. Grassland conservation quality, in terms of extent of high-diversity protected habitats, was the key factor shaping the use of grasslands by bats of all the considered guilds, together with several terrain and landscape features, which showed more guild-specific effects. Moreover, our results indicate that bat assemblages are functionally shifted along an ecological gradient from highly modified to well-conserved grassland sites, indicating a prevalence of opportunistic taxa in the former, and higher abundance of species of conservation concern in the latter. Overall, we demonstrate that the effects of EU-listed habitats may extend also onto bats in the case of Mediterranean dry grasslands, highlighting the importance of preserving such habitats as a tool for conserving highly mobile species.

Listening to ShotSpotter: Acoustics and Predictive Policing in Chicago

This project hopes to highlight a moving, multidirectional story about acoustic surveillance tools and how they are used to predict and conduct the future. I focus on ShotSpotter Inc’s occupation in Chicago, which has grown omnipresent following the signature of the city’s current contract with ShotSpotter for $33 million in the summer of 2021, lasting through August 2023 (Del Vecchio and Chapman 2022).2 Chicago is now blanketed with microphones and Strategic Decision Support Centers (“SDSC’s”) installed to 21 of the city’s 22 police districts. (Ibid and CBS 2020)3. Chicago accounted for 14% of ShotSpotter’s total revenue in 2021, as the company’s second largest customer (MarketScreener 2022)4. Activists in the city have protested this relationship, especially after ShotSpotter acquired predictive policing company HunchLab in 2018.5 Legislation in Illinois has yet to adequately grapple with advanced acoustic surveillance and predictive policing techniques, further ripening this moment for this dialogue.6 This project is not a study of the borne and exacerbated ills of ShotSpotter’s collaborations with Chicago Police and their rise of their technocratic methodologies, nor is the project much concerned with ShotSpotter’s credence as a technology that is successful even by its own objectives.7 Rather, this project is an experimental inquiry into how our futures may be patrolled by means of acoustic capture and prediction. I explored this by placing ShotSpotter’s approach in contrast with other shapes and styles of listening that nurture the past, present, and future.

Real-Time Tunnel Abnormal Sound Detection Algorithm Using Convolutional Neural Networks

In the traffic industry, the automatic accident detection system is a major concern. Although image-based and radar-based traffic accident detection systems are commonly employed, they have several drawbacks, including the need to secure the camera’s field of view, a high rate of false alarms, and a lengthy detection time. Using a real-time acoustic surveillance system and the classification algorithm via Convolutional Neural Network (CNN), this article proposes several methods for identifying abnormal situations, such as a car crash or tire skid sound, to overcome the limitations of existing methods. We create an audio database by collecting sounds from two tunnels in South Korea using self-made microphones for eight months and classifying them into three categories: car crash, tire skid, and normal environmental sounds. We establish a three-step classification procedure using an algorithm. We compare the detection rate and false alarm rate of our proposed method to those of deep learning techniques including MLP (Multi-Layer Perceptron), Long-Short Term Memory, ShuffleNetv2, and MobileNetv2. In addition, we present a method for filtering out irrelevant sound data to improve the computational efficiency of our approach.

An Acoustic Camera for Use on UAVs.

Airborne acoustic surveillance would enable and ease several applications, including security surveillance, urban and industrial noise monitoring, rescue missions, and wildlife monitoring. Airborne surveillance with an acoustic camera mounted on an airship would provide the deployment flexibility and utility required by these applications. Nevertheless, and problematically for these applications, there is not a single acoustic camera mounted on an airship yet. We make significant advances towards solving this problem by designing and constructing an acoustic camera for direct mounting on the hull of a UAV airship. The camera consists of 64 microphones, a central processing unit, and software for data acquisition and processing dedicatedly developed for far-field low-level acoustic signal detection. We demonstrate a large-aperture mock-up camera operation on the ground, although all preparations have been made to integrate the camera onto an airship. The camera has an aperture of 2 m and has been designed for surveillance from a height up to 300 m, with a spatial resolution of 12 m.

Sound Event Classification Based on Frequency-Energy Feature Representation and Two-Stage Data Dimension Reduction

The classification of environmental sound events is of great significance for applications such as machine hearing and acoustic surveillance. Feature representation and feature vector dimension directly affect system performance. To better extract features and reduce computational burden, a novel frequency-energy feature representation and two-stage dimension reduction system were proposed. First, a frequency-energy diagram is generated. Based on this, the importance screening is done and only the energy bins of high importance are retained, which reduces the dimension of feature vector while extracting key information. Then the Bicubic interpolation method is used to further reduce the dimension. And the appropriate feature vector dimension is determined based on the change of information entropy. The proposed frequency-energy feature representation and two-stage dimension reduction system are evaluated with Real Word Computing Partnership sound scene database (RWCP-SSD), UrbanSound8K, and ESC-50 datasets, which demonstrate that the robustness is satisfactory under low signal-to-noise ratios (SNRs) and 15 noise types from NOISEX-92 database.

Performance evaluation of the smartphone-based AI cough monitoring app - Hyfe Cough Tracker against solicited respiratory sounds

Background: Emerging technologies to remotely monitor patients’ cough show promise for various clinical applications. Currently available cough detection systems all represent a trade-off between convenience and performance. The accuracy of such technologies is highly contingent on the clinical settings in which they are intended to be used. Moreover, establishing gold standards to measure this accuracy is challenging. Objectives: We present the first performance evaluation study of the Hyfe Cough Tracker app, a passive cough monitoring smartphone application. We evaluate performance for cough detection using continuous audio recordings and cough counting by trained individuals as the gold standard. We propose standard procedures to use multi-observer cough sound annotation from continuous audio recordings as the gold standard for evaluating automated cough detection devices. Methods: This study was embedded in a larger digital acoustic surveillance study (clinicaltrial.gov NCT04762693). Forty-nine participants were included and instructed to produce a diverse series of solicited sounds in 10-minute sessions. Simultaneously, continuous audio recording was performed using a MP3 recorder and two smartphones running Hyfe Cough Tracker app monitored and identified cough events. All continuous audio recordings were independently labeled by three medically-trained researchers. Results: Hyfe Cough Tracker app showed sensitivity of 91% and specificity of 98% with a very high correlation between the cough rate measured by Hyfe and that of human annotators (Pearson correlation of 0.968). A standardized approach to establish an acoustic gold standard for identifying cough sounds with multiple observers is presented. Conclusion: This is the first performance evaluation of a new smartphone-based cough monitoring system. Hyfe Cough Tracker can detect, record and count coughs from solicited cough-like explosive sounds in controlled acoustic environments with very high accuracy. Additional steps are required to validate the system in clinical and community settings.

Acoustic surveillance of cough for detecting respiratory disease using artificial intelligence.

Research questionCan smartphones be used to detect individual and population-level changes in cough frequency that correlate with the incidence of coronavirus disease 2019 (COVID-19) and other respiratory infections?MethodsThis was a prospective cohort study carried out in Pamplona (Spain) between 2020 and 2021 using artificial intelligence cough detection software. Changes in cough frequency around the time of medical consultation were evaluated using a randomisation routine; significance was tested by comparing the distribution of cough frequencies to that obtained from a model of no difference. The correlation between changes of cough frequency and COVID-19 incidence was studied using an autoregressive moving average analysis, and its strength determined by calculating its autocorrelation function (ACF). Predictors for the regular use of the system were studied using a linear regression. Overall user experience was evaluated using a satisfaction questionnaire and through focused group discussions.ResultsWe followed-up 616 participants and collected >62 000 coughs. Coughs per hour surged around the time cohort subjects sought medical care (difference +0.77 coughs·h−1; p=0.00001). There was a weak temporal correlation between aggregated coughs and the incidence of COVID-19 in the local population (ACF 0.43). Technical issues affected uptake and regular use of the system.InterpretationArtificial intelligence systems can detect changes in cough frequency that temporarily correlate with the onset of clinical disease at the individual level. A clearer correlation with population-level COVID-19 incidence, or other respiratory conditions, could be achieved with better penetration and compliance with cough monitoring.

Effective governance of a remote marine protected area: The case of Dongsha Atoll National Park

Many fisheries in Taiwan’s Dongsha Atoll National Park (DANP), a marine protected area (MPA), are subject to illicit fishing by fishers from countries around Taiwan. This study examined the governance of the DANP in response to this problem. We reviewed documents, conducted a field survey of the Dongsha Islands, held semistructured interviews with local interviewees, and analyzed records of automatic identification system (AIS) signals. We found that insufficient enforcement and a lack of interagency cooperation have impeded MPA governance in DANP since 2002. We proposed solutions in three areas. The first is detection: (1) participatory monitoring mechanisms and (2) an integrated monitoring, control, and surveillance system should be established to compensate for the limitations of coastal radar. This counteracts AIS manipulation and the use of motor-powered skiffs by illegal fishers. The second is response. To respond swiftly to illegal activity, target identification is crucial. Thus, the use of technological tools, such as unmanned aerial vehicles, optical imaging systems, and acoustic surveillance systems, should supplement AIS-based identification. The third is adjudication. Because most illegal fishers are not Taiwanese, the Taiwanese government can use international agreements to prosecute (and deter) illegal fishers. This study elucidates challenges in regional MPA governance.

On the Importance of Passive Acoustic Monitoring Filters

Passive acoustic monitoring (PAM) is a noninvasive technique to supervise wildlife. Acoustic surveillance is preferable in some situations such as in the case of marine mammals, when the animals spend most of their time underwater, making it hard to obtain their images. Machine learning is very useful for PAM, for example to identify species based on audio recordings. However, some care should be taken to evaluate the capability of a system. We defined PAM filters as the creation of the experimental protocols according to the dates and locations of the recordings, aiming to avoid the use of the same individuals, noise patterns, and recording devices in both the training and test sets. It is important to remark that the filters proposed here were not intended to improve the accuracy rates. Indeed, these filters tended to make it harder to obtain better rates, but at the same time, they tended to provide more reliable results. In our experiments, a random division of a database presented accuracies much higher than accuracies obtained with protocols generated with PAM filters, which indicates that the classification system learned other components presented in the audio. Although we used the animal vocalizations, in our method, we converted the audio into spectrogram images, and after that, we described the images using the texture. These are well-known techniques for audio classification, and they have already been used for species classification. Furthermore, we performed statistical tests to demonstrate the significant difference between the accuracies generated with and without PAM filters with several well-known classifiers. The configuration of our experimental protocols and the database were made available online.

Efficient multichannel detection of impulsive audio events for wireless networks

Impulsive audio events such as gunshots, explosions or glass shattering, are commonly associated with security threats, thus they are of particular interest for automated acoustic surveillance. Even though impulsive audio events are greatly influenced by their propagation path, little work has been done in multichannel detection, and most precedents available in the literature deal with single-channel detection systems. Unfortunately, the spatial dependence of impulsive sound recordings proves as a problem for robust performance under real conditions. It is possible, however, to take advantage of the spatial diversity provided by a wireless sensor network to counteract this problem. In this paper, we show how an ensemble of spatially diverse detectors can greatly improve the performance of the system. We propose an efficient multichannel detection system of impulsive audio events intended for low-cost Wireless Acoustic Sensor Networks. Our proposal is based on a low complexity classification algorithm and an efficient method to include temporal context into the feature vector. The obtained results show that the proposed detection system is capable of achieving a more than adequate performance without incurring large computational loads.

Towards an enhanced management of recreational whale watching: The use of ecological and behavioural data to support evidence-based management actions

Recognition of the potential disturbance from whale-based tourism has stimulated the discussion of whether whale watching remains ‘no take’. Here we use observations and passive acoustic surveillance of tourism activities of foraging gray whales (Eschrichtius robustus) on the west coast of Vancouver Island to better understand the pressures on whales that are repeatedly subject to aerial and vessel-based viewing. Pressure from the fleet was greatest for low prey years, whereby fewer whales were subject to a greater number of encounters with whale watching vessels. The pressure on operators for close encounters was also greater during these years. The acoustic recordings found that whale watching vessels shaped the soundscape, with a distinct diurnal pattern. Acoustic reactions by the whales were also noted. Cow-calf pairs showed increased calling rate in the presence of vessels in the feeding bay, with calling rate of single foraging whales increasing as vessel number, particularly those engaged in whale watching, increased. These responses have implications for energetics, communication success. The implications of acoustic disturbance from whale watching has yet to be well defined, but it is only one of many stressors that gray whales are subject to. The results of this study lead us to argue for greater evidence-based management of whale watching; restrictions on encounters; noise emission limitations; requiems for the whales, and a greater interpretative component for passengers. The aggregate and cumulative effects of disturbance should also be emphasized in vessel-whale interaction guidelines, with the most vulnerable individuals removed from the viewing resource altogether.

Passive Shallow Water Automated Target Recognition using Deep Convolutional Bi directional Long Short Term Memory

The extremely challenging nature of passive acoustic surveillance makes it a key area of research in NavalNon-Co-operative Target Recognition especially in Anti-Submarine Warfare systems. In shallow waters, thecomplex acoustics due to the highly varying ambient background noise as well as the multi-modal propagation in the surface-bottom bounded channel makes surveillance even difficult. In this work, an ensemble of Convolutional Neural Networks and Bidirectional Long Short Term Memory stages employing soft attention is used to effectively capture the spectro-temporal dynamics of the target signature. In order to alleviate the overall computational cost associated with the optimal model search in the extensive hyperparameter space, a recursive model elimination scheme, making frugal use of the available resources, is also proposed. Experimental analysis on acoustic target records, collected from the shallows of Arabian Sea, has yielded encouraging results in terms of model accuracy, precision and recall.

Optimal deployment of a distributed IoT acoustic surveillance system

In this study, the deployment of distributed Internet of Things (IoT) acoustic sensors using a real-time scenario is rigorously analysed. Simulation results indicate that in an ideal situation, wit...

Optimal deployment of a distributed IoT acoustic surveillance system

Optimal deployment of a distributed IoT acoustic surveillance system

Illegal Logging Detection Based on Acoustic Surveillance of Forest

In this article, we present a framework for automatic detection of logging activity in forests using audio recordings. The framework was evaluated in terms of logging detection classification performance and various widely used classification methods and algorithms were tested. Experimental setups, using different ratios of sound-to-noise values, were followed and the best classification accuracy was reported by the support vector machine algorithm. In addition, a postprocessing scheme on decision level was applied that provided an improvement in the performance of more than 1%, mainly in cases of low ratios of sound-to-noise. Finally, we evaluated a late-stage fusion method, combining the postprocessed recognition results of the three top-performing classifiers, and the experimental results showed a further improvement of approximately 2%, in terms of absolute improvement, with logging sound recognition accuracy reaching 94.42% when the ratio of sound-to-noise was equal to 20 dB.

Acoustic Module for Direction Finding of Small Aircraft

Introduction. The studied problem is relevant due to the growing use of unmanned small aircraft for mobile services of the mail and goods delivery, surveillance and security. At the same time, the danger of their use in events/campaigns, which pose a threat to both public and private interests, is increasing. Principally different devices are applied for surveillance; each type has its advantages and disadvantages. Passive acoustic surveillance tools provide stealth observation, but have a small detection range. It is possible to increase detection distance by improving the receiving microphones module design and developingeffective monitoring and localization algorithms. Theoretical Results. The paper presents the design of an acoustic module in the form of a tetrahedron with microphones located on its vertices. It is proposed to scan the space by angular coordinates using algorithms to generate a controlled power characteristic or controlled correlation characteristic. The theoretical justification of scanning algorithms, the results of the calculation of directivity characteristics and estimates of the error in determining the angular coordinates for each method are provided. Laboratory studies were conducted on direction finding of a noise source by the acoustic module prototype. Despite of reverberation in the room, it was possible to verify the theoretical results and confirm the advantage of the algorithm for the correlation formation of directivity. Conclusions. The results of the work can be used to create acoustic direction finders and systems for localizing noise objects. The proposed algorithm of controlled correlation characteristic can be used in monitoring tools to assess the acoustic characteristics of premises, for example, to experimentally determine the location and estimate sound absorption of enclosing constructions/the obstacles, to localize breaches in sound isolation.

Investigation of using quadcopter as a means of acoustic and optic surveillance

The main problem in performing optic and acoustic surveillance is the distance. Using quadcopters for gaining access to optic and acoustic information allows coming closer to the places of storing, processing and transmitting of information, suggesting that the interception of this information might be unauthorized. The object of surveillance can be located in the places traditionally inaccessible for normal surveillance – on (restricted areas) closed grounds or on a high floor. The objective of this article is theoretical and practical investigation of using civil unmanned aircrafts to perform surveillance. The basic task of acoustic surveillance is to determine the possibility of receiving and recording the signal of appropriate quality in spite of the noises of the quadcopter itself on the basis of the methods of calculation of discrete word intelligibility index developed by N. B. Pokrovsky. The basic task of optic surveillance is to determine the distance of text recognition on the monitor display or paper using traditional cameras installed on quadcopters. The special task is to choose the parameters of a quadcopter, which can be used for the objectives, to determine its possible characteristics of noise, weight, energy and size. In conclusion, the authors determined critical conditions of quadcopter use in acoustic and optic surveillance for understanding the range of necessary preventive measures against technical leak of information.