Biomimetic Sonar Head Research Articles

A validation study for a bat-inspired sonar sensing simulator.

Many species of bats rely on echoes to forage and navigate in densely vegetated environments. Foliage echoes in some cases can help bats gather information about the environment, whereas in others may generate clutter that can mask prey echoes during foraging. It is therefore important to study foliage echoes and their role in bat's sensory ecology. In our prior work, a foliage echo simulator has been developed; simulated echoes has been compared with field recordings using a biomimetic sonar head. In this work, we improve the existing simulator by allowing more flexible experimental setups and enabling a closer match with the experiments. Specifically, we add additional features into the simulator including separate directivity patterns for emitter and receiver, the ability to place emitter and receiver at distinct locations, and multiple options to orient the foliage to mimic natural conditions like strong wind. To study how accurately the simulator can replicate the real echo-generating process, we compare simulated echoes with experimental echoes measured by ensonifying a single leaf across four different species of trees. We further extend the prior work on estimating foliage parameters to estimating a map of the environment.

Autonomous localization and mapping based on biomimetic sonar in natural environments

Bats navigating in dense vegetation based on biosonar have to obtain the necessary sensory information from “clutter echoes,” i.e., echoes that are superpositions of contributions of many reflecting facets (e.g., leaves). Since the locations and reflective properties of the individual facets are unknown, clutter echoes have to be treated as random signals that can neither be predicted nor—under most practical circumstance—be replicated. Nevertheless, prior research has shown that deep neural networks are capable of extracting fairly precise location information from clutter echoes. This raises the question whether clutter echoes could be used to provide navigational guidance in a conventional simultaneous localization and mapping (SLAM) framework, which is to a large extent dependent on precise and deterministic measurement to generate a localized map. Our hypothesis is that biomimetic sonar echoes indeed contain the information that is necessary support to local path planning, which can be utilized by a suitable deep learning architecture and training process. To investigate this issue, we have collected data in natural, heavily vegetated environments using a biomimetic sonar head that mimics the periphery of the biosonar system in horseshoe bats. The annotation of the data and the network training process is currently undergoing.

Investigating of the impact of biommetic pinna dynamics on sonar tasks in natural environments

Bats are known to perform incredible feats using their echolocation abilities, which include natural tasks, such as navigation through foliage and hunting prey, as well as tasks they have been trained to perform, such as classification of man-made objects. Much work has been done to engineer biomimetic systems that can perform these and similar tasks in a comparable way to bats. Biomimetic sonar devices have performed tasks such as identifying a geographic location, object classification, and foliage gap finding. Particular species of bats (notably those of the families Hipposideridae and Rhinolophidae) use rapid ear motion while making sonar calls, and the full effect and purpose of this motion is not fully understood. Our research goal is to perform sonar tasks with a biomimetic sonar head capable of both static and dynamic ear motion and investigate the effect of dynamic ear motion on captured echoes and performance on tasks. To this end, we have collected echo data in natural habitats with dense tropical vegetation in Brunei on the island of Borneo. Tasks currently under investigation include location identification and detection of man-made objects in foliage, and are performed by deep learning classifiers trained on echoes taken by the biomimetic sonar device.

Bioinspired solution to finding passageways in foliage with sonar

Finding narrow gaps in foliage is an important component skill for autonomous navigation in densely vegetated environments. Traditional approaches are based on collecting large amounts of data with high spatial resolution. However, the biosonar systems of bats that live in dense habitats demonstrate that finding gaps is possible based on sensors with angular resolutions that are poor compared to technologies such as man-made sonar and lidar. To investigate these capabilities, we have used a biomimetic sonar head to ensonify artificial hedges in the laboratory. We found that a conventional approach based on echo energy performed poorly on detecting gaps with the area under the receiver operating characteristic (ROC) curve ranging from 0.69 to 0.75 depending on the distance to the hedge and gap width. A deep-learning approach based on a convolutional neural network (CNN) operating on the echo spectrograms achieved area under the ROC curve (AUC) values between 0.94 and 0.97. Class activation mapping indicated that the rising flank of the echoes was critical for detecting the gaps. As a consequence, a simple code consisting of first threshold-crossing times was able to almost reproduce the performance of the CNN classifier (AUC 0.9 to 0.95). This demonstrates that the echo waveforms contained patterns that were indicative of a gap in the foliage but did not suffer from the comparatively large beamwidth used.

Deep-learning exploration of the acoustic granularity of bat habitats

Many bat species are able to find their way through densely vegetated habitats and select their microhabitats to suit their respective needs. This raises the question as to how granular the natural habitats of bats are when sensed through the echoes triggered by the animals' biosonar pulses. To investigate this question, a portable biomimetic sonar head has been used to collect about 40 000 foliage echoes across a natural forest site on the Virginia Tech campus that was approximately 250 × 140 m2 in size. Each recorded echo was assigned a spatial location that was estimated from the outputs of a concurrently operated GPS receiver. These spatial locations were then used to cluster the echoes into a varying number of compact patches using a k-means clustering algorithm. To determine whether these spatial patches could be determined from the echoes, a convolutional neural network based on the ResNet50 architecture was trained to classify the echoes with respect to their spatial labels. Even based on just a single echo, up to 100 different spatial patches could be distinguished in this way with accuracies greater than 95%. These results demonstrate that bat biosonar can capture sensory information for small-scale navigation in natural forest habitats.

Dynamic echo signatures created by a biomimetic sonar head

Certain bat species (e.g. horseshoe bats, family Rhinolophidae) are known for conspicuous deformations of the emission baffles (noseleaves) and reception baffles (ears). Previously reported numerical studies and experiments with biomimetic reproductions of these baffles have shown that such deformations can result in time-variant emitter/receiver characteristics. However, it has not been investigated whether these time-variant characteristics could also manifest themselves in likewise time-variant properties in echoes from targets of varying complexity. To investigate this question, a biomimetic sonar head complete with deformable emission and reception baffles has been used to ensonify targets with different simple geometries (sphere, cylinder, and cube) as well as random, more natural target geometries (artificial plants) from distances of about 1 meter. Time-variant echo signatures were found in all these cases, i.e. irrespective of target complexity and whether the time-variance was injected into the emission, the reception, or into both. This demonstrates that although the time-variant emission/reception characteristics had been previously measured only under careful conditions, they are capable of impacting real-world echoes. Even targets with distributed clouds of scattering facets did not obscure the effects of the changing conformation states. Hence these changes in ear position created by baffle deformations could serve the animals or man-made sonar systems that mimic them to encode additional echo information through time-variant echo signatures.

Deep learning of biosonar landmarks for navigation in forest environments

Many bat species live in densely vegetated habitats which pose challenges to their biosonar systems. Among these challenges, the problem of identifying prey in clutter has received the most attention. Here, the far less well-studied problem of landmark identification in forest environments has been investigated. To this end, a large data set of about 220 000 foliage echoes has been collected along different tracks located in forested area. The echoes were recorded using a biomimetic sonarhead with flexible noseleaf and pinnae modeled on the periphery of horseshoe bats. Low-dimensional representations of these foliage echoes were created with the encoder portion of a variational autoencoder deep neural network architecture. The feature vectors obtained in this manner were subjected to clustering to determine whether the echo recordings exhibited continuous variability or fell into discernible clusters. The data silhouettes indicate the presence of a small number of distinguishable clusters in the echo data. Furthermore, mapping the assigned cluster labels to the geographical coordinates of the respective echo recordings revealed that different tracks were characterized by a different “fingerprints” of echo classes. Hence, these fingerprints could be a hypothetical basis for biosonar-based navigation in forest environments.Many bat species live in densely vegetated habitats which pose challenges to their biosonar systems. Among these challenges, the problem of identifying prey in clutter has received the most attention. Here, the far less well-studied problem of landmark identification in forest environments has been investigated. To this end, a large data set of about 220 000 foliage echoes has been collected along different tracks located in forested area. The echoes were recorded using a biomimetic sonarhead with flexible noseleaf and pinnae modeled on the periphery of horseshoe bats. Low-dimensional representations of these foliage echoes were created with the encoder portion of a variational autoencoder deep neural network architecture. The feature vectors obtained in this manner were subjected to clustering to determine whether the echo recordings exhibited continuous variability or fell into discernible clusters. The data silhouettes indicate the presence of a small number of distinguishable clusters in the echo data...

Integrating a brain into a bat robot

The sophisticated biosonar systems of horseshoe bats have enabled these animals to navigate and pursue prey in complex environments. A conspicuous peripheral dynamics in which the animals' noseleaves and pinnae change during biosonar behaviors could play an important role in enabling these capabilities. It may be hypothesized that for the integration between peripheral dynamics and neural signal processing/estimation to be maximally effective, the periphery should be controlled by feedback from the output of the subsequent neural echo processing. In the way, the specifics of sensory information encoding in the periphery could be controlled based on the needs of the neural signal processing. As a first step towards such an integration in a biomimetic sonar head, a computational model for the inner ear and the auditory nerve's spike code has been integrated with a dynamic periphery that—like the computational models—mimics horseshoe bats. For each model stage, alternative versions with different levels of complexity have been implemented to test how module complexity and the values of the associated parameters affect the capacity of the echo representation to encode sensory information. These effects have been tested based on a large dataset of 220 000 echoes collected in natural forest environments.

Biomimetic solutions to finding passageways in foliage

Many bats species live in densely vegetated habitats. Hence, they must have evolved the ability to detect passageways through the foliage. To identify echo features that bats have at their disposal to accomplish this sensing task, a biomimetic sonar head was used to ensonify artificial foliages in the laboratory. These foliages consisted of plastic vines that were arranged to create gaps of defined width and height. The conventional sonar approach to detecting gaps between scatterers is based on the drop in energy that occurs when the sonar beam is aimed at a gap. However, the performance of such an energy-detector decrease as the sonar-beam width increases relative to the angle subtended by the gap edges. Hence, reliable detection of a narrow gap with a wide beam and/or from a distance is not feasible with this approach. Here, a machine-learning approach based on convolutional neural networks was employed to identify features in energy-normalized spectrogram that could support passageway-detection independently of echo energy. The results indicate that features other than echo energy exist in the spectrogram that by themselves support much better passageway detection performance than the energy-based reference. Work to understand the features learned by the convolutional neural network is currently underway.

Emission baffle deformations in bat biosonar and biomimetic systems

Old-World leaf-nosed bats (Hipposideridae) and horseshoe bats (Rhinolophidae) are two families of echolocating bats that emit their biosonar pulses through nostrils which are surrounded by elaborated baffle shapes (“noseleaves” ). Prior work has shown that the noseleaves change shape in synchrony with ultrasound emission. These deformations involve at least two noseleaf parts with static (geometric) as well as dynamic complexity. To investigate how these deformations could be used in bioinspired systems, data has been collected from bats recorded with microphone/camera arrays as well as a biomimetic sonar head with a biomimetic noseleaf model inspired by Pratt's roundleaf bats (Hipposideros pratti). Both sets of experiments have produced qualitatively similar results that show an emitted wave field which depends on frequency, direction, and also time. Furthermore, the results obtained with the biomimetic noseleaf have demonstrated that noseleaf deformations during ultrasound emission can result in an enhanced information-encoding capacity. Current work is aimed at establishing whether this enhanced coding capacity can be utilized to improve performance, especially in sensing tasks that are related to the natural environments in which the bats' biosonar systems operate. It also needs to be established how much detail needs to be mimicked in an engineered system to harness these effects.

Biomimetic foliage echo simulation

Many echolocating bat species are capable of navigating through highly cluttered environments, such as dense foliage—apparently without difficulty. Since interpreting foliage echoes must hence be key to the animals' navigation capabilities, simulating such echoes would be a major step towards understanding bat navigation. This is a difficult task, as vegetation echoes are highly complex and stochastic signals. Prior work on modeling foliage echoes frequently relied on physical approximations, since full physics models are computationally infeasible. For simulating the echo from a leaf, a round disk approximation has been used, since it’s echo has a known physical solution. These disks have been distributed according to the locations of leaves in a tree using a Lindenmayer System (L-System), thus creating a full tree echo model. Successful prior work in using a machine learning approach for echo based tree classification, shows that a data driven approach may also be useful in modeling echoes. Hence, future research will be using machine learning methods, such as auto-encoders and generative adversarial networks, to simulate the echoes of leaves and trees. The echo data to be fed into these methods will be of leaves recorded in an anechoic chamber using a biomimetic sonar head.

A design for a dynamic biomimetic sonarhead inspired by horseshoe bats

The noseleaf and pinnae of horseshoe bats (Rhinolophus ferrumequinum) have both been shown to actively deform during biosonar operation. Since these baffle structures directly affect the properties of the animals biosonar system, this work mimics horseshoe bat sonar system with the goal of developing a platform to study the dynamic sensing principles horseshoe bats employ. Consequently, two robotic devices were developed to mimic the dynamic emission and reception characteristics of horseshoe bats. The noseleaf and pinnae shapes were modeled as smooth blanks matched to digital representations of a horseshoe bat specimens noseleaf and pinnae. Local shape features mimicking structures on the pinnae and noseleaf were added digitally. Flexible baffles with local shape feature combinations were manufactured and paired with actuation mechanisms to mimic pinnae and noseleaf deformations in vivo. Two noseleaves with and without local shape features were considered. Each noseleaf baffle was mounted to a platform called the dynamic emission head to actuate three surface elements of the baffle. Similarly, 12 pinna realizations composed of combinations of three local shape features were mounted to a platform called the dynamic reception head to deform the left and right pinnae independently. Motion of the noseleaf and pinnae were synchronized to the incoming and outgoing sonar waveform, and the joint time-frequency properties of the noseleaf and pinnae local feature combinations and pairs of pinnae and noseleaf thereof were characterized across spatial direction. Amplitude modulations to the outgoing and incoming sonar pulse information across spatial direction were observed for all pinnae and noseleaf local shape feature combinations. Peak modulation variance generated by motion of the pinnae and combinations of the noseleaf and pinnae approached a white Gaussian noise variance bound. It was found the dynamic emitter generated less modulation than either the combined or reception scenarios.

System integration for a biomimetic dynamic sonar head

Horseshoe bats (Rhinolophidae) have dynamic biosonar systems with interfaces for ultrasonic emission/reception that deform while diffracting the outgoing/incoming sound waves. This peripheral dynamics has been shown to enhance sensory information encoding. To investigate the properties and potential uses of these system features in the real world, a biomimetic sonar head is being developed with the goal of mimicking as much of the bats' peripheral dynamics as faithfully as possible. Further constraints on this development were suitability for integration with mobile platforms and support for interactive quantitative/in-depth experimentation. Flexible noseleaf and pinna shapes along with an actuation system have been designed to mimic the static geometry of the respective structures in bats as well as several of the animals' degrees of freedom in deforming them. The electrical subsystem has been designed to support high-quality ultrasound generation and recording. Generation of all source voltages and signal amplification have been consolidated into a single circuit board to reduce weight and complexity of the overall system. The frame has been altered to reduce weight and add modularity to the noseleaf and pinna elements. The software architecture is based on a back-end (Python, LabVIEW) combined with a flexible front-end (MATLAB) for experimental design and data analysis.Horseshoe bats (Rhinolophidae) have dynamic biosonar systems with interfaces for ultrasonic emission/reception that deform while diffracting the outgoing/incoming sound waves. This peripheral dynamics has been shown to enhance sensory information encoding. To investigate the properties and potential uses of these system features in the real world, a biomimetic sonar head is being developed with the goal of mimicking as much of the bats' peripheral dynamics as faithfully as possible. Further constraints on this development were suitability for integration with mobile platforms and support for interactive quantitative/in-depth experimentation. Flexible noseleaf and pinna shapes along with an actuation system have been designed to mimic the static geometry of the respective structures in bats as well as several of the animals' degrees of freedom in deforming them. The electrical subsystem has been designed to support high-quality ultrasound generation and recording. Generation of all source voltages and sign...

Integration of emitter and receiver dynamics in a biomimetic sonar head

Horseshoe bats (family Rhinolophidae) transmit and receive sonar information about the environment using morphologically complex and dynamic noseleaf and pinnae. Both local shape features and dynamic deformations of the noseleaf and pinnae contribute to properties of the bat biosonar system. This work systematically explores the effect of local shape features on the dynamic properties of emission and reception baffles using a biomimetic robotic system inspired by the horseshoe bat. Parametric models of the greater horseshoe bat (Rhinolophus ferrumequinum) noseleaf and pinnae with selectable combinations of local shape features were developed. Baffles with feature realizations were cast out of flexible silicone and mounted to a platform to dynamically actuate the emission and reception baffle surfaces with motions patterns similar to the greater horseshoe bat. Motions of the baffle surfaces were synchronized to the outgoing and incoming sonar waveform, and the time-frequency properties of the emission and ...

Dynamic periphery in a biomimetic sonar system introduces time-variant signatures into targets echoes

Bats have developed unique and refined systems of echolocation throughout the course of their evolutionary history, giving them the ability to navigate and hunt in extremely cluttered environments. While the mechanisms behind many of these abilities remain unknown, it has been observed that the most effective biosonar systems in bats use a variety of dynamic sensing mechanisms. One conspicuous manifestation of this dynamics can be seen in changes to the shapes of the baffle that diffract the emitted biosonar pulses (noseleaves) and the returning echoes (outer ears). Using numerical predictions as well as measurements with biomimetic hardware, our own prior work has established that the dynamics in these baffles can create time-variant emitter and receiver characteristics. However, it has yet to be demonstrated that these time-variant device characteristics have a substantial impact on the received echoes. To address this question, a biomimetic sonar head with dynamic emission and reception baffles was use...

A design for a biomimetic dynamic sonar head

The biosonar system of horseshoe bats (family Rhinolophidae) has been shown to employ unusual dynamics upon the emission as well as the reception of the ultrasonic pulses. Non-rigid changes to the shapes of the noseleaves (emission baffles) as well as the outer ears (pinnae, reception baffles) have been demonstrated to affect the properties of the emitted and received ultrasonic signals. These effects have been found in the results of numerical simulations as well as experimentation with physical prototypes. In the present work, a next-generation prototype of a biomimetic sonar head inspired by horseshoe bats is being developed. The goals for this system are to create more comprehensive and life-like dynamic baffle shape geometries as well as a better acoustic coupling between the ultrasonic transducers and the time-variant baffle shapes. Particular attention has been paid to geometry of the transition between nostrils and the noseleaf baffle. A single biomimetic system that incorporates these dynamic emission and reception baffles will enable an experimental investigation of how these two dynamic stages could be used in an integrated fashion to enhance sonar performance in real-world sonar sensing scenarios.

Airborne ultrasonic image generation with biomimetic sonar head

This work presents the results of airborne ultrasonic image generation based on biomimetic sonar heads. Ultrasonic images refer to the representation of the environment with the help of ultrasonic waves and based on sonar principle. The generation of ultrasonic images is based on the classical methodology of the sonar, having target detection and classification purposes as main applications. An element of the image (which could be assimilated as pixel) has the intensity proportional to signals reflected by the surface of objects in the explored environment. Sonar heads with biomimetic functional behaviour (i.e. having one emitter and two receivers) were built. The key motivation in deploying such sonar heads is efficiency, that is, simplicity combined with acceptable performances based on ultrasonic image processing. Simplicity is related to modularity and portable structure, in the sense of computer's portability. Biomimetism has a double meaning, by the structure of the sonar head and by the nature of the exploring algorithms (which use observations on bat behaviour). This work considers airborne ultrasonic images at distances less than one metre and with fixed working frequency (40 kHz). The preliminary results show the feasibility of the approach and consolidate a research direction focused on navigation based on airborne ultrasonic image processing.

A novel biomimetic sonarhead using beamforming technology to mimic bat echolocation

A novel biomimetic sonarhead has been developed to allow researchers of bat echolocation behavior and biomimetic sonar to perform experiments with a system similar to the bat¿s sensory system. The bat's echolocation-related transfer function (ERTF) is implemented using an array of receivers to implement the head-related transfer function (HRTF), and an array of emitters mounted on a cylindrical manifold to implement the emission pattern of the bat. The complete system is controlled by a field-programmable gate array (FPGA) based embedded system connected through a USB interface.

Narrowband target tracking using a biomimetic sonarhead

This work presents a target tracking method that exploits both the physics of narrowband echolocation and appropriate movements of the sensor, taking inspiration from narrowband (CF-FM) bats. The platform used in this work is RoBat, a biomimetic sonarhead mounted on a mobile platform. Two different techniques are integrated in the tracker. First, Barshan’s and Kuc’s method for estimating target range and azimuth angle is adapted to the geometry of the sonarhead. Next, Walker’s arc scanning hypothesis for target elevation estimation in CF-FM bats is implemented, applying curve fitting for an accurate estimation of the peak-delay. The integration of both techniques in a preliminary 3D tracking system is performed applying both methods independently.

The use of Doppler in Sonar-based mobile robot navigation: inspirations from biology

So-called CF–FM bats are highly mobile creatures who emit long calls in which much of the energy is concentrated in a single frequency. These bats face sensor interpretation problems very similar to those of mobile robots provided with ultrasonic sensors, while navigating in cluttered environments. This paper presents biologically-inspired engineering on the use of narrowband Sonar in mobile robotics. It replicates, using robotics as a modelling medium, methods CF–FM bats use to exploit Doppler-shifts––a rich source of information not used by commercial robotic ultrasonic range sensors––in different tasks. The experimental platform for the work is RoBat, a 6 DOF biomimetic sonarhead mounted on a commercial 3 DOF mobile platform. The platform is provided with signal processing capabilities inspired by the bat's auditory system. The CF–FM bat modifies––increasing or decreasing––the carrier frequency of its own calls, compensating the Doppler-shift produced when the bat, the reflector or both are moving. This echolocating behaviour, called Doppler-shift compensation, is successfully implemented in RoBat. Inspired by this behaviour, a convoy navigation controller following a set of simple Doppler-dependent rules is successfully devised. The performance of the controller is satisfactory despite low Doppler-shift resolution caused by the lower velocity of the robot when compared to real bats. Finally, Müller's hypothesis on the use of acoustic flow by CF–FM bats for obstacle avoidance is also implemented in RoBat, resulting in a crude estimation of the target's passing distance at small bearing angles, which improves as the angle increases, nevertheless sufficing for avoiding the two reflectors of the experiment.