Biosonar System Research Articles

A robotic platform for integrating biomimetic sonar and AI

To gain a first-hand view of the biosonar inputs that a bat receives when operating in a complex natural environment, a bioinspired sonar head to mimic the auditory periphery of the bats' biosonar system is currently under development. The sonar head seeks to replicate the static geometric complexity of the baffles that emit the biosonar pulses and receive the returning echoes in horseshoe bats. Furthermore, some of the dynamic deformations of these structures are mimicked using tension-driven soft-robotics actuated by a set of 12 motors (two for the noseleaf and five for the each pinna). The periphery has been interfaced with acoustic emission and reception systems as well as a modular system control system for the acoustic input and output as well as the motors. Besides low-level control, the sonar head has onboard computing resources that can support deep-learning inference in the loop with acoustic data acquisition and pulse generation. All functional components have been integrated into a custom-designed shell that allows operating the system in natural outdoor environments. A wireless user interface allows remote control over the system so that the tasks of carrying and orienting the sonar head and controlling its function can be shared between two individuals.

Spatiotemporal dynamics of biosonar in navigating Bornean Rhinolophid and Hipposiderid bats

To investigate the active sensing strategies used by echolocating bats of the genera Rhinolophus and Hipposideros, we have constructed a 9-meter long flight tunnel, which incorporates an array of 32 ultrasonic microphones distributed throughout the tunnel. Rhinolophus and Hipposideros are of special interest because of their highly flexible biosonar system; these bats emit pulses from their nasal cavities, using complex noseleaf structures to quickly and precisely alter the beam-form and direction of emissions. Additionally, each species utilizes a unique combination of constant-frequency (CF) and frequency-modulated (FM) ultrasonic signals with varying durations, repetition rates, and frequencies. We plan to trap several species of wild Bornean bats of these genera and fly individual bats through the tunnel; a time-of-arrival algorithm will be used to localize the position of each bat at the time of each biosonar pulse emission, and an amplitude-comparison method to measure the horizontal and vertical direction of each emission from the bat’s noseleaf. We will also incorporate relatively simple foliage obstacles into the tunnel; this will create complex acoustic clutter and allow us to determine how bats of different species adjust their biosonar sampling strategies in order to navigate around novel obstacles in a cluttered environment.

Autonomous localization and mapping based on biomimetic sonar in natural environments

Bats navigating in dense vegetation based on biosonar have to obtain the necessary sensory information from “clutter echoes,” i.e., echoes that are superpositions of contributions of many reflecting facets (e.g., leaves). Since the locations and reflective properties of the individual facets are unknown, clutter echoes have to be treated as random signals that can neither be predicted nor—under most practical circumstance—be replicated. Nevertheless, prior research has shown that deep neural networks are capable of extracting fairly precise location information from clutter echoes. This raises the question whether clutter echoes could be used to provide navigational guidance in a conventional simultaneous localization and mapping (SLAM) framework, which is to a large extent dependent on precise and deterministic measurement to generate a localized map. Our hypothesis is that biomimetic sonar echoes indeed contain the information that is necessary support to local path planning, which can be utilized by a suitable deep learning architecture and training process. To investigate this issue, we have collected data in natural, heavily vegetated environments using a biomimetic sonar head that mimics the periphery of the biosonar system in horseshoe bats. The annotation of the data and the network training process is currently undergoing.

Numerical-modeling-based investigation of sound transmission and reception in the short-finned pilot whale (Globicephala macrorhynchus)

The sound-transmission, beam-formation, and sound-reception processes of a short-finned pilot whale (Globicephala macrorhynchus) were investigated using computed tomography (CT) scanning and numerical simulation. The results showed that sound propagations in the forehead were modulated by the upper jaw, air components, and soft tissues, which attributed to the beam formation in the external acoustic field. These structures owned different acoustic impedance and formed a multiphasic sound transmission system that can modulate sounds into a beam. The reception pathways composed of the solid mandible and acoustic fats in the lower head conducted sounds into the tympano-periotic complex. In the simulations, sounds were emitted in the forehead transmission system and propagated into water to interrogate a steel cylinder. The resulting echoes can be interpreted from multiple perspectives, including amplitude, waveform, and spectrum, to obtain the acoustic cues of the steel cylinder. By taking the short-finned pilot whale as an example, this study provides meaningful information to further deepen our understanding of biosonar system operations, and may expand sound-reception theory in odontocetes.

A simulation framework for bio-inspired sonar sensing with Unmanned Aerial Vehicles.

We introduce a unified simulation framework that generates natural sensing environments and produces biosonar echoes under various sensing scenarios. This framework produces rich sensory data with environmental information completely known, thus can be used for the training of robotic algorithms for biosonar-based Unmanned Aerial Vehicles. The simulated environment consists of random trees with full geometry of the tree foliage. To simulate a single tree, we adopt the Lindenmayer system to generate the initial branching pattern and integrate that with the available measurements of the 3D computer-aided design object files to create natural-looking branches, sub-branches, and leaves. A forest is formed by simulating trees at random locations generated by using an inhomogeneous Poisson process. While our simulated environments can be generally used for testing other sensors and training robotic algorithms, in this study we focus on testing bat-inspired Unmanned Aerial Vehicles that recreate bat’s flying behavior through biosonar sensors. To this end, we also introduce an foliage echo simulator that produces biosonar echoes while mimicking bat’s biosonar system. We demonstrate the application of the proposed simulation framework by generating real-world scenarios with multiple trees and computing the resulting impulse responses under static or dynamic motions of an Unmanned Aerial Vehicle.

Physical implementation of dolphin biosonar to facilitate ultrasound control

We demonstrated that the feats of the dolphin biosonar system can be achieved through physical implementation. Numerical and experimental results suggested that dolphins have evolved to intelligently manipulate physical laws. Gradient distributions of sound speed and density in the forehead counterpart can enhance the main beam by gathering more sound energy to reinforce the main beam and lowering side lobes. As dolphins prove to accomplish efficient control on their biosonar capabilities in multiple ways, this paper provides an additional aspect to enrich our understanding of how one of the best natural biosonar systems works and build a step to inspire additional advanced sound control systems.

An experimental link between fast noseleaf deformations and biosonar pulse dynamics in hipposiderid bats

Old-World leaf-nosed bats (Hipposideridae) are echolocating bats with peculiar emission-side dynamics where beamforming baffles ("noseleaves") that surround the points of ultrasound emission (nostrils) change shape while diffracting the outgoing biosonar pulses. While prior work with numerical and robotic models has suggested that these noseleaf deformations could have an impact on the output characteristics of the bat's biosonar system, testing the hypothesis that this is the case in bats remains a critical step to be taken. The work presented here has tested the hypothesis that the noseleaf dynamics in a species of hipposiderid bat (Pratt's roundleaf bat, H. pratti) leads to time-variant acoustical properties on the output side of the bats' biosonar emission system. The time-variant effects of the noseleaf motion could be detected even in the presence of other sources of variability by comparing the distribution of pulse energy over the angle at different points in time. Furthermore, a convolutional neural network was able to classify the noseleaf motion state based on microphone array recordings with 85.3% accuracy. These results hence demonstrate that these nose-emitting bats have access to a substrate for behavioral flexibility on the emission-side of their biosonar systems.

Sound reception pathway of the Indo-Pacific humpback dolphin

The Indo-Pacific humpback dolphins (Sousa chinensis) are nearshore odontocetes, distributed in tropical and sub-tropical oceans. This species has been studied to unveil its ability to echolocate. Indo-Pacific humpback dolphin, like its Odontocetes companion, relies on echolocation system to navigate and detect targets, which contains a sound transmitting system in the forehead and a sound reception in the jaw. Their soft tissues present gradient sound speed and density distributions in the forehead. Solid skull, air structures and soft tissues form a natural multi-phase meta-material to modulate sounds into energy focused beams. This multi-phase property is also applied to the hearing system as revealed in current papers. Here in this work, the physical mechanism of sound reception in the Indo-Pacific humpback dolphin is studied by using the computed tomography (CT) scanning, physical measurements and numerical simulation. Hounsfield units (HUs) of the forehead tissues are extracted from CT scanning results. A linear relationship is revealed between HU and sound speed, HU and density, which are combined with HU distribution to reconstruct the sound speed and density distribution of the sound reception system. The CT scanning shows that the sound reception system located at lower head is composed of external mandibular fat, internal mandibular fat, mandible and hearing bones. Model of sound reception system is developed on the basis of CT scanning results and used in subsequent simulations. The physical process of sound reception reveals that the hearing system can guide sounds through variable pathways to reach hearing bones. Sounds can enter into the reception system along the acoustic pathways composed of mandible, external mandibular fat and internal mandibular fat. Mandibular fat and mandible form a unique sound pathway. In addition, another pathway which is composed of external mandibular fat, pan bone and internal mandibular fat can lead the sound to propagate and finally arrive at hearing bones. The diversity of acoustic pathways is applicable to a range of frequencies from 30 to 120 kHz. The variability of acoustic pathways in Indo-Pacific humpback dolphin shows the complexity of its biosonar system. The anatomy and simulation results can deepen our understanding of the mechanism of echolocation of Indo-Pacific humpback dolphin and provide references for designing man-made sound reception devices.

Learning the relationship between biosonar pulse trains and peripheral dynamics in hipposiderid bats

The biosonar system of hipposiderid bats is characterized by a peripheral dynamics consisting of variable noseleaf and pinna motions as well as temporal variability in the emitted pulse trains. It may be hypothesized that a well-integrated biosonar emission system should coordinate noseleaf and pinna motions with the biosonar pulse trains if this leads to synergistic effects. To investigate whether a relationship between these two sources of variability in the emission system of hipposiderid bats exists, a data set with high-speed stereo video recordings of noseleaf and pinna motions that were synchronized with ultrasonic recordings of the biosonar pulses has been analyzed. Different similarity metrics that were developed for analyzing neural spike sequences have been tested for comparing the individual biosonar pulse trains. Based on these metrics, pulse trains were clustered and the results were compared to the different categories of noseleaf and pinna motions that were found in the video recordings. The results obtained so far for 80 recorded sequences show that noseleaf/pinna motions and pulse trains can be organized into joint clusters, i.e., certain categories of peripheral motions cluster with specific types of biosonar pulse trains. These links could indicate a synergistic relationship between noseleaf/pinna motions and pulse timings.The biosonar system of hipposiderid bats is characterized by a peripheral dynamics consisting of variable noseleaf and pinna motions as well as temporal variability in the emitted pulse trains. It may be hypothesized that a well-integrated biosonar emission system should coordinate noseleaf and pinna motions with the biosonar pulse trains if this leads to synergistic effects. To investigate whether a relationship between these two sources of variability in the emission system of hipposiderid bats exists, a data set with high-speed stereo video recordings of noseleaf and pinna motions that were synchronized with ultrasonic recordings of the biosonar pulses has been analyzed. Different similarity metrics that were developed for analyzing neural spike sequences have been tested for comparing the individual biosonar pulse trains. Based on these metrics, pulse trains were clustered and the results were compared to the different categories of noseleaf and pinna motions that were found in the video recordings. T...

Transmission beam pattern and dynamics of a spinner dolphin (Stenella longirostris).

Toothed whales possess a sophisticated biosonar system by which ultrasonic clicks are projected in a highly directional transmission beam. Beam directivity is an important biosonar characteristic that reduces acoustic clutter and increases the acoustic detection range. This study measured click characteristics and the transmission beam pattern from a small odontocete, the spinner dolphin (Stenella longirostis). A formerly stranded individual was rehabilitated and trained to station underwater in front of a 16-element hydrophone array. On-axis clicks showed a mean duration of 20.1 μs, with mean peak and centroid frequencies of 58 and 64 kHz [standard deviation (s.d.) ±30 and ±12 kHz], respectively. Clicks were projected in an oval, vertically compressed beam, with mean vertical and horizontal beamwidths of 14.5° (s.d. ± 3.9) and 16.3° (s.d. ± 4.6), respectively. Directivity indices ranged from 14.9 to 27.4 dB, with a mean of 21.7 dB, although this likely represents a broader beam than what is normally produced by wild individuals. A click subset with characteristics more similar to those described for wild individuals exhibited a mean directivity index of 23.3 dB. Although one of the broadest transmission beams described for a dolphin, it is similar to other small bodied odontocetes.

Biomimetic sonar and the problem of finding deterministic targets in foliage

The ability of (bio)sonars to find targets-of-interest is often hampered by a cluttered environment. For example, naval sonars encounter difficulties finding mines partially or fully buried within sand. Such situations pose target identification challenges are much harder than target detection and resolution problems. There are many bat species which navigate and hunt in dense vegetation and thus must be able to identify targets-of-interest within clutter. Evolutionary adaptation of the bat biosonar system is likely to have resulted in the “discovery” of features that support making distinctions between clutter and echoes of interest. The most well-established case is given by cf-fm bats that use Doppler shifts caused by the wingbeat of a flying insect prey to identify the prey in foliage. Other bat species have been shown to use a passive sonar approach that is based on unique prey-generated acoustic signals. Some of the most interesting cases can be found in bat species that are successful in finding preys that apparently does not emit any distinguishing sounds themselves and would hence limit the bats to an active-sonar approach. Such bat species could provide model species for new ways in which the target-identification problem in clutter can be solved with active sonar.

Ultrasound navigation based on minimally designed vehicle inspired by the bio-sonar strategy of bats

ABSTRACTIn this study, we conducted vehicle experiments and a numerical simulation based on a simple algorithm inspired by the bio-sonar system of bats to investigate how the behavioral strategy employed by bats contributes to acoustic navigation for minimal-design sensing. In particular, a double-pulse scanning method inspired from the echolocation behavior of bats was proposed, in which (1) the direction of ultrasound emission by a vehicle equipped with 1 transmitter and 2 receivers was alternately shifted between the movement direction of the vehicle and the direction of the nearest obstacle, and (2) the movement direction of the vehicle was calculated for every double-pulse emission based on integrated information from all echoes detected. As a result of 100 repeated drives in a practical course, the success rate of an obstacle-avoidance drive improved from 13% with the conventional single-pulse scanning method to 73% with the proposed method. Furthermore, the numerical simulation demonstrated that the proposed method achieves robust path planning by suppressing the localization ambiguity due to the interference of multiple echoes. The practical experiments and numerical simulation suggest that bats employ a simple behavioral solution in the operation of acoustic sensing for various problems occurring in the real world.

Dynamic relationship between noseleaf and pinnae in echolocating hipposiderid bats

Old World leaf-nosed bats (family Hipposideridae) can deform the shapes of their 'noseleaves' (i.e. ultrasonic emission baffles) and outer ears during echolocation behaviors. Prior work has shown that deformations on the emission as well as on the reception side can have an impact on the properties of the emitted/received sonar signals. The occurrence of the deformations on the emission and reception sides raises the question of whether the bats coordinate these two dynamic biosonar features to achieve synergistic effects. To address this question, simultaneous three-dimensional reconstructions of the trajectories of landmarks on the dynamic noseleaf and pinna geometries have been obtained in great roundleaf bats (Hipposideros pratti). These joint kinematics data on the noseleaf and pinnae have shown both qualitative and quantitative relationships between the noseleaf and pinna motions: large noseleaf deformations (opening or closing) tended to be associated with non-rigid pinna motions. Furthermore, closing deformations of the noseleaves tended to co-occur with closing motions of the pinna. Finally, a canonical correlation analysis of the motion trajectories has revealed a tight correlation between the motions of the landmarks on the noseleaf and both pinnae. These results demonstrate that the biosonar system of hipposiderid bats includes coordinated emission and reception dynamics.

Mobile targets and sensor mobility in bat biosonar

Many species of echolocating bats pursue highly mobile prey such as flying insects. Target mobility poses a challenge when a prey executes escape maneuvers. In these cases, bats and their prey find themselves locked in evolutionary arms races. However, prey mobility also provides the bats with opportunities for solving the problem of detecting prey buried in clutter. This applies to active as well as passive biosonar. In passive biosonar, bats frequently listen to sounds that are generated involuntarily by the motion of a prey. In active biosonar, certain bat groups have evolved highly specialized biosonar systems to detect flying prey insects by virtue of unique Doppler signatures that are generated by the insect's wing beat. Since the Doppler shifts concerned are very small, this sensory feat requires a well-integrated suit of evolutionary specializations that include sonar signal design, cochlear filtering, representations all through the auditory system, and even adaptive behavioral control of the emitted frequencies. In addition to this complexity, bats with such a biosonar system also employ a peripheral mobility where the baffles for pulse emission and reception move rapidly during biosonar operation. It remains to be seen how these mobilities fit together to support the animal's sensory abilities.Many species of echolocating bats pursue highly mobile prey such as flying insects. Target mobility poses a challenge when a prey executes escape maneuvers. In these cases, bats and their prey find themselves locked in evolutionary arms races. However, prey mobility also provides the bats with opportunities for solving the problem of detecting prey buried in clutter. This applies to active as well as passive biosonar. In passive biosonar, bats frequently listen to sounds that are generated involuntarily by the motion of a prey. In active biosonar, certain bat groups have evolved highly specialized biosonar systems to detect flying prey insects by virtue of unique Doppler signatures that are generated by the insect's wing beat. Since the Doppler shifts concerned are very small, this sensory feat requires a well-integrated suit of evolutionary specializations that include sonar signal design, cochlear filtering, representations all through the auditory system, and even adaptive behavioral control of the emi...

A design for a dynamic biomimetic sonarhead inspired by horseshoe bats

The noseleaf and pinnae of horseshoe bats (Rhinolophus ferrumequinum) have both been shown to actively deform during biosonar operation. Since these baffle structures directly affect the properties of the animals biosonar system, this work mimics horseshoe bat sonar system with the goal of developing a platform to study the dynamic sensing principles horseshoe bats employ. Consequently, two robotic devices were developed to mimic the dynamic emission and reception characteristics of horseshoe bats. The noseleaf and pinnae shapes were modeled as smooth blanks matched to digital representations of a horseshoe bat specimens noseleaf and pinnae. Local shape features mimicking structures on the pinnae and noseleaf were added digitally. Flexible baffles with local shape feature combinations were manufactured and paired with actuation mechanisms to mimic pinnae and noseleaf deformations in vivo. Two noseleaves with and without local shape features were considered. Each noseleaf baffle was mounted to a platform called the dynamic emission head to actuate three surface elements of the baffle. Similarly, 12 pinna realizations composed of combinations of three local shape features were mounted to a platform called the dynamic reception head to deform the left and right pinnae independently. Motion of the noseleaf and pinnae were synchronized to the incoming and outgoing sonar waveform, and the joint time-frequency properties of the noseleaf and pinnae local feature combinations and pairs of pinnae and noseleaf thereof were characterized across spatial direction. Amplitude modulations to the outgoing and incoming sonar pulse information across spatial direction were observed for all pinnae and noseleaf local shape feature combinations. Peak modulation variance generated by motion of the pinnae and combinations of the noseleaf and pinnae approached a white Gaussian noise variance bound. It was found the dynamic emitter generated less modulation than either the combined or reception scenarios.

Quantitative approaches to sensory information encoding by bat noseleaves and pinnae

The biosonar systems of horseshoe bats (Rhinolophidae) and Old World round leaf-nosed bats (Hipposideridae) incorporate a pervasive dynamic at the interfaces for ultrasound emission (noseleaves) and reception (pinnae). Changes in the shapes of these structures alter the acoustic characteristics of the biosonar system and could hence influence the encoding of sensory information. The focus of the present work is on approaches that can be used to investigate the hypothesis that the interface dynamic effects sensory information encoding. Mutual information can be used as a metric to quantify the extent to which the different ultrasonic emission and reception characteristics (beampatterns) provide independent views of the environment. Two different quantitative approaches have been taken to evaluate the relationship between dynamically encoded additional sensory information and sensing performance in finding the direction of a biosonar target. The first approach is to determine an upper bound on the number of different directions that can be distinguished by virtue of distinct spectral signatures. The second approach is based on a lower bound (Cramér–Rao) on the variance of direction estimates. All these different metrics demonstrate that the peripheral dynamics seen in bats result in the encoding of additional sensory information that is suitable for enhancing biosonar performance.

Integration of dynamic emission and reception in the biosonar system of horseshoe bats

Bats have evolved unique mechanisms by which to navigate and hunt in their environments. Some species of the microchiroptera, such as members of the horseshoe bat family (Rhinolophidae), which live in particularly challenging, cluttered environments, have been observed to move their peripheral structures for emission and reception (noseleaf and pinna) during echolocation very rapidly, e.g., within a tenth of a second. With these behaviors, the bats could create time-variant properties in their biosonar systems to provide additional degrees of freedom to be used for enhancing the encoding of sensory information. To make full use of this opportunity, a tight sensorimotor integration is likely to be required. In order to explore the possible role of sensorimotor integration in a dynamic biosonar system, we have developed biomimetic robotic models to replicate the effects of the dynamics of the baffle structures. Data obtained with these systems indicates that a coordinated emitter and receiver dynamics can both contribute to the embedding of time-variant signatures in the signals that encode information about target class. Ongoing research seeks to transition insights from work on the biomimetic design paradigms to data from brain recordings made in from the auditory brainstem and the superior colliculus of horseshoe bats.

Why hipposiderid biosonar is worth studying

Although the genus Hipposideros contains a diverse set of more than 70 species of echolocating bats, the biosonar system of this group has received far less attention than that of the related horseshoe bats (family Rhinolophidae) which share the same basic cf-fm biosonar. Only a relatively small number of field observations can be found in the literature and even fewer laboratory studies on hipposiderids have been reported. The Shandong University—Virginia Tech International Laboratory has been working with two of the larger hipposiderid species, the great roundleaf bat (Hipposideros armiger) and Pratt's roundleaf bat (Hipposideros pratti) and has conducted biosonar as well as flight experiments with individuals from both species. It was observed that the bats from both species have highly dynamic biosonar systems that employ large non-rigid noseleaf motions as well as large rigid and non-rigid motions of the pinnae. The motions seen in the hipposiderids appear to be relatively larger and more frequent than those in the greater horseshoe bats with which similar experiments were conducted. In addition, the hipposiderids bats tested were found to be maneuverable fliers that should make an excellent model system for the integration of dynamic sonar with a highly capable flight system.

Bats adjust their pulse emission rates with swarm size in the field.

Flying in swarms, e.g., when exiting a cave, could pose a problem to bats that use an active biosonar system because the animals could risk jamming each other's biosonar signals. Studies from current literature have found different results with regard to whether bats reduce or increase emission rate in the presence of jamming ultrasound. In the present work, the number of Eastern bent-wing bats (Miniopterus fuliginosus) that were flying inside a cave during emergence was estimated along with the number of signal pulses recorded. Over the range of average bat numbers present in the recording (0 to 14 bats), the average number of detected pulses per bat increased with the average number of bats. The result was interpreted as an indication that the Eastern bent-wing bats increased their emission rate and/or pulse amplitude with swarm size on average. This finding could be explained by the hypothesis that the bats might not suffer from substantial jamming probabilities under the observed density regimes, so jamming might not have been a limiting factor for their emissions. When jamming did occur, the bats could avoid it through changing the pulse amplitude and other pulse properties such as duration or frequency, which has been suggested by other studies. More importantly, the increased biosonar activities may have addressed a collision-avoidance challenge that was posed by the increased swarm size.

Information-theoretic assessment of the peripheral dynamics in the biosonar system of horseshoe bats

Horseshoe bat biosonar is characterized by a pronounced dynamics at the interfaces of ultrasound emission and reception: The bats can alter the shapes of their noseleaves and outer ears (pinnae) through muscular action while they are emitting or receiving biosonar signals. An information-theoretic analysis has been conducted to assess whether these non-rigid baffle motions could affect the ability of the bats' biosonar system to encode sensory information. This analysis was based on acoustic characterizations (beampatterns) obtained either through numerical predictions or from measurements of physical models. In total, four time-variant baffle shapes, for emission and reception, numerical and physical each, have been analyzed. The similarity of beampattern across different baffle shape conformations was quantified using normalized mutual information that was found to be <20% for distant stages. Hence, the changes in the baffle shapes were capable of providing different views of a biosonar environment. The information added by these views was found to enhance performance bounds for sonar estimation tasks related to target direction. In particular, it was found that the number of resolvable directions and the accuracy of direction-finding (as measured by the Cramer Rao lower bound) increased when beampatterns from across a sequence of shape conformation were combined.