Acoustic Data Research Articles

In pop pursuit: social bond strength predicts vocal synchrony during cooperative mate guarding in bottlenose dolphins.

Vocal communication is an emblematic feature of group-living animals, used to share information and strengthen social bonds. Vocalizations are also used to coordinate group-level behaviours in many taxa, but little is known of the factors that may influence vocal behaviour during cooperative acts. Allied male Indo-Pacific bottlenose dolphins (Tursiops aduncus) use the 'pop' vocalization as a coercive signal when working together to herd single oestrous females. Using long-term association and acoustic data, we examined the influence of social and non-social factors on pop use by allied male dolphins in this context. Neither pop rate nor pop bout duration were influenced by any of the factors examined. However, allied males with stronger social bonds engaged in higher rates of vocal synchrony; whereby they actively matched the timing of their pop production. Hence, social bond strength influenced pop use in a cooperative context, suggesting dual functions of pop use: to induce the female to remain close, and to promote social bond maintenance and cooperation among males. This article is part of the theme issue 'The power of sound: unravelling how acoustic communication shapes group dynamics'.

Assessing the potential of BirdNET to infer European bird communities from large-scale ecoacoustic data

1.Passive acoustic monitoring has become increasingly popular as a practical and cost-effective way of obtaining highly reliable acoustic data in ecological research projects. Increased ease of collecting these data means that, currently, the main bottleneck in ecoacoustic monitoring projects is often the time required for the manual analysis of passively collected recordings. In this study we evaluate the potential and current limitations of BirdNET-Analyzer v2.4, the most advanced and generic deep learning algorithm for bird recognition to date, as a tool to assess bird community composition through the automated analysis of large-scale ecoacoustic data.2.To this end, we study 3 acoustic datasets comprising a total of 629 environmental soundscapes collected in 194 different sites spread across a 19° latitude span in Europe. We analyze these soundscapes using both BirdNET and manual listening by local expert birders, and we then compare the results obtained through the two methods to evaluate the performance of the algorithm both at the level of each single vocalization and for entire recording sequences (1, 5 or 10 min).3.Since BirdNET provides a confidence score for each identification, minimum confidence thresholds can be used to filter out identifications with low scores, thus retaining only the most reliable ones. The volume of ecoacoustic data used in this study did not allow us to estimate species-specific minimum confidence thresholds for most taxa, so we instead used and evaluated global confidence thresholds selected for optimized results when consistently applied across all species.4.Our analyses reveal that BirdNET identifications can be highly reliable if a sufficiently high minimum confidence threshold is used. However, the inevitable trade-off between precision and recall does not allow to obtain satisfactory results for both metrics at the same time. We found that F1-scores remain moderate (<0.5) for all datasets and confidence thresholds studied, and that acoustic datasets of extended duration seem to be currently necessary for BirdNET to provide a reliable and minimally comprehensive picture of the target bird community. We estimate, however, that the usage of species- and context-specific minimum confidence thresholds would allow to substantially improve the global performance benchmarks obtained in this study.5.We conclude that a judicious use of AI-based identifications provided by BirdNET can represent a powerful method to assist in the assessment of bird community composition through the automated analysis of ecoacoustic data, especially when applied to acoustic datasets of extended duration.

Reconstructing the sound speed profile of South China Sea using remote sensing data and long short-term memory neural networks

IntroductionSound waves are refracted along the direction of their propagation owing to spatial and temporal fluctuations in the speed of sound in seawater. Errors are compounded when sound speed profiles (SSPs) with low precision are used to detect and locate distant underwater targets because an accurate SSP is critical for the identification of underwater objects based on acoustic data. Only sparse historical spatiotemporal data on the SSP of the South China Sea are available owing to political issues, its complex atmospheric system, and the unique topography of its seabed, because of which frequent oceanic movements at the mesoscale affect the accuracy of inversion of its SSP.MethodIn this study, we propose a method for the inversion of the SSP of the South China Sea based on a long short-term memory model. We use continuous-time data on the SSP of the South China Sea as well as satellite observations of the height and temperature of the sea surface to make use of the long-term and short-term memory-related capacities of the proposed model.ResultIt can achieve highly accurate results while using a small number of samples by virtue of the unique structure of its memory. Compared with the single empirical orthogonal function regression method, the inversion accuracy of this model is improved by 24.5%, and it performed exceptionally well in regions with frequent mesoscale movements.DiscussionThis enables it to effectively address the challenges posed by the sparse sample distribution and the frequent mesoscale movements of the South China Sea.

An acoustic exploration of sibilant contrasts and sibilant merger in Mixean Basque

Abstract This exploratory study investigates sibilants in Mixean Low Navarrese, an endangered variety of Basque. This variety has been described with ten different contrastive sibilants: /s̻, s̺, ʃ, t͡s̻, t͡s̺, t͡ʃ, z̻, z̺, ʒ, d͡z̺/. The objective of the paper is to (a) provide a detailed description of the acoustics of Mixean sibilants, and (b) elucidate whether ten categories can be proposed based only on acoustical data, or whether fewer categories should be considered. The study is based on free-conversation data of ten subjects (three females, seven males) aged between 80 and 85 years. We analyze metrics reflecting the place of articulation (spectral moments, and especially the center of gravity (CoG)), including also the temporal dynamics of CoG (using the discrete cosine transform of CoG measurements of nine intervals of each phone). We also explore the acoustic correlates of the contrasts between (a) voiced and voiceless sounds and (b) fricative and affricate sounds. The results show that only seven categories can be proposed based on acoustic measurements. The lamino-alveolar series reliably contrasts with the rest, but the distinction does not hold between the apico-alveolar and the postalveolar series. We found minimal differences in the analysis of dynamic data, and none in the static analysis.

Hierarchical denoising representation disentanglement and dual-channel cross-modal-context interaction for multimodal sentiment analysis

Multimodal sentiment analysis aims to extract sentiment cues from various modalities, such as textual, acoustic, and visual data, and manipulate them to determine the inherent sentiment polarity in the data. Despite significant achievements in multimodal sentiment analysis, challenges persist in addressing noise features in modal representations, eliminating substantial gaps in sentiment information among modal representations, and exploring contextual information that expresses different sentiments between modalities. To tackle these challenges, our paper proposes a new Multimodal Sentiment Analysis (MSA) framework. Firstly, we introduce the Hierarchical Denoising Representation Disentanglement module (HDRD), which employs hierarchical disentanglement techniques. This ensures the extraction of both common and private sentiment information while eliminating interference noise from modal representations. Furthermore, to address the uneven distribution of sentiment information among modalities, our Inter-Modal Representation Enhancement module (IMRE) enhances non-textual representations by extracting sentiment information related to non-textual representations from textual representations. Next, we introduce a new interaction mechanism, the Dual-Channel Cross-Modal Context Interaction module (DCCMCI). This module not only mines correlated contextual sentiment information within modalities but also explores positive and negative correlation contextual sentiment information between modalities. We conducted extensive experiments on two benchmark datasets, MOSI and MOSEI, and the results indicate that our proposed method offers state-of-the-art approaches.

Image analysis of acoustic data and interpretation of rock stress orientations for geothermal exploration in Gothenburg borehole GE-1, SW Sweden

Abstract This study contributes to geothermal exploration in 1660–1520 Ma old, reworked bedrock in Sweden. Our primary objectives are to constrain the orientation of horizontal stresses, and to discuss implications for geothermal exploration. High-resolution acoustic televiewer image data reveal the downhole distribution of stress indicators (borehole breakouts, drilling-induced fractures and petal centreline fractures) and pre-existing structures (natural fractures, foliation). About 135 m of stress indicators are measured from 0.2–1.0 km. The results suggest a uniform NNW–SSE mean maximum horizontal stress orientation. A total of 1525 pre-existing structures (natural fractures, foliation) are mapped in borehole GE-1. The prevailing stress regime controls whether natural fractures and foliation are well-oriented for stimulation. For strike-slip and normal faulting stress regimes, well-oriented fractures steeply dip towards the WSW. For a reverse faulting stress regime, shallow dipping fractures are well-oriented for stimulation. The downhole distribution of stress indicators and other stress measurements in the region and other parts of Fennoscandia tentatively suggest a strike-slip stress regime, but additional studies are needed to constrain the complete stress field at study depth and towards engineered geothermal systems reservoir target depths. Our secondary objective is to highlight that interpretation of high-resolution acoustic data, particularly in metamorphic crystalline rocks, is subjective and that more guidelines for data interpretation are needed. The interactive interpretation of the images is based on visual analyses of complex pre-existing structures and stress indicators with highly variable shapes. The application of three methods for data analyses in the GE-1 borehole proposes that drilling-induced fractures are little influenced by the method applied. Interpretations on individual borehole breakout azimuths may, however, result in over 10° differences in orientation.

Semi-Supervised Training for (Pre-Stack) Seismic Data Analysis

A lack of labeled examples is a problem in different domains, such as text and image processing, medicine, and static reservoir characterization, because supervised learning relies on vast volumes of these data to perform successfully, but this is quite expensive. However, large amounts of unlabeled data exist in these domains. The deep semi-supervised learning (DSSL) approach leverages unlabeled data to improve supervised learning performance using deep neural networks. This approach has succeeded in image recognition, text classification, and speech recognition. Nevertheless, there have been few works on pre-stack seismic reservoir characterization, in which knowledge of rock and fluid properties is fundamental for oil exploration. This paper proposes a methodology to estimate acoustic impedance using pre-stack seismic data and DSSL with a recurrent neural network. The few labeled datasets for training were pre-processed from raw seismic and acoustic impedance data from five borehole logs. The results showed that the acoustic impedance estimation at the well location and outside it was better predicted by the DSSL compared to the supervised version of the same neural network. Therefore, employing a large amount of unlabeled data can be helpful in the development of seismic data interpretation systems.

Diagnosing dominance: Problematic sandhi types in the Chinese Wu dialect of Jinshan

The concept of phonological dominance plays an important role in typologizing tone sandhi behavior in Sinitic languages. We confront the diagnostic criteria for dominance with data from Jinshan, a northern Wu dialect with complex disyllabic lexical tone sandhi. Auditory and acoustic data for the seven Jinshan tones in monosyllabic words are presented and compared with their realization in several different types of disyllabic word tone. It is argued that the current criteria for deciding the dominance of tone sandhi require refinement, which, when applied, reveal examples of both left and right dominance within the same lexical tone sandhi system.

Acoustic emission data based modelling of fracture of glassy polymer

Acoustic emission (AE) activity data resulting from the fracture processes of brittle materials is valuable real time information regarding the evolving state of damage in the material. Here, through a combined experimental and computational study we explore the possibility of utilising the statistical signatures of AE activity data for characterisation of disorder parameter in simulation of tensile fracture of epoxy based polymer. For simulations we use a square random spring network model with quasi-brittle spring behaviour and a normally distributed failure strain threshold. We show that the disorder characteristics while have marginal effect on the power law exponent of the avalanche size distribution, are strongly correlated with the waiting time interval between consecutive record breaking avalanches as well as the total number of records. This sensitivity to disorder is exploited in estimating the disorder parameter suitable for the experiments on tensile failure of epoxy based polymer. The disorder parameter is estimated assuming equivalence between the amplitude distribution of AE data and avalanche size distribution of the simulations. The chosen disorder parameter is shown to well reproduce the failure characteristics in terms of the peak load of the macroscopic response, the power-law behaviour with avalanche dominated fracture type as well as realistic fracture paths.

Enhancing Emergency Vehicle Detection: A Deep Learning Approach with Multimodal Fusion

Emergency vehicle detection plays a critical role in ensuring timely responses and reducing accidents in modern urban environments. However, traditional methods that rely solely on visual cues face challenges, particularly in adverse conditions. The objective of this research is to enhance emergency vehicle detection by leveraging the synergies between acoustic and visual information. By incorporating advanced deep learning techniques for both acoustic and visual data, our aim is to significantly improve the accuracy and response times. To achieve this goal, we developed an attention-based temporal spectrum network (ATSN) with an attention mechanism specifically designed for ambulance siren sound detection. In parallel, we enhanced visual detection tasks by implementing a Multi-Level Spatial Fusion YOLO (MLSF-YOLO) architecture. To combine the acoustic and visual information effectively, we employed a stacking ensemble learning technique, creating a robust framework for emergency vehicle detection. This approach capitalizes on the strengths of both modalities, allowing for a comprehensive analysis that surpasses existing methods. Through our research, we achieved remarkable results, including a misdetection rate of only 3.81% and an accuracy of 96.19% when applied to visual data containing emergency vehicles. These findings represent significant progress in real-world applications, demonstrating the effectiveness of our approach in improving emergency vehicle detection systems.

Population health implications of exposure to pervasive military aircraft noise pollution.

While the adverse health effects of civil aircraft noise are relatively well studied, impacts associated with more intense and intermittent noise from military aviation have been rarely assessed. In recent years, increased training at Naval Air Station Whidbey Island, USA has raised concerns regarding the public health and well-being implications of noise from military aviation. This study assessed the public health risks of military aircraft noise by developing a systematic workflow that uses acoustic and aircraft operations data to map noise exposure and predict health outcomes at the population scale. Acoustic data encompassing seven years of monitoring efforts were integrated with flight operations data for 2020-2021 and a Department of Defense noise simulation model to characterize the noise regime. The model produced contours for day-night, nighttime, and 24-h average levels, which were validated by field monitoring and mapped to yield the estimated noise burden. Established thresholds and exposure-response relationships were used to predict the population subject to potential noise-related health effects, including annoyance, sleep disturbance, hearing impairment, and delays in childhood learning. Over 74,000 people within the area of aircraft noise exposure were at risk of adverse health effects. Of those exposed, substantial numbers were estimated to be highly annoyed and highly sleep disturbed, and several schools were exposed to levels that place them at risk of delay in childhood learning. Noise in some areas exceeded thresholds established by federal regulations for public health, residential land use and noise mitigation action, as well as the ranges of established exposure-response relationships. This study quantified the extensive spatial scale and population health burden of noise from military aviation.We employed a novel GIS-based workflow for relating mapped distributions of aircraft noise exposure to a suite of public health outcomes by integrating acoustic monitoring and simulation data with a dasymetric population density map. This approach enables the evaluation of population health impacts due to past, current, and future proposed military operations. Moreover, itcanbemodified for application to other environmental noise sources and offers an improved open-sourcetool to assess the population health implications of environmental noise exposure, inform at-risk communities, and guide efforts in noise mitigation and policy governing noise legislation, urban planning, and land use.

Merging experiment data and simulation data for parameter identification of shaft seal

A parameter identification method which merges experimentally monitored signals and physics-based simulation data is proposed, targeting the identification tasks in shaft seals which are challenging due to the high-dimensional parameter space. Features were extracted from the monitored acoustic emission signals following a proposed cross-timescale analysis routine, and the simulation data were augmented using Kriging surrogate model to obtain a dataset with stratified fidelity. Then, a transferable architecture of convolutional neural network modified for periodical data was proposed, with which part of the parameters trained by simulation data were reserved when training model using acoustic emission data acquired in experiments. Cross validation shows that transfer learning can effectively improve the performance, provided data augmentation and proper transfer mode. In conclusion, the study provides an effective parameter identification method which merges the simulation data which carry the physical knowledge and the experiment data which carry directly monitored results.

Audio–visual speech recognition based on regulated transformer and spatio–temporal fusion strategy for driver assistive systems

This article presents a research methodology for audio–visual speech recognition (AVSR) in driver assistive systems. These systems necessitate ongoing interaction with drivers while driving through voice control for safety reasons. The article introduces a novel audio–visual speech command recognition transformer (AVCRFormer) specifically designed for robust AVSR. We propose (i) a multimodal fusion strategy based on spatio–temporal fusion of audio and video feature matrices, (ii) a regulated transformer based on iterative model refinement module with multiple encoders, (iii) a classifier ensemble strategy based on multiple decoders. The spatio–temporal fusion strategy preserves contextual information of both modalities and achieves their synchronization. An iterative model refinement module can bridge the gap between acoustic and visual data by leveraging their impact on speech recognition accuracy. The proposed multi-prediction strategy demonstrates superior performance compared to traditional single-prediction strategy, showcasing the model’s adaptability across diverse audio–visual contexts. The transformer proposed has achieved the highest values of speech command recognition accuracy, reaching 98.87% and 98.81% on the RUSAVIC and LRW corpora, respectively. This research has significant implications for advancing human–computer interaction. The capabilities of AVCRFormer extend beyond AVSR, making it a valuable contribution to the intersection of audio–visual processing and artificial intelligence.

Marine predators segregate interspecifically by space and time in a sheltered coastal bay.

Marine predators are vital to the healthy functioning of coastal ecosystems, but to understand their roles, it is necessary to elucidate their movement ecology, particularly in relation to one another. A decade's worth of acoustic telemetry data (2011-2020) from Algoa Bay, South Africa, was investigated to determine how two mesopredatory species (teleosts: dusky kob Argyrosomus japonicus, n = 11, and leervis Lichia amia, n = 16) and two top predatory species (sharks: ragged-tooth sharks Carcharias taurus, n = 45, and white sharks Carcharodon carcharias, n = 31) used and shared this bay ecosystem. Multi-annual seasonal fidelity to the bay was exhibited by all species, but differences in residency were observed among species. Similarly, species used space in the bay differently-the teleosts moved less and had movements restricted to the central and western inshore regions of the bay. Conversely, the sharks roamed more, but detections were concentrated in the western part of the bay for C. taurus and in the eastern part of the bay for C. carcharias. Social network analysis showed that species segregated in space and time on a fine scale. However, there was some interaction observed between C. taurus, L. amia, and A. japonicus, but to varying degrees. This is likely because of strong habitat preferences exhibited by each species and predator-prey relationships between these predatory guilds. Results highlight that the sheltered marine Algoa Bay is a resource-rich environment, supporting multiple predators with different hunting strategies albeit similar prey preferences. Finally, these species are likely afforded some protection by the current Greater Addo Elephant National Park Marine Protected Area in the bay but are vulnerable to fishing pressure when they leave this ecosystem.

MBCFNet: A Multimodal Brain–Computer Fusion Network for human intention recognition

Accurate recognition of human intent is crucial for effective human–computer speech interaction. Numerous intent understanding studies were based on speech-to-text transcription, which often overlook the influence of paralinguistic cues (such as speaker’s emotion, attitude, etc.), leading to misunderstandings, especially when identical textual content conveys multiple intents by means of different paralinguistic information. Considering that interaction intent is produced from the human brain, we propose a novel Multimodal Brain–Computer Fusion Network (MBCFNet) to discriminate the different intents carried by the identical textual information, in which the acoustic-textual representation was adapted by brain functional information through a cross-modal transformer. In the output module, a joint multi-task learning method was used to optimize both the primary task of intent recognition and the auxiliary task of emotion recognition of the speaker. To evaluate the model performance, we constructed a multimodal dataset CMSLIU that consists of acoustic, textual, and electroencephalograph (EEG) data of the same Chinese texts with varying intents. Experimental results on the self-constructed dataset indicate that the proposed model achieves state-of-the-art (SOTA) performance compared with other competing models. Ablation experiment results reveal that the model performance declines without information from EEG or audio modality, and also when the auxiliary task of emotion recognition is removed. All these results suggest the effectiveness of the proposed information fusion strategy in spoken language intent recognition and the brain–computer information fusion idea can also be extended to other similar fields.

Geographic variation in the acoustic signals of lesser treefrogs Dendropsophus minutus (Anura, Hylidae)

Biological sound archives are a major source to investigate geographic variation in animal acoustic signals and their evolutionary drivers. The acoustic signals of anuran species with wide distribution ranges often vary geographically as a result of isolation by distance and climate amongst other factors. We examined whether the acoustic structure of call notes would vary geographically in lesser treefrogs Dendropsophus minutus using recordings from bioacoustics repositories. We also tested whether climate (mean annual temperature and annual precipitation) drive geographic variation in those signals. The acoustic distance was unrelated to geographic distance, suggesting that isolation by distance solely cannot explain geographic variation in call structure. Overall, lesser treefrogs uttered call notes with lower frequencies and bandwidths in the west of their range. In addition, frogs produced shorter call notes in hotter, wetter sites and narrow-bandwidth notes in hotter environments. We suggest that frogs produce more calls (not measured here) of shorter durations to maximize transmission and minimize the metabolic costs of calling at high air temperatures. We also suggest that hotter environments favor the propagation of lower-bandwidth calls. This study reinforces the feasibility and power of citizen science and acoustic data repositories for understanding the evolution of anuran acoustic signals.

Elevated fish densities extend kilometres from oil and gas platforms.

Thousands of offshore oil and gas platforms have been installed throughout the world's oceans and more structures are being installed as part of the transition to renewable energy. These structures increase the availability of ecological niches by providing hard substrate in midwater and complex 3D habitat on the seafloor. This can lead to 'hotspots' of biodiversity, or increased densities of flora and fauna, which potentially spill over into the local area. However, the distances over which these higher densities extend (the 'range of influence') can be highly variable. Fish aggregate at such structures, but the range of influence and any implications for wider fish populations, are unclear. We investigated the relationship between fish and platform areal densities using high resolution fisheries acoustic data. Data were collected in the waters surrounding the vessel exclusions zones around 16 oil and gas platforms in the North Sea, and throughout the wider area. We estimated densities of schooling fish using echo-integration, and densities of non-schooling fish using echo-counting. At 10 platforms, non-schooling fish densities were elevated near the platform relative to background levels in the equivalent wider area. The range of influence, defined here as the range to which fish densities were elevated above background, varied from 0.8 to 23 km. In areas of high platform density, fish schools were encountered more often, and non-schooling fish densities were higher, when controlling for other sources of environmental variation. This is the first time such long-range effects have been identified; previously, ranges of influence have been reported in the order of just 10s-100s of metres. These findings suggest that the environmental impact of these structures may extend further than previously thought, which may be relevant in the context of upcoming management decisions around the decommissioning of these structures.

A Methodology for Speaker Diazaration System Based on LSTM and MFCC Coefficients

Research on Speaker Identification is always difficult. A speaker may be automatically identified using by comparing their voice sample with their previously recorded voice, the machine learning strategy has grown in favor in recent years. Convolutional neural networks (CNN) , deep neural networks (DNN) are some of the machine learning techniques that has employed recently. The article will discuss a successful speaker verification system based on the d-vector to construct a new approach based on speaker diarization. In particular, in this article, we use the concept of LSTM to cluster the speech segments using MFCC coefficients and identify the speakers in the diarization system. The proposed system will be evaluated using benchmark performance metrics, and a comparative study will be made with other models. The need to consider the LSTM neural network using acoustic data and linguistic dialect is considered. LSTM networks could produce reliable speaker segmentation outputs.

Assessing learners’ English public speaking anxiety with multimodal deep learning technologies

Public speaking anxiety (PSA) is a common phenomenon for language learners involving both psychological and physiological symptoms. Timely and effective PSA assessment can help diagnose learners’ speaking anxiety, offer learners feedback to alleviate their anxiety, and improve their public speaking competence. However, it is still a challenging issue to achieve accurate automated assessment of learners’ PSA due to the lack of large-scale and open-source multimodal datasets based on real classroom settings. This study collected the public speaking videos of English as a foreign language (EFL) learners in public speaking courses, and constructed a large-scale multimodal dataset named Speaking Anxiety in Real Classrooms (SARC) with three modalities of acoustic, visual, and textual data (including 1,158 manually-annotated speech videos and corresponding speech drafts of 382 participants). A multimodal deep learning model for automated assessment of learners’ speaking anxiety was proposed, and an online formative assessment platform was then developed to realize the automated assessment of PSA for classroom teaching. A pilot survey study involving 78 participants was then conducted to investigate learners’ acceptance of the platform. Experimental results verified the validity of the deep learning model and the consistency between automated assessment and teacher assessment. Learners’ acceptance data further indicated that collaboration between automated and human assessment provided them with the most satisfactory experience of using the platform to improve their English public speaking.

Cross-modal zero-sample diagnosis framework utilizing non-contact sensing data fusion

Gearboxes, fundamental components in the domains of manufacturing, transportation, and aerospace apparatus, are highly susceptible to impairments. The emerging technique of non-contact sensing measurement holds promise as a valuable tool for real-time monitoring and diagnosis of gearboxes, especially in challenging, dynamic environments. However, historical non-contact diagnostic techniques often face difficulties due to load-variation-induced interference, which leads to inconsistent diagnostic results. Moreover, acquiring sufficient non-contact sensing data for specific rare or severe gearbox faults can be exceedingly challenging and, at times, unattainable. This data scarcity limits the application of existing supervised and semi-supervised non-contact diagnostic methods. To address these limitations, this paper develops a novel diagnostic model, called Non-Contact Sensing Data Fusion Driven Cross-Modal Zero-sample Diagnostic Network. Specifically, we first develop a novel central fusion module to facilitate feature-level fusion at both global and local scales utilizing infrared thermal and acoustic data, thereby establishing a rich and comprehensive fault representation. Following that, we construct an end-to-end cross-modal zero-sample diagnostic architecture to mine and fuse complementary fault information. This approach enhances zero-sample diagnostic performance through efficient information utilization and cross-modal feature fusion. Afterward, we adopt a cross-modal self-enhancing fusion strategy during the training phase, which improves the information fusion performance of intra-class features from various modalities. This strategy effectively reduces misclassification risks and augments the robustness of the proposed zero-sample diagnostic network against interference from load variations. Comprehensive experimental results confirm that our proposed NCFZD sets the new state-of-the-art in multiple non-contact, zero-sample diagnostic scenarios, reaching HM scores of 86.06 %, 81.53 %, and 62.56 % respectively. By incorporating the information fusion theory, this model advances gearbox diagnostics in non-contact sensing and contributes to the broader field of information fusion theory by demonstrating its practical application in real-world problem-solving scenarios.