Time Delay Neural Network Architecture Research Articles

Attention-Based Fusion of Ultrashort Voice Utterances and Depth Videos for Multimodal Person Identification

Multimodal deep learning, in the context of biometrics, encounters significant challenges due to the dependence on long speech utterances and RGB images, which are often impractical in certain situations. This paper presents a novel solution addressing these issues by leveraging ultrashort voice utterances and depth videos of the lip for person identification. The proposed method utilizes an amalgamation of residual neural networks to encode depth videos and a Time Delay Neural Network architecture to encode voice signals. In an effort to fuse information from these different modalities, we integrate self-attention and engineer a noise-resistant model that effectively manages diverse types of noise. Through rigorous testing on a benchmark dataset, our approach exhibits superior performance over existing methods, resulting in an average improvement of 10%. This method is notably efficient for scenarios where extended utterances and RGB images are unfeasible or unattainable. Furthermore, its potential extends to various multimodal applications beyond just person identification.

Rainfall prediction using time-delay wavelet neural network (TDWNN) model for assessing agrometeorological risk

In an agriculturally dependent nation like India, accurate and effective rainfall forecasting methods are crucial for assessing agrometeorological risk. Forecasting rainfall is perhaps one amongst the most arduous tasks in this context due to the prevalence of a non-linear pattern. One of the most promising and frequently employed approaches for forecasting rainfall data is the Time-Delay Neural Network (TDNN) model. TDNN's non-parametric, data-driven, and self-adaptive characteristics make it increasingly attractive for modelling nonlinear dynamics and generating nonlinear forecasts. Nevertheless, since the conventional TDNN uses the sigmoid activation function, there is always a chance that the training process may converge to local minima. This study addresses the usage of a Time-delay wavelet neural network (TDWNN), which employs a TDNN architecture with a hidden layer activation function derived from the orthonormal wavelet family, in order to circumvent this issue. TDWNN has been empirically demonstrated using annual rainfall data from two districts in the Indian state of West Bengal. According to the findings of this study, the TDWNN model is superior than the conventional TDNN method to assess agrometeorological risk.

EfficientTDNN: Efficient Architecture Search for Speaker Recognition

Convolutional neural networks (CNNs), such as the time-delay neural network (TDNN), have shown their remarkable capability in learning speaker embedding. However, they meanwhile bring a huge computational cost in storage size, processing, and memory. Discovering the specialized CNN that meets a specific constraint requires a substantial effort of human experts. Compared with hand-designed approaches, neural architecture search (NAS) appears as a practical technique in automating the manual architecture design process and has attracted increasing interest in spoken language processing tasks such as speaker recognition. In this paper, we propose EfficientTDNN, an efficient architecture search framework consisting of a TDNN-based supernet and a TDNN-NAS algorithm. The proposed supernet introduces temporal convolution of different ranges of the receptive field and feature aggregation of various resolutions from different layers to TDNN. On top of it, the TDNN-NAS algorithm quickly searches for the desired TDNN architecture via weight-sharing subnets, which surprisingly reduces computation while handling the vast number of devices with various resources requirements. Experimental results on the VoxCeleb dataset show the proposed EfficientTDNN enables approximate $10^{13}$ architectures concerning depth, kernel, and width. Considering different computation constraints, it achieves a 2.20% equal error rate (EER) with 204M multiply-accumulate operations (MACs), 1.41% EER with 571M MACs as well as 0.94% EER with 1.45G MACs. Comprehensive investigations suggest that the trained supernet generalizes subnets not sampled during training and obtains a favorable trade-off between accuracy and efficiency.

Hindi speech recognition using time delay neural network acoustic modeling with i-vector adaptation

It is a need of time to build an Automatic Speech Recognition (ASR) system for low and limited resource languages. Usually, statistical techniques such as Hidden Markov Models (HMM) have been applied for Indian language ASR systems for the last two decades. In this work, we have selected the Time-delay Neural Network (TDNN) based acoustic modeling with i-vector adaptation for limited resource Hindi ASR. The TDNN can capture the extended temporal context of acoustic events. To reduce the training time, we used sub-sampling based TDNN architecture in this work. Further, data augmentation techniques have been applied to extend the size of training data developed by TIFR, Mumbai. The results show that data augmentation significantly improves the performance of the Hindi ASR. Further, $$\approx$$ 4% average improvement has been recorded by applying i-vector adaptation in this work. We found the best system accuracy of 89.9% with TDNN based acoustic modeling with i-vector adaptation.

Vehicle classification by acoustic signature

The aim of this research is to investigate the feasibility of developing a traffic monitoring detector for the purpose of reliable on-line vehicle classification to aid traffic management systems. The detector used was a directional microphone connected to a DAT (Digital Audio Tape) recorder. The digital signal was preprocessed by LPC (Linear Predictive Coding) parameter conversion based on autocorrelation analysis. A Time Delay Neural Network (TDNN) was chosen to classify individual travelling vehicles based on their speed-independent acoustic signature. The paper provides a description of the TDNN architecture and training algorithm, and an overview of the LPC preprocessing and feature extraction technique as applied to audio monitoring of road traffic. The performance of TDNN vehicle classification, convergence, and accuracy for the training patterns are fully illustrated. To establish the viability of this classification approach, initially, recordings were carried out on a strip of airfield for four types of vehicles under controlled conditions. A TDNN network was successfully trained with 100% accuracy in classification for the training patterns, as well as the test patterns. The net was also robust to changes in the starting position of the acoustic waveforms with 86% accuracy for the same test data set. In the second phase of the experiment, roadside recordings were made at a two-way urban road site in the city of Leeds with no control over the environmental parameters such as background noise, interference from other travelling vehicles, or the speed of the recorded vehicle. A second TDNN network was also successfully trained with 96% accuracy for the training patterns and 84% accuracy for the test patterns.

A field trial of acoustic signature analysis for vehicle classification

The aim of this research is to investigate the feasibility of developing a traffic monitoring detector for the purpose of reliable on-line vehicle classification to aid traffic management systems. The detector used was a directional microphone connected to a DAT (Digital Audio Tape) recorder. The digital signal was pre-processed by LPC (Linear Predictive Coding) parameter conversion based on autocorrelation analysis. A Time Delay Neural Network (TDNN) was chosen to classify individual travelling vehicles based on their speed-independent acoustic signature. Locations for data acquisition included roadside recordings at a number of two-way urban road sites in the city of Leeds with no control over the environmental parameters such as background noise, interference from other travelling vehicles or the speed of the recorded vehicles. The results and performance analysis of TDNN vehicle classification, the convergence for training patterns and accuracy of test patterns are fully illustrated. The paper also provides a description of the TDNN architecture and training algorithm, and an overview of the LPC pre-processing and feature extraction technique as applied to audio monitoring of road traffic. In the final phase of the experiment, the four broad categorisations of vehicles for training the network consisted of: buses or lorries; small or large saloons; various types of motorcycles; and light goods vehicles or vans. A TDNN network was successfully trained with 94% accuracy for the training patterns and 82.4% accuracy for the test patterns.

A time-delay neural network architecture for isolated word recognition

A translation-invariant back-propagation network is described that performs better than a sophisticated continuous acoustic parameter hidden Markov model on a noisy, 100-speaker confusable vocabulary isolated word recognition task. The network's replicated architecture permits it to extract precise information from unaligned training patterns selected by a naive segmentation rule.