Time Delay Neural Network Research Articles

A Study of Blockchain and Machine Learning-Enabled IoT Security in Time-Delayed Neural Network Vocal Pattern Recognition to Improve Web-Based Vocal Teaching

With the development of information technology, online vocal teaching is becoming more and more popular, but the sound quality of teaching is also becoming more and more demanding. As online vocal instruction becomes more popular, the need for high-quality sound in these digital environments becomes more critical. This research tackles the problem of improving sound quality in real-time vocal teaching by integrating advanced technologies such as Blockchain and Machine Learning within the Internet of Things (IoT) security framework. We created a vocal recognition model using Time-Delay Neural Network (TDNN) and improved it with Generated Feature Vector (GFV). This integration yields a strong GTDNN vocal recognition system that is specifically designed to secure and optimize web-based vocal teaching. Our experiments show that GTDNN outperforms traditional TDNN and i-vector methods in feature vector extraction, adapting well to different speech environments. In various speech settings, GTDNN's Error Rates (EERs) are impressively low at 11.3%, 12.0%, 4.9%, 6.2%, and 6.1%, indicating superior performance over comparison models. GTDNN has an EER of 9.6% for short-duration speech and 2.3% for long-duration speech. Furthermore, the GTDNN system achieves an overall pass rate of 94% for target speech and an impressive rejection rate for non-target speech, ensuring high accuracy in a variety of speech environments.

MECOS: A bilingual Manipuri–English spontaneous code-switching speech corpus for automatic speech recognition

In this study, we introduce a new code-switched speech database with 57h of Manipuri–English annotated spontaneous speech. Manipuri is an official language of India and is primarily spoken in the north–eastern Indian state of Manipur. Most Manipur native speakers today are bilingual and frequently use code switching in everyday discussions. By carefully assessing the amount of code-switched speech in each video, recordings from YouTube are gathered. 21,339 utterances and 291,731 instances of code switching are present in the database. Given the code-switching nature of the data, a proper annotation procedure is used, and the data are manually annotated using the Meitei Mayek unicode font and the roman alphabets for Manipuri and English, respectively. The transcription includes the information of the speakers, non-speech information, and the corresponding annotation. The aim of this research is to construct an automatic speech recognition (ASR) system as well as offer a thorough analysis and details of the speech corpus. We believe that our research is the first to use an ASR system for Manipuri–English code-switched speech. To evaluate the performance, ASR systems based on hybrid deep neural network and hidden Markov model (DNN–HMM), time delay neural network (TDNN), hybrid time delay neural network and long short-term memory (TDNN–LSTM) and three end-to-end (E2E) models i.e. hybrid connectionist temporal classification and attention model (CTC-Attention), Conformer, wav2vec XLSR are developed for Manipuri–English language. In comparison to other models, pure TDNN produces outcomes that are clearly superior.

Phantom in the opera: adversarial music attack for robot dialogue system

This study explores the vulnerability of robot dialogue systems' automatic speech recognition (ASR) module to adversarial music attacks. Specifically, we explore music as a natural camouflage for such attacks. We propose a novel method to hide ghost speech commands in a music clip by slightly perturbing its raw waveform. We apply our attack on an industry-popular ASR model, namely the time-delay neural network (TDNN), widely used for speech and speaker recognition. Our experiment demonstrates that adversarial music crafted by our attack can easily mislead industry-level TDNN models into picking up ghost commands with high success rates. However, it sounds no different from the original music to the human ear. This reveals a serious threat by adversarial music to robot dialogue systems, calling for effective defenses against such stealthy attacks.

Exponential Stability of Stochastic Time-Delay Neural Networks with Random Delayed Impulses

The mean square exponential stability of stochastic time-delay neural networks (STDNNs) with random delayed impulses (RDIs) is addressed in this paper. Focusing on the variable delays in impulses, the notion of average random delay is adopted to consider these delays as a whole, and the stability criterion of STDNNs with RDIs is developed by using stochastic analysis idea and the Lyapunov method. Taking into account the impulsive effect, interference function and stabilization function of delayed impulses are explored independently. The results demonstrate that delayed impulses with random properties take a crucial role in dynamics of STDNNs, not only making stable STDNNs unstable, but also stabilizing unstable STDNNs. Our conclusions, specifically, allow for delays in both impulsive dynamics and continuous subsystems that surpass length of impulsive interval, which alleviates certain severe limitations, such as presence of upper bound for impulsive delays or requirement that impulsive delays can only exist between two impulsive events. Finally, feasibility of the theoretical results is verified through three simulation examples.

A research study on methods for improving the service quality of a network system based on time information

The network delay is one of the important criteria for selecting computing and networking capabilities. Network delay is one of the important bases of computing network selection. With the continuous development and integration of digital technology, communication technology, and control theory, the research on control theory for networked systems has rapidly advanced. This study proposes an event-triggered mixed time-delay neural network (NN) synchronization method, aiming to reduce network transmission delays through the optimization of NNs, thereby enhancing service responsiveness or reducing errors in data transmission. Firstly, a discrete-driven NN model subject to mixed time delays is established, and the corresponding response network is constructed. In the design of the synchronization controller, a quadratic function is constructed based on the measurement of error states, serving as a threshold for designing the event-trigger mechanism. For the synchronization error system, by constructing a Lyapunov–Krasovskii functional dependent on time delays and a general µ-function, analytical techniques and matrix theory are applied to obtain sufficient criteria for the µ-stability. The validity of the theoretical findings is proven through numerical examples.

Joint speaker encoder and neural back-end model for fully end-to-end automatic speaker verification with multiple enrollment utterances

Conventional automatic speaker verification systems can usually be decomposed into a front-end model such as time delay neural network (TDNN) for extracting speaker embeddings and a back-end model such as statistics-based probabilistic linear discriminant analysis (PLDA) or neural network-based neural PLDA (NPLDA) for similarity scoring. However, the sequential optimization of the front-end and back-end models may lead to a local minimum, which theoretically prevents the whole system from achieving the best optimization. Although some methods have been proposed for jointly optimizing the two models, such as the generalized end-to-end (GE2E) model and NPLDA E2E model, most of these methods have not fully investigated how to model the intra-relationship between multiple enrollment utterances. In this paper, we propose a new E2E joint method for speaker verification especially designed for the practical scenario of multiple enrollment utterances. To leverage the intra-relationship among multiple enrollment utterances, our model comes equipped with frame-level and utterance-level attention mechanisms. Additionally, focal loss is utilized to balance the importance of positive and negative samples within a mini-batch and focus on the difficult samples during the training process. We also utilize several data augmentation techniques, including conventional noise augmentation using MUSAN and RIRs datasets and a unique speaker embedding-level mixup strategy for better optimization.

Examining the effectiveness of artificially replicated lake systems in predicting eutrophication indicators: a comparative data-driven analysis

Abstract Data-driven models for the prediction of lake eutrophication essentially rely on water quality datasets for a longer duration. If such data are not readily available, lake management through data-driven modeling becomes impractical. So, a novel approach is presented here for the prediction of eutrophication indicators, such as dissolved oxygen, Secchi depth, total nitrogen, and total phosphorus, in the waterbodies of Assam, India. These models were developed using water quality datasets collected through laboratory investigation in artificially simulated lake systems. Two artificial prototype lakes were eutrophied in a controlled environment with the gradual application of wastewater. A periodic assessment of water quality was done for model development. Data-driven modeling in the form of multilayer perceptron (MLP), time-delay neural network (TDNN), support vector regression (SVR), and Gaussian process regression (GPR) were utilized. The trained model's accuracy was evaluated based on statistical parameters and a reasonable correlation was observed between targeted and model predicted values. Finally, the trained models were tested against some natural waterbodies in Assam and a satisfactory prediction accuracy was obtained. TDNN and GPR models were found superior compared to other methods. Results of the study indicate feasibility of the adopted modeling approach in predicting lake eutrophication when periodic water quality data are limited for the waterbody under consideration.

Stabilization of nonlinear hybrid stochastic time-delay neural networks with Lévy noise using discrete-time feedback control

<p>This paper aims to formulate a class of nonlinear hybrid stochastic time-delay neural networks (STDNNs) with Lévy noise. Specifically, the coefficients of networks grow polynomially instead of linearly, and the time delay of given neural networks is non-differentiable. In many practical situations, nonlinear hybrid STDNNs with Lévy noise are unstable. Hence, this paper uses feedback control based on discrete-time state and mode observations to stabilize the considered nonlinear hybrid STDNNs with Lévy noise. Then, we establish stabilization criteria of $ H_{\infty} $ stability, asymptotic stability, and exponential stability for the controlled nonlinear hybrid STDNNs with Lévy noise. Finally, a numerical example illustrating the usefulness of theoretical results is provided.</p>

DS-TDNN: Dual-Stream Time-Delay Neural Network With Global-Aware Filter for Speaker Verification

DS-TDNN: Dual-Stream Time-Delay Neural Network With Global-Aware Filter for Speaker Verification

Improving the Performance of Low-resourced Speaker Identification with Data Preprocessing

Automatic speaker identification is done to tackle daily security problems. Speech data collection is an essential but very challenging task for under-resourced languages like Burmese. The speech quality is crucial to accurately recognize the speaker’s identity. This work attempted to find the optimal speech quality appropriate for Burmese tone to enhance identification compared with other more richy resourced languages on Mel-frequency cepstral coefficients (MFCCs). A Burmese speech dataset was created as part of our work because no appropriate dataset available for use. In order to achieve better performance, we preprocessed the foremost recording quality proper for not only Burmese tone but also for nine other Asian languages to achieve multilingual speaker identification. The performance of the preprocessed data was evaluated by comparing with the original data, using a time delay neural network (TDNN) together with a subsampling technique that can reduce time complexity in model training. The experiments were investigated and analyzed on speech datasets of ten Asian languages to reveal the effectiveness of the data preprocessing. The dataset outperformed the original dataset with improvements in terms of equal error rate (EER). The evaluation pointed out that the performance of the system with the preprocessed dataset improved that of the original dataset.

Hybrid time series models with exogenous variable for improved yield forecasting of major Rabi crops in India

Accurate and in-time prediction of crop yield plays a crucial role in the planning, management, and decision-making processes within the agricultural sector. In this investigation, utilizing area under irrigation (%) as an exogenous variable, we have made an exertion to assess the suitability of different hybrid models such as ARIMAX (Autoregressive Integrated Moving Average with eXogenous Regressor)–TDNN (Time-Delay Neural Network), ARIMAX–NLSVR (Non-Linear Support Vector Regression), ARIMAX–WNN (Wavelet Neural Network), ARIMAX–CNN (Convolutional Neural Network), ARIMAX–RNN (Recurrent Neural Network) and ARIMAX–LSTM (Long Short Term Memory) as compared to their individual counterparts for yield forecasting of major Rabi crops in India. The accuracy of the ARIMA model has also been considered as a benchmark. Empirical outcomes reveal that the ARIMAX–LSTM hybrid modeling combination outperforms all other time series models in terms of root mean square error (RMSE) and mean absolute percentage error (MAPE) values. For these models, an average improvement of RMSE and MAPE values has been observed to be 10.41% and 12.28%, respectively over all other competing models and 15.83% and 18.42%, respectively over the benchmark ARIMA model. The incorporation of the area under irrigation (%) as an exogenous variable in the ARIMAX framework and the inbuilt capability of the LSTM model to process complex non-linear patterns have been observed to significantly enhance the accuracy of forecasting. The performance supremacy of other hybrid models over their individual counterparts has also been evident. The results also suggest avoiding any performance generalization of individual models for their hybrid structures.

Prediction of tail biting in pigs using partial least squares regression and artificial neural networks

Tail biting in pig farming causes economic losses as well as pain and suffering to the affected pigs. The aim of the study was to predict tail lesions to provide the farmer with the opportunity to intervene early. To do this, tail lesions were recorded during rearing (REA) and fattening (FAT) on two farms (Farm 1: 10 batches REA, 5 batches FAT; Farm 2: 7 batches REA, 4 batches FAT). In addition, tail posture, skin lesions, musculoskeletal system issues, treatment index for suckling, weight at weaning, water consumption, activity time, humidity, NH3 and CO2 concentrations were observed. Processed to daily values forming a time series, these data was used to predict the prevalence of tail lesions per pen using partial least squares (PLS) regression models, focused time-delay neural networks (FTDNN) and nonlinear autoregressive neural networks with exogenous inputs (NARX). After training, validating and testing the methods for Farm 1, the respective methods were applied to the data of Farm 2. In addition, it was investigated whether the prevalence of tail lesions per pen could be predicted for an offset of one to five days before with PLS. The networks were used to test whether a delay of one to five days before should be included for prediction. NARX achieved the best prediction performance on both farms in REA and FAT: accuracy between true and estimated values (R 0.85–0.97), recall (REC 0.63–0.97), false positive rate (FPR 0.01–0.27), precision (PRC 0.43–0.92), F1-score (0.54–0.94) and area under the precision-recall curve (AUPRC 0.66–0.97). PLS (R 0.49–0.81, REC 0.32–0.86, FPR 0.02–0.36, PRC 0.42–0.70, F1-score 0.40–0.77, AUPRC 0.43–0.85) and FTDNN (R 0.45–0.90, REC 0.19–1.00, FPR 0.00–0.97, PRC 0.30–0.84, F1-score 0.28–0.75, AUPRC 0.30–0.86) performed less accurately than NARX but still with sufficient prediction performance, whereby the results for Farm 2 were worse than for Farm 1. Therefore, a generalisation of the methods to other farms is possible. No noticeable difference was observed between offsets 1 to 5 and delays 1 to 5. PLS, FTDNN and NARX with all offsets and delays are suitable for predicting the prevalence of tail lesions per pen in several farms as well as in REA and FAT.

A multimodal screening system for elderly neurological diseases based on deep learning

In this paper, we propose a deep-learning-based algorithm for screening neurological diseases. We proposed various examination protocols for screening neurological diseases and collected data by video-recording persons performing these protocols. We converted video data into human landmarks that capture action information with a much smaller data dimension. We also used voice data which are also effective indicators of neurological disorders. We designed a subnetwork for each protocol to extract features from landmarks or voice and a feature aggregator that combines all the information extracted from the protocols to make a final decision. Multitask learning was applied to screen two neurological diseases. To capture meaningful information about these human landmarks and voices, we applied various pre-trained models to extract preliminary features. The spatiotemporal characteristics of landmarks are extracted using a pre-trained graph neural network, and voice features are extracted using a pre-trained time-delay neural network. These extracted high-level features are then passed onto the subnetworks and an additional feature aggregator that are simultaneously trained. We also used various data augmentation techniques to overcome the shortage of data. Using a frame-length staticizer that considers the characteristics of the data, we can capture momentary tremors without wasting information. Finally, we examine the effectiveness of different protocols and different modalities (different body parts and voice) through extensive experiments. The proposed method achieves AUC scores of 0.802 for stroke and 0.780 for Parkinson’s disease, which is effective for a screening system.

A novel PAPR reduction scheme based on time-delay neural network for CO-OFDM systems

In this paper, we propose a low-complexity peak-to-average power ratio (PAPR) reduction scheme based on time-delay neural network (TDNN) for coherent optical orthogonal frequency division multiplexing (CO-OFDM) systems to reduce the limitation of PAPR on the total power of laser diodes and the impact of fiber nonlinear effects. The proposed scheme is jointly trained on traditional iterative clipping and filtering (ICF) scheme and simplified clipping and filtering (SCF) scheme, and changes the objective function by adjusting the given training weight α. Compared to the original orthogonal frequency division multiplexing (OFDM) signal of existing system, our method achieves an adjustable PAPR reduction of 5.31 dB to 6.12 dB for 10−4 complementary cumulative distribution function (CCDF). When bit error rate (BER) Pe=10−3, the proposed scheme (α=0.5) achieves additional transmission distances of 220 km, 55 km, 105 km, and 100 km, compared to the original OFDM signal, partial transmission sequence (PTS) scheme, ICF scheme, and feedforward neural network (FNN)-based ICF scheme, respectively. Besides, our method achieves 86.3% and 80.0% floating-point operations (FLOPs) reductions, compared with PTS scheme and ICF scheme, respectively, which effectively reduces the complexity of PAPR reduction module, saving the system computational time.

Non-fragile output-feedback control for time-delay neural networks with persistent dwell time switching: A system mode and time scheduler dual-dependent design

This paper is concerned with non-fragile output-feedback control for time-delay neural networks with persistent dwell time (PDT) switching in a continuous-time setting. The main purpose is to design an output-feedback controller subject to gain fluctuations, guaranteeing both asymptotic stability and L2-gain of the closed-loop control system. To achieve reduced conservatism, the controller is formulated to depend not only on the system mode but also on a time scheduler constructed based on the PDT switching rule and minimum time span. A criterion for the asymptotic stability and L2-gain analysis is established through the application of the Gronwall–Bellman inequality and mathematical induction. Then, a numerically tractable design approach for the desired controller is proposed, utilizing a four-section piecewise time-dependent Lyapunov–Krasovskii functional and several nonlinearity decoupling techniques. For comparative purposes, a simple case, independent of the time scheduler, is also investigated, and the corresponding controller design approach is presented. Finally, a simulation example is given to illustrate the effectiveness and superiority of the proposed system mode and time scheduler dual-dependent controller design approach.

Calculating the distance between languages with deep learning

Artificial intelligence (AI) has been implemented in various fields, including speech recognition. In this paper, a computational method is proposed for calculating the similarity between different languages or language varieties, with their similarity represented in terms of distance. In this process, we extracted mel spectrogram features from speech signals to provide the feature vectors and derived pairs of signal tokens based on vectors. Then, we trained a Siamese time-delay neural network to calculate the distance between two signal tokens. If the token pairs are from the same language group, the distance obtained using this Siamese network model is zero. In this preliminary experiment, three types of regional Mandarin Chinese (BJ, FJ, GD) were used as the dataset. The results gave the F1-score of 0.794, 0.623, and 0.715 for the classification task with respect to BJ, FJ, and GD dataset. In addition, 10 Taiwan Mandarin (TM) native speakers participated in identification and a pair-wise discrimination experiment to allow comparison with the Siamese network model. The 10 TM natives tended to misidentify GD-accented Mandarin as FJ-accented Mandarin resulting in a much greater distance between the two in the FJ-GD discrimination task compared to the Siamese network model. Overall, the results show that the performance of our model is better than the comparative experiment in completing the identification task, with the distance between the Siamese network model and the experiment having a mean absolute error (MAE) of 0.35. The familiarity might be the reason why the 10 TM natives displayed a bias towards BJ-accented Mandarin Chinese. To sum up, we provide a computational method to calculate the distance between two languages or language varieties, which can help linguists pre-classify sound files.

Speaker diarization with variants of self-attention and joint speaker embedding extractor

The speaker diarization task pertains to the automated differentiation of speakers within an audio recording, while lacking any prior information regarding the speakers. The introduction of the self-attention mechanism in End-to-End Neural Speaker Diarization (EEND) has elegantly resolved the issue of overlapping speakers. The Transformer model equipped with self-attention mechanism has shown great potential in collecting global information, yielding remarkable outcomes in various tasks. However, the individual speaker characteristics are predominantly reflected in the contextual information, which conventional self-attention would not adequately address. In this study, we propose a hierarchical encoders model to augment the encoders’ acquisition of speaker information in two distinct ways: (1) Constraining the perceptual field of the self-attentive mechanism with left-right windows or Gaussian weights to highlight contextual information; (2) Utilizing a pre-trained time-delay neural network based speaker embedding extractor to alleviate the shortcomings of speaker feature extraction ability. We evaluate the proposed methods on a simulated dataset of two speakers and a real conversation dataset. The model with the most favorable outcomes among the proposed enhancements achieves a diarization error rate of 7.74% on the simulated dataset and 21.92% on MagicData-RAMC after adaptation. These results compellingly demonstrate the efficacy of the proposed methods.

Soft sensor model predictive control for azeotropic distillation of the separation of DIPE/IPA/water mixture

BackgroundA novel advanced control scheme for azeotropic distillation (AD) shows superiority in improving product quality and economic benefit. MethodsA soft sensor model predictive control (SS-MPC) strategy was developed for the separation of the diisopropyl ether (DIPE)/ isopropyl alcohol (IPA)/water mixture via AD. The soft sensor model was built based on time-delayed neural networks (TDNN), while the weight of the SS-MPC controller was optimized by multi-objective genetic algorithm. Significant findingsProduct purity of two distillation columns was controlled simultaneously by the proposed soft sensor model predictive controller, meanwhile, avoiding the measurement of composition in practice. The dynamic performance was compared with the conventional PID scheme and MPC scheme in the face of ±10 % feed disturbances. The results show that SS-MPC can better control the product purity than PID without using the composition detection instruments.

Advancements in end-to-end isolated Kannada ASR system by combining robust noise elimination technique and TDNN

The demonstration of the latest enhancements in the end-to-end (E2E) isolated Kannada automatic speech recognition (ASR) system, achieved by combining a robust background noise elimination technique and a time delay neural network (TDNN), is presented in this work. An E2E Kannada ASR system consists of an interactive voice response system (IVRS), ASR models, and databases containing weather and agricultural commodity prices information. In the earlier spoken query system (SQS), the presence of babble, street noise, and other background noises led to a decrease in both offline and online speech recognition accuracies. To properly train the models, we increase the size of the database by collecting the Kannada speech data from an additional 500 farmers under real-time conditions. Moreover, the proposed noise elimination technique is employed to enhance the degraded speech data. Additionally, the efficacy of the TDNN is explored to improve the recognition accuracy of ASR models and the SQS system.The outcomes of the proposed speech enhancement algorithm demonstrate the absence of audible musical noise and other types of background noises in enhanced NOIZEUS and isolated Kannada speech databases. Leveraging Kannada language resources and the amalgamation of the proposed noise reduction technique and TDNN, a significant 1.1% enhancement in speech recognition accuracy is achieved in comparison with the previously developed deep neural network-hidden Markov model (DNN-HMM) based SQS. The enhanced isolated E2E ASR SQS system undergoes testing by 500 farmers, enabling them to access real-time agricultural commodity prices and weather forecasting information in their native Kannada language/dialects. This practical validation highlights the applicability and effectiveness of our advancements in real-world scenarios. The source code of proposed noise elimination technique, experimental results of ASR models and demo conversation of SQS are made publicly available at: https://sites.google.com/view/thimmarajayadavag/downloads

Aircraft Actuator Performance Analysis Based on Dynamic Neural Network

Monitoring the condition of the aircraft actuators in various operating and environmental circumstances, this paper presents a method for measuring the surface roughness of aircraft actuators. The proposed method starts with the current and vibration signal as failure indicators and a dual-tree complex wavelet transformation (DTCWT) to generate the necessary features. Time-delay neural networks (TDNNs) have been developed for real-time performance monitoring to categorize problems and determine their severity. The simulation results show that the suggested method can accurately identify various faults.