Convolutional Recurrent Network Research Articles

Genotype sampling for deep-learning assisted experimental mapping of a combinatorially complete fitness landscape.

Experimental characterization of fitness landscapes, which map genotypes onto fitness, is important for both evolutionary biology and protein engineering. It faces a fundamental obstacle in the astronomical number of genotypes whose fitness needs to be measured for any one protein. Deep learning may help to predict the fitness of many genotypes from a smaller neural network training sample of genotypes with experimentally measured fitness. Here I use a recently published experimentally mapped fitness landscape of more than 260,000 protein genotypes to ask how such sampling is best performed. I show that multilayer perceptrons, recurrent neural networks (RNNs), convolutional networks, and transformers, can explain more than 90 percent of fitness variance in the data. In addition, 90 percent of this performance is reached with a training sample comprising merely ≈103 sequences. Generalization to unseen test data is best when training data is sampled randomly and uniformly, or sampled to minimize the number of synonymous sequences. In contrast, sampling to maximize sequence diversity or codon usage bias reduces performance substantially. These observations hold for more than one network architecture. Simple sampling strategies may perform best when training deep learning neural networks to map fitness landscapes from experimental data. The fitness landscape data analyzed here is publicly available as described previously (Papkou, et al., 2023). All code used to analyze this landscape is publicly available at https://github.com/andreas-wagner-uzh/fitness_landscape_sampling. Supplementary data are available at Bioinformatics online.

Adaptive Graph Convolutional Recurrent Network with Transformer and Whale Optimization Algorithm for Traffic Flow Prediction

Accurate traffic flow prediction plays a crucial role in the development of intelligent traffic management. Despite numerous investigations into spatio-temporal methods, achieving high accuracy in traffic flow prediction remains challenging. This challenge arises from the complex dynamic spatio-temporal correlations within the traffic road network and the limitations imposed by the selection of hyperparameters based on experiments and manual experience, which can affect the performance of the network architecture. This paper introduces a novel transformer-based adaptive graph convolutional recurrent network. The proposed network automatically infers the interdependencies among different traffic sequences and incorporates the capability to capture global spatio-temporal correlations. This enables the dynamic capture of long-range temporal correlations. Furthermore, the whale optimization algorithm is employed to efficiently design an optimal network structure that aligns with the requirements of the traffic domain and maximizes the utilization of limited computational resources. This design approach significantly enhances the model’s performance and improves the accuracy of traffic flow prediction. The experimental results on four real datasets demonstrate the efficacy of our approach. In PEMS03, it improves MAE by 2.6% and RMSE by 1.4%. In PEMS04, improvements are 1.6% in MAE and 1.4% in RMSE, with a similar MAPE score to the best baseline. For PEMS07, our approach shows a 4.1% improvement in MAE and 2.2% in RMSE. On PEMS08, it surpasses the current best baseline with a 3.4% improvement in MAE and 1.6% in RMSE. These results confirm the good performance of our model in traffic flow prediction across multiple datasets.

Squeeze-and-excitation 3D convolutional attention recurrent network for end-to-end speech emotion recognition

Speech emotion recognition (SER) is difficult since emotions are complex and dynamic processes involving multiple dimensions and sub-dimensions. Feature extraction is a challenging step in SER, where relevant features are extracted from the speech to identify emotional states accurately. Overcoming these challenges is essential to ensuring the effectiveness and robustness of the SER system. 3D-convolutional neural networks (3D-CNNs) are successfully used for feature extraction in SER. Speech signals can be converted into spectrogram-like representations where one axis represents time, another frequency, and the third can represent additional context or features. For modeling the dynamic nature of speech, a squeeze-and-excitation-based 3D-CNN model is employed for capturing temporal and spatial features of speech. Attention Gated Recurrent Units (AGRU) are applied to the extracted features for learning long-range temporal dependencies and selecting more informative features. The extraction and selection of the spatio-temporal feature representation lead to a hierarchical representation of the input speech. The fusion of squeeze-and-excitation 3D-CNN and AGRU (named SE3D-CARN) is evaluated on two datasets, EMO-BD and IEMOCAP, to identify various emotional states in speech. The proposed SER model reached an accuracy of 94.2% and 81.1% on the EMO-BD and IEMOCAP datasets, respectively.

A traffic flow prediction method based on constrained dynamic graph convolutional recurrent networks

To enhance the efficiency of predicting future traffic flow trends within a transportation network, we propose a traffic flow prediction approach based on Constrained Dynamic Graph Convolutional Recurrent Network (C-DGCRN). Firstly, we employ matrix linear constraint and low-rank constraint optimization techniques to assist the Dynamic Graph Convolutional Recurrent Network (DGCRN) in extracting relevant internal correlations within the transportation network, leading to the formation of an affinity matrix. Secondly, we utilize the Hadamard product operation to effectively transfer weights from the affinity matrix to DGCRN, further enhancing its predictive accuracy. The experiments conducted an error assessment on the Metro Traffic Los Angeles and Performance Measurement System-Bay Area datasets. The Mean Absolute Error, Root Mean Square Error, and Mean Absolute Percentage Error of C-DGCRN were 2.66, 5.06, and 6.83, respectively. Compared to state-of-the-art methods like Adaptive Graph Convolutional Recurrent Network, C-DGCRN showed a reduction of 7%, 9%, and 11% in these metrics.

Indonesian Lip-Reading Detection and Recognition Based on Lip Shape Using Face Mesh and Long-Term Recurrent Convolutional Network

Indonesian Lip-Reading Detection and Recognition Based on Lip Shape Using Face Mesh and Long-Term Recurrent Convolutional Network

Recurrent Neural Networks and classical machine learning methods for concentrations prediction of aluminum alloy in laser Induced breakdown spectroscopy

Recurrent Neural Networks are classes of Artificial Neural Networks that establish connections between various nodes in a directed or undirected graph for investigation of the temporal dynamical. In this study, different Recurrent Neural Network (RNN) architectures are utilized for quantitative analysis of aluminum alloys by the laser induced breakdown spectroscopy (LIBS) technique. The fundamental harmonic (1064 nm) of a nanosecond Nd:YAG laser pulse is employed to generate the LIBS plasma for the prediction of constituent concentrations of the aluminum standard samples. For the purpose of predicting concentration, Recurrent Neural Networks based on different networks, such as Long Short Term Memory (LSTM), Gated Recurrent Unit (GRU), Simple Recurrent Neural Network (Simple RNN), and as well as Recurrent Convolutional Networks comprising of Conv-SimpleRNN, Conv-LSTM and Conv-GRU are employed. Then, a comparison is made among prediction by classical machine learning methods of support vector regressor (SVR), the Multilayer Perceptron (MLP), Decision Tree algorithm, Gradient Boosting Regression (GBR), Random Forest Regression (RFR), Linear Regression, and k-Nearest Neighbor (KNN) algorithm. Results demonstrated that the machine learning tools based on Convolutional Recurrent Networks had the best efficiencies in prediction of the most of the elements among other multivariate methods.

Eye-LRCN: A Long-Term Recurrent Convolutional Network for Eye Blink Completeness Detection.

Computer vision syndrome causes vision problems and discomfort mainly due to dry eye. Several studies show that dry eye in computer users is caused by a reduction in the blink rate and an increase in the prevalence of incomplete blinks. In this context, this article introduces Eye-LRCN, a new eye blink detection method that also evaluates the completeness of the blink. The method is based on a long-term recurrent convolutional network (LRCN), which combines a convolutional neural network (CNN) for feature extraction with a bidirectional recurrent neural network that performs sequence learning and classifies the blinks. A Siamese architecture is used during CNN training to overcome the high-class imbalance present in blink detection and the limited amount of data available to train blink detection models. The method was evaluated on three different tasks: blink detection, blink completeness detection, and eye state detection. We report superior performance to the state-of-the-art methods in blink detection and blink completeness detection, and remarkable results in eye state detection.

Sign Language Word Detection Using LRCN

Sign language is the most effective communication for deaf or hard-of-hearing people. Specialized training is required to understand sign language, and as such, people without disabilities around them cannot communicate effectively. The main objective of this study is to develop a mechanism for streamlining the deep learning model for sign language recognition by utilizing the 30 most prevalent words in our everyday lives. The dataset was designed through 30 ASL (American Sign Language) words consisting of custom-processed video sequences, which consist of 5 subjects and 50 sample videos for each class. The CNN model can be applied to video frames to extract spatial properties. Using CNN’s acquired data, the LSTM model may then predict the action being performed in the video. We present and evaluate the results of two separate datasets—the Pose dataset and the Raw video dataset. The dataset was trained with the Long-term Recurrent Convolutional Network (LRCN) approach. Finally, a test accuracy of 92.66% was reached for the raw dataset, while 93.66% for the pose dataset.

Brain Tumor Detection Using Classification Model

This study aims to develop automated methods for detecting and segmenting brain tumors in MRI scans using neural networks and conventional image processing. Accurately identifying tumor location and boundaries is crucial for diagnosis and determining the most effective treatment approach. Manual review by radiologists is time-consuming and subjective. However, recent advances in deep learning now enable more efficient analysis of large MRI datasets.
 A variety of neural network models have been explored for tumor segmentation, including recurrent networks, autoencoders and convolutional networks. RNNs are well-suited to capture contextual information across sequential MRI slices. Autoencoders perform unsupervised feature learning from the images. CNNs have shown promise due to their ability to directly learn visual patterns from pixel values. Unsupervised clustering techniques such as fuzzy c-means and k-means are also investigated to group tumor pixels based on intensity similarity without labeled data.
 Previous research combining these techniques has achieved high segmentation accuracy between 84-97% using additional image processing steps. Automated segmentation provides benefits over manual inspection such as reduced workload and more consistent delineation of tumor margins. It can precisely outline growths to aid radiologists in diagnosis and pathologists in assessing malignancy.
 The detection and segmentation of brain tumors has applications in both radiology and oncology. Radiologists could benefit from faster scan review and automated report generation. Neurosurgeons would gain insights into a tumor's size, location and infiltration to determine the most suitable surgical approach. Oncologists could utilize quantitative tumor features to select the most targeted radiation therapy or chemotherapy protocols. Overall, continued advances in deep and machine learning hold promise to improve diagnosis and management of brain cancer patients.

R-GCN: a residual-gated recurrent unit convolution network model for anomaly detection in blockchain transactions

R-GCN: a residual-gated recurrent unit convolution network model for anomaly detection in blockchain transactions

An adaptive adjacency matrix-based graph convolutional recurrent network for air quality prediction

In recent years, air pollution has become increasingly serious and poses a great threat to human health. Timely and accurate air quality prediction is crucial for air pollution early warning and control. Although data-driven air quality prediction methods are promising, there are still challenges in studying spatial–temporal correlations of air pollutants to design effective predictors. To address this issue, a novel model called adaptive adjacency matrix-based graph convolutional recurrent network (AAMGCRN) is proposed in this study. The model inputs Point of Interest (POI) data and meteorological data into a fully connected neural network to learn the weights of the adjacency matrix thereby constructing the self-ringing adjacency matrix and passes the pollutant data with this matrix as input to the Graph Convolutional Network (GCN) unit. Then, the GCN unit is embedded into LSTM units to learn spatio-temporal dependencies. Furthermore, temporal features are extracted using Long Short-Term Memory network (LSTM). Finally, the outputs of these two components are merged and air quality predictions are generated through a hidden layer. To evaluate the performance of the model, we conducted multi-step predictions for the hourly concentration of PM2.5, PM10 and O3 at Fangshan, Tiantan and Dongsi monitoring stations in Beijing. The experimental results show that our method achieves better predicted effects compared with other baseline models based on deep learning. In general, we designed a novel air quality prediction method and effectively addressed the shortcomings of existing studies in learning the spatio-temporal correlations of air pollutants. This method can provide more accurate air quality predictions and is expected to provide support for public health protection and government environmental decision-making.

Single image super-resolution based on trainable feature matching attention network

Convolutional Neural Networks (CNNs) have been widely employed for image Super-Resolution (SR) in recent years. Various techniques enhance SR performance by altering CNN structures or incorporating improved self-attention mechanisms. Interestingly, these advancements share a common trait. Instead of explicitly learning high-frequency details, they learn an implicit feature processing mode that utilizes weighted sums of a feature map’s own elements for reconstruction, akin to convolution and non-local. In contrast, early dictionary-based approaches learn feature decompositions explicitly to match and rebuild Low-Resolution (LR) features. Building on this analysis, we introduce Trainable Feature Matching (TFM) to amalgamate this explicit feature learning into CNNs, augmenting their representation capabilities. Within TFM, trainable feature sets are integrated to explicitly learn features from training images through feature matching. Furthermore, we integrate non-local and channel attention into our proposed Trainable Feature Matching Attention Network (TFMAN) to further enhance SR performance. To alleviate the computational demands of non-local operations, we propose a streamlined variant called Same-size-divided Region-level Non-Local (SRNL). SRNL conducts non-local computations in parallel on blocks uniformly divided from the input feature map. The efficacy of TFM and SRNL is validated through ablation studies and module explorations. We employ a recurrent convolutional network as the backbone of our TFMAN to optimize parameter utilization. Comprehensive experiments on benchmark datasets demonstrate that TFMAN achieves superior results in most comparisons while using fewer parameters. The code is available at https://github.com/qizhou000/tfman.

End-to-end attention convolutional recurrent network for online handwritten Chinese text recognition

End-to-end attention convolutional recurrent network for online handwritten Chinese text recognition

Flood prediction with optimized gated recurrent unit-temporal convolutional network and improved KDE error estimation

<abstract> <p>Flood time series forecasting stands a critical challenge in precise predictive models and reliable error estimation methods. A novel approach utilizing a hybrid deep learning model for both point and interval flood prediction is presented, enhanced by improved kernel density estimation (KDE) for prediction comparison and error simulation. Firstly, an optimized gated recurrent unit-time convolutional network (GRU-TCN) is constructed by tuning the internal structure of the TCN, the activation function, the L2 regularization, and the optimizer. Then, Pearson Correlation is used for feature selection, and the hyperparameters of the improved GRU-TCN are optimized by the subtraction-average-based optimizer (SABO). To further assess the prediction uncertainty, interval predictions are provided via Non-parametric KDE, with an optimized bandwidth setting for accurate error distribution simulation. Experimental comparisons are made on 5-year hydro-meteorological daily data from two stations along the Yangtze River. The proposed model surpasses long short-term memory network (LSTM), TCN, GRU, TCN-LSTM, and GRU-TCN, with a reduction of more than 13% in root mean square error (RMSE) and approximately 15% in mean absolute error (MAE), resulting in better interval estimation and error control. The improved kernel density estimation curves for the errors are closer to the mean value of the confidence intervals, better reflecting the trend of the error distribution. This research enhances the accuracy and reliability of flood predictions and improves the capacity of humans to cope with climate and environmental changes.</p> </abstract>

On the Predictive Power of Objective Intelligibility Metrics for the Subjective Performance of Deep Complex Convolutional Recurrent Speech Enhancement Networks

On the Predictive Power of Objective Intelligibility Metrics for the Subjective Performance of Deep Complex Convolutional Recurrent Speech Enhancement Networks

Deep neural network for learning wave scattering and interference of underwater acoustics

It is challenging to construct generalized physical models of underwater wave propagation owing to their complex physics and widely varying environmental parameters and dynamical scales. In this article, we present a deep convolutional recurrent autoencoder network (CRAN) for data-driven learning of complex underwater wave scattering and interference. We specifically consider the dynamics of underwater acoustic scattering from various non-uniform seamount shapes leading to complex wave interference patterns of back-scattered and forward-propagated waves. The CRAN consists of a convolutional autoencoder for learning low-dimensional system representation and a long short-term memory (LSTM)-based recurrent neural network for predicting system evolution in low dimensions. The convolutional autoencoder enables efficient dimension reduction of wave propagation by independently learning global and localized wave features. To improve the time horizon of wave dynamics prediction, we introduce an LSTM architecture with a single-shot learning mechanism and optimal time-delayed data embedding. On training the CRAN over 30 cases containing various seamount geometries and acoustic source frequencies, we can predict wave propagation up to a time horizon of 5 times the initiation sequence length for 15 out-of-training cases with a mean L2 error of approximately 10%. For selected out-of-training cases, the prediction time horizon could be increased to 6 times the initiation sequence length. Importantly, such predictions are obtained with physically consistent wave scattering and wave interference patterns and at 50% lower L2 error compared to routinely use standard LSTMs. These results demonstrate the potential of employing such deep neural networks for learning complex underwater ocean acoustic propagation physics.

Extended Multi-Component Gated Recurrent Graph Convolutional Network for Traffic Flow Prediction

Extended Multi-Component Gated Recurrent Graph Convolutional Network for Traffic Flow Prediction

Automatic 3D Lamina Curve Extraction from Freehand 3D Ultrasound Data using Sequential Localization Recurrent Convolutional Networks.

Freehand 3D ultrasound imaging is emerging as a promising modality for regular spine exams due to its non-invasiveness and affordability. The laminae landmarks play a critical role in depicting the 3D shape of the spine. However, the extraction of the 3D lamina curves from transverse ultrasound sequences presents a challenging task, primarily attributed to the presence of diverse contrast variations, imaging artifacts, the complex surface of vertebral bones, and the difficulties associated with probe manipulation. This paper proposes Sequential Localization Recurrent Convolutional Networks (SL-RCN), a novel deep learning model that takes the contextual relationships into account and embeds the transformation matrix feature as a 3D knowledge base to enhance accurate ultrasound sequence analysis. The assessment involved the analysis of 3D ultrasound sequences obtained from 10 healthy adult human participants, covering both the lumbar and thoracic regions. The performance of SL-RCN is evaluated through 7-fold cross-validation, employing the leave-one-participant-out strategy. The validity of the AI model training is assessed on test data from 3 participants. Normalized Discrete Fréchet Distance (NDFD) is employed as the main metric to evaluate the disparity of the extracted 3D lamina curves. In contrast to our previous 2D image analysis method, SL-RCN generates reduced left/right mean distance errors from 1.62/1.63mm to 1.41/1.40mm, and NDFDs from 0.5910/0.6389 to 0.4276/0.4567. The increase in the mean NDFD value from 7-fold cross-validation to the test-data experiment is less than 0.05. The experiments demonstrate the SL-RCN's capability in extracting accurate paired smooth lamina landmark curves.

Investigating the Influence of Convolutional Operations on LSTM Networks in Video Classification

Video classification holds a foundational position in the realm of computer vision, involving the categorization and labeling of videos based on their content. Its significance resonates across various applications, including video surveil-lance, content recommendation, action recognition, video indexing, and more. The primary objective of video classification is to automatically analyze and comprehend the visual information embedded in videos, facilitating the efficient organization, retrieval, and interpretation of extensive video collections. The integration of convolutional neural networks (CNNs) and long short-term memory (LSTM) networks has brought about a revolution in video classification. This fusion effectively captures both spatial and temporal dependencies within video sequences, leveraging the strengths of CNNs in extracting spatial features and LSTMs in modeling sequential and temporal information. ConvLSTM and LRCN (Long-term Recurrent Convolutional Networks) are two widely embraced architectures that embody this fusion. This paper seeks to investigate the impact of convolutions on LSTM networks in the context of video classification, aiming to compare the performance of ConvLSTM and LRCN.

Multi-level Graph Memory Network Cluster Convolutional Recurrent Network for traffic forecasting

Traffic forecasting plays a vital role in the management of urban road networks and the development of intelligent transportation systems. To effectively capture spatial and temporal dependencies within traffic data, Graph Neural Networks (GNNs) emerge by combining recurrent neural networks and dynamic graph learning. However, traditional GNNs face challenges in efficiently memorizing patterns at different levels of these dependencies, including long-term shared patterns and short-term specific patterns. We propose a traffic forecasting model, the Multi-level Graph Memory Network Cluster Convolutional Recurrent Network (MMNCCRN). The MMNCCRN consists of four modules: Encoder Module (EM), Attention Module (AM), Memory Network Cluster Module (MNCM), and Decoder Module (DM). To enhance the model’s performance, AM incorporates a concise yet efficient attention mechanism that augments important information in the output of EM. This alleviates the pressure on MNCM to identify patterns at different levels. Meanwhile, in MNCM, we extensively leverage memory networks and introduce the concept of clustering. By storing and memorizing patterns implicit in spatial and temporal dependencies, MNCM assists the model in learning underlying graph structures and discovering new hidden patterns. Experimental results show that our model outperforms others in most metrics on four datasets.