Autism Spectrum Disorder Detection by Hybrid Convolutional Recurrent Neural Networks from Structural and Resting State Functional MRI Images

In hospitals, brain-related disorders such as Parkinson's disease (PD) could be diagnosed by analyzing electroencephalograms (EEG). However, conventional EEG-based diagnosis for PD relies on handcrafted feature extraction, which is laborious and time-consuming. With the emergence of deep learning, automated analysis of EEG signals can be realized by exploring the inherent information in data, and outputting the results of classification from the hidden layer. In the present study, four deep learning algorithm architectures, including two convention deep learning models (convolutional neural network, CNN; and recurrent neural network, RNN) and two hybrid convolutional recurrent neural networks (2D-CNN-RNN and 3D-CNN-RNN), were designed to detect PD based on task-state EEG signals. Our results showed that the hybrid models outperformed conventional ones (fivefold average accuracy: 3D-CNN-RNN 82.89%, 2D-CNN-RNN 81.13%, CNN 80.89%, and RNN 76.00%) as they combine the strong modeling power of CNN in temporal feature extraction, and the advantage of RNN in processing sequential information. This study represents the an attempt to use hybrid convolutional recurrent neural networks in classifying PD and normal take-state EEG signals, which carries important implications to the clinical practice.

For women, most common cause of death is Breast tumour and in worldwide, it is the second leading reason for cancer deaths. Due the requirement of breast cancer’s early detection and false diagnosis impact on patients, made researchers to investigate Deep Learning (DL) techniques for mammograms. There are four stages in this proposed HIRResCNN framework, namely, Pre-processing, reduction of dimensionality, segmentation and classification. From images, noises are removed using two filtering algorithms called Median and mean filtering in pre-processing stage. Then canny edge detector is used for detecting edges. Gaussian filtering is used in canny edge detector to smoothen the images. In the next dimensionality reduction stage, attributes are correlated using Principal Component Analysis (PCA) inclusive of related features. So, this huge dataset is minimized and only few variables are used for expressing it. In order to detect the breast cancer accurately, foreground and background subtraction is done in the third stage called segmentation stage. At last, for detecting and classifying breast cancer, a Hybrid Inception Recurrent Residual Convolutional Neural Network (HIRResCNN) is introduced, which integrates Harmony Search Optimization (HSO) to tune bias and weight parameters and classification accuracy is enhanced using HIRResCNN-HSO model. Strength of Recurrent Convolutional Neural Network (RCNN), Residual Network (ResNet) and Inception Network (Inception-v4), are combined in a powerful Deep Convolutional Neural Network (DCNN) model called HIRResCNN. using Mammographic Image Analysis Society (MIAS) dataset, various experiments are conduced and results are compared with other available techniques. Around 92.6% accuracy rate is produced using this proposed HIRResCNN classifier in finding breast cancer.

In this paper, we investigate the performance of two deep learning paradigms for the audio-based tasks of acoustic scene, environmental sound and domestic activity classification. In particular, a convolutional recurrent neural network (CRNN) and pre-trained convolutional neural networks (CNNs) are utilised. The CRNN is directly trained on Mel-spectrograms of the audio samples. For the pre-trained CNNs, the activations of one of the top layers of various architectures are extracted as feature vectors and used for training a linear support vector machine (SVM).Moreover, the predictions of the two models—the class probabilities predicted by the CRNN and the decision function of the SVM—are combined in a decision-level fusion to achieve the final prediction. For the pre-trained CNN networks we use as feature extractors, we further evaluate the effects of a range of configuration options, including the choice of the pre-training corpus. The system is evaluated on the acoustic scene classification task of the IEEE AASP Challenge on Detection and Classification of Acoustic Scenes and Events (DCASE 2017) workshop, ESC-50 and the multi-channel acoustic recordings from DCASE 2018, task 5. We have refrained from additional data augmentation as our primary goal is to analyse the general performance of the proposed system on different datasets. We show that using our system, it is possible to achieve competitive performance on all datasets and demonstrate the complementarity of CRNNs and ImageNet pre-trained CNNs for acoustic classification tasks. We further find that in some cases, CNNs pre-trained on ImageNet can serve as more powerful feature extractors than AudioSet models. Finally, ImageNet pre-training is complimentary to more domain-specific knowledge, either in the form of the convolutional recurrent neural network (CRNN) trained directly on the target data or the AudioSet pre-trained models. In this regard, our findings indicate possible benefits of applying cross-modal pre-training of large CNNs to acoustic analysis tasks.

To design a convolutional recurrent neural network (CRNN) that calculates three-dimensional (3D) positions of lung tumors from continuously acquired cone beam computed tomography (CBCT) projections, and facilitates the sorting and reconstruction of 4D-CBCT images. Under an IRB-approved clinical lung protocol, kilovoltage (kV) projections of the setup CBCT were collected in free-breathing. Concurrently, an electromagnetic signal-guided system recorded motion traces of three transponders implanted in or near the tumor. Convolutional recurrent neural network was designed to utilize a convolutional neural network (CNN) for extracting relevant features of the kV projections around the tumor, followed by a recurrent neural network for analyzing the temporal patterns of the moving features. Convolutional recurrent neural network was trained on the simultaneously collected kV projections and motion traces, subsequently utilized to calculate motion traces solely based on the continuous feed of kV projections. To enhance performance, CRNN was also facilitated by frequent calibrations (e.g., at 10° gantry rotation intervals) derived from cross-correlation-based registrations between kV projections and templates created from the planning 4DCT. Convolutional recurrent neural network was validated on a leave-one-out strategy using data from 11 lung patients, including 5500kV images. The root-mean-square error between the CRNN and motion traces was calculated to evaluate the localization accuracy. Three-dimensional displacement around the simulation position shown in the Calypso traces was 3.4±1.7mm. Using motion traces as ground truth, the 3D localization error of CRNN with calibrations was 1.3±1.4mm. CRNN had a success rate of 86±8% in determining whether the motion was within a 3D displacement window of 2mm. The latency was 20ms when CRNN ran on a high-performance computer cluster. CRNN is able to provide accurate localization of lung tumors with aid from frequent recalibrations using the conventional cross-correlation-based registration approach, and has the potential to remove reliance on the implanted fiducials.

Autism spectrum disorder (ASD) is a neurodevelopmental disorder accompanied by structural and functional changes in the brain. However, the relationship between brain structure and function in children with ASD remains largely obscure. In the current study, parallel independent component analysis (pICA) was performed to identify inter-modality associations by drawing on information from different modalities. Structural and resting-state functional magnetic resonance imaging data from 105 children with ASD and 102 typically developing children (obtained from the open-access Autism Brain Imaging Data Exchange database) were combined through the pICA framework. Features of structural and functional modalities were represented by the voxel-based morphometry (VBM) and amplitude of low-frequency fluctuations (ALFF), respectively. The relationship between the structural and functional components derived from the pICA was investigated by Pearson's correlation analysis, and between-group differences in these components were analyzed through the two-sample t-test. Finally, multivariate support vector regression analysis was used to analyze the relationship between the structural/functional components and Autism Diagnostic Observation Schedule (ADOS) subscores in the ASD group. This study found a significant association between VBM and ALFF components in ASD. Significant between-group differences were detected in the loading coefficients of the VBM component. Furthermore, the ALFF component loading coefficients predicted the subscores of communication and repetitive stereotypic behaviors of the ADOS. Likewise, the VBM component loading coefficients predicted the ADOS communication subscore in ASD. These findings provide evidence of a link between brain function and structure, yielding new insights into the neural mechanisms of ASD.

Autism spectrum disorder (ASD) is a collection of a way of continuing neurodevelopment illnesses identified by limited and repetitive behavioral patterns, as well as social and communication impairments. Despite the fact that symptoms are most common in childhood, diagnosis is sometimes delayed. Because the current ASD diagnostic technique is solely uncertain and questionnaire, requiring the physician to review the behavior and developmental history of a child. It's been suggested that behavioral symptoms in ASD are linked to brain findings of increased short-distance and diminished long-distance connections. The suggested approach makes use of brain imaging data from Autism Brain Imaging Data Exchange (ABIDE), a global multi-site database. For ASD identification, the proposed approach uses functional connectivity characteristics extracted from resting state functional MRI data. ASD patients’ brain connections could be disturbed. The presented method extracts time series from 122 regions of interest defined by the Bootstrap Analysis of Stable Clusters (BASC) and also extracts time series from 48 regions of interest defined by the Harvard Oxford (HO) brain atlas to create an efficient functional connectivity matrix for all individuals. Synthetic Minority Over-sampling Technique (SMOTE) an oversampling procedure employed for artificial data generation. The classification challenge is carried out using a Recurrent Convolutional Neural Network (RCNN) and Convolutional Neural Network (CNN) and results were compared. In both models, fivefold Cross-Validation is applied. The model performed better in the modified experiment, with RCNN achieving an accuracy of nearly 85%.

In this paper we study how a Convolutional Recurrent Neural Network performs for predicting the gene expression levels from histone modification signals. Moreover, we consider two simplified variants of the Convolutional Recurrent Neural Network: Convolutional Neural Network and Recurrent Neural Network. The performance of the methods is evaluated with histone modification signal and gene expression data derived from Roadmap Epigenomics Mapping Consortium database, and compared against the state of the art method: the DeepChrome. It is shown that the proposed models give a statistically significant improvement over the baseline.

Heart failure (HF) is a complex clinical syndrome that poses a major hazard to human health. Patients with different types of HF have great differences in pathogenesis and treatment options. Therefore, HF typing is of great significance for timely treatment of patients. In this paper, we proposed an automatic approach for HF typing based on heart sounds (HS) and convolutional recurrent neural networks, which provides a new non-invasive and convenient way for HF typing. Firstly, the collected HS signals were preprocessed with adaptive wavelet denoising. Then, the logistic regression based hidden semi-Markov model was utilized to segment HS frames. For the distinction between normal subjects and the HF patients with preserved ejection fraction or reduced ejection fraction, a model based on convolutional neural network and recurrent neural network was built. The model can automatically learn the spatial and temporal characteristics of HS signals. The results show that the proposed model achieved a superior performance with an accuracy of 97.64%. This study suggests the proposed method could be a useful tool for HF recognition and as a supplement for HF typing.

In this paper, we present a novel method, called four-dimensional convolutional recurrent neural network, which integrating frequency, spatial and temporal information of multichannel EEG signals explicitly to improve EEG-based emotion recognition accuracy. First, to maintain these three kinds of information of EEG, we transform the differential entropy features from different channels into 4D structures to train the deep model. Then, we introduce CRNN model, which is combined by convolutional neural network (CNN) and recurrent neural network with long short term memory (LSTM) cell. CNN is used to learn frequency and spatial information from each temporal slice of 4D inputs, and LSTM is used to extract temporal dependence from CNN outputs. The output of the last node of LSTM performs classification. Our model achieves state-of-the-art performance both on SEED and DEAP datasets under intra-subject splitting. The experimental results demonstrate the effectiveness of integrating frequency, spatial and temporal information of EEG for emotion recognition.

Most speech separation studies in monaural channel use only a single type of network, and the separation effect is typically not satisfactory, posing difficulties for high quality speech separation. In this study, we propose a convolutional recurrent neural network with an attention (CRNN-A) framework for speech separation, fusing advantages of two networks together. The proposed separation framework uses a convolutional neural network (CNN) as the front-end of a recurrent neural network (RNN), alleviating the problem that a sole RNN cannot effectively learn the necessary features. This framework makes use of the translation invariance provided by CNN to extract information without modifying the original signals. Within the supplemented CNN, two different convolution kernels are designed to capture information in both the time and frequency domains of the input spectrogram. After concatenating the time-domain and the frequency-domain feature maps, the feature information of speech is exploited through consecutive convolutional layers. Finally, the feature map learned from the front-end CNN is combined with the original spectrogram and is sent to the back-end RNN. Further, the attention mechanism is further incorporated, focusing on the relationship among different feature maps. The effectiveness of the proposed method is evaluated on the standard dataset MIR-1K and the results prove that the proposed method outperforms the baseline RNN and other popular speech separation methods, in terms of GNSDR (gloabl normalised source-to-distortion ratio), GSIR (global source-to-interferences ratio), and GSAR (gloabl source-to-artifacts ratio). In summary, the proposed CRNN-A framework can effectively combine the advantages of CNN and RNN, and further optimise the separation performance via the attention mechanism. The proposed framework can shed a new light on speech separation, speech enhancement, and other related fields.

In this paper, alternative direction finding methods have been proposed using deep learning techniques. Firstly, Regeression and Classification models have created by using Convolutional Neural Networks (CNNs). In the second Convolutional Neural Networks and Recurrent Neural Networks (RNNs) have been utilized in the proposed methods. Despite having vast amount of direction finding studies, utilization of neural networks is scarce in literature and past works mostly only includes usage of CNNs. In this study, direction finding is performed by learning signals reaching multiple antenna arrays by networks. Created neural networks have been fed with different data formats and their performances against noisy and no-noise data have been shown. In addition, comparative analysis of the developed models were made in the similar Signal-to-Noise Ratio (SNR) range with the subspace based MUSIC algorithm, which is frequently used in direction finding.

EEG emotion recognition based on TQWT-features and hybrid convolutional recurrent neural network

Automatic machine anomaly sound detection is important for machine maintenance in display manufacturing factory. Recently, unsupervised anomaly detection approach based on autoencoder and reconstruction error has been proposed, which has assumption that the network trained only with the normal data produces higher reconstruction errors for the anomalies than the normal inputs. However, most approaches based on autoencoder consist of Convolutional Neural Network (CNN) which deals with the spectrogram of the sound like an image. In this paper, we propose Convolutional Recurrent Neural Network to use the characteristics of the spectrogram. To effectively integrate time and frequency information, we apply a 1‐dimensional convolutional layers and recurrent neural network to the network structure. Also, we introduce a prediction loss to prevent the network from learning to reconstruct anomalies well. Experimental results on the ToyADMOS and MIMII datasets demonstrate that the proposed approach achieves promising performance.

Nowadays, network attacks are the most crucial problem of modern society. All networks, from small to large, are vulnerable to network threats. An intrusion detection (ID) system is critical for mitigating and identifying malicious threats in networks. Currently, deep learning (DL) and machine learning (ML) are being applied in different domains, especially information security, for developing effective ID systems. These ID systems are capable of detecting malicious threats automatically and on time. However, malicious threats are occurring and changing continuously, so the network requires a very advanced security solution. Thus, creating an effective and smart ID system is a massive research problem. Various ID datasets are publicly available for ID research. Due to the complex nature of malicious attacks with a constantly changing attack detection mechanism, publicly existing ID datasets must be modified systematically on a regular basis. So, in this paper, a convolutional recurrent neural network (CRNN) is used to create a DL-based hybrid ID framework that predicts and classifies malicious cyberattacks in the network. In the HCRNNIDS, the convolutional neural network (CNN) performs convolution to capture local features, and the recurrent neural network (RNN) captures temporal features to improve the ID system’s performance and prediction. To assess the efficacy of the hybrid convolutional recurrent neural network intrusion detection system (HCRNNIDS), experiments were done on publicly available ID data, specifically the modern and realistic CSE-CIC-DS2018 data. The simulation outcomes prove that the proposed HCRNNIDS substantially outperforms current ID methodologies, attaining a high malicious attack detection rate accuracy of up to 97.75% for CSE-CIC-IDS2018 data with 10-fold cross-validation.

Statistics show that the risk of autism spectrum disorder (ASD) is increasing in the world. Early diagnosis is most important factor in treatment of ASD. Thus far, the childhood diagnosis of ASD has been done based on clinical interviews and behavioral observations. There is a significant need to reduce the use of traditional diagnostic techniques and to diagnose this disorder in the right time and before the manifestation of behavioral symptoms. The purpose of this study is to present the intelligent model to diagnose ASD in young children based on resting-state functional magnetic resonance imaging (rs-fMRI) data using convolutional neural networks (CNNs). CNNs, which are by far one of the most powerful deep learning algorithms, are mainly trained using datasets with large numbers of samples. However, obtaining comprehensive datasets such as ImageNet and achieving acceptable results in medical imaging domain have become challenges. In order to overcome these two challenges, the two methods of "combining classifiers," both dynamic (mixture of experts) and static (simple ‌Bayes) approaches, and "transfer learning" were used in this analysis. In addition, since diagnosis of ASD will be much more effective at an early age, samples ranging in age from 5 to 10years from global Autism Brain Imaging Data Exchange I and II (ABIDE I and ABIDE II) datasets were used in this research. The accuracy, sensitivity, and specificity of presented model outperform the results of previous studies conducted on ABIDE I dataset (the best results obtained from Adamax optimization technique: accuracy = 0.7273, sensitivity = 0.712, specificity = 0.7348). Furthermore, acceptable classification results were obtained from ABIDE II dataset (the best results obtained from Adamax optimization technique: accuracy = 0.7, sensitivity = 0.582, specificity = 0.804) and the combination of ABIDE I and ABIDE II datasets (the best results obtained from Adam optimization technique: accuracy = 0.7045, sensitivity = 0.679, specificity = 0.7421). We can conclude that the proposed architecture can be considered as an efficient tool for diagnosis of ASD in young children. From another perspective, this proposed method can be applied to analyzing rs-fMRI data related to brain dysfunctions.