ConvNet Architecture Research Articles

Time-Frequency Decomposition of Scalp Electroencephalograms Improves Deep Learning-Based Epilepsy Diagnosis.

Epilepsy diagnosis based on Interictal Epileptiform Discharges (IEDs) in scalp electroencephalograms (EEGs) is laborious and often subjective. Therefore, it is necessary to build an effective IED detector and an automatic method to classify IED-free versus IED EEGs. In this study, we evaluate features that may provide reliable IED detection and EEG classification. Specifically, we investigate the IED detector based on convolutional neural network (ConvNet) with different input features (temporal, spectral, and wavelet features). We explore different ConvNet architectures and types, including 1D (one-dimensional) ConvNet, 2D (two-dimensional) ConvNet, and noise injection at various layers. We evaluate the EEG classification performance on five independent datasets. The 1D ConvNet with preprocessed full-frequency EEG signal and frequency bands (delta, theta, alpha, beta) with Gaussian additive noise at the output layer achieved the best IED detection results with a false detection rate of 0.23/min at 90% sensitivity. The EEG classification system obtained a mean EEG classification Leave-One-Institution-Out (LOIO) cross-validation (CV) balanced accuracy (BAC) of 78.1% (area under the curve (AUC) of 0.839) and Leave-One-Subject-Out (LOSO) CV BAC of 79.5% (AUC of 0.856). Since the proposed classification system only takes a few seconds to analyze a 30-min routine EEG, it may help in reducing the human effort required for epilepsy diagnosis.

Deep Learning with ConvNet Predicts Imagery Tasks Through EEG

Deep learning with convolutional neural networks (ConvNets) has dramatically improved the learning capabilities of computer vision applications just through considering raw data without any prior feature extraction. Nowadays, there is a rising curiosity in interpreting and analyzing electroencephalography (EEG) dynamics with ConvNets. Our study focused on ConvNets of different structures, the efficiency of multiple machine learning algorithms with optimization on ConvNets, constructing for predicting imagined left and right movements on a subject-independent basis through raw EEG data. We adapted novel lower-upper triangularization based extreme learning machines (LuELM) to the ConvNet architecture. Results showed that recently advanced methods in machine learning field, i.e. adaptive moments and batch normalization together with dropout strategy, improved ConvNets predicting ability, outperforming that of conventional fully-connected neural networks with widely-used spectral features. The proposed prediction model achieved improvements in classification performances with the rates of 90.33%, 91.00%, and 89.67% for accuracy, recall, and specificity, respectively.

ConvNet frameworks for multi-modal fake news detection

An upsurge of false information revolves around the internet. Social media and websites are flooded with unverified news posts. These posts are comprised of text, images, audio, and videos. There is a requirement for a system that detects fake content in multiple data modalities. We have seen a considerable amount of research on classification techniques for textual fake news detection, while frameworks dedicated to visual fake news detection are very few. We explored the state-of-the-art methods using deep networks such as CNNs and RNNs for multi-modal online information credibility analysis. They show rapid improvement in classification tasks without requiring pre-processing. To aid the ongoing research over fake news detection using CNN models, we build textual and visual modules to analyze their performances over multi-modal datasets. We exploit latent features present inside text and images using layers of convolutions. We see how well these convolutional neural networks perform classification when provided with only latent features and analyze what type of images are needed to be fed to perform efficient fake news detection. We propose a multi-modal Coupled ConvNet architecture that fuses both the data modules and efficiently classifies online news depending on its textual and visual content. We thence offer a comparative analysis of the results of all the models utilized over three datasets. The proposed architecture outperforms various state-of-the-art methods for fake news detection with considerably high accuracies.

Accurate Imputation of Greenhouse Environment Data for Data Integrity Utilizing Two-Dimensional Convolutional Neural Networks.

Greenhouses require accurate and reliable data to interpret the microclimate and maximize resource use efficiency. However, greenhouse conditions are harsh for electrical sensors collecting environmental data. Convolutional neural networks (ConvNets) enable complex interpretation by multiplying the input data. The objective of this study was to impute missing tabular data collected from several greenhouses using a ConvNet architecture called U-Net. Various data-loss conditions with errors in individual sensors and in all sensors were assumed. The U-Net with a screen size of 50 exhibited the highest coefficient of determination values and the lowest root-mean-square errors for all environmental factors used in this study. U-Net50 correctly learned the changing patterns of the greenhouse environment from the training dataset. Therefore, the U-Net architecture can be used for the imputation of tabular data in greenhouses if the model is correctly trained. Growers can secure data integrity with imputed data, which could increase crop productivity and quality in greenhouses.

Convolutional Neural Network Based Symbol Detector for Two-Dimensional Magnetic Recording

Data detection in magnetic recording (MR) channels can be viewed as an image processing problem, proceeding from the 2-D image of readback bits, to higher level abstractions of features using convolutional layers that finally allow classification of individual bits. In this work, convolutional neural networks (ConvNets) are employed in place of the typical partial response equalizer and maximum-likelihood detector with noise prediction to directly process the un-equalized readback signals and output soft estimates. Several variations of ConvNets are compared in terms of network complexity and performance. The best performing ConvNet detector with two convolutional layers provides a data storage density of up to 3.7489 Terabits/in <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> on a low track pitch two-dimensional MR channel simulated with a grain-flipping-probability (GFP) model. An alternate ConvNet architecture reduces the network complexity by about 74%, yet results in only a 2.09% decrease in density compared to the best performing detector.

Optimizing Hardware Accelerated General Matrix-Matrix Multiplication for CNNs on FPGAs

Convolution is inarguably the most complex operation utilized in Convolutional Neural Networks (convnets). Owing to the billions of independent multiply-adds involved, convolution is being massively parallelized by the simultaneous utilization of many cores of Graphical Processing Units (GPUs). Although GPUs have shown significant performance improvements in both training and inference stages, they are not well-suited for mobile vision applications where both energy and real-time constraints need to be satisfied. In contrast, Field Programmable Gate Arrays (FPGAs) have demonstrated massive parallelization capabilities, with fast DSPs and on-chip memory, at a lower energy cost than GPUs. Hence, they are being utilized to design convnet accelerators for embedded applications. In this brief, we design an FPGA-based accelerator for general matrix-matrix multiplication (GeMM) to improve the efficiency of convolutional layers of Shufflenet, an efficient convnet architecture. Experimental results show significant performance improvements against the state-of-the-art FPGA-based implementations of both efficient convnets that are tailored towards mobile vision applications, and complex convnets that are used in traditional applications.

Study on a Fast Solver for Poisson’s Equation Based on Deep Learning Technique

Fast and efficient computational electromagnetic simulation is a long-standing challenge. In this article, we propose a data-driven model to solve Poisson’s equation that leverages the learning capacity of deep learning techniques. A deep convolutional neural network (ConvNet) is trained to predict the electric potential with different excitations and permittivity distribution in 2-D and 3-D models. With a careful design of cost function and proper training data generated from finite-difference solvers, the proposed network enables a reliable simulation with significant speedup and fairly good accuracy. Numerical experiments show that the same ConvNet architecture is effective for both 2-D and 3-D models, and the average relative prediction error of the proposed ConvNet model is less than 3% in both 2-D and 3-D simulations with a significant reduction in computation time compared to the finite-difference solver. This article shows that deep neural networks have a good learning capacity for numerical simulations. This could help us to build some fast solvers for some computational electromagnetic problems.

OutdoorSent

Opinion mining in outdoor images posted by users during different activities can provide valuable information to better understand urban areas. In this regard, we propose a framework to classify the sentiment of outdoor images shared by users on social networks. We compare the performance of state-of-the-art ConvNet architectures and one specifically designed for sentiment analysis. We also evaluate how the merging of deep features and semantic information derived from the scene attributes can improve classification and cross-dataset generalization performance. The evaluation explores a novel dataset—namely, OutdoorSent—and other publicly available datasets. We observe that the incorporation of knowledge about semantic attributes improves the accuracy of all ConvNet architectures studied. Besides, we found that exploring only images related to the context of the study—outdoor, in our case—is recommended, i.e., indoor images were not significantly helpful. Furthermore, we demonstrated the applicability of our results in the United States city of Chicago, Illinois, showing that they can help to improve the knowledge of subjective characteristics of different areas of the city. For instance, particular areas of the city tend to concentrate more images of a specific class of sentiment, which are also correlated with median income, opening up opportunities in different fields.

Efficient Parallel Inflated 3D Convolution Architecture for Action Recognition

Deep neural networks have received increasing attention in human action recognition. Previous research has established that utilizing 3D convolution is a reasonable approach to learn spatio-temporal representation. Nevertheless, constructing effective 3D ConvNets usually need an expensive pre-training process that performing on a huge-scale video dataset. To avoid this burdensome situation, one major issue is to determine whether the pre-trained parameters of 2D convolution networks can be directly bootstrapped into 3D. In this paper, we devise a 2D-Inflated operation and a parallel 3D ConvNet architecture to solve this problem. The 2D-Inflated operation is used for converting pre-trained 2D ConvNets into 3D ConvNets, which avoiding video data pre-training. We further explore the optimal quantity of 3D ConvNet in the parallel architecture, and the results suggest that 6-nets architecture is an excellent solution for recognition. Another contribution of our study is two practical and valid skills, accumulated gradient descent and video sequence decomposition. Either of those techniques can promote the improvement of performance. The recognition results of UCF101 and HMDB51 reveal that, without the video data pre-training, our 3D ConvNets still can achieve competitive performance to the other generic and recent methods of using 3D ConvNets in the RGB image domain.

Improved Traffic Sign Recognition Using Deep ConvNet Architecture

Everyday more and more accidents between vehicles accrue mainly because of poorly visible road signs, that is why it is important to have a system that can help drivers for safer driving. In order to improve road safety and significantly reduce the risk of these accidents, numerous works have been undertaken in computer vision. In this paper, we propose a contribution to this issue by taking advantage of the remarkable results of deep convolutional neural networks in computer vision. We propose an automatic recognition system of road signs based on a modified model inspired by LeNet model. The results obtained by comparison of LeNet model and two proposed modified models on the German traffic dataset is about 99% accuracy which is promising compared to the state-of-the-art results.

Sensor Classification Using Convolutional Neural Network by Encoding Multivariate Time Series as Two-Dimensional Colored Images

This paper proposes a framework to perform the sensor classification by using multivariate time series sensors data as inputs. The framework encodes multivariate time series data into two-dimensional colored images, and concatenate the images into one bigger image for classification through a Convolutional Neural Network (ConvNet). This study applied three transformation methods to encode time series into images: Gramian Angular Summation Field (GASF), Gramian Angular Difference Field (GADF), and Markov Transition Field (MTF). Two open multivariate datasets were used to evaluate the impact of using different transformation methods, the sequences of concatenating images, and the complexity of ConvNet architectures on classification accuracy. The results show that the selection of transformation methods and the sequence of concatenation do not affect the prediction outcome significantly. Surprisingly, the simple structure of ConvNet is sufficient enough for classification as it performed equally well with the complex structure of VGGNet. The results were also compared with other classification methods and found that the proposed framework outperformed other methods in terms of classification accuracy.

Clinical Interpretable Deep Learning Model for Glaucoma Diagnosis.

Despite the potential to revolutionise disease diagnosis by performing data-driven classification, clinical interpretability of ConvNet remains challenging. In this paper, a novel clinical interpretable ConvNet architecture is proposed not only for accurate glaucoma diagnosis but also for the more transparent interpretation by highlighting the distinct regions recognised by the network. To the best of our knowledge, this is the first work of providing the interpretable diagnosis of glaucoma with the popular deep learning model. We propose a novel scheme for aggregating features from different scales to promote the performance of glaucoma diagnosis, which we refer to as M-LAP. Moreover, by modelling the correspondence from binary diagnosis information to the spatial pixels, the proposed scheme generates glaucoma activations, which bridge the gap between global semantical diagnosis and precise location. In contrast to previous works, it can discover the distinguish local regions in fundus images as evidence for clinical interpretable glaucoma diagnosis. Experimental results, performed on the challenging ORIGA datasets, show that our method on glaucoma diagnosis outperforms state-of-the-art methods with the highest AUC (0.88). Remarkably, the extensive results, optic disc segmentation (dice of 0.9) and local disease focus localization based on the evidence map, demonstrate the effectiveness of our methods on clinical interpretability.

Two-Stream Deep Hashing With Class-Specific Centers for Supervised Image Search.

Hashing has been widely used for large-scale approximate nearest neighbor search due to its storage and search efficiency. Recent supervised hashing research has shown that deep learning-based methods can significantly outperform nondeep methods. Most existing supervised deep hashing methods exploit supervisory signals to generate similar and dissimilar image pairs for training. However, natural images can have large intraclass and small interclass variations, which may degrade the accuracy of hash codes. To address this problem, we propose a novel two-stream ConvNet architecture, which learns hash codes with class-specific representation centers. Our basic idea is that if we can learn a unified binary representation for each class as a center and encourage hash codes of images to be close to the corresponding centers, the intraclass variation will be greatly reduced. Accordingly, we design a neural network that leverages label information and outputs a unified binary representation for each class. Moreover, we also design an image network to learn hash codes from images and force these hash codes to be close to the corresponding class-specific centers. These two neural networks are then seamlessly incorporated to create a unified, end-to-end trainable framework. Extensive experiments on three popular benchmarks corroborate that our proposed method outperforms current state-of-the-art methods.

Spatio-Temporal Residual Networks for Slide Transition Detection in Lecture Videos

In this paper, we present an approach for detecting slide transitions in lecture videos by introducing the spatio-temporal residual networks. Given a lecture video which records the digital slides, the speaker, and the audience by multiple cameras, our goal is to find keyframes where slide content changes. Since temporal dependency among video frames is important for detecting slide changes, 3D Convolutional Networks has been regarded as an efficient approach to learn the spatio-temporal features in videos. However, 3D ConvNet will cost much training time and need lots of memory. Hence, we utilize ResNet to ease the training of network, which is easy to optimize. Consequently, we present a novel ConvNet architecture based on 3D ConvNet and ResNet for slide transition detection in lecture videos. Experimental results show that the proposed novel ConvNet architecture achieves the better accuracy than other slide progression detection approaches.

Hyperparameter Optimization and Regularization on Fashion-MNIST Classification

Nowadays the most exciting technology breakthrough has been the rise of the deep learning. In computer vision Convolutional Neural Networks (CNN or ConvNet) are the default deep learning model used for image classification problems. In these deep network models, feature extraction is figure out by itself and these models tend to perform well with huge amount of samples. Herein we explore the impact of various Hyper-Parameter Optimization (HPO) methods and regularization techniques with deep neural networks on Fashion-MNIST (F-MNIST) dataset which is proposed by Zalando Research. We have proposed deep ConvNet architectures with Data Augmentation and explore the impact of this by configuring the hyperparameters and regularization methods. As deep learning requires a lots of data, the insufficiency of image samples can be expand through various data augmentation methods like Cropping, Rotation, Flipping, and Shifting. The experimental results show impressive results on this new benchmarking dataset F-MNIST.

DEEPCON: protein contact prediction using dilated convolutional neural networks with dropout.

Exciting new opportunities have arisen to solve the protein contact prediction problem from the progress in neural networks and the availability of a large number of homologous sequences through high-throughput sequencing. In this work, we study how deep convolutional neural networks (ConvNets) may be best designed and developed to solve this long-standing problem. With publicly available datasets, we designed and trained various ConvNet architectures. We tested several recent deep learning techniques including wide residual networks, dropouts and dilated convolutions. We studied the improvements in the precision of medium-range and long-range contacts, and compared the performance of our best architectures with the ones used in existing state-of-the-art methods. The proposed ConvNet architectures predict contacts with significantly more precision than the architectures used in several state-of-the-art methods. When trained using the DeepCov dataset consisting of 3456 proteins and tested on PSICOV dataset of 150 proteins, our architectures achieve up to 15% higher precision when L/2 long-range contacts are evaluated. Similarly, when trained using the DNCON2 dataset consisting of 1426 proteins and tested on 84 protein domains in the CASP12 dataset, our single network achieves 4.8% higher precision than the ensembled DNCON2 method when top L long-range contacts are evaluated. DEEPCON is available at https://github.com/badriadhikari/DEEPCON/.

3D Convolutional Neural Networks for Event-Related Potential detection.

Deep learning techniques have recently been successful in the classification of brain evoked responses for multiple applications, including brain-machine interface. Single-trial detection in the electroencephalogram (EEG) of brain evoked responses, like event-related potentials (ERPs), requires multiple processing stages, in the spatial and temporal domains, to extract high level features. Convolutional neural networks, as a type of deep learning method, have been used for EEG signal detection as the underlying structure of the EEG signal can be included in such system, facilitating the learning step. The EEG signal is typically decomposed into 2 main dimensions: space and time. However, the spatial dimension can be decomposed into 2 dimensions that better represent the relationships between the sensors that are involved in the classification. We propose to analyze the performance of 2D and 3D convolutional neural networks for the classification of ERPs with a dataset based on 64 EEG channels. We propose and compare 6 conv net architectures: 4 using 3D convolutions, that vary in relation to the number of layers and feature maps, and 2 using 2D convolutions. The results support the conclusion that 3D convolutions provide better performance than 2D convolutions for the binary classification of ERPs.

KekuleScope: prediction of cancer cell line sensitivity and compound potency using convolutional neural networks trained on compound images

The application of convolutional neural networks (ConvNets) to harness high-content screening images or 2D compound representations is gaining increasing attention in drug discovery. However, existing applications often require large data sets for training, or sophisticated pretraining schemes. Here, we show using 33 IC50 data sets from ChEMBL 23 that the in vitro activity of compounds on cancer cell lines and protein targets can be accurately predicted on a continuous scale from their Kekulé structure representations alone by extending existing architectures (AlexNet, DenseNet-201, ResNet152 and VGG-19), which were pretrained on unrelated image data sets. We show that the predictive power of the generated models, which just require standard 2D compound representations as input, is comparable to that of Random Forest (RF) models and fully-connected Deep Neural Networks trained on circular (Morgan) fingerprints. Notably, including additional fully-connected layers further increases the predictive power of the ConvNets by up to 10%. Analysis of the predictions generated by RF models and ConvNets shows that by simply averaging the output of the RF models and ConvNets we obtain significantly lower errors in prediction for multiple data sets, although the effect size is small, than those obtained with either model alone, indicating that the features extracted by the convolutional layers of the ConvNets provide complementary predictive signal to Morgan fingerprints. Lastly, we show that multi-task ConvNets trained on compound images permit to model COX isoform selectivity on a continuous scale with errors in prediction comparable to the uncertainty of the data. Overall, in this work we present a set of ConvNet architectures for the prediction of compound activity from their Kekulé structure representations with state-of-the-art performance, that require no generation of compound descriptors or use of sophisticated image processing techniques. The code needed to reproduce the results presented in this study and all the data sets are provided at https://github.com/isidroc/kekulescope.

Transfer learning with pre-trained deep convolutional neural networks for serous cell classification

Serous effusion is a condition of excess accumulation of fluids in serous cavities due to different underlying pathological conditions. The basis of cytopathological assessment of serous effusions is the identification of cells in the fluid based on their morphology and texture. This assessment is a physically and mentally laborious task, and it can also lead to variability among pathologists. In literature, only a small number of feature-based methods are conducted for automated serous cell classification. In this study, a transfer learning with pre-trained deep convolutional neural networks (ConvNets) is proposed to automatically identify 11 different categories of serous cells in effusion cytology. Unlike the methods which rely on the extraction of cellular features such as morphology and texture, this method is an appearance-based machine learning approach. We fine-tuned four pre-trained ConvNet architectures that are AlexNet, GoogleNet, ResNet and DenseNet on the serous cell dataset. To reduce the overfitting effect, we augmented the data by image rotation, translation, and mirroring. The proposed method was evaluated on both original and augmented sets of serous cells derived from a publicly available dataset. Among the four ConvNet models, ResNet and DenseNet obtained the highest accuracies of 93.44% and 92.90%. However, when two models were compared in terms of accuracy and model complexity, ResNet-TL was selected as the best network model. When compared to the results without data augmentation, data augmentation increased the accuracy rate approximately 10%. Results show that higher classification results were achieved than other traditional methods without requiring precise segmentation.

FeatherNet

Convolutional Neural Network (ConvNet or CNN) algorithms are characterized by a large number of model parameters and high computational complexity. These two requirements have made it challenging for implementations on resource-limited FPGAs. The challenges are magnified when considering designs for low-end FPGAs. While previous work has demonstrated successful ConvNet implementations with high-end FPGAs, this article presents a ConvNet accelerator design that enables the implementation of complex deep ConvNet architectures on resource-constrained FPGA platforms aimed at the IoT market. We call the design “FeatherNet” for its light resource utilization. The implementations are VHDL-based providing flexibility in design optimizations. As part of the design process, new methods are introduced to address several design challenges. The first method is a novel stride-aware graph-based method targeted at ConvNets that aims at achieving efficient signal processing with reduced resource utilization. The second method addresses the challenge of determining the minimal precision arithmetic needed while preserving high accuracy. For this challenge, we propose variable-width dynamic fixed-point representations combined with a layer-by-layer design-space pruning heuristic across the different layers of the deep ConvNet model. The third method aims at achieving a modular design that can support different types of ConvNet layers while ensuring low resource utilization. For this challenge, we propose the modules to be relatively small and composed of computational filters that can be interconnected to build an entire accelerator design. These model elements can be easily configured through HDL parameters (e.g., layer type, mask size, stride, etc.) to meet the needs of specific ConvNet implementations and thus they can be reused to implement a wide variety of ConvNet architectures. The fourth method addresses the challenge of design portability between two different FPGA vendor platforms, namely, Intel/Altera and Xilinx. For this challenge, we propose to instantiate the device-specific hardware blocks needed in each computational filter, rather than relying on the synthesis tools to infer these blocks, while keeping track of the similarities and differences between the two platforms. We believe that the solutions to these design challenges further advance knowledge as they can benefit designers and other researchers using similar devices or facing similar challenges. Our results demonstrated the success of addressing the design challenges and achieving low (30%) resource utilization for the low-end FPGA platforms: Zedboard and Cyclone V. The design overcame the limitation of designs targeted for high-end platforms and that cannot fit on low-end IoT platforms. Furthermore, our design showed superior performance results (measured in terms of [Frame/s/W] per Dollar) compared to high-end optimized designs.