Small Number Of Trainable Parameters Research Articles

Deep-Unfolded Sparse CDMA: Multiuser Detector and Sparse Signature Design

Sparse code division multiple access (SCDMA) is a promising non-orthogonal multiple access technique for future wireless communications. In SCDMA, transmitted symbols from multiple users are coded by their own sparse signature sequences, and a base station attempts to detect those symbols using the signature sequences. In this paper, we present a new deep-unfolded multiuser detector called a complex sparse trainable projected gradient (C-STPG) detector for SCDMA systems. Deep unfolding is a deep learning method that tunes trainable parameters in iterative algorithms using supervised data and deep learning techniques. The proposed detector provides a much superior detection performance over that of the LMMSE detector. Other advantages of the proposed detector include a low computational complexity in execution and a low training cost owing to the small number of trainable parameters. In addition, we propose a novel joint learning strategy called gradual sparsification for designing sparse signature sequences based on deep unfolding. This method is computationally efficient in optimizing a set of sparse signature sequences. Numerical results show that the gradual sparsification successfully yields sparse signature sequences with a smaller symbol error rate than those of randomly designed sparse signature sequences.

Feature Selection Module for CNN Based Object Detector

In the field of computer vision, the detection of multiple objects with different scales within a single image is challenging. To target this problem, feature pyramids are a basic component commonly found in multi-scale object detectors. In the construction of standard feature pyramids, different semantic features are simply connected to rebuild a new feature map, regardless of whether these features have a positive effect to the output or not. In order to avoid introducing too many redundant features within the feature fusion stage, a new feature fusion module called the Feature Selection Module (FSM) was proposed in this paper, which can automatically detect the most representative features for the rebuilding of feature maps. The channel attention mechanism in FSM is able to process and score each channel, filtering out irrelevant features while focusing on features with high contribution. Moreover, FSM can be easily embedded within feature pyramids. Simply adding a small number of trainable parameters to the network can significantly improve the ability of feature extraction. We validated our FSM with the VOC 2007 object detection dataset, based on Yolo series detectors. Findings from the present study demonstrates that for a small computational cost, our method is able to consistently improve the performance of Yolo detectors.

Hybrid Deep Learning-Gaussian Process Network for Pedestrian Lane Detection in Unstructured Scenes.

Pedestrian lane detection is an important task in many assistive and autonomous navigation systems. This article presents a new approach for pedestrian lane detection in unstructured environments, where the pedestrian lanes can have arbitrary surfaces with no painted markers. In this approach, a hybrid deep learning-Gaussian process (DL-GP) network is proposed to segment a scene image into lane and background regions. The network combines a compact convolutional encoder-decoder net and a powerful nonparametric hierarchical GP classifier. The resulting network with a smaller number of trainable parameters helps mitigate the overfitting problem while maintaining the modeling power. In addition to the segmentation output for each test image, the network also generates a map of uncertainty-a measure that is negatively correlated with the confidence level with which we can trust the segmentation. This measure is important for pedestrian lane-detection applications, since its prediction affects the safety of its users. We also introduce a new data set of 5000 images for training and evaluating the pedestrian lane-detection algorithms. This data set is expected to facilitate research in pedestrian lane detection, especially the application of DL in this area. Evaluated on this data set, the proposed network shows significant performance improvements compared with several existing methods.

Automated inter-patient seizure detection using multichannel Convolutional and Recurrent Neural Networks

We present an end-to-end deep learning model that can automatically detect epileptic seizures in multichannel electroencephalography (EEG) recordings. Our model combines a Convolutional Neural Network (CNN) and a Bidirectional Long Short-Term Memory (BLSTM) network to efficiently mine information from the EEG data using a small number of trainable parameters. Specifically, the CNN learns a latent encoding for each one second window of raw multichannel EEG data. In conjunction, the BLSTM learns the temporal evolution of seizure presentations given the CNN encodings. The combination of these architectures allows our model to capture both the short time scale EEG features indicative of seizure activity as well as the long term correlations in seizure presentations. Unlike most prior work in seizure detection, we mimic an in-patient monitoring setting through a leave-one-patient-out cross validation procedure, attaining an average seizure detection sensitivity of 0.91 across all patients. This strategy verifies that our model can generalize to new patients. We demonstrate that our CNN–BLSTM outperforms both conventional feature extraction methods and state-of-the-art deep learning approaches that rely on larger and more complex network architectures.

Convolutional neural network-based models for diagnosis of breast cancer.

Breast cancer is the most prevailing cancer in the world and each year affecting millions of women. It is also the cause of largest number of deaths in women dying in cancers. During the last few years, researchers are proposing different convolutional neural network models in order to facilitate diagnostic process of breast cancer. Convolutional neural networks are showing promising results to classify cancers using image datasets. There is still a lack of standard models which can claim the best model because of unavailability of large datasets that can be used for models’ training and validation. Hence, researchers are now focusing on leveraging the transfer learning approach using pre-trained models as feature extractors that are trained over millions of different images. With this motivation, this paper considers eight different fine-tuned pre-trained models to observe how these models classify breast cancers applying on ultrasound images. We also propose a shallow custom convolutional neural network that outperforms the pre-trained models with respect to different performance metrics. The proposed model shows 100% accuracy and achieves 1.0 AUC score, whereas the best pre-trained model shows 92% accuracy and 0.972 AUC score. In order to avoid biasness, the model is trained using the fivefold cross validation technique. Moreover, the model is faster in training than the pre-trained models and requires a small number of trainable parameters. The Grad-CAM heat map visualization technique also shows how perfectly the proposed model extracts important features to classify breast cancers.

A Reservoir Computing Approach to Word Sense Disambiguation

Reservoir computing (RC) has emerged as an alternative approach for the development of fast trainable recurrent neural networks (RNNs). It is considered to be biologically plausible due to the similarity between randomly designed artificial reservoir structures and cortical structures in the brain. The paper continues our previous research on the application of a member of the family of RC approaches—the echo state network (ESN)—to the natural language processing (NLP) task of Word Sense Disambiguation (WSD). A novel deep bi-directional ESN (DBiESN) structure is proposed, as well as a novel approach for exploiting reservoirs’ steady states. The models also make use of ESN-enhanced word embeddings. The paper demonstrates that our DBiESN approach offers a good alternative to previously tested BiESN models in the context of the word sense disambiguation task having smaller number of trainable parameters. Although our DBiESN-based model achieves similar accuracy to other popular RNN architectures, we could not outperform the state of the art. However, due to the smaller number of trainable parameters in the reservoir models, in contrast to fully trainable RNNs, it is to be expected that they would have better generalization properties as well as higher potential to increase their accuracy, which should justify further exploration of such architectures.

Cardiac MRI Segmentation With a Dilated CNN Incorporating Domain-Specific Constraints

Semantic segmentation of cardiac MR images is a challenging task due to its importance in medical assessment of heart diseases. Having a detailed localization of specific regions of interest such as Right and Left Ventricular Cavities and Myocardium, doctors can infer important information about the presence of cardiovascular diseases, which are today a major cause of death globally. This paper addresses the problem of semantic segmentation in cardiac MR images using a dilated Convolutional Neural Network. Opting for dilated convolutions allowed us to work in full resolution throughout the network's layers, preserving localization accuracy, while maintaining a relatively small number of trainable parameters. To assist the network's training process we designed a custom loss function. Furthermore, we developed new augmentation techniques and also adapted existing ones, to cope for the lack of sufficient training images. Consequently, the training set increases not only by amount, but by substance as well, and the network trains quickly and efficiently without overfitting. Our pre- and post-processing steps are also crucial to the whole process. We apply our methodology for the Right and Left Ventricles (RV, LV) and also the Myocardium (MYO) according to the Automated Cardiac Diagnosis Challenge (ACDC) with promising results. Submitting our algorithm's predictions to the Post-2017-MICCAI-challenge testing phase, we achieved similar scores (average Dice coefficient 0.916) on the test data set compared to the state of the art featured in the ACDC leaderboard, but with significantly fewer parameters than the leading method. Our approach outperforms other methods featuring dilated convolutions in this challenge up until now.

Fully Convolutional Neural Network for Detection and Counting of Diatoms on Coatings after Short-Term Field Exposure

While the use of deep learning is a valuable technology for automatic detection systems for medical data and images, the biofouling community is still lacking an analytical tool for the detection and counting of diatoms on samples after short-term field exposure. In this work, a fully convolutional neural network was implemented as a fast and simple approach to detect diatoms on two-channel (fluorescence and phase-contrast) microscopy images by predicting bounding boxes. The developed approach performs well with only a small number of trainable parameters and a F1 score of 0.82. Counting diatoms was evaluated on a data set of 600 microscopy images of three different surface chemistries (hydrophilic and hydrophobic) and is very similar to counting by humans while demanding only a fraction of the analysis time.

Deep Learning COVID-19 Features on CXR Using Limited Training Data Sets.

Under the global pandemic of COVID-19, the use of artificial intelligence to analyze chest X-ray (CXR) image for COVID-19 diagnosis and patient triage is becoming important. Unfortunately, due to the emergent nature of the COVID-19 pandemic, a systematic collection of CXR data set for deep neural network training is difficult. To address this problem, here we propose a patch-based convolutional neural network approach with a relatively small number of trainable parameters for COVID-19 diagnosis. The proposed method is inspired by our statistical analysis of the potential imaging biomarkers of the CXR radiographs. Experimental results show that our method achieves state-of-the-art performance and provides clinically interpretable saliency maps, which are useful for COVID-19 diagnosis and patient triage.

Two-Step CNN Framework for Text Line Recognition in Camera-Captured Images

In this paper, we introduce an “on the device” text line recognition framework that is designed for mobile or embedded systems. We consider per-character segmentation as a language-independent problem and individual character recognition as a language-dependent one. Thus, the proposed solution is based on two separate artificial neural networks (ANN) and dynamic programming instead of employing image processing methods for the segmentation step or end-to-end ANN. To satisfy the tight constraints on memory size imposed by embedded systems and to avoid overfitting, we employ ANNs with a small number of trainable parameters. The primary purpose of our framework is the recognition of low-quality images of identity documents with complex backgrounds and a variety of languages and fonts. We demonstrate that our solution shows high recognition accuracy on natural datasets even being trained on purely synthetic data. We use MIDV-500 and Census 1961 Project datasets for text line recognition. The proposed method considerably surpasses the algorithmic method implemented in Tesseract 3.05, the LSTM method (Tesseract 4.00), and unpublished method used in the ABBYY FineReader 15 system. Also, our framework is faster than other compared solutions. We show the language-independence of our segmenter with the experiment with Cyrillic, Armenian, and Chinese text lines.

Generalized neuron: Feedforward and recurrent architectures

Feedforward neural networks such as multilayer perceptrons (MLP) and recurrent neural networks are widely used for pattern classification, nonlinear function approximation, density estimation and time series prediction. A large number of neurons are usually required to perform these tasks accurately, which makes the MLPs less attractive for computational implementations on resource constrained hardware platforms. This paper highlights the benefits of feedforward and recurrent forms of a compact neural architecture called generalized neuron (GN). This paper demonstrates that GN and recurrent GN (RGN) can perform good classification, nonlinear function approximation, density estimation and chaotic time series prediction. Due to two aggregation functions and two activation functions, GN exhibits resilience to the nonlinearities of complex problems. Particle swarm optimization (PSO) is proposed as the training algorithm for GN and RGN. Due to a small number of trainable parameters, GN and RGN require less memory and computational resources. Thus, these structures are attractive choices for fast implementations on resource constrained hardware platforms.