Discriminating single-molecule binding events from diffraction-limited fluorescence.

Classification of videos based on its content is one of the challenging and significant research problems. In this paper, a simple and efficient model is proposed for classification of sports videos using deep learned convolution neural networks. In the proposed research, the gray scale variants of image frames are employed for classification process through convolution technique at varied levels of abstraction by adapting it through a sequence of hidden layers. The image frames considered for classification are obtained after the duplicate frame elimination and each frame is further rescaled to dimension 120x240. The sports videos categories used for experimentation include badminton, football, cricket and tennis which are downloaded from various sources of google and YouTube. The classification in the proposed method is performed with Deep Convolution Neural Networks (DCNN) with around 20 filters each of size 5x5 with around stride length of2 and its outcomes are compared with Local Binary Patterns (LBP), Bag of Words Features (BWF) technique. The SURF features are extracted from the BWF technique and further 80% of strongest feature points are employed for clustering the image frames using K-Means clustering technique with an average accuracy achieved of about 87% in classification. The LBF technique had produced an average accuracy of 73% in differentiating one image frame to other whereas the DCNN had shown a promising outcome with accuracy of about 91% in case of 40% training and 60% test datasets, 99% accuracy in case of 60% training an 40% test datasets. The results depict that the proposed method outperforms the image processing-based techniques LBP and BWF.

Advanced video classification systems decode video frames to derive the necessary texture and motion representations for ingestion and analysis by spatio-temporal deep convolutional neural networks (CNNs). However, when considering visual Internet-of-Things applications, surveillance systems and semantic crawlers of large video repositories, the video capture and the CNN-based semantic analysis parts do not tend to be co-located. This necessitates the transport of compressed video over networks and incurs significant overhead in bandwidth and energy consumption, thereby significantly undermining the deployment potential of such systems. In this paper, we investigate the trade-off between the encoding bitrate and the achievable accuracy of CNN-based video classification models that directly ingest AVC/H.264 and HEVC encoded videos. Instead of retaining entire compressed video bitstreams and applying complex optical flow calculations prior to CNN processing, we only retain motion vector and select texture information at significantly-reduced bitrates and apply no additional processing prior to CNN ingestion. Based on three CNN architectures and two action recognition datasets, we achieve 11%-94% saving in bitrate with marginal effect on classification accuracy. A model-based selection between multiple CNNs increases these savings further, to the point where, if up to 7% loss of accuracy can be tolerated, video classification can take place with as little as 3 kbps for the transport of the required compressed video information to the system implementing the CNN models.

Advanced video classification systems decode video frames to derive texture and motion representations for ingestion and analysis by spatio–temporal deep convolutional neural networks (CNNs). However, when considering visual Internet-of-Things applications, surveillance systems, and semantic crawlers of large video repositories, the video capture and the CNN-based semantic analysis parts do not tend to be co-located. This necessitates the transport of compressed video over networks and incurs significant overhead in bandwidth and energy consumption, thereby significantly undermining the deployment potential of such systems. In this paper, we investigate the trade-off between the encoding bitrate and the achievable accuracy of CNN-based video classification models that directly ingest AVC/H.264 and HEVC encoded videos. Instead of retaining entire compressed video bitstreams and applying complex optical flow calculations prior to CNN processing, we only retain motion vector and select texture information at significantly reduced bitrates and apply no additional processing prior to CNN ingestion. Based on three CNN architectures and two action recognition datasets, we achieve 11%–94% savings in bitrate with marginal effect on classification accuracy. A model-based selection between multiple CNNs increases these savings further to the point where, if up to 7% loss of accuracy can be tolerated, video classification can take place with as little as 3 kb/s for the transport of the required compressed video information to the system implementing the CNN models.

Convolutional Neural Networks (CNNs) have been demonstrated to be able to produce the best performance in image classification problems. Recurrent Neural Networks (RNNs) have been utilized to make use of temporal information for time series classification. The main goal of this paper is to examine how temporal information be- tween frame sequences can be used to improve the performance of video classification using RNNs. Using transfer learning, this paper presents a comparative study of seven video classification network architectures, which utilize either global or local features extracted by VGG-16, a very deep CNN pre-trained for image classification. Hold-out validation has been used to optimize the ratio of dropout and the number of units in the fully-connected layers in the proposed architectures. Each network architecture for video classification has been executed a number of times using different data splits, with the best architecture identified using the independent T-test. Experimental results show that the network architecture using local features extracted by the pre-trained CNN and ConvLSTM for making use of temporal information can achieve the best accuracy in video classification.

Convolutional Neural Networks (CNNs) are the state-of-the-art in computer vision for different purposes such as image and video classification, recommender systems and natural language processing. The connectivity pattern between CNNs neurons is inspired by the structure of the animal visual cortex. In order to allow the processing, they are realized with multiple parallel 2-dimensional FIR filters that convolve the input signal with the learned feature maps. For this reason, a CNN implementation requires highly parallel computations that cannot be achieved using traditional general-purpose processors, which is why they benefit from a very significant speed-up when mapped and run on Field Programmable Gate Arrays (FPGAs). This is because FPGAs offer the capability to design full customizable hardware architectures, providing high flexibility and the availability of hundreds to thousands of on-chip Digital Signal Processing (DSP) blocks. This paper presents an FPGA implementation of a hand-written number recognition system based on CNN. The system has been characterized in terms of classification accuracy, area, speed, and power consumption. The neural network was implemented on a Xilinx XC7A100T FPGA, and it uses 29.69% of Slice LUTs, 4.42% of slice registers and 52.50% block RAMs. We designed the system using a 9-bit representation that allows for avoiding the use of DSP. For this reason, multipliers are implemented using LUTs. The proposed architecture can be easily scaled on different FPGA devices thank its regularity. CNN can reach a classification accuracy of 90%.

Monitoring tool usage in surgery videos using boosted convolutional and recurrent neural networks.

Logistic regression (LR) is a popular method that is used for estimating causal effects in observational studies using propensity scores. We examine the use of deep learning models such as the deep neural network (DNN), PropensityNet (PN), convolutional neural network (CNN), and convolutional neural network-long short-term memory network (CNN-LSTM) to estimate propensity scores and evaluate causal inference. We conducted studies using simulated data with different sample sizes (N = 500, N = 1000, N = 2000), 15 covariates, a continuous outcome and a binary exposure. These data were used in seven scenarios that were different in the degree of nonlinearity and nonadditivity associations between the exposure and covariates. Estimation of propensity scores was considered a classification task and performance metrics that included classification accuracy, receiver operating characteristic curve area under the curve (AUCROC), covariate balance, standard error, absolute bias, and the 95% confidence interval coverage were evaluated for each model. Our simulation results show that deep learning models (CNN, DNN, and CNN-LSTM) outperformed LR in the estimation of the propensity score. CNN and CNN-LSTM achieved good results for covariate balance, classification accuracy, AUCROC, and Cohen’s Kappa. Although LR provided substantially better bias reduction, it produced subpar performance based on classification accuracy, AUCROC, Cohen’s Kappa, and 95% confidence interval coverage compared to the deep learning models. The results suggest that deep learning methods, especially CNN, may be useful for estimating propensity scores that are used to estimate causal effects.

BackgroundEpilepsy is one of the diseases of the nervous system, which has a large population in the world. Traditional diagnosis methods mostly depended on the professional neurologists’ reading of the electroencephalogram (EEG), which was time-consuming, inefficient, and subjective. In recent years, automatic epilepsy diagnosis of EEG by deep learning had attracted more and more attention. But the potential of deep neural networks in seizure detection had not been fully developed.MethodsIn this article, we used a one-dimensional convolutional neural network (1-D CNN) to replace the residual network architecture’s traditional convolutional neural network (CNN). Moreover, we combined the Independent recurrent neural network (indRNN) and CNN to form a new residual network architecture-independent convolutional recurrent neural network (RCNN). Our model can achieve an automatic diagnosis of epilepsy EEG. Firstly, the important features of EEG were learned by using the residual network architecture of 1-D CNN. Then the relationship between the sequences were learned by using the recurrent neural network. Finally, the model outputted the classification results.ResultsOn the small sample data sets of Bonn University, our method was superior to the baseline methods and achieved 100% classification accuracy, 100% classification specificity. For the noisy real-world data, our method also exhibited powerful performance.ConclusionThe model we proposed can quickly and accurately identify the different periods of EEG in an ideal condition and the real-world condition. The model can provide automatic detection capabilities for clinical epilepsy EEG detection. We hoped to provide a positive significance for the prediction of epileptic seizures EEG.

Convolutional Neural Networks (CNN) are useful methods for identification of previously unknown embedded patterns in images. Several object and facial recognition along with image segmentation tasks have benefited from the non-linear abstraction of hybrid features using CNN. This work presents a novel CNN model parametrization work-flow developed on the cloud-computing platform of Microsoft Azure Machine Learning Studio (MAMLS) that is capable of learning from the feature maps and classifying multi-modal images with different variabilities using one common flow. This two-step work-flow trains CNN models using 70/30 data split. First, the CNN layers are fixed and the optimal kernel and normalization parameters are identified that maximize classification accuracy on the test data. Next, using the optimal kernel and normalization parameters, the best CNN architecture that maximizes classification accuracy is detected. Finally, the activated feature maps (AFMs) from the optimally parameterized CNN model are analyzed to learn new features that can enhance image-based classification accuracies. The proposed flow achieves classification accuracies in the range of 92.5–99.2% that can be further enhanced by doubling the samples based on the features learned from the AFMs. The proposed non-deep CNN models in the MAMLS platform are capable of processing image data sets with 400–4 million samples using a common flow without exponential increase in the computation time. Thus, optimally parametrized non-deep CNN models are capable of identifying novel features that may enhance image-based classification accuracies.

Clinically Relevant Vulnerabilities of Deep Machine Learning Systems for Skin Cancer Diagnosis

Convolutional Neural Networks (CNNs) present drawbacks for modeling geometric transformations, caused by the convolution operation’s locality. Deformable convolution (DCON) is a mechanism that solves these drawbacks and improves the robustness. In this study, we clarify the optimal way to replace the standard convolution with its deformable counterpart in a CNN model. To this end, we conducted several experiments using DCONs applied in the layers that conform a small four-layer CNN model and on the four-layers of several ResNets with depths 18, 34, 50, and 101. The models were tested in binary balanced classes with 2D and 3D data. If DCON is used on the first layers of the proposal of model, the computational resources will tend to increase and produce bigger misclassification than the standard CNN. However, if the DCON is used at the end layers, the quantity of Flops will decrease, and the classification accuracy will improve by up to 20% about the base model. Moreover, it gains robustness because it can adapt to the object of interest. Also, the best kernel size of the DCON is three. With these results, we propose a guideline and contribute to understanding the impact of DCON on the robustness of CNNs.

Deep learning has been successfully applied in classification of white blood cells (WBCs), however, accuracy and processing time are found to be less than optimal hindering it from getting its full potential. This is due to imbalanced dataset, intra-class compactness, inter-class separability and overfitting problems. The main research idea is to enhance the classification and prediction accuracy of blood images while lowering processing time through the use of deep convolutional neural network (DCNN) architecture by using the modified loss function. The proposed system consists of a deep neural convolution network (DCNN) that will improve the classification accuracy by using modified loss function along with regularization. Firstly, images are pre-processed and fed through DCNN that contains different layers with different activation function for the feature extraction and classification. Along with modified loss function with regularization, weight function aids in the classification of WBCs by considering weights of samples belonging to each class for compensating the error arising due to imbalanced dataset. The processing time will be counted by each image to check the time enhancement. The classification accuracy and processing time are achieved using the dataset-master. Our proposed solution obtains better classification performance in the given dataset comparing with other previous methods. The proposed system enhanced the classification accuracy of 98.92% from 96.1% and a decrease in processing time from 0.354 to 0.216 s. Less time will be required by our proposed solution for achieving the model convergence with 9 epochs against the current convergence time of 13.5 epochs on average, epoch is the formation white blood cells (WBCs) and the development of granular cells. The proposed solution modified loss function to solve the adverse effect caused due to imbalance dataset by considering weight and use regularization technique for overfitting problem.

In this paper, we propose a novel fusion strategy of prediction vectors obtained from two different deep neural networks designed for the task of event recognition in unconstrained videos. Videos are suitably represented by a set of key-frames. Two types of features, namely spatial and temporal, are computed from a hybrid pre-trained CNN-RNN (Convolutional Neural Networks - Recurrent Neural Networks) framework for each video. These features are able to capture both the transient and long-term dependencies for understanding of the events. Frame-level and video-level prediction vectors are generated from two separate CNN-RNN (ResNet50-LSTM) frameworks exploiting spatial and temporal features respectively. The fusion is performed on these prediction vectors at different levels. The entire fusion framework relies on a concept of consolidation of probability distributions. This consolidation is implemented using conflation and a biasing technique. A multi-level fusion is achieved and at each level, a significant amount of classification accuracy is observed as improving. The experiment is performed on four benchmark datasets, namely, Columbia Consumer Video (CCV), Kodak’s Consumer Video, UCF-101 and Human Motion Database (HMDB). The increment in mAP values achieved by the proposed fusion strategy is much higher than the conventional fusion strategies in use. Also, the classification accuracies of all the four datasets are comparable to other state-of-the-art methods for event classification in unconstrained videos

Objects and processes classification is a common experimental problem. Its solution, first of all, is needed in automatic diagnosis systems, for example, to determine the equipment operation state through the diagnostic signal or to identify abnormalities with medical images. With the development of convolutional neural networks, new prospects for solving such problems have risen up. However, the classification accuracy that can be achieved on these networks is not sufficient enough for all diagnosis issues. It is subject to, for example, timely diagnosis of the onset of transient phenomena. At the same time, another type of neural network, Kohonen self-organizing maps, has a conceptual property for training on unclassified set of classes, that is, giving the opportunity to solve such an issue. Therefore, the accuracy enhancement of the classification on the basis of Kohonen networks implementation in the architecture of convolutional networks is a relevant objective and has practical significance. The article analyzes the means of improving the accuracy of convolutional neural networks and arising problems solution. The ways of increasing the proportion of correct clustering on Kohonen networks due to the ‘growth’ of its grid shape in determining new classes in the learning process are also given. It is this property that makes it possible to recognize transient phenomena. It is determined that the existing solutions of the combination of Kohonen networks and convolutional networks are aimed at improving the efficiency of only self-organizing maps, with the purpose to improve the accuracy of classification by convolutional networks, it became necessary to develop a new architecture. The paper provides a description of this issue. Since the initial information of the Kohonen networks is the weighting matrix values of the grid shape neurons, it was necessary to associate it with the representation of the images in order to process the diagnostic images. The paper proposes the concept of a built-in associative array block based on Kohonen networks. According to the proposed method, a software implementation of a hybrid neural network is developed. The formulation and results of computational experiments are presented. The efficiency of the proposed method is experimentally proved. Keywords: convolutional neural networks, CNN, Kohonen self-organizing maps, SOM, signal classification, image classification.

With the rapid development of the Internet of things and e-commerce, feature-based image retrieval and classification have become a serious challenge for shoppers searching websites for relevant product information. The last decade has witnessed great interest in research on content-based feature extraction techniques. Moreover, semantic attributes cannot fully express the rich image information. This paper designs and trains a deep convolutional neural network that the convolution kernel size and the order of network connection are based on the high efficiency of the filter capacity and coverage. To solve the problem of long training time and high resource share of deep convolutional neural network, this paper designed a shallow convolutional neural network to achieve the similar classification accuracy. The deep and shallow convolutional neural networks have data pre-processing, feature extraction and softmax classification. To evaluate the classification performance of the network, experiments were conducted using a public database Caltech256 and a homemade product image database containing 15 species of garment and 5 species of shoes on a total of 20,000 color images from shopping websites. Compared with the classification accuracy of combining content-based feature extraction techniques with traditional support vector machine techniques from 76.3% to 86.2%, the deep convolutional neural network obtains an impressive state-of-the-art classification accuracy of 92.1%, and the shallow convolutional neural network reached a classification accuracy of 90.6%. Moreover, the proposed convolutional neural networks can be integrated and implemented in other colour image database.