Deep Neural Network Features Research Articles

Cross-modal prototype learning for zero-shot handwritten character recognition

Traditional methods of handwritten character recognition rely on extensive labeled data. However, humans can generalize to unseen handwritten characters by watching a few printed examples in textbooks. To simulate this ability, we propose a cross-modal prototype learning method (CMPL) to realize zero-shot recognition. For each character class, a prototype is generated by mapping the printed character into a deep neural network feature space. For unseen character class, its prototype can be directly produced from a printed character sample, therefore, not requiring any handwritten samples to realize class-incremental learning. Specifically, CMPL considers different modalities simultaneously - online handwritten trajectories, offline handwritten images, and auxiliary printed character images. The joint learning of the above modalities is achieved through sharing printed prototypes between online and offline data. In zero-shot inference, we feed CMPL the printed samples to obtain corresponding class prototypes, and then the unseen handwritten character can be recognized by the nearest prototype. Our experimental results demonstrate that CMPL outperforms the state-of-the-art methods in both online and offline zero-shot handwritten Chinese character recognition. Moreover, we also show the cross-domain generalization of CMPL from two perspectives: cross-language and modern-to-ancient handwritten character recognition, focusing on the transferability between different languages and different styles (i.e., modern and historical handwritings).

M3BTCNet: multi model brain tumor classification using metaheuristic deep neural network features optimization

M3BTCNet: multi model brain tumor classification using metaheuristic deep neural network features optimization

Deep neural network modeling identifies biomarkers of response to immune-checkpoint therapy

SummaryImmunotherapy has shown significant promise as a treatment for cancer, such as lung cancer and melanoma. However, only 10%–30% of the patients respond to treatment with immune checkpoint blockers (ICBs), underscoring the need for biomarkers to predict response to immunotherapy. Here, we developed DeepGeneX, a computational framework that uses advanced deep neural network modeling and feature elimination to reduce single-cell RNA-seq data on ∼26,000 genes to six of the most important genes (CCR7, SELL, GZMB, WARS, GZMH, and LGALS1), that accurately predict response to immunotherapy. We also discovered that the high LGALS1 and WARS-expressing macrophage population represent a biomarker for ICB therapy nonresponders, suggesting that these macrophages may be a target for improving ICB response. Taken together, DeepGeneX enables biomarker discovery and provides an understanding of the molecular basis for the model’s predictions.

Double compression detection in HEVC-coded video with the same coding parameters using picture partitioning information

Detection of double compression, particularly in the high-efficiency video coding (HEVC) compressed domain, is one of the most operative and efficacious ways of authenticating the validity of videos in the field of forensic analysis. The strength of identifying abnormalities in the videos confides in diverse coding parameters (such as quantization parameters, size and structure of the group of pictures, and modes of compression). Many methods have been introduced to dig up HEVC double compression with different coding parameters. However, the revelation of the HEVC double compression under the same coding environments still remains a competitive task, as recompressions leave small footprints. In this paper, we introduce a novel method based on frame partitioning information to distinguish between single and double compressions with the same coding parameters. We propose extracting statistical and deep convolution neural network (DCNN) features from partition pictures and prediction modes, including coding unit, prediction unit, transform unit, and most probable modes information. Finally, machine learning technology is integrated to categorize videos into two classes, single and double compressions, by combining the statistical and DCNN features. We obtain the best experimental results by assembling the statistical and DCNN features for wide video graphics array (WVGA) and high-definition (HD) sequences with average accuracies of 99.66% and 99.60% in all-intra and 99.46% and 99.33% in low-delay P modes respectively. Experimental results of the proposed system show the effectiveness and efficiency over the state-of-the-art techniques in video forensic.

Correction to: A Fused Heterogeneous Deep Neural Network and Robust Feature Selection Framework for Human Actions Recognition

A Correction to this paper has been published: 10.1007/s13369-021-05881-4

An Information Theoretic Interpretation to Deep Neural Networks.

With the unprecedented performance achieved by deep learning, it is commonly believed that deep neural networks (DNNs) attempt to extract informative features for learning tasks. To formalize this intuition, we apply the local information geometric analysis and establish an information-theoretic framework for feature selection, which demonstrates the information-theoretic optimality of DNN features. Moreover, we conduct a quantitative analysis to characterize the impact of network structure on the feature extraction process of DNNs. Our investigation naturally leads to a performance metric for evaluating the effectiveness of extracted features, called the H-score, which illustrates the connection between the practical training process of DNNs and the information-theoretic framework. Finally, we validate our theoretical results by experimental designs on synthesized data and the ImageNet dataset.

An Efficient Hyperparameter Control Method for a Network Intrusion Detection System Based on Proximal Policy Optimization

The complexity of network intrusion detection systems (IDSs) is increasing due to the continuous increases in network traffic, various attacks and the ever-changing network environment. In addition, network traffic is asymmetric with few attack data, but the attack data are so complex that it is difficult to detect one. Many studies on improving intrusion detection performance using feature engineering have been conducted. These studies work well in the dataset environment; however, it is challenging to cope with a changing network environment. This paper proposes an intrusion detection hyperparameter control system (IDHCS) that controls and trains a deep neural network (DNN) feature extractor and k-means clustering module as a reinforcement learning model based on proximal policy optimization (PPO). An IDHCS controls the DNN feature extractor to extract the most valuable features in the network environment, and identifies intrusion through k-means clustering. Through iterative learning using the PPO-based reinforcement learning model, the system is optimized to improve performance automatically according to the network environment, where the IDHCS is used. Experiments were conducted to evaluate the system performance using the CICIDS2017 and UNSW-NB15 datasets. In CICIDS2017, an F1-score of 0.96552 was achieved and UNSW-NB15 achieved an F1-score of 0.94268. An experiment was conducted by merging the two datasets to build a more extensive and complex test environment. By merging datasets, the attack types in the experiment became more diverse and their patterns became more complex. An F1-score of 0.93567 was achieved in the merged dataset, indicating 97% to 99% performance compared with CICIDS2017 and UNSW-NB15. The results reveal that the proposed IDHCS improved the performance of the IDS by automating learning new types of attacks by managing intrusion detection features regardless of the network environment changes through continuous learning.

Attention modulates neural representation to render reconstructions according to subjective appearance

Stimulus images can be reconstructed from visual cortical activity. However, our perception of stimuli is shaped by both stimulus-induced and top-down processes, and it is unclear whether and how reconstructions reflect top-down aspects of perception. Here, we investigate the effect of attention on reconstructions using fMRI activity measured while subjects attend to one of two superimposed images. A state-of-the-art method is used for image reconstruction, in which brain activity is translated (decoded) to deep neural network (DNN) features of hierarchical layers then to an image. Reconstructions resemble the attended rather than unattended images. They can be modeled by superimposed images with biased contrasts, comparable to the appearance during attention. Attentional modulations are found in a broad range of hierarchical visual representations and mirror the brain–DNN correspondence. Our results demonstrate that top-down attention counters stimulus-induced responses, modulating neural representations to render reconstructions in accordance with subjective appearance.

Interweave features of Deep Convolutional Neural Networks for semantic segmentation

Although semantic segmentation based on Deep Convolutional Neural Networks (DCNN) has made great signs of progress, the issue that features generated by deep models are of low-resolution and will negatively affect the final semantic segmentation performances is not fully addressed yet. In this paper, we propose to adaptively combine high-level and low-level features of DCNN to improve the quality of the features used for semantic segmentation. To this end, we design a feature interweaving neural network module to fuse features from different layers of DCNN to effectively take advantage of their complementary properties. And, in order to enhance complementarity and diminish contradiction of the features for better feature fusion, we propose a feature modulation neural network module to modulate the features before interweaving. Furthermore, global information of images is summarized and used to augment the features for providing guidance for feature interweaving. The proposed method is extensively evaluated and compared to state-of-the-art methods based on two benchmark semantic segmentation datasets Cityscapes and PASCAL VOC 2012 in the experiments. Obtained results demonstrate the effectiveness of the proposed method.

Comparative Features of MIA GMDH and Deep Feed-Forward Neural Networks

Comparative Features of MIA GMDH and Deep Feed-Forward Neural Networks

Big data-enabled intelligent synchronisation for the complex production logistics system under the opti-state control strategy

Diversified customer needs make the production system more susceptible to high-frequency fluctuations of uncertain factors (UFs), which puts forward higher requirements for the real-time and systematic decision-making of the system. The opti-state control strategy enables the system to maintain the adaptive optimal state after being disturbed. The core intelligent synchronisation of the opti-state control strategy needs to perceive the state of the affected system and its degree of change. Aiming at the challenge of difficulty in evaluating the uncertain factors impact degree (UFID) of the complex production logistics system, this work proposes a big data-enabled intelligent synchronisation under the opti-state control strategy. Based on the simulation data of system operation, big data is used to mine the relationship between the UFID and the system states, then use wrapper GA-DNN (Deep Neural Network) feature selection and classification method evaluates the UFID, which will be applied to the synchronisation decision. The results show that the method can accurately evaluate the UFID and avoid the waste of resources and the increase in operating costs caused by excessive evaluation of the UFID, thereby also improves the effectiveness and efficiency of the opti-state control strategy.

Speech enhancement from fused features based on deep neural network and gated recurrent unit network

Speech is easily interfered by external environment in reality, which results in the loss of important features. Deep learning has become a popular speech enhancement method because of its superior potential in solving nonlinear mapping problems for complex features. However, the deficiency of traditional deep learning methods is the weak learning capability of important information from previous time steps and long-term event dependencies between the time-series data. To overcome this problem, we propose a novel speech enhancement method based on the fused features of deep neural networks (DNNs) and gated recurrent unit (GRU). The proposed method uses GRU to reduce the number of parameters of DNNs and acquire the context information of the speech, which improves the enhanced speech quality and intelligibility. Firstly, DNN with multiple hidden layers is used to learn the mapping relationship between the logarithmic power spectrum (LPS) features of noisy speech and clean speech. Secondly, the LPS feature of the deep neural network is fused with the noisy speech as the input of GRU network to compensate the missing context information. Finally, GRU network is performed to learn the mapping relationship between LPS features and log power spectrum features of clean speech spectrum. The proposed model is experimentally compared with traditional speech enhancement models, including DNN, CNN, LSTM and GRU. Experimental results demonstrate that the PESQ, SSNR and STOI of the proposed algorithm are improved by 30.72%, 39.84% and 5.53%, respectively, compared with the noise signal under the condition of matched noise. Under the condition of unmatched noise, the PESQ and STOI of the algorithm are improved by 23.8% and 37.36%, respectively. The advantage of the proposed method is that it uses the key information of features to suppress noise in both matched and unmatched noise cases and the proposed method outperforms other common methods in speech enhancement.

Data-Driven Deep Learning-Based Attention Mechanism for Remaining Useful Life Prediction: Case Study Application to Turbofan Engine Analysis

Accurately predicting the remaining useful life (RUL) of the turbofan engine is of great significance for improving the reliability and safety of the engine system. Due to the high dimension and complex features of sensor data in RUL prediction, this paper proposes four data-driven prognostic models based on deep neural networks (DNNs) with an attention mechanism. To improve DNN feature extraction, data are prepared using a sliding time window technique. The raw data collected after normalizing is simply fed into the suggested network, requiring no prior knowledge of prognostics or signal processing and simplifying the proposed method’s applicability. In order to verify the RUL prediction ability of the proposed DNN techniques, the C-MAPSS benchmark dataset of the turbofan engine system is validated. The experimental results showed that the developed long short-term memory (LSTM) model with attention mechanism achieved accurate RUL prediction in both scenarios with a high degree of robustness and generalization ability. Furthermore, the proposed model performance outperforms several state-of-the-art prognosis methods, where the LSTM-based model with attention mechanism achieved an RMSE of 12.87 and 11.23 for FD002 and FD003 subset of data, respectively.

Contrast sensitivity is formed by visual experience and task demands

The contrast sensitivity function (CSF) is a fundamental signature of the visual system and has been measured extensively in numerous species. It is defined by measurements of the visibility threshold for sinusoidal gratings at all spatial frequencies. Here, we measured the CSF in artificial visual systems, namely deep neural networks (DNNs), using the same contrast discrimination paradigm as used in human psychophysical experiments. During training, networks are exclusively exposed to natural images, and the task is to identify which one of two input images has higher contrast. The contrast discriminator networks learn a linear classifier over frozen features of pretrained DNNs. At the testing stage, we measured the network's CSF by presenting sinusoidal gratings of different orientations and spatial frequencies. Our results demonstrate that the pretrained DNNs show the band-limited, inverted U-shaped CSF, that is characteristic of the human CSF. The exact shape of the DNNs' CSF appears to be task-dependent. The human CSF is better captured by scene segmentation DNNs than image classification ones (tested under identical settings such as the architecture and computational complexity). When the network was trained to discriminate the contrast in natural images from scratch (i.e., with no previous training task), a relatively flat CSF emerged, dissimilar to the human CSF. The environment of pretrained DNNs proved influential as well. Those with a diet of natural images, similar to human experience (e.g. objects or faces), obtain more human-like CSF. Pretrained DNNs with aerial pictures, such as a bird flying in the sky would see, obtain a shifted CSF towards higher spatial frequencies, similar to that of an eagle. In conclusion, the CSFs derived from the DNNs are the result of efficient processing of the natural world around us. Different visual environments have succinct effects on the shape of the CSFs.

Statistical Mechanics of Deep Linear Neural Networks: The Backpropagating Kernel Renormalization

The success of deep learning in many real-world tasks has triggered an intense effort to understand the power and limitations of deep learning in the training and generalization of complex tasks, so far with limited progress. In this work, we study the statistical mechanics of learning in Deep Linear Neural Networks (DLNNs) in which the input-output function of an individual unit is linear. Despite the linearity of the units, learning in DLNNs is nonlinear, hence studying its properties reveals some of the features of nonlinear Deep Neural Networks (DNNs). Importantly, we solve exactly the network properties following supervised learning using an equilibrium Gibbs distribution in the weight space. To do this, we introduce the Back-Propagating Kernel Renormalization (BPKR), which allows for the incremental integration of the network weights starting from the network output layer and progressing backward until the first layer's weights are integrated out. This procedure allows us to evaluate important network properties, such as its generalization error, the role of network width and depth, the impact of the size of the training set, and the effects of weight regularization and learning stochasticity. BPKR does not assume specific statistics of the input or the task's output. Furthermore, by performing partial integration of the layers, the BPKR allows us to compute the properties of the neural representations across the different hidden layers. We have proposed an extension of the BPKR to nonlinear DNNs with ReLU. Surprisingly, our numerical simulations reveal that despite the nonlinearity, the predictions of our theory are largely shared by ReLU networks in a wide regime of parameters. Our work is the first exact statistical mechanical study of learning in a family of DNNs, and the first successful theory of learning through successive integration of DoFs in the learned weight space.

Glassboxing Deep Learning to Enhance Aircraft Detection from SAR Imagery

Although deep learning has achieved great success in aircraft detection from SAR imagery, its blackbox behavior has been criticized for low comprehensibility and interpretability. Such challenges have impeded the trustworthiness and wide application of deep learning techniques in SAR image analytics. In this paper, we propose an innovative eXplainable Artificial Intelligence (XAI) framework to glassbox deep neural networks (DNN) by using aircraft detection as a case study. This framework is composed of three parts: hybrid global attribution mapping (HGAM) for backbone network selection, path aggregation network (PANet), and class-specific confidence scores mapping (CCSM) for visualization of the detector. HGAM integrates the local and global XAI techniques to evaluate the effectiveness of DNN feature extraction; PANet provides advanced feature fusion to generate multi-scale prediction feature maps; while CCSM relies on visualization methods to examine the detection performance with given DNN and input SAR images. This framework can select the optimal backbone DNN for aircraft detection and map the detection performance for better understanding of the DNN. We verify its effectiveness with experiments using Gaofen-3 imagery. Our XAI framework offers an explainable approach to design, develop, and deploy DNN for SAR image analytics.

ECG signal classification via combining hand-engineered features with deep neural network features

ECG signal classification via combining hand-engineered features with deep neural network features

UAV Remote Sensing Image Automatic Registration Based on Deep Residual Features

With the rapid development of unmanned aerial vehicle (UAV) technology, UAV remote sensing images are increasing sharply. However, due to the limitation of the perspective of UAV remote sensing, the UAV images obtained from different viewpoints of a same scene need to be stitched together for further applications. Therefore, an automatic registration method of UAV remote sensing images based on deep residual features is proposed in this work. It needs no additional training and does not depend on image features, such as points, lines and shapes, or on specific image contents. This registration framework is built as follows: Aimed at the problem that most of traditional registration methods only use low-level features for registration, we adopted deep residual neural network features extracted by an excellent deep neural network, ResNet-50. Then, a tensor product was employed to construct feature description vectors through exacted high-level abstract features. At last, the progressive consistency algorithm (PROSAC) was exploited to remove false matches and fit a geometric transform model so as to enhance registration accuracy. The experimental results for different typical scene images with different resolutions acquired by different UAV image sensors indicate that the improved algorithm can achieve higher registration accuracy than a state-of-the-art deep learning registration algorithm and other popular registration algorithms.

Deep learning method for prediction of patient-specific dose distribution in breast cancer

BackgroundPatient-specific dose prediction improves the efficiency and quality of radiation treatment planning and reduces the time required to find the optimal plan. In this study, a patient-specific dose prediction model was developed for a left-sided breast clinical case using deep learning, and its performance was compared with that of conventional knowledge-based planning using RapidPlan™.MethodsPatient-specific dose prediction was performed using a contour image of the planning target volume (PTV) and organs at risk (OARs) with a U-net-based modified dose prediction neural network. A database of 50 volumetric modulated arc therapy (VMAT) plans for left-sided breast cancer patients was utilized to produce training and validation datasets. The dose prediction deep neural network (DpNet) feature weights of the previously learned convolution layers were applied to the test on a cohort of 10 test sets. With the same patient data set, dose prediction was performed for the 10 test sets after training in RapidPlan. The 3D dose distribution, absolute dose difference error, dose-volume histogram, 2D gamma index, and iso-dose dice similarity coefficient were used for quantitative evaluation of the dose prediction.ResultsThe mean absolute error (MAE) and one standard deviation (SD) between the clinical and deep learning dose prediction models were 0.02 ± 0.04%, 0.01 ± 0.83%, 0.16 ± 0.82%, 0.52 ± 0.97, − 0.88 ± 1.83%, − 1.16 ± 2.58%, and − 0.97 ± 1.73% for D95%, Dmean in the PTV, and the OARs of the body, left breast, heart, left lung, and right lung, respectively, and those measured between the clinical and RapidPlan dose prediction models were 0.02 ± 0.14%, 0.87 ± 0.63%, − 0.29 ± 0.98%, 1.30 ± 0.86%, − 0.32 ± 1.10%, 0.12 ± 2.13%, and − 1.74 ± 1.79, respectively.ConclusionsIn this study, a deep learning method for dose prediction was developed and was demonstrated to accurately predict patient-specific doses for left-sided breast cancer. Using the deep learning framework, the efficiency and accuracy of the dose prediction were compared to those of RapidPlan. The doses predicted by deep learning were superior to the results of the RapidPlan-generated VMAT plan.

RETRACTED ARTICLE: A Fused Heterogeneous Deep Neural Network and Robust Feature Selection Framework for Human Actions Recognition

RETRACTED ARTICLE: A Fused Heterogeneous Deep Neural Network and Robust Feature Selection Framework for Human Actions Recognition