Deep Neural Network Classifier Research Articles

Constructing infinite deep neural networks with flexible expressiveness while training

The depth of the deep neural network (DNN) refers to the number of hidden layers between the input and output layers of an artificial neural network. It usually indicates a certain degree of complexity of the computational cost (parameters and floating point operations per second) and expressiveness once the network structure is settled. In this study, we experimentally investigate the effectiveness of using neural ordinary differential equations (NODEs) as a component to provide further depth in a continuous way to relatively shallower networks rather than stacking more layers (discrete depth), which achieved an improvement with fewer parameters. Experiments are conducted on classic DNNs, the residual networks. Moreover, we construct infinite deep neural networks with flexible complexity based on NODEs, enabling the system to adjust its complexity during training. On a better hidden-space provided by adaptive step DNNs, adaptive step ResNet with NODE (ResODE) is managed to achieve better performances in terms of convergence and accuracy than standard networks, and the improvements are widely observed in popular benchmarks.

Jadeite: A novel image-behavior-based approach for Java malware detection using deep learning

Java malware exploiting language vulnerabilities has become increasingly prevalent in the recent past. Since Java is a platform-independent language, these security threats open up the opportunity for multi-platform exploitation. Although security researchers continuously develop different approaches for protecting against Java malware programs, the presence of complicated Java malware properties, such as code obfuscation, makes these malware programs fly under the radar. These challenges present the need to develop new approaches that are resilient to such properties. This article presents Jadeite, a novel approach for detecting Java bytecode malware programs using static analysis and recent advancements in the image-based, deep-learning classification space. In particular, Jadeite extracts the Interprocedural Control Flow Graph (ICFG) from a given Java bytecode file and then prunes the ICFG and converts it into an adjacency matrix. Finally, Jadeite constructs a grayscale image from this matrix. We leverage an object detection algorithm in a deep Convolutional Neural Network (CNN) classifier to determine maliciousness. Also, Jadeite extracts an additional set of features from the Java malware program to improve the accuracy of malware classification. These features are consolidated with the extracted images and used as inputs to the CNN classifier. Experimental results demonstrate that Jadeite achieves high accuracy (98.4%) compared to other Java malware detection approaches and is capable of detecting both known and previously-unseen real-world malicious Java programs.

An Improved Deep Neural Network Algorithm for the Prediction of Limited Proteolysis in Native Protein

Protease is a proteolytic enzyme that hydrolyzes the amino acid where the cleavage only occurs at specific sites of the amino acid substrate. By discovering the nick site, the prediction on the function of proteases can be identified and enable humans to control the protein's hydrolysis by their corresponding protease. It is very contributed to controlling protein production especially viral protein. The experts may alter the production of viral protein by reducing the viral proteases to undergo proteolysis. With the rise of computational methods in this era, deep learning is becoming more famous and applied in every field of study, including the biological area. Conventional techniques such as mass spectrometry and two-dimensional gel electrophoresis are being replaced by computational methods due to time-consuming. Thus, this study improves the deep learning algorithm by proposing the Hybrid model of Random Forest + Deep Neural Network (Hybrid RF+DNN) to classify nick sites. The classification in this study is compared with the other machine learning algorithms such as Random Forest (RF), Support Vector Machine (SVM), and Deep Neural Network (DNN). The proposed method is believed to enhance the classification results in identifying the positive and negative nick sites. The RF is a feature-selector that gathers the most important feature before entering the DNN classifier. This approach reduces the data dimensionality and speeds up the execution time of the training process. The performance of the models was measured by confusion matrix, specificity, sensitivity, etc. However, the proposed method is not the best performer among the mentioned classifiers from the result. The proposed method may become the best performer as the parameter tuning is done more precisely, even after the feature selection by the RF algorithm. Thus, the proposed method with the enhancement appears to be an alternative to the researcher discovering nick site.

Abstract 10161: Predicting Life-Threatening Ventricular Arrhythmias in Patients with Non-Ischemic Dilated Cardiomyopathy Using Electrocardiogram-Based Deep Neural Networks

Introduction: Patients with non-ischemic DCM are at risk of life-threatening ventricular arrhythmias. Improved patient selection for ICD implantation is warranted. Deep neural networks (DNN) can discover complex ECG features without the need for hand-crafted feature extraction. This study aimed to distinguish patients at risk of arrhythmias and detect and visualize ECG characteristics that attribute to the DNN classification. Methods: We trained a DNN model on baseline raw ECG samples of adult patients with DCM for the composite outcome of sustained ventricular arrhythmia, (aborted) sudden cardiac death and appropriate ICD therapy in the University Medical Centre Utrecht. Gradient-Weighted Class Activation Mapping was used for visual interpretation of the network. DNN performance was validated both internally and in an external cohort from the Maastricht University Medical University Center. Results: 26.628 ECGs were used for pre-training the network. In total, 317 patients with DCM were included in the primary cohort with a median age of 52 years (IQR 42 - 61), median LVEF of 25% (IQR 20-33), NYHA II and III (32% and 25%). 233 patients were ICD recipients. Subsequently 84 patients reached the outcome (median follow-up duration of 3.96 years (IQR 1.94 - 7.00)). The DNN focused on subtle but predictive differences in the terminal QRS-segment, the T-wave, and the P-wave (figure). The AUC of the DNN was 0.70 in cross-validation. External validation was similar, with an AUC of 0.66. Conclusion: In this study we trained and validated a deep neural network for prediction of life-threatening arrhythmic events. Even though the discriminatory changes on ECG segments were very subtle, the trained DNN was able to predict life-threatening events, focusing on the terminal QRS-segment, the T-wave, and the P-wave. Future studies are warranted to elucidate the identified ECG abnormalities and understand their electrophysiological substrates for LTA prediction in DCM.

An efficient epileptic seizure classification system using empirical wavelet transform and multi-fuse reduced deep convolutional neural network with digital implementation

Epilepsy, an incurable brain disorder portrayed by seizures, is the most common neurological disease worldwide. The embryonic detection of epileptic action helps the psychologist for the diagnosis of epileptic seizure and reduces the seizure effect on the patient’s life. Empirical wavelet transform (EWT), and a novel multi-fuse reduced deep convolutional neural network (MF-RDCNN) classifier are integrated to design a computer-aided-diagnosis (CAD) system for meticulous classification of epileptic seizure from electroencephalogram (EEG) signals. The EWT is enforced on recorded EEG signals to extract three distinct band-limited modes (BLMs) and the proposed classifier is used to compute the most informative unsupervised signatures automatically by taking the extracted BLMs as inputs. The experimental results are carried out using three EEG databases provided by Bonn-University, Germany single-channel EEG (dataset-A), Neurology and Sleep Center, New Delhi single-channel EEG (dataset-B), and Boston Children’s Hospital multichannel scalp EEG (dataset-C) to verify the effectiveness of the proposed algorithm. The performance of the proposed method is superior over other prevalent methods and has 100%,99.82% overall mean classification accuracy for two class and three class classification problems respectively using ten-fold cross-validation. Moreover, the proposed method has specificity (SPE) of 99.29%, sensitivity (SEN) of 99.86%, and classification accuracy (ACC) of 99.29% with 0.71% of false positive rate per hour (FPR/h) by taking 40% of data for training, 40% of data for testing, and the remaining 20% of data for validation from the total data incorporated with the database-C. The lesser computational complexity, higher learning speed, short event recognition time, remarkable overall mean classification accuracy, and outstanding overall performance are the major advantages of the proposed EWT-MF-RDCNN method over EWT-RDCNN and RDCNN method for accurate classification of seizure EEG epochs. The digital architecture of the MF-RDCNN classifier is also implemented in a high-speed FPGA processor to develop a CAD system for efficacious diagnosis of epileptic seizure activity online.

Mian: interactive web-based microbiome data table visualization and machine learning platform.

Mian is a web application to interactively visualize, run statistical tools and train machine learning models on operational taxonomic unit (OTU) or amplicon sequence variant (ASV) datasets to identify key taxonomic groups, diversity trends or taxonomic composition shifts in the context of provided categorical or numerical sample metadata. Tools, including Fisher's exact test, Boruta feature selection, alpha and beta diversity, and random forest and deep neural network classifiers, facilitate open-ended data exploration and hypothesis generation on microbial datasets. Mian is freely available at: miandata.org. Mian is an open-source platform licensed under the MIT license with source code available at github.com/tbj128/mian. Supplementary data are available at Bioinformatics online.

Filtering airborne LIDAR data by using fully convolutional networks

The classification of LIDAR point clouds has always been a challenging task. Classification refers to label each point in different categories, such as ground, vegetation or building. The success of deep learning techniques in image processing tasks have encouraged researchers to use deep neural networks for classification of LIDAR point clouds. In this paper, we proposed a U-Net based architecture capable of classifying LIDAR data. The results indicated that our network model achieved an average F1 score of 91% over all three classes (ground, vegetation and building) for our best model.

A novel deep learning technique for analysis and detection of ARMD using OCT scan images

Age-Related Macular Degeneration (ARMD) is a medical situation resulting in blurred or no vision in the middle of the eye view. Though this disease doesn’t make the person completely blind, it makes it very difficult for the person to perform day to day activities like reading, driving, recognizing people etc. This paper aims to detect ARMD though Optical Coherence Tomography (OCT) scans where the drusen in the macula is detected and identify the infected. The images are first passed though Directional Total Variation (DTV) Denoising followed by Active contour algorithm to mark the boundaries of the layers in macula. In deep learning, a convolutional neural network is a class of deep neural networks, most commonly applied to analyzing visual imagery. Then these images categorized as healthy and infected using Convolution Neural Network. Different CNN variant algorithms like Alexnet, VggNet and GoogleNet have been compared in the experiments and the results obtained are better compared to traditional methods.

Internet of things and deep learning enabled healthcare disease diagnosis using biomedical electrocardiogram signals

AbstractWith recent advancements in the internet of things (IoT), wearables, and sensing technologies, the quality of healthcare services gets improved and it caused a shift from conventional clinical‐based healthcare to real‐time monitoring. The sensors are commonly integrated into several medical gadgets to save the bio‐signals produced by the physiological activities of the human body. At the same time, a biomedical electrocardiogram (ECG) signal is employed as a familiar way to examine and diagnose cardiovascular diseases (CVDs), which is rapid and non‐invasive. Since the increasing number of patients degrades the classification performance due to high differences in the ECG signal patterns among several patients, computer‐assisted automated diagnostic tools are essential for ECG signal classification. With this motivation, this paper introduces a new IoT and deep learning (DL) enabled healthcare disease diagnosis (IoTDL‐HDD) model using biomedical ECG signals. The proposed IoTDL‐HDD model aims to detect the presence of CVDs by the use of DL models in biomedical ECG signals. In addition, the proposed IoTDL‐HDD model utilizes a BiLSTM feature extraction technique to extract useful feature vectors from the ECG signals. For improving the efficiency of the BiLSTM technique, the artificial flora optimization (AFO) algorithm is employed as a hyperparameter optimizer. Besides, a fuzzy deep neural network (FDNN) classifier is employed for assigning proper class labels to the ECG signals. The performance of the IoTDL‐HDD model is examined on biomedical ECG signals and the outcomes are inspected in distinct features. The resultant experimental outcomes pointed out the supremacy of the IoTDL‐HDD model with the maximum accuracy of 93.452%.

Border Collie Cat Optimization for Intrusion Detection System in Healthcare IoT Network Using Deep Recurrent Neural Network

Abstract Attacks are the major problems in the Internet of Things (IoT) applications and communication networks. The undetected intruders affect the availability of the system for end-users, increase identity theft and data breaches. Hence, it is required to detect the attacks in the IoT systems to ensure effective defense and security. In this research, the Border Collie Cat Optimization-based Deep Recurrent Neural Network is proposed to detect intrusion in the IoT networks. Here, the proposed Border Collie Cat Optimization algorithm is derived by the integration of Border Collie Optimization and Cat Swarm Optimization. At first, the messages are authenticated at the authentication phase using the hashing and encryption function. After authenticating the device, the communication between the server and user is carried out at the communication phase to make the IoT device eligible for data transfer within the network. Then, the Deep Recurrent Neural Network classifier is employed to detect the intruders in the IoT network in such a way that the training process is carried out using the proposed Border Collie Optimization algorithm. The proposed approach obtained higher performance with the metrics, like detection rate, sensitivity, specificity and accuracy with the values of 0.9375, 0.9539, 0.8791 and 0.9263, respectively.

A Novel Framework using Zero Shot Learning Technique for Non-Factoid Question Answering System

Non-Factoid Question Answering (QA) is the next generation of textual QA systems, which gives passage level summaries for a natural language query, posted by the user. The main issue lies in the appropriateness of the generated summary. This paper proposes a framework for non-factoid QA system, which has three main components: (i) A deep neural network classifier, which produces sentence vector considering word correlation and context. (ii) Zero shot classifier that uses a multi-channel Convolutional Neural Network (CNN), to extract knowledge from multiple sources in the knowledge accumulator. This output acts as a knowledge enhancer that strengthens the passage level summary. (iii) Summary generator that uses Maximal Marginal Relevance (MMR) algorithm, which computes similarity among the query related answer and the sentences from zero shot classifier. This model is applied on the datasets WikiPassageQA and ANTIQUE. The experimental analysis shows that this model gives comparatively better results for WikiPassageQA dataset.

Thyroid Nodule Segmentation and Classification Using Deep Convolutional Neural Network and Rule-based Classifiers.

Thyroid cancer has a high prevalence all over the world. Accurate thyroid nodule diagnosis can lead to effective treatment and decrease the mortality rate. Ultrasound imaging is a safe, portable, and inexpensive tool for thyroid nodule monitoring. However, the widespread use of ultrasound has also resulted in over-diagnosis and over-treatment of nodules. There is also large variability in the assessment and characterization of nodules. Thyroid nodule classification requires precise delineation of the nodule boundary which is tedious and time- consuming. Automatic segmentation of nodule boundaries is highly desirable, however, it is challenging due to the wide range of nodule appearances, shapes, and sizes. In this study, we propose an end-to-end pipeline for nodule segmentation and classification. A residual dilated UNet (resDUnet) model is proposed for nodule segmentation. The output of resDUnet is fed to two rule-based classifiers to categorize the composition and echogenicity of the segmented nodule. We evaluate our segmentation method on a large dataset of 352 ultrasound images reviewed by a certified radiologist. When compared with ground-truth, resDUnet gives a higher Dice score than the standard UNet (82% vs. 81%). Our method requires minimal user interaction and it is robust to reasonable variations in the user-specified region-of-interest. We expect the proposed method to reduce variability in thyroid nodule assessment which results in more efficient and cost-effective monitoring of thyroid cancer.

Breast Cancer Histopathological Image Classification with Adversarial Image Synthesis.

Data limitation is one of the major challenges in applying deep learning to medical images. Data augmentation is a critical step to train robust and accurate deep learning models for medical images. In this research, we increase the size of a small dataset by using an Auxiliary Classifier Generative Adversarial Network (ACGAN) which generates realistic images along with their class labels.We evaluate the effectiveness of our ACGAN augmentation method by performing breast cancer histopathological image classification with deep convolutional neural network (dCNN) classifiers trained on our enhanced dataset. For our classifier, we use a transfer learning approach where the convolutional features are extracted from a pertained model and subsequently fed into several extreme gradient boosting (XGBoost) classifiers. Our experimental results on Breast Cancer Histopathological (BreakHis) dataset show that ACGAN data augmentation, along with our XGBoost classifier increases the classification accuracy by 9.35% for binary classification (benign vs. malignant) and 8.88% for four-class tumor sub-type classification compared with standard transfer learning approach.

A Data Augmentation Method for War Trauma Using the War Trauma Severity Score and Deep Neural Networks

The demand for large-scale analysis and research of data on trauma from modern warfare is increasing day by day, but the amount of existing data is not sufficient to meet such demand. In this study, an integrated modeling approach incorporating a war trauma severity scoring algorithm (WTSS) and deep neural networks (DNN) is proposed. First, the proposed WTSS, which uses multiple non-linear regression based on the characteristics of war trauma data and the medical evaluation by an expert panel, performed a standardized assessment of an injury and predicts its trauma consequences. Second, to generate virtual injury, based on the probability of occurrence, the injured parts, injury types, and complications were randomly sampled and combined, and then WTSS was used to assess the consequences of the virtual injury. Third, to evaluate the accuracy of the predicted injury consequences, we built a DNN classifier and then trained it with the generated data and tested it with real data. Finally, we used the Delphi method to filter out unreasonable injuries and improve data rationality. The experimental results verified that the proposed approach surpassed the traditional artificial generation methods, achieved a prediction accuracy of 84.43%, and realized large-scale and credible war trauma data augmentation.

A Deep Learning Approach for Voice Disorder Detection for Smart Connected Living Environments

Edge Analytics and Artificial Intelligence are important features of the current smart connected living community. In a society where people, homes, cities, and workplaces are simultaneously connected through various devices, primarily through mobile devices, a considerable amount of data is exchanged, and the processing and storage of these data are laborious and difficult tasks. Edge Analytics allows the collection and analysis of such data on mobile devices, such as smartphones and tablets, without involving any cloud-centred architecture that cannot guarantee real-time responsiveness. Meanwhile, Artificial Intelligence techniques can constitute a valid instrument to process data, limiting the computation time, and optimising decisional processes and predictions in several sectors, such as healthcare. Within this field, in this article, an approach able to evaluate the voice quality condition is proposed. A fully automatic algorithm, based on Deep Learning, classifies a voice as healthy or pathological by analysing spectrogram images extracted by means of the recording of vowel /a/, in compliance with the traditional medical protocol. A light Convolutional Neural Network is embedded in a mobile health application in order to provide an instrument capable of assessing voice disorders in a fast, easy, and portable way. Thus, a straightforward mobile device becomes a screening tool useful for the early diagnosis, monitoring, and treatment of voice disorders. The proposed approach has been tested on a broad set of voice samples, not limited to the most common voice diseases but including all the pathologies present in three different databases achieving F1-scores, over the testing set, equal to 80%, 90%, and 73%. Although the proposed network consists of a reduced number of layers, the results are very competitive compared to those of other “cutting edge” approaches constructed using more complex neural networks, and compared to the classic deep neural networks, for example, VGG-16 and ResNet-50.

Electric and fuel car identification based on UAV thermal infrared images using deep convolutional neural networks

ABSTRACT Electric cars, as an alternative to fuel cars, are growing rapidly in number and help reduce carbon dioxide emissions and build green cities. However, due to the appearance similarities between electric and fuel cars, it is still a challenge to count the proportion of electric cars on the road. Considering that thermal infrared (TIR) imaging technology can obtain the temperature characteristics of cars as well as work at night, this paper proposes a new solution to identify electric and fuel cars in Unmanned aerial vehicle (UAV)-based TIR images. First, a semiautomated labelling method is implemented for training data set generation, which is based on a traditional object tracking algorithm to improve labelling efficiency. Then, two classic deep convolutional neural networks, YOLOv5 and SSD, are used to verify the reliability of the dataset. Experiments prove that the proposed solution can effectively identify electric and fuel cars on the road, with mean of average precision (mAP) up to 0.99. This study is the first attempt to apply UAV-based TIR imaging to electric and fuel car identification and propose the first open data set for relevant research.

AoP-LSE: Antioxidant Proteins Classification Using Deep Latent Space Encoding of Sequence Features.

It is of utmost importance to develop a computational method for accurate prediction of antioxidants, as they play a vital role in the prevention of several diseases caused by oxidative stress. In this correspondence, we present an effective computational methodology based on the notion of deep latent space encoding. A deep neural network classifier fused with an auto-encoder learns class labels in a pruned latent space. This strategy has eliminated the need to separately develop classifier and the feature selection model, allowing the standalone model to effectively harness discriminating feature space and perform improved predictions. A thorough analytical study has been presented alongwith the PCA/tSNE visualization and PCA-GCNR scores to show the discriminating power of the proposed method. The proposed method showed a high MCC value of 0.43 and a balanced accuracy of 76.2%, which is superior to the existing models. The model has been evaluated on an independent dataset during which it outperformed the contemporary methods by correctly identifying the novel proteins with an accuracy of 95%.

Efficient Face Spoofing Detection With Flash

In light of the rising demand for biometric-authentication systems, preventing face spoofing attacks is a critical issue for the safe deployment of face recognition systems. Here, we propose an efficient face presentation attack detection (PAD) algorithm that requires minimal hardware and only a small database, making it suitable for resource-constrained devices such as mobile phones. Utilizing one monocular visible-light camera, the proposed algorithm takes two facial photos, taken with and without a flash, respectively. The proposed SpecDiff descriptor is constructed by leveraging two types of reflection: (i) specular reflections from the iris region that have a specific intensity distribution depending on liveness, and (ii) diffuse reflections from the entire face region that represents the 3D structure of a subject’s face. Classifiers trained with the SpecDiff descriptor outperform other flash-based PAD algorithms on both an in-house database and four publicly available databases: NUAA, Replay-Attack, Spoofing in the Wild, and OULU-NPU. Furthermore, the proposed algorithm achieves statistically significantly better accuracy to that of an end-to-end, deep neural network classifier, while being approximately six-times faster execution speed. The limitation of the proposed algorithm is also quantified under various adversarial lighting conditions, to guide users for the safe deployment of the algorithm. The code is publicly available at https://github.com/Akinori-F-Ebihara/SpecDiff-spoofing-detector. Example images of in-house database are also available at https://github.com/Akinori-F-Ebihara/SpecDiff in house database sample.

A 10.13µJ/Classification 2-Channel Deep Neural Network Based SoC for Negative Emotion Outburst Detection of Autistic Children.

An electroencephalogram (EEG)-based non-invasive 2-channel neuro-feedback SoC is presented to predict and report negative emotion outbursts (NEOB) of Autistic patients. The SoC incorporates area-and-power efficient dual-channel Analog Front-End (AFE), and a deep neural network (DNN) emotion classification processor. The classification processor utilizes only the two-feature vector per channel to minimize the area and overfitting problems. The 4-layers customized DNN classification processor is integrated on-sensor to predict the NEOB. The AFE comprises two entirely shared EEG channels using sampling capacitors to reduce the area by 30%. Moreover, it achieves an overall integrated input-referred noise, NEF, and crosstalk of 0.55 µVRMS, 2.71, and -79 dB, respectively. The 16 mm2 SoC is implemented in 0.18 um 1P6M, CMOS process and consumes 10.13 μJ/classification for 2 channel operation while achieving an average accuracy of >85% on multiple emotion databases and real-time testing.

WBA-DNN: A hybrid weight bat algorithm with deep neural network for classification of poisonous and harmful wild plants

In this study, a deep neural network structure is proposed for the classification of poisonous and harmful wild plants in fields. Furthermore, a novel metaheuristic weight bat-inspired algorithm is developed for training the devised deep neural network. The harmful wild plant dataset is obtained from the agricultural field contains the purslane plants and harmful plants. The feature of the plants extract with mean absolute deviation, the dataset is consists of four features, two classes information, and contains 3452 samples, one-third of these samples are classified as purslane plants. Firstly, the performance of the proposed weight bat-inspired algorithm based deep neural network is evaluated by using ten UCI data repository datasets and the obtained results are compared with state-of-the-art classification algorithms. Then, classification of the harmful wild plant dataset is performed, results of the proposed weight bat-inspired algorithm based deep neural network are compared to two categories of classification algorithms, including (i) the most well-known classification algorithms, including decision tree, k-nearest neighbors, backpropagation based deep neural network, naïve bayes, random forest, AdaBoost, and support vector machine; (ii) optimization-based deep neural network, including bat algorithm, genetic algorithm, particle swarm optimization, equilibrium optimizer, A bio-inspired based optimization algorithm, and salp swarm algorithm. The proposed weight bat-inspired algorithm based deep neural network has outperformed the most well-known classification algorithms and optimization-based deep neural network in terms of CA, FPR, REC, PRE, TNR, AUC, F1-M, and F-M by 0.980, 0.020, 0.980, 0.980, 0.980, 0.980, 0.980, and 0.980, respectively. The highest performance has indicated that the training of deep neural networks by the weight bat-inspired algorithm is proven to be a very effective and useful tool for the classification of poisonous and harmful wild plants.