Deep Neural Network Techniques Research Articles

Animal image identification and classification using deep neural networks techniques

Animal image identification and classification using deep neural networks techniques

A Neural Network-Based Method for Surface Metallization of Polymer Materials

It’s no secret that polymers have been employed extensively in a variety of industries. Polymers, on the other hand, have faced difficulties in their development because of their complicated chemical composition and structure. Data-driven approaches in polymer science and technology have resulted in new directions in research leading to the implementation of deep learning models and vast data assets. In the growing area of polymer informatics, deep learning methods based on factual data are being used to speed up the performance assessment and process improvement of new polymers. Using a deep neural network (DNN), we can now forecast the surface metallization properties of polymer materials, which we describe in this research. First, we collect a raw dataset of polymer materials’ characteristics. The raw data are filtered and normalized using the min–max normalization approach. To convert normalized data into numerical characteristics, principal component analysis (PCA) is employed. Polymer surface metallization characteristics can then be predicted using a suggested DNN technique. The proposed and conventional approaches are also compared so that our research can be done to its full potential.

Missing value imputation and the effect of feature normalisation on financial distress prediction

ABSTRACT In this paper, we focus on comparing the imputation performance of different deep and machine learning techniques on nine related datasets containing different missing rates ranging from 10% to 50%. Moreover, since each feature value ranges differently, such as total liability/equity ratio and earnings per share, the effect of feature normalisation on the imputation results is also examined to see whether normalising the feature values after missing value imputation can improve the prediction model performance. The experimental results show that the deep neural network technique does not necessarily perform better than the traditional machine learning technique for missing value imputation. In particular, the random forest imputation model performs the best, whereas the k-nearest neighbour method is the second best imputation model in terms of the AUC rates and type II errors. The performance in improvement of prediction models after performing feature normalisation is heavily dependent on the chosen classification technique. There is thus no need to consider the normalisation step when the random forest classifier is used. It is found that the deep neural network and support vector machine classifiers can significantly outperform those without feature normalisation.

Accurate classification of lung nodules on CT images using the TransUnet.

Computed tomography (CT) is an effective way to scan for lung cancer. The classification of lung nodules in CT screening is completely doctor dependent, which has drawbacks, including difficulty classifying tiny nodules, subjectivity, and high false-positive rates. In recent years, deep convolutional neural networks, a deep learning technology, have been shown to be effective in medical imaging diagnosis. Herein, we propose a deep convolutional neural network technique (TransUnet) to automatically classify lung nodules accurately. TransUnet consists of three parts: the transformer, the Unet, and global average pooling (GAP). The transformer encodes discriminative features via global self-attention modeling on CT image patches. The Unet, which collects context by constricting route, enables exact lunge nodule localization. The GAP categorizes CT images, assigning each sample a score. Python was employed to pre-process all CT images in the LIDI-IDRI, and the obtained 8,474 images (3,259 benign and 5,215 lung nodules) were used to evaluate the method's performance. The accuracies of TransUnet in the training and testing sets were 87.90 and 84.62%. The sensitivity, specificity, and AUC of the proposed TransUnet on the testing dataset were 70.92, 93.17, and 0.862%, respectively (0.844-0.879). We also compared TransUnet to three well-known methods, which outperformed these methods. The experimental results on LIDI-IDRI demonstrated that the proposed TransUnet has a great performance in classifying lung nodules and has a great potential application in diagnosing lung cancer.

Deep Learning–based Fast Spectral Inversion of Hα and Ca ii 8542 Line Spectra

A multilayer spectral inversion (MLSI) model has recently been proposed for inferring the physical parameters of plasmas in the solar chromosphere from strong absorption lines taken by the Fast Imaging Solar Spectrograph (FISS). We apply a deep neural network (DNN) technique in order to produce the MLSI outputs with reduced computational costs. We train the model using two absorption lines, Hα and Ca ii 8542 Å, taken by FISS, and 13 physical parameters obtained from the application of MLSI to 49 raster scans (∼2,000,000 spectra). We use a fully connected network with skip connections and multi-branch architecture to avoid the problem of vanishing gradients and to improve the model’s performance. Our test shows that the DNN successfully reproduces the physical parameters for each line with high accuracy and a computing time of about 0.3–0.4 ms per line, which is about 250 times faster than the direct application of MLSI. We also confirm that the DNN reliably reproduces the temporal variations of the physical parameters generated by the MLSI inversion. By taking advantage of the high performance of the DNN, we plan to provide physical parameter maps for all the FISS observations, in order to understand the chromospheric plasma conditions in various solar features.

Deep Neural Network Technique Based Field Digitizing Units (FDUs) Instruments Fault Detection and Isolation

Deep Neural Network Technique Based Field Digitizing Units (FDUs) Instruments Fault Detection and Isolation

Predicting shale mineralogical brittleness index from seismic and elastic property logs using interpretable deep learning

The mineralogical brittleness index (MBI) of organic-rich shale formations is one of the key parameters to identify the optimal production well locations and optimize hydraulic fracturing. Since we as a community don't understand the exact physical relationship between the MBI and seismic properties from well logs, we have used traditional approaches like the log-based brittleness index (LBI) and the elastic brittleness index (EBI) to quantify the rock brittleness from seismic data and well logs. The LBI method is easy to use but is empirically derived from the porosity and sonic logs. On the other hand, the EBI method is dependent on the average values of Young's modulus and Poisson's ratio but is not physically meaningful in practice. Therefore, we develop a deep learning approach to obtain a more reliable MBI model from seismic properties and enhance the interpretability with Shapley values. First, we analyze the statistical relationship between the MBI and eight seismic properties from well logs and distinguish the influential input variables for the MBI prediction, such as bulk density, Young's modulus, and Poisson's ratio. Second, we find a multivariate linear regression (MLR) model with three input properties and quantify the relative statistical contribution of each input based on Shapley values. Third, we use a deep neural network technique to derive the nonlinear estimation model with a better fit to the MBI data than the traditional methods. We test and verify our approach on field log and core data from the Wolfcamp shales in the Permian Basin, Texas. In conclusion, this workflow can provide a more interpretable and accurate MBI estimation from seismic properties to enhance unconventional shale reservoir characterization.

Measurement Error Models: From Nonparametric Methods to Deep Neural Networks

The success of deep learning has inspired a lot of recent interests in exploiting neural network structures for statistical inference and learning. In this paper, we review some popular deep neural network structures and techniques under the framework of nonparametric regression with measurement errors. In particular, we demonstrate how to use a fully connected feed-forward neural network to approximate the regression function f(x), explain how to use a normalizing flow to approximate the prior distribution of X, and detail how to construct an inference network to generate approximate posterior samples of X. After reviewing recent advances in variational inference for deep neural networks, such as the importance weighted autoencoder, doubly reparametrized gradient estimator, and nonlinear independent components estimation, we describe an inference procedure built upon these advances. An extensive numerical study is presented to compare the neural network approach with classical nonparametric methods, which suggests that the neural network approach is more flexible in accommodating different classes of regression functions and performs superior or comparable to the best available method in many settings.

Detection of Planting Systems in Olive Groves Based on Open-Source, High-Resolution Images and Convolutional Neural Networks

This paper aims to evaluate whether an automatic analysis with deep learning convolutional neural networks techniques offer the ability to efficiently identify olive groves with different intensification patterns by using very high-resolution aerial orthophotographs. First, a sub-image crop classification was carried out. To standardize the size and increase the number of samples of the data training (DT), the crop images were divided into mini-crops (sub-images) using segmentation techniques, which used a different threshold and stride size to consider the mini-crop as suitable for the analysis. The four scenarios evaluated discriminated the sub-images efficiently (accuracies higher than 0.8), obtaining the largest sub-images (H = 120, W = 120) for the highest average accuracy (0.957). The super-intensive olive plantings were the easiest to classify for most of the sub-image sizes. Nevertheless, although traditional olive groves were discriminated accurately, too, the most difficult task was to distinguish between the intensive plantings and the traditional ones. A second phase of the proposed system was to predict the crop at farm-level based on the most frequent class detected in the sub-images of each crop. The results obtained at farm level were slightly lower than at the sub-images level, reaching the highest accuracy (0.826) with an intermediate size image (H = 80, W = 80). Thus, the convolutional neural networks proposed made it possible to automate the classification and discriminate accurately among traditional, intensive, and super-intensive planting systems.

Unsupervised Deep-Learning-Based Reconfigurable Intelligent Surface-Aided Broadcasting Communications in Industrial IoTs

This article presents a general system framework that lays the foundation for reconfigurable intelligent surface (RIS)-enhanced broadcast communications in Industrial Internet of Things (IIoTs). In our system model, we consider multiple sensor clusters co-existing in a smart factory where the direct links between these clusters and a central base station (BS) are blocked completely. In this context, a RIS is utilized to reflect signals broadcast from BS toward cluster heads (CHs) which act as a representative of clusters, where BS only has access to the statistical distribution of the channel state information (CSI). An analytical upper bound of the total ergodic spectral efficiency (SE) and an approximation of outage probability are derived. Based on these analytical results, two algorithms are introduced to control the phase shifts at RIS, which are the Riemannian conjugate gradient (RCG) method and the deep neural network (DNN) method. While the RCG algorithm operates based on the conventional iterative method, and the DNN technique relies on unsupervised deep learning (DL). Our numerical results show that both algorithms achieve satisfactory performance based on only statistical CSI. In addition, compared to the RCG scheme, using DL reduces the computational latency by more than ten times with an almost identical total ergodic SE achieved. These numerical results reveal that while using the conventional RCG method may provide unsatisfactory latency, and the DNN technique shows much promise for enabling RIS in ultrareliable and low-latency communications (URLLC) in the context of IIoTs.

An effective network intrusion detection and classification system for securing WSN using VGG-19 and hybrid deep neural network techniques

In today’s world, a Network Intrusion Detection System (NIDS) plays a vital role in order to secure the Wireless Sensor Network (WSN). However, the traditional NIDS model faced critical constraints with network traffic data due to growth in the complexity of modern attacks. These constraints have a direct impact on the overall performance of the WSN. In this paper, a new robust network intrusion classification framework based on the enhanced Visual Geometry Group (VGG-19) pre-trained model has been proposed to prolong the performance of WSN. Primarily, the pre-trained weights from the ImageNet dataset are utilized to train the parameters of the VGG-19. Afterward, a Hybrid Deep Neural Network based on Convolutional Neural Network (CNN) and Long Short-Term Memory (LSTM) will be employed to extract the influential features from network traffic data to enlarge the intrusion detection accuracy. The proposed VGG-19 + Hybrid CNN-LSTM model exploits both binary classification and multi-classification to classify attacks as either normal or attacked. A network intrusion benchmark dataset is used to assess the performance of the suggested system. The results reveal that the proposed VGG-19 + Hybrid CNN-LSTM learning system surpasses other pre-trained models with a superior accuracy of 98.86% during the multi-classification test.

Design of the Music Intelligent Management System Based on a Deep CNN

Music is a common art form in people’s life, and it is closely related to people’s living conditions. Since ancient times, music has been closely related to people’s lives. The music intelligent management system is convenient and user-friendly, and it can meet the demand for music. However, it has major flaws. Collaborative filtering algorithm can achieve the recommendation performance of music intelligent management system, which can recommend the same type of music to users with related preferences. Deep learning technology has developed relatively maturely, and it has been successfully applied in people’s life and production. Deep convolutional neural network (CNN) techniques can extract deeper features than simple CNN techniques. Although there is a weak nonlinear relationship between people’s behavioral characteristics and living habits and the music intelligent management system, the advantage of deep CNN technology is to deal with the nonlinear relationship between large amounts of data. This study uses deep CNN technology to extract the relationship between people’s living habits, living environment, and behavior characteristics and the music intelligent management system. The deep CNN technology helps the music intelligent management system to further realize the active recommendation function of the music intelligent management system. The research results also show that the deep CNN technology has good feasibility and high accuracy in the music intelligent management system. It can well map the relationship between people’s behavioral characteristics and living habits and the music intelligent management system. The deep CNN technology can also realize the active recommendation function of the music intelligent management system. For the prediction of the music intelligent management system, the largest prediction error is only 2.17%. This part of the error is for the prediction of song genres. The prediction errors for the other two features are both within 2%.

A Color-Texture-Based Deep Neural Network Technique to Detect Face Spoofing Attacks

Abstract Given the face spoofing attack, adequate protection of human identity through face has become a significant challenge globally. Face spoofing is an act of presenting a recaptured frame before the verification device to gain illegal access on behalf of a legitimate person with or without their concern. Several methods have been proposed to detect face spoofing attacks over the last decade. However, these methods only consider the luminance information, reflecting poor discrimination of spoofed face from the genuine face. This article proposes a practical approach combining Local Binary Patterns (LBP) and convolutional neural network-based transfer learning models to extract low-level and high-level features. This paper analyzes three color spaces (i.e., RGB, HSV, and YCrCb) to understand the impact of the color distribution on real and spoofed faces for the NUAA benchmark dataset. In-depth analysis of experimental results and comparison with other existing approaches show the superiority and effectiveness of our proposed models.

The Study of Render Farm Image Classification Using Deep Neural Network

Rendering on the making of movie animation is a process of combining the imagination of animators and artistic creativity by turning the graphic display into millions of moving images. To perform the rendering process, the render farm is used, which is a combination of a group of high-performance computers (super-computers) commonly referred to as Computer Generated Imager. Throughout this time, the render artist spends a considerable plenty of time on doing image analysis from render farms manually. Reflecting on that, the study presented in this paper propose a deep learning method can separate images of the results of render farms more quickly and accurately. The technique proposed in this study is the classification of approved or revised images as the results of the render farm machine by using Deep Neural Network (DNN) technique. To verify the DNN that best fits the dataset, experiments were carried out on several layer depths and adjustment of epoch. In terms of treatment of the dataset, the experiment scenario selected was percentage and cross validation. The best performance in the experiment results is provided by layer depth configuration of 10, epoch value 100. The configuration provides the value of accuracy, precision and recall of 90%, 96% and 92% respectively.

Deep LSTM Model for Diabetes Prediction with Class Balancing by SMOTE

Diabetes is an acute disease that happens when the pancreas cannot produce enough insulin. It can be fatal if undiagnosed and untreated. If diabetes is revealed early enough, it is possible, with adequate treatment, to live a healthy life. Recently, researchers have applied artificial intelligence techniques to the forecasting of diabetes. As a result, a new SMOTE-based deep LSTM system was developed to detect diabetes early. This strategy handles class imbalance in the diabetes dataset, and its prediction accuracy is measured. This article details investigations of CNN, CNN-LSTM, ConvLSTM, and deep 1D-convolutional neural network (DCNN) techniques and proposed a SMOTE-based deep LSTM method for diabetes prediction. Furthermore, the suggested model is analyzed towards machine-learning, and deep-learning approaches. The proposed model’s accuracy was measured against the diabetes dataset and the proposed method achieved the highest prediction accuracy of 99.64%. These results suggest that, based on classification accuracy, this method outperforms other methods. The recommendation is to use this classifier for diabetic patients’ clinical analysis.

Deep Convolutional Neural Network Mechanism Assessment of COVID-19 Severity.

As an epidemic, COVID-19's core test instrument still has serious flaws. To improve the present condition, all capabilities and tools available in this field are being used to combat the pandemic. Because of the contagious characteristics of the unique coronavirus (COVID-19) infection, an overwhelming comparison with patients queues up for pulmonary X-rays, overloading physicians and radiology and significantly impacting the quality of care, diagnosis, and outbreak prevention. Given the scarcity of clinical services such as intensive care and motorized ventilation systems in the aspect of this vastly transmissible ailment, it is critical to categorize patients as per their risk categories. This research describes a novel use of the deep convolutional neural network (CNN) technique to COVID-19 illness assessment seriousness. Utilizing chest X-ray images as contribution, an unsupervised DCNN model is constructed and suggested to split COVID-19 individuals into four seriousness classrooms: low, medium, serious, and crucial with an accuracy level of 96 percent. The efficiency of the DCNN model developed with the proposed methodology is demonstrated by empirical findings on a suitably huge sum of chest X-ray scans. To the evidence relating, it is the first COVID-19 disease incidence evaluation research with four different phases, to use a reasonably high number of X-ray images dataset and a DCNN with nearly all hyperparameters dynamically adjusted by the variable selection optimization task.

CovMnet-Deep Learning Model for classifying Coronavirus (COVID-19).

Diagnosing COVID-19, current pandemic disease using Chest X-ray images is widely used to evaluate the lung disorders. As the spread of the disease is enormous many medical camps are being conducted to screen the patients and Chest X-ray is a simple imaging modality to detect presence of lung disorders. Manual lung disorder detection using Chest X-ray by radiologist is a tedious process and may lead to inter and intra-rate errors. Various deep convolution neural network techniques were tested for detecting COVID-19 abnormalities in lungs using Chest X-ray images. This paper proposes deep learning model to classify COVID-19 and normal chest X-ray images. Experiments are carried out for deep feature extraction, fine-tuning of convolutional neural networks (CNN) hyper parameters, and end-to-end training of four variants of the CNN model. The proposed CovMnet provide better classification accuracy of 97.4% for COVID-19 /normal than those reported in the previous studies. The proposed CovMnet model has potential to aid radiologist to monitor COVID-19 disease and proves to be an efficient non-invasive COVID-19 diagnostic tool for lung disorders.

Jellyfish Search-Optimized Deep Learning for Compressive Strength Prediction in Images of Ready-Mixed Concrete.

Most building structures that are built today are built from concrete, owing to its various favorable properties. Compressive strength is one of the mechanical properties of concrete that is directly related to the safety of the structures. Therefore, predicting the compressive strength can facilitate the early planning of material quality management. A series of deep learning (DL) models that suit computer vision tasks, namely the convolutional neural networks (CNNs), are used to predict the compressive strength of ready-mixed concrete. To demonstrate the efficacy of computer vision-based prediction, its effectiveness using imaging numerical data was compared with that of the deep neural networks (DNNs) technique that uses conventional numerical data. Various DL prediction models were compared and the best ones were identified with the relevant concrete datasets. The best DL models were then optimized by fine-tuning their hyperparameters using a newly developed bio-inspired metaheuristic algorithm, called jellyfish search optimizer, to enhance the accuracy and reliability. Analytical experiments indicate that the computer vision-based CNNs outperform the numerical data-based DNNs in all evaluation metrics except the training time. Thus, the bio-inspired optimization of computer vision-based convolutional neural networks is potentially a promising approach to predict the compressive strength of ready-mixed concrete.

A Study on the Impact of Memory DoS Attacks on Cloud Applications and Exploring Real-Time Detection Schemes

Even though memory denial-of-service attacks can cause severe performance degradations on <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">co-located</i> virtual machines, a previous detection scheme against such attacks cannot accurately detect the attacks and also generates high detection delay and high performance overhead since it assumes that cache-related statistics of an application follow the same probability distribution at all times, which may not be true for all types of applications. In this paper, we present the experimental results showing the impacts of memory DoS attacks on different types of cloud-based applications. Based on these results, we propose two lightweight and responsive Statistical based Detection Schemes (SDS/B and SDS/P) that can detect such attacks accurately. SDS/B constructs a profile of normal range of cache-related statistics for all applications and use statistical methods to infer an attack when the real-time collected statistics exceed this normal range, while SDS/P exploits the increased periods of access patterns for periodic applications to infer an attack. Upon SDS, we further leverage deep neural network (DNN) techniques to design a DNN-based detection scheme that is general to various types of applications and more robust to adaptive attack scenarios. Our evaluation results show that SDS/B, SDS/P and DNN outperform the state-of-the-art detection scheme, e.g., with 65% higher specificity, 40% shorter detection delay, and 7% less performance overhead. We also discuss how to use SDS and DNN-based detection schemes under different situations.

Application of Convolution Neural Network for Adaptive Traffic Controller System

This paper presents an adaptive traffic control system based on deep convolution neural network (DCNN) technique for a multimodal traffic environment. The deep learning controller technique is based on feature mapping to estimate the optimal state-action value function. The controller executes acyclic phase assignment based on minimum green time duration for traffic signal operation. Moreover, the research implements Discrete Lane Cells (DLC) approach for state representation. A number of traffic performance measures were examined and selected traffic conditions were tested. The findings indicate that the DCNN system has superior performance in over-saturated traffic condition. The DCNN agent has achieved significant 85% to 95% lower waiting time, l7% to 38% shorter travel time, and it has mitigated the highest median flow rate at 295 veh/s in comparison to other traffic systems. For under-saturated test scenarios, a fair comparable performance is measured for the proposed controller. Overall, the DCNN system provides stable performance across all the tested signal junction scenarios in comparison to other controller systems.