Application Of Deep Neural Networks Research Articles

Application of deep neural survival networks to the development of risk prediction models for diabetes mellitus, hypertension, and dyslipidemia.

: Although numerous risk prediction models have been proposed, few such models have been developed using neural network-based survival analysis. We developed risk prediction models for three cardiovascular disease risk factors (diabetes mellitus, hypertension, and dyslipidemia) among a working-age population in Japan using DeepSurv, a deep feed-forward neural network. : Data were obtained from the Japan Epidemiology Collaboration on Occupational Health Study. A total of 51 258, 44 197, and 31 452 individuals were included in the development of risk models for diabetes mellitus, hypertension, and dyslipidemia, respectively; two-thirds of whom were used to develop prediction models, and the rest were used to validate the models. We compared the performances of DeepSurv-based models with those of prediction models based on the Cox proportional hazards model. : The area under the receiver-operating characteristic curve was 0.878 [95% confidence interval (CI) = 0.864-0.892] for diabetes mellitus, 0.835 (95% CI = 0.826-0.845) for hypertension, and 0.826 (95% CI = 0.817-0.835) for dyslipidemia. Compared with the Cox proportional hazards-based models, the DeepSurv-based models had better reclassification performance [diabetes mellitus: net reclassification improvement (NRI) = 0.474, P ≤ 0.001; hypertension: NRI = 0.194, P ≤ 0.001; dyslipidemia: NRI = 0.397, P ≤ 0.001] and discrimination performance [diabetes mellitus: integrated discrimination improvement (IDI) = 0.013, P ≤ 0.001; hypertension: IDI = 0.007, P ≤ 0.001; and dyslipidemia: IDI = 0.043, P ≤ 0.001]. : This study suggests that DeepSurv has the potential to improve the performance of risk prediction models for cardiovascular disease risk factors.

Application of Deep Neural Network and Human-Computer Interaction Technology Based on Multimodal Perception in Art Design

Abstract The essence of art design is the process of emotional interaction with humans. In this paper, emotions are classified using multimodal fusion features, and abstract fusion features are obtained by sampling the multimodal matching tensor using average pooling. The matching fusion matrix in the tensor operator is used to convert from two-modal to multi-modal matching. Virtual interaction model in art design controls the design objectives, and the optimized virtual world is constructed by using virtual reality technology, so as to build an immersive art design model. Finally, a study was conducted to examine the impact of the use of emotion perception and interaction technology in art product design with students from the School of Design and Art, University of G. The results show that the liking degree of three-color matching in the cognitive experiment test is 0.307, which is higher than the liking degree of two-color matching of 0.223, indicating that overall three-color matching samples are more popular in art design. This study provides effective evaluation and guidance for designing art products that are emotional and innovative.

Semantic-Layout-Guided Image Synthesis for High-Quality Synthetic-Aperature Radar Detection Sample Generation

With the widespread application and functional complexity of deep neural networks (DNNs), the demand for training samples is increasing. This elevated requirement also extends to DNN-based SAR object detection. Most public SAR object detection datasets are oriented to marine targets such as ships, while data sets oriented to land targets are relatively rare, though they are an effective way to improve the land object detection capability of deep models through SAR sample generation. In this paper, a synthesis generation collaborative SAR sample augmentation framework is proposed to achieve flexible and diverse high-quality sample augmentation. First, a semantic-layout-guided image synthesis strategy is proposed to generate diverse detection samples. The issues of object location rationality and object layout diversity are also addressed. Meanwhile, a pix2pixGAN network guided by layout maps is utilized to achieve diverse background augmentation. Second, a progressive training strategy of diffusion models is proposed to achieve semantically controllable SAR sample generation to further improve the diversity of scene clutter. Finally, a sample cleaning method considering distribution migration and network filtering is employed to further improve the quality of detection samples. The experimental results show that this semantic synthesis generation method can outperform existing sample augmentation methods, leading to a comprehensive improvement in the accuracy metrics of classical detection networks.

Hashing Fake: Producing Adversarial Perturbation for Online Privacy Protection Against Automatic Retrieval Models

The wide application of deep neural networks (DNNs) has significantly improved the performance of hashing models on multimodal retrieval issues. DNN-based deep models can automatically learn semantic features from raw data to make human-level decisions. However, the superior generalization leads to potential privacy leakage risks. Strong DNN-based retrieval models enable malicious crawlers to search for nontag private information based on semantic similarity matching. Hence, executing effective privacy protection mechanisms against those retrieval software is essential for reliable social website construction. In this article, we propose a retrieval task-based adversarial perturbation generation method called Hashing Fake to meet this request. Specifically, DNNs are recently found to be vulnerable to a specific set of attacks called adversarial perturbations, which denote some magnitude-restricted signals added on objective samples to misguide well-crafted DNN models, and perturbations’ magnitudes are small enough that will not induce humans’ perception. Moreover, since existing adversarial perturbation generation methods are designed for supervised tasks, Hashing Fake constructs a differential approximation substitution for perturbation production on unsupervised retrieval tasks. Through extensive experiments on several deep retrieval benchmarks, we demonstrate that well-crafted perturbations using Hashing Fake can effectively misguide objective models’ recognitions to make false predictions. The added norm-restricted perturbations on objective samples will not alter humans’ perception; hence, Hashing Fake can be applied on real-world social websites to protect subscribers’ privacy against malicious retrieval software.

Two‐stage architectural fine‐tuning for neural architecture search in efficient transfer learning

AbstractIn many deep neural network (DNN) applications, the difficulty of gathering high‐quality data in industry fields hinders the practical use of DNN. Thus, the concept of transfer learning (TL) has emerged, which leverages the pretrained knowledge of the DNN which was built based on large‐scale datasets. For this TL objective, this paper suggests two‐stage architectural fine‐tuning for reducing the costs and time while exploring the most efficient DNN model, inspired by neural architecture search (NAS). The first stage is mutation, which reduces the search costs using a priori architectural information. Moreover, the next stage is early‐stopping, which reduces NAS costs by terminating the search process in the middle of computation. The data‐intensive experimental results verify that the proposed method outperforms benchmarks.

Intracranial hemorrhage detection: how much can windowing, transfer learning and data augmentation affect a deep learning model’s performance?

The application of Deep Neural Networks to detect intracranial hemorrhage from computed tomography images has been widely used in clinical medicine. In general, building these solutions combines three types of approaches: Preprocessing, Transfer Learning, and Data Augmentation. This study’s goal was to measure the contribution of each of these approaches and thus highlight which approach has more margin of improvement. An experimental study was conducted on a public dataset containing computerized tomography images. The comparison used the stratified ten-fold cross-validation process to set confidence intervals evaluating performance measured by the area under the receiver operating characteristic curve. The ResNet-50 was the deep learning model selected. The results showed that all the approaches raise the generalization power when applied in isolation, and Data Augmentation offers the most significant gain to the baseline. The experiment also showed an opportunity to improve the detection of intracranial hemorrhage by applying new preprocessing techniques since this was the approach that showed the smallest increase in discriminatory power among the investigated approaches. The paired Wilcoxon’s Signed-Rank Test showed that not all the differences were statistically significant, with a confidence level of 95%.

Application of Deep Neural Network with Frequency Domain Filtering in the Field of Intrusion Detection

In the field of intrusion detection, existing deep learning algorithms have limited capability to effectively represent network data features, making it challenging to model the complex mapping relationship between network data and attack behavior. This limitation, in turn, impacts the detection accuracy of intrusion detection systems. To address this issue and further enhance detection accuracy, this paper proposes an algorithm called the Fourier Neural Network (FNN). The core of FNN consists of a Deep Fourier Neural Network Block (DFNNB), which is composed of a Hadamard Neural Network (HNN) and a Fourier Neural Network Layer (FNNL). In a DFNNB, the HNN is responsible for sampling the network intrusion data samples in different time domain spaces. The FNNL, on the other hand, performs a Fourier transform on the samples outputted by the HNN and maps them to the frequency domain space, followed by a filtering process. Finally, the data processed by filtering are transformed back to the time domain space for subsequent feature extraction work by the DFNNB. Additionally, to enhance the algorithm’s detection accuracy and filter out noise signals, this paper also introduces a High-energy Filtering Process (HFP), which eliminates noise signals from the data signal and reduces interference on the final detection result. Due to the ability of FNN to process network data in both the time domain space and the frequency domain space, it possesses a stronger capability in expressing data features. Finally, this paper conducts performance evaluations on the KDD Cup99, NSL-KDD, UNSW-NB15, and CICIDS2017 datasets. The results demonstrate that the proposed FNN-based IDS model achieves higher detection rates, lower false alarm rates, and better detection performance than classical deep learning and machine learning methods.

Unifying Gradients to Improve Real-World Robustness for Deep Networks

The wide application of deep neural networks (DNNs) demands an increasing amount of attention to their real-world robustness, i.e., whether a DNN resists black-box adversarial attacks, among which score-based query attacks (SQAs) are the most threatening since they can effectively hurt a victim network with only access to model outputs. Defending against SQAs requires a slight but artful variation of outputs due to the service purpose for users, who share the same output information with SQAs. In this article, we propose a real-world defense by Unifying Gradients (UniG) of different data so that SQAs could only probe a much weaker attack direction that is similar for different samples. Since such universal attack perturbations have been validated as less aggressive than the input-specific perturbations, UniG protects real-world DNNs by indicating to attackers a twisted and less informative attack direction. We implement UniG efficiently by a Hadamard product module, which is plug-and-play. According to extensive experiments on 5 SQAs, 2 adaptive attacks and 7 defense baselines, UniG significantly improves real-world robustness without hurting clean accuracy on CIFAR10 and ImageNet. For instance, UniG maintains a model of 77.80% accuracy under a 2500-query Square attack while the state-of-the-art adversarially trained model only has 67.34% on CIFAR10. Simultaneously, UniG outperforms all compared baselines in terms of clean accuracy and achieves the smallest modification of the model output. The code is released at https://github.com/snowien/UniG-pytorch .

Application of Multiple Deep Neural Networks to Multi-Solution Synthesis of Linkage Mechanisms

This paper studies the problem of linkage-bar synthesis by means of multiple deep neural networks (DNNs), which requires the inverse solution of linkage parameters based on a desired trajectory curve. This problem is highly complex due to the fact that the solution space is nonlinear and may contain multiple solutions, while a good quality of learning cannot be obtained by a single neural network approach. Therefore, this paper proposes employing Fourier descriptors to represent trajectory curves in a systematic and normalized form, developing a multi-solution distribution evaluation by random restart local searches (MDE-RRLS) to examine a better solution-space partitioning scheme, utilizing multiple DNNs to learn subspace regions separately, and creating a multi-facet query (MFQuery) to cooperatively predict multiple solutions. The experiments demonstrate that the proposed approach can obtain better or at least competitive outcomes compared to previous work in the literature. Furthermore, to verify the effectiveness and applicability, this paper investigates the design problem of an industrial six-linkage-bar ladle mechanism used in a die-casting system, and the proposed method can obtain several superior design solutions and offer alternatives in a short period of time when faced with redesign requirements.

FTG: Score-based black-box watermarking by fragile trigger generation for deep model integrity verification

Deep neural networks (DNNs) are widely used in real-world applications, thanks to their exceptional performance in image recognition. However, their vulnerability to attacks, such as Trojan and data poison, can compromise the integrity and stability of DNN applications. Therefore, it is crucial to verify the integrity of DNN models to ensure their security. Previous research on model watermarking for integrity detection has encountered the issue of overexposure of model parameters during embedding and extraction of the watermark. To address this problem, we propose a novel score-based black-box DNN fragile watermarking framework called fragile trigger generation (FTG). The FTG framework only requires the prediction probability distribution of the final output of the classifier during the watermarking process. It generates different fragile samples as the trigger, based on the classification prediction probability of the target classifier and a specified prediction probability mask to watermark it. Different prediction probability masks can promote the generation of fragile samples in corresponding distribution types. The whole watermarking process does not affect the performance of the target classifier. When verifying the watermarking information, the FTG only needs to compare the prediction results of the model on the samples with the previous label. As a result, the required model parameter information is reduced, and the FTG only needs a few samples to detect slight modifications in the model. Experimental results demonstrate the effectiveness of our proposed method and show its superiority over related work. The FTG framework provides a robust solution for verifying the integrity of DNN models, and its effectiveness in detecting slight modifications makes it a valuable tool for ensuring the security and stability of DNN applications.

Applications of deep neural networks in exploration seismology: A technical survey

Exploration seismology uses reflected and refracted seismic waves, emitted from a controlled (active) source into the ground, and recorded by an array of seismic sensors (receivers) to image the subsurface geologic structures. These seismic images are the main resources for energy and resource exploration and scientific investigation of the crust and upper mantle. We survey recent advances in applications of machine-learning methods, more specifically deep neural networks (DNNs), in exploration seismology. We provide a technically oriented review of DNN applications for seismic data acquisition; data preprocessing tasks such as interpolation/extrapolation, denoising, first-break picking, velocity picking, and seismic migration; data processing tasks such as geologic and structural interpretations; and data modeling tasks such as the inference of subsurface structures and lithologic and petrophysical properties. DNNs have entered almost every sector of exploration seismology. They have outperformed many traditional algorithms for the automation of seismic data acquisition, data preprocessing, data processing, interpretations, and data modeling tasks. However, despite the impressive performances of DNN-based approaches, the out-of-distribution generalization and interpretability of these models remain challenging. To overcome these challenges, incorporating domain knowledge into the DNNs is a promising path and a focus of current deep-learning research in seismology.

Robust Facial Emotion Recognition using Marine Predators Algorithm with Deep Convolutional Neural Network

Facial emotion recognition (FER) is a technology that includes the automatic identification and categorization of human sentiments depending on facial emotions. It leverages deep learning (DL), computer vision (CV), and machine learning (ML) methods to analyze the features of an individual's face, like the place of the mouth, eyes, eyebrows, and complete facial movements for determining their emotional conditions. Popular emotions that FER can recognize comprise surprise, sadness, happiness, fear, disgust, and anger. FER employing DL has an advanced application of artificial intelligence (AI) and deep neural networks (DNNs) that contain training methods to automatically recognize and categorize human expressions based on facial expressions. This approach normally comprises convolutional neural networks (CNNs) or highly complex models namely recurrent neural networks (RNNs) and convolutional RNNs (CRNNs) for analyzing and interpreting complex facial features and dynamics. This study introduces a new Robust Facial Emotion Recognition employing the Marine Predators Algorithm with Deep Learning (RFER-MPADL) approach. The main aim of the RFER-MPADL technique is to detect and categorize different kinds of emotions expressed in facial images. To accomplish this, the RFER-MPADL technique initially applies a bilateral filtering (BF) approach for the preprocessing step. Additionally, the RFER-MPADL technique uses the EfficientNet-B0 method for feature extraction. Moreover, the tuning process of the EfficientNet-B0 method was implemented using the MPA. Finally, the classification of facial emotions can be performed by the use of a deep autoencoder (DAE), in turn augments the overall performance of the RFER-MPADL method. The experimental analysis of the RFER-MPADL methodology is assessed on a standard facial expression dataset. The extensive outcomes exhibited the effectual performance of the RFER-MPADL methodology over other methods.

Ultrasonic characterization of small defects based on Res-ViT and unsupervised domain adaptation

This paper investigated the application of deep neural networks and domain adaptation for ultrasonic characterization of elliptical defects that are small and inclined. Based on performance evaluation of deep residual network (ResNet) and vision transformer (ViT), we proposed a novel Res-ViT architecture which fuses deep representative features of both models. Furthermore, we developed an unsupervised domain adaptation method to minimize the distance between the source and target domains, which is measured by maximum mean discrepancy. This approach serves to improve the generalizability of the proposed Res-ViT model in noisy environments. Simulation studies were performed at various noise levels to evaluate robustness of different deep neural networks. The proposed Res-ViT model was shown to reduce the characterization uncertainty of various defect parameters, including size, angle, and aspect ratio. Experiments were performed on three elliptical defects which have large orientation angles of 60∘ relative to the array direction. The proposed method achieved a 61% reduction in the root-mean-square error (RMSE) of defect size compared to a benchmark approach, which is based on principal component analysis and the nearest neighbor method.

Relative frequencies of perovskite solar cells strengthened by composite materials: Application of deep neural networks for solving the vibration problem

Perovskite solar cells have shown competitive power conversion efficiencies (PCE) with the potential for better performance when compared to the most widely used photovoltaic technology. Perovskites may deteriorate when exposed for an extended amount of time to light, heat, or electrical current, as well as when they interact with moisture and oxygen. To increase stability, researchers are examining deterioration in both the perovskite material itself and the surrounding device layers. Cell durability has to be increased in order to produce perovskite solar goods for sale. Improved cell durability is critical for the development of commercial perovskite solar products. So, improving the dynamic properties of this kind of green energy device is a key issue in engineering. In the current report, the stability zone and relative frequency change of the perovskite solar cells reinforced by three-dimensional functionally graded material (3D-FGM) as the last layer is presented. For exact modeling of the perovskite solar cell structure due to its size (micro size), it is crucial to take the size effect into account. Modified couple stress theory (MCST) as a size-dependent theory for micro-scale systems with curvature derivate parameters is offered. Once the study’s findings have mastered the mathematical discoveries, they are instructed to produce the machine learning technique’s outputs. In actuality, the fast-converging numerical approach is used to derive the characteristics of the design points. The momentum-based optimizer is used to find the optimal values of the parameters influencing how the fully connected neural network functions. The method used in this research is efficient since it quickly and properly trains the multi-layered network. The funding now available may be used to promote future research on the importance of the dynamic stability of perovskite solar cells employing 3D-FG material as the constructed layer.

S2AC: Self-Supervised Attention Correlation Alignment Based on Mahalanobis Distance for Image Recognition

Susceptibility to domain changes for image classification hinders the application and development of deep neural networks. Domain adaptation (DA) makes use of domain-invariant characteristics to improve the performance of a model trained on labeled data from one domain (source domain) on an unlabeled domain (target) with a different data distribution. But existing DA methods simply use pretrained models (e.g., AlexNet, ResNet) for feature extraction, which are convolutional models that are trapped in localized features and fail to acquire long-distance dependencies. Furthermore, many approaches depend too much on pseudo-labels, which can impair adaptation efficiency and lead to unstable and inconsistent results. In this research, we present S2AC, a novel approach for unsupervised deep domain adaptation, that makes use of a stacked attention architecture as a feature map extractor. Our method can fuse domain discrepancy with minimizing a linear transformation of the second statistics (covariances) extended by the p-norm, while simultaneously designing pretext tasks on heuristics to improve the generality of the learning representation. In addition, we have developed a new trainable relative position embedding that not only reduces the model parameters but also enhances model accuracy and expedites the training process. To illustrate our method’s efficacy and controllability, we designed extensive experiments based on the Office31, Office_Caltech_10, and OfficeHome datasets. To the best of our knowledge, the proposed method is the first attempt at incorporating attention-based networks and self-supervised learning for image domain adaptation, and has shown promising results.

Research on improving the generalization ability of NLP categorization models: A comparison of Mixup and generative adversarial networks

The application of deep neural networks in tasks related to natural language processing has grown in recent years. However, training deep learning models often requires a large amount of data to produce accurate and generalizable results. When the data are insufficient, categorization models may suffer from overfitting or poor performance. This paper compares two methods for enhancing the generalization ability of natural language processing categorization models when data are insufficient: Mixup and Generative Adversarial Networks (GANs). This paper uses the Internet Movie Database (IMDB) dataset for a binary classification task, Mixup method, and Relational Generative Adversarial Networks (RelGAN) model to generate new data, respectively, and compares them with the original dataset. Experimental results indicate that Mixup method can reduce overfitting risk effectively and enhance the models robustness, while the GAN model does not show obvious advantages. This paper supports the idea that using noise-adding operations to enhance the dataset is feasible in the case of small samples rather than using generative models.

FlowDyn: A daily streamflow prediction pipeline for dynamical deep neural network applications

This paper presents a dynamical neural network framework to understand how catchment systems respond to daily rainfall-runoff processes over time. We developed an interactive Python-based deep neural network (DNN) package called FlowDyn (presented through a JS-based web platform) to simulate and forecast daily streamflow data for >180 gauging stations across the globe. Several DNN models, including long short-term memory (LSTM), Gated Recurrent Unit (GRU), and a hybrid network of convolutional neural network and LSTM (ConvLSTM), as well as an auto encoder (AE) model were developed and integrated into the FlowDyn pipeline to analyze and forecast sequential daily streamflow values that are embedded within a web-based application for demonstration and visualization. Inputs were gathered from different web services, including the catchment attributes and meteorology for large-sample studies (CAMELS), the national climatic data center (NCDC), and the global runoff data center (GRDC). DNN configurations were trained and tested with an average accuracy rating of 0.83 across 183 river basins globally. FlowDyn simulation and performance demonstrated that different DNN models were able to learn both regionally consistent and location-specific hydrological behaviors. Through the findings of this paper, we advocate the merit of applying FlowDyn package in the field of daily rainfall-runoff prediction at both local and global scales.

How Deep Learning and Neural Networks can Improve Prosthetics and Exoskeletons: A Review of State-of-the-Art Methods and Challenges

Deep learning and neural networks are powerful computational methods that have been widely applied in various fields, such as healthcare and robotics. In this paper, we review some of the recent research studies that use deep learning and neural networks in healthcare and robotics, particularly focusing on their application in prosthetics and exoskeletons. The main source of data for this review is Scopus, which is a large and multidisciplinary database of peer-reviewed literature. The search criteria for this review are exoskeleton AND prosthetic AND deep AND learning. The search is limited to documents published from 2014 to 2023, as this period covers the recent developments and trends in the field4. The search results in 488 documents that match the criteria. We selected 20 papers that represent the state-of-the-art methods and applications of deep learning and neural networks in prosthetics and exoskeletons. We categorized these papers by various attributes, such as document type, subject area, sensor type, respondent, condition, etc. The main finding of this paper was that deep learning techniques and neural networks have diverse and transformative potential in healthcare and robotics, especially in the development and improvement of prosthetics and exoskeletons. The paper highlighted how these advanced computational methods can be harnessed to interpret complex biological signals, improve device functionality, enhance user safety, and ultimately improve quality of life for individuals using these devices. The paper also identified some possible future directions for this topic, such as exploring the impact of deep learning techniques and neural networks on the performance, usability, and user satisfaction of prosthetics and exoskeletons. This paper provided a valuable insight into the current state-of-the-art and future prospects of deep learning techniques and neural networks in healthcare and robotics

Online Meta-Learning for Hybrid Model-Based Deep Receivers

Recent years have witnessed growing interest in the application of deep neural networks (DNNs) for receiver design, which can potentially be applied in complex environments without relying on knowledge of the channel model. However, the dynamic nature of communication channels often leads to rapid distribution shifts, which may require periodically retraining. This paper formulates a data-efficient two-stage training method that facilitates rapid online adaptation. Our training mechanism uses a predictive meta-learning scheme to train rapidly from data corresponding to both current and past channel realizations. Our method is applicable to any deep neural network (DNN)-based receiver, and does not require transmission of new pilot data for training. To illustrate the proposed approach, we study DNN-aided receivers that utilize an interpretable model-based architecture, and introduce a modular training strategy based on predictive meta-learning. We demonstrate our techniques in simulations on a synthetic linear channel, a synthetic non-linear channel, and a COST 2100 channel. Our results demonstrate that the proposed online training scheme allows receivers to outperform previous techniques based on self-supervision and joint-learning by a margin of up to 2.5 dB in coded bit error rate in rapidly-varying scenarios.

Active attenuation of free field noise in a spatial domain using neural networks inferred controller

In this talk, we explore the application of deep neural networks (DNNs) to infer the decoupling of an incoming acoustic array, distinguishing between the signal of interest and noise components. The objective is to leverage this decoupling information to develop an effective controller that can attenuate the noise, thereby improving the quality of the desired signal. The decoupled noise is then fed into a specially designed controller to attenuate the noise while preserving the desired signal. The controller leverages the DNN's predictions to adaptively adjust its parameters, allowing it to adapt to changing noise conditions and improve noise reduction performance. The approach is evaluated using simulations and real-world acoustic array measurements. The results demonstrate the performance of the DNN-based decoupling and noise attenuation system in reducing noise while preserving the quality of the desired signal. The proposed system offers effective adaptive solutions for enhancing the quality of desired signals in the presence of noise, with potential applications in areas such as speech recognition, audio communication systems, and environmental noise suppression.