Deep Neural Network-based Methods Research Articles

A hybrid physics-informed machine learning approach for time-dependent reliability assessment of electromagnetic relays

Electromagnetic relays (EMRs) are intricate micro-electromechanical systems characterized by nonlinear behavior and coupling effects between electromagnetic and mechanical forces. Accurately modeling degradation and assessing reliability are crucial yet challenging tasks for ensuring their safe and efficient operation. Current data-driven methods for degradation modeling and reliability assessment often neglect the known physical knowledge regarding EMRs, leading to inaccuracies in modeling and assessment outcomes when data is incomplete. While physics-informed machine learning (PIML) approaches offer a potential solution, common regression models like Gaussian processes (GP) and long short-term memory (LSTM) suffer from underfitting and overfitting, respectively. To address these issues, we presents a hybrid PIML approach for time-dependent reliability assessment based on the emerging variational autoencoder (VAE) framework. This approach combines the advantages of GP-based methods that enable probabilistic representation with deep neural network-based methods that are more flexible and computationally efficient. Finally, we validate our proposed approach using real-world engineering data, demonstrating its superior accuracy and computational efficiency compared to state-of-the-art methods.

High Fidelity Makeup via 2D and 3D Identity Preservation Net

In this article, we address the challenging makeup transfer task, aiming to transfer makeup from a reference image to a source image while preserving facial geometry and background consistency. Existing deep neural network-based methods have shown promising results in aligning facial parts and transferring makeup textures. However, they often neglect the facial geometry of the source image, leading to two adverse effects: (1) alterations in geometrically relevant facial features, causing face flattening and loss of personality, and (2) difficulties in maintaining background consistency, as networks cannot clearly determine the face-background boundary. To jointly tackle these issues, we propose the High Fidelity Makeup via two-dimensional (2D) and 3D Identity Preservation Network (IP23-Net), to the best of our knowledge, a novel framework that leverages facial geometry information to generate more realistic results. Our method comprises a 3D Shape Identity Encoder, which extracts identity and 3D shape features. We incorporate a 3D face reconstruction model to ensure the three-dimensional effect of face makeup, thereby preserving the characters’ depth and natural appearance. To preserve background consistency, our Background Correction Decoder automatically predicts an adaptive mask for the source image, distinguishing the foreground and background. In addition to popular benchmarks, we introduce a new large-scale High Resolution Synthetic Makeup Dataset containing 335,230 diverse high-resolution face images to evaluate our method’s generalization ability. Experiments demonstrate that IP23-Net achieves high-fidelity makeup transfer while effectively preserving background consistency. The code will be made publicly available.

A video object detector with Spatio-Temporal Attention Module for micro UAV detection

Many deep neural network-based methods have recently been proposed for object detection due to the significant success of deep learning in computer vision. However, existing object detection methods typically extract the appearance features of objects from single image so that they usually suffer from poor performance in detecting micro Unmanned Aerial Vehicle (UAV), because micro UAV lacks of rich color, shape and texture information. To address this issue, we introduce the temporal information of objects from videos and develop a Spatio-Temporal Attention Module (STAM) to efficiently enhance feature map extraction for detecting micro UAV, and then integrate STAM into YOLOX to develop a video object detector for micro UAV. Meanwhile, we propose a lightweight Spatial Pyramid Pooling (SPP) module termed Group Simplified Spatial Pyramid Pooling-Fast with Cross Stage Partial (Group SimSPPFCSP) for the backbone’s final stage layer to efficiently and lightly extract more semantic information, and we propose a neck with rich propagation pathways (NRPP) to facilitate the effective propagation of spatial and temporal information across different levels. Furthermore, we propose two data augmentation operations including SeqMosaic and SeqMixUp, to augment video data for video object detection. Experimental results show that our model can achieve competitive precision (with 5.0 mAP and 8.1 mAPSmall improvement) while maintaining real-time inference speed (35.3 fps).

FlexDTI: flexible diffusion gradient encoding scheme-based highly efficient diffusion tensor imaging using deep learning

Objective. Most deep neural network-based diffusion tensor imaging methods require the diffusion gradients’ number and directions in the data to be reconstructed to match those in the training data. This work aims to develop and evaluate a novel dynamic-convolution-based method called FlexDTI for highly efficient diffusion tensor reconstruction with flexible diffusion encoding gradient scheme. Approach. FlexDTI was developed to achieve high-quality DTI parametric mapping with flexible number and directions of diffusion encoding gradients. The method used dynamic convolution kernels to embed diffusion gradient direction information into feature maps of the corresponding diffusion signal. Furthermore, it realized the generalization of a flexible number of diffusion gradient directions by setting the maximum number of input channels of the network. The network was trained and tested using datasets from the Human Connectome Project and local hospitals. Results from FlexDTI and other advanced tensor parameter estimation methods were compared. Main results. Compared to other methods, FlexDTI successfully achieves high-quality diffusion tensor-derived parameters even if the number and directions of diffusion encoding gradients change. It reduces normalized root mean squared error by about 50% on fractional anisotropy and 15% on mean diffusivity, compared with the state-of-the-art deep learning method with flexible diffusion encoding gradient scheme. Significance. FlexDTI can well learn diffusion gradient direction information to achieve generalized DTI reconstruction with flexible diffusion gradient scheme. Both flexibility and reconstruction quality can be taken into account in this network.

Research on the application of graph neural networks in text classification

With the rapid development of the Internet and social media, a large amount of text data is constantly generated, and text classification has become an important task. Text classification is an important branch in the field of natural language processing and has attracted the attention of many researchers in recent years. Due to the ease of editing text data, most texts in the network are manually constructed and uploaded by users. Therefore, the standardization of online texts and the classification of texts with different granularity are of great significance in the field of information retrieval. This article introduces the basic concepts of text classification and three text classification methods: knowledge engineering (rule) based text classification methods, traditional machine learning-based text classification methods, and deep neural network-based text classification methods. This article also analyzes the advantages and disadvantages of the three methods. Subsequently, the basic concepts and common models of graph neural networks were explored. An application case of graph neural networks in text classification is provided. This article provides effective suggestions for text classification applications in the field of natural language processing.

Hyperspectral image dynamic range reconstruction using deep neural network-based denoising methods

Hyperspectral (HS) measurement is among the most useful tools in agriculture for early disease detection. However, the cost of HS cameras that can perform the desired detection tasks is prohibitive-typically fifty thousand to hundreds of thousands of dollars. In a previous study at the Agricultural Research Organization’s Volcani Institute (Israel), a low-cost, high-performing HS system was developed which included a point spectrometer and optical components. Its main disadvantage was long shooting time for each image. Shooting time strongly depends on the predetermined integration time of the point spectrometer. While essential for performing monitoring tasks in a reasonable time, shortening integration time from a typical value in the range of 200 ms to the 10 ms range results in deterioration of the dynamic range of the captured scene. In this work, we suggest correcting this by learning the transformation from data measured with short integration time to that measured with long integration time. Reduction of the dynamic range and consequent low SNR were successfully overcome using three developed deep neural networks models based on a denoising auto-encoder, DnCNN and LambdaNetworks architectures as a backbone. The best model was based on DnCNN using a combined loss function of ℓ2\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\ell _{2}$$\\end{document} and Kullback–Leibler divergence on images with 20 consecutive channels. The full spectrum of the model achieved a mean PSNR of 30.61 and mean SSIM of 0.9, showing total improvement relatively to the 10 ms measurements’ mean PSNR and mean SSIM values by 60.43% and 94.51%, respectively.

Contrastive Multiscale Transformer for Image Dehazing.

Images obtained in an unfavorable environment may be affected by haze or fog, leading to fuzzy image details, low contrast, and loss of important information. Recently, significant progress has been achieved in the realm of image dehazing, largely due to the adoption of deep learning techniques. Owing to the lack of modules specifically designed to learn the unique characteristics of haze, existing deep neural network-based methods are impractical for processing images containing haze. In addition, most networks primarily focus on learning clear image information while disregarding potential features in hazy images. To address these limitations, we propose an innovative method called contrastive multiscale transformer for image dehazing (CMT-Net). This method uses the multiscale transformer to enable the network to learn global hazy features at multiple scales. Furthermore, we introduce feature combination attention and a haze-aware module to enhance the network's ability to handle varying concentrations of haze by assigning more weight to regions containing haze. Finally, we design a multistage contrastive learning loss incorporating different positive and negative samples at various stages to guide the network's learning process to restore real and non-hazy images. The experimental findings demonstrate that CMT-Net provides exceptional performance on established datasets and exhibits superior visual outcomes.

Tracking an untracked space debris after an inelastic collision using physics informed neural network

With the sustained rise in satellite deployment in Low Earth Orbits, the collision risk from untracked space debris is also increasing. Often small-sized space debris (below 10 cm) are hard to track using the existing state-of-the-art methods. However, knowing such space debris’ trajectory is crucial to avoid future collisions. We present a Physics Informed Neural Network (PINN)—based approach for estimation of the trajectory of space debris after a collision event between active satellite and space debris. In this work, we have simulated 8565 inelastic collision events between active satellites and space debris. To obtain the states of the active satellite, we use the TLE data of 1647 Starlink and 66 LEMUR satellites obtained from space-track.org. The velocity of space debris is initialized using our proposed velocity sampling method, and the coefficient of restitution is sampled from our proposed Gaussian mixture-based probability density function. Using the velocities of the colliding objects before the collision, we calculate the post-collision velocities and record the observations. The state (position and velocity), coefficient of restitution, and mass estimation of un-tracked space debris after an inelastic collision event along with the tracked active satellite can be posed as an optimization problem by observing the deviation of the active satellite from the trajectory. We have applied the classical optimization method, the Lagrange multiplier approach, for solving the above optimization problem and observed that its state estimation is not satisfactory as the system is under-determined. Subsequently, we have designed Deep Neural network-based methods and Physics Informed Neural Network (PINN) based methods for solving the above optimization problem. We have compared the performance of the models using root mean square error (RMSE) and interquartile range of the predictions. It has been observed that the PINN-based methods provide a better estimation performance for position, velocity, mass and coefficient of restitution of the space debris compared to other methods.

DM-Fusion: Deep Model-Driven Network for Heterogeneous Image Fusion.

Heterogeneous image fusion (HIF) is an enhancement technique for highlighting the discriminative information and textural detail from heterogeneous source images. Although various deep neural network-based HIF methods have been proposed, the most widely used single data-driven manner of the convolutional neural network always fails to give a guaranteed theoretical architecture and optimal convergence for the HIF problem. In this article, a deep model-driven neural network is designed for this HIF problem, which adaptively integrates the merits of model-based techniques for interpretability and deep learning-based methods for generalizability. Unlike the general network architecture as a black box, the proposed objective function is tailored to several domain knowledge network modules to model the compact and explainable deep model-driven HIF network termed DM-fusion. The proposed deep model-driven neural network shows the feasibility and effectiveness of three parts, the specific HIF model, an iterative parameter learning scheme, and data-driven network architecture. Furthermore, the task-driven loss function strategy is proposed to achieve feature enhancement and preservation. Numerous experiments on four fusion tasks and downstream applications illustrate the advancement of DM-fusion compared with the state-of-the-art (SOTA) methods both in fusion quality and efficiency. The source code will be available soon.

PL-Net: progressive learning network for medical image segmentation.

In recent years, deep convolutional neural network-based segmentation methods have achieved state-of-the-art performance for many medical analysis tasks. However, most of these approaches rely on optimizing the U-Net structure or adding new functional modules, which overlooks the complementation and fusion of coarse-grained and fine-grained semantic information. To address these issues, we propose a 2D medical image segmentation framework called Progressive Learning Network (PL-Net), which comprises Internal Progressive Learning (IPL) and External Progressive Learning (EPL). PL-Net offers the following advantages: 1) IPL divides feature extraction into two steps, allowing for the mixing of different size receptive fields and capturing semantic information from coarse to fine granularity without introducing additional parameters; 2) EPL divides the training process into two stages to optimize parameters and facilitate the fusion of coarse-grained information in the first stage and fine-grained information in the second stage. We conducted comprehensive evaluations of our proposed method on five medical image segmentation datasets, and the experimental results demonstrate that PL-Net achieves competitive segmentation performance. It is worth noting that PL-Net does not introduce any additional learnable parameters compared to other U-Net variants.

Deep Neural Network-based Methods for Brain Image De-noising: A Short Comparison

Deep Neural Network-based Methods for Brain Image De-noising: A Short Comparison

A fatigue crack growth prediction method on small datasets based on optimized deep neural network and Delaunay data augmentation

The neural network-based prediction method has shown great potential in the field of fatigue crack growth prediction due to its powerful nonlinear processing and generalization capabilities. However, traditional deep neural network-based methods often encounter problems such as overfitting, underfitting, and instability when predicting FCG rates on small datasets. To address these issues, this study proposes a SA-DNN prediction method for metal fatigue crack growth by introducing Delaunay data augmentation and simulated annealing algorithm into the basic DNN framework. The proposed SA-DNN model is then used to predict the fatigue crack growth process of AM60B magnesium alloy and 7055 Al alloy, and the intrinsic and extrinsic generalization abilities of the model are evaluated. In both the evaluation of intrinsic and extrinsic generalization abilities, the mean squared errors of the training, validation, and testing sets rapidly converge to the target error in various epochs. The lowest mean squared errors achieved are 1.07 e-5 and 1.241 e-5, respectively. Moreover, the results also indicate that the proposed model can accurately predict the fatigue crack growth process of metal materials with varying degrees of nonlinearity.

NmPLS-Net: Segmenting Pulmonary Lobes Using nmODE

Pulmonary lobe segmentation is vital for clinical diagnosis and treatment. Deep neural network-based pulmonary lobe segmentation methods have seen rapid development. However, there are challenges that remain, e.g., pulmonary fissures are always not clear or incomplete, especially in the complex situation of the trilobed right pulmonary, which leads to relatively poor results. To address this issue, this study proposes a novel method, called nmPLS-Net, to segment pulmonary lobes effectively using nmODE. Benefiting from its nonlinear and memory capacity, we construct an encoding network based on nmODE to extract features of the entire lung and dependencies between features. Then, we build a decoding network based on edge segmentation, which segments pulmonary lobes and focuses on effectively detecting pulmonary fissures. The experimental results on two datasets demonstrate that the proposed method achieves accurate pulmonary lobe segmentation.

UNet-rootMUSIC: A high accuracy direction of arrival estimation method under array imperfection

In the practical direction-finding systems, the accuracy and resolution of the direction of arrival (DOA) estimation are affected not only by the Gaussian noise and array size but also by hardware configuration imperfections, such as errors in element manufacturing and mounting. These impairments cause phase and amplitude errors in estimating the DOA of signal sources. To address this issue, this paper proposes to combine a U-shape deep neural network (UNet) with the multiple signal classification via the root of the polynomial (rootMUSIC) algorithm (so-called UNet-rootMUSIC) to improve the DOA estimation accuracy. In this approach, the UNet model plays a role in converting a covariance matrix of received signals containing phase and gain errors into a nearly perfect one of the ideal antenna array. The rootMUSIC algorithm is then employed to estimate the signal DOA based on the converted covariance matrix. The DOA estimation performance of the uniform linear array of eight elements with an inter-element distance of λ/2 is analyzed through experimental simulations. The simulation results demonstrate that our method can significantly reduce the root mean square error of DOA estimation compared to the conventional MUSIC, rootMUSIC, ESPRIT methods and two deep neural network-based angular classification methods.

Automated Small Tumor Segmentation by a Template-Based Global Hierarchical Attention Method.

Accurate segmentation of tumors is significant for radiation therapy treatment planning and clinical decision-making. While deep convolutional neural network-based methods have found valuable applications in automatic medical segmentation, tumor segmentation, especially small tumor segmentation, remains challenging due to deficiencies of current deep learning in convolutional and pooling operations, which often results in the loss of small object information. This research proposes a global hierarchical attention-based method for accurate and automated segmentation of small tumors by exploiting the associations between small tumors and the feature maps of large tumors. This study included 131 patients with liver cancer. The in-plane resolution of the patients' CTs is from 0.55 mm to 1.0 mm and slice spacing from 0.45 mm to 6.0 mm. We randomly selected 100 CT scans as the training set and others as the testing set. Each CT slice of the testing set was separated into groups according to tumor size as follows: 0.1-2.0, 2.1-5.0, 5.1-10.0, and 10.1-20.0 cm. The CT slice without tumor or tumor size > 20 cm were excluded. This work presents a tumor template-based hierarchical attention method to quantify the relation between small and large tumors by computing their feature maps. The relation of small-large tumors can compensate for the information loss of small tumors during the convolutional and pooling operations and improve the performance of small tumor segmentation. Among 20,693 CT slices of the 31 testing patients, 3.0% CT slices with tumors ≤2 cm, 6.7% ≤5 cm, 10.6% ≤10 cm, and 13.4%≤20 cm. We compared our method with six widely used segmentation models. The results show our model outperforms other methods on all sizes of liver tumors, especially for small size tumors: For the 0.1-2.0 cm liver tumor, it achieved 8.4%, 10.0%, 11.3%, 9.1%, 10.9%, and 9.6% improvement compared to Unet, PAN, DeepLabV3, FPN, LinkNet, and PSPNet, respectively. We found that the small-large tumors relation can significantly improve small tumor segmentation, which is valuable for treatment planning, and clinical decision-making. Our experimental results show that our method can significantly improve the accuracy of segmenting small liver tumors compared to existing deep-learning-based models. The method is quite general and can be extended to other types of tumor detection and segmentation. We discovered that the relationship between small and large tumors can significantly enhance the segmentation of small tumors, which has significant value for treatment planning and clinical decision-making. Our experiments demonstrate that our approach significantly improves the accuracy of small liver tumor segmentation compared to existing deep learning-based models. Our method is quite versatile and can be extended to other types of tumor detection and segmentation.

Convenient intelligent diagnosis for rotating machinery: An improved deep forest method based on feature reconstruction

Due to the advantages of feature autoextraction and deep architecture, deep learning-based intelligent fault diagnosis has attracted increasing attention. However, a variety of complex hyper-parameter settings greatly limit its practical applications. Moreover, it is more critical and difficult to diagnose multiple mixed faults of rotating machinery under small training samples. To bridge these gaps, this paper proposes a convenient intelligent diagnostic method based on the improved deep forest, where a feature reconstruction algorithm is used to address the high computational cost and feature submergence caused by the long time series characteristics of vibration data. Comparison experiments with typical deep neural network-based methods are implemented, and the results validate the effectiveness and superiority of the proposed method, as well as the robustness of the hyper-parameters.

Learning Template-Constraint Real-Time Siamese Tracker for Drone AI Devices via Concatenation

Significant progress has been made in object tracking tasks thanks to the application of deep learning. However, current deep neural network-based object tracking methods often rely on stacking sub-modules and introducing complex structures to improve tracking accuracy. Unfortunately, these approaches are inefficient and limit the feasibility of deploying efficient trackers on drone AI devices. To address these challenges, this paper introduces ConcatTrk, a high-speed object tracking method designed specifically for drone AI devices. ConcatTrk utilizes a lightweight network architecture, enabling real-time tracking on edge devices. Specifically, the proposed method primarily uses the concatenation operation to construct its core tracking steps, including multi-scale feature fusion, intra-frame feature matching, and dynamic template updating, which aim to reduce the computational overhead of the tracker. To ensure tracking performance in UAV tracking scenarios, ConcatTrk implements a learnable feature matching operator along with a simple and efficient template constraint branch, which enables accurate tracking by discriminatively matching features and incorporating periodic template updates. Results of comprehensive experiments on popular benchmarks, including UAV123, OTB100, and LaSOT, show that ConcatTrk has achieved promising accuracy and attained a tracking speed of 41 FPS on an edge AI device, Nvidia AGX Xavier. ConcatTrk runs 8× faster than the SOTA tracker TransT while using 4.9× fewer FLOPs. Real-world tests on the drone platform have strongly validated its practicability, including real-time tracking speed, reliable accuracy, and low power consumption.

Level set guided region prototype rectification network for retinal vessel segmentation

Retinal vessel segmentation refers to extracting the vessel region with continuous and smooth boundaries from retinal images, which is of great significance in clinical practices. However, due to the weak and blurry edges of targets as well as interference (such as optic cup and disc) in the background, current deep neural network-based methods struggle in extracting features with discriminative semantics while preserving continuous edges. To enforce continuous predictions of weak edges, we propose a level set guided region prototype rectification (LSRPR) framework and a novel level set loss (LS-loss) with learnable and self-guided mechanisms. Specifically, the LSRPR firstly takes features of the last layer from the decoders of a U-Net version as input and rectified the region prototype by an auxiliary self-supervised level set loss, then the pre-trained model is fine-tuned by using supervised level set loss. The LS-loss facilitates the model to generate reliable guidance and enhances the continuous of edges among the decoders of neural network model. The proposed method is simple, yet effective, which can easily be extended to other frameworks. The quantitative and qualitative experimental results on public retinal vessel datasets indicate the effectiveness of the region prototype rectification compared to other SOTA models. Our code is available at Github:https://github.com/tweedlemoon/LSRPR.

Dual branch deep interactive UNet for monaural noisy-reverberant speech enhancement

Noise and reverberation can severely degrade speech quality and intelligibility, so many deep neural network-based noisy-reverberant speech enhancement methods have been proposed, among which classic methods include spectral masking and spectral mapping. Spectrum masking and spectrum mapping have their advantages and disadvantages in different noise environments, and they are complementary. This paper proposes a dual branch deep interactive UNet (DBDIUNet) for monaural speech enhancement to combine the advantages of spectral mapping and spectral masking. The DBDIUNet uses a classical encoder-decoder architecture, including a shared encoder and two decoders. One decoder outputs the complex ideal ratio mask (cIRM), and the other outputs the enhanced complex spectrum. The two signals are coupled by coherent averaging to get the enhanced speech signal. A novel deep interaction structure is proposed for the interaction of information between the two decoders, which achieves a very significant performance improvement at the minimal cost of computational consumption and hyperparameters. Compared with the noisy speech on the Interspeech 2020 deep noise suppression challenge blind test set, DBDIUNet improves the WB-PESQ, NB-PESQ, STOI, SI-SDR indicators by 1.575, 0.955, 7.9%, 8.67 respectively. In the noisy-reverberant speech enhancement test, DBDIUNet improves the WB-PESQ, STOI, SI-SDR, DNSMOS, and SRMR by 0.98, 10.24%, 5.43, 1.51, 3.43, respectively, which exceeds the state-of-the-art model.

Pyramid Attention Network for Image Restoration

Self-similarity refers to the image prior widely used in image restoration algorithms that small but similar patterns tend to occur at different locations and scales. However, recent advanced deep convolutional neural network-based methods for image restoration do not take full advantage of self-similarities by relying on self-attention neural modules that only process information at the same scale. To solve this problem, we present a novel Pyramid Attention module for image restoration, which captures long-range feature correspondences from a multi-scale feature pyramid. Inspired by the fact that corruptions, such as noise or compression artifacts, drop drastically at coarser image scales, our attention module is designed to be able to borrow clean signals from their “clean” correspondences at the coarser levels. The proposed pyramid attention module is a generic building block that can be flexibly integrated into various neural architectures. Its effectiveness is validated through extensive experiments on multiple image restoration tasks: image denoising, demosaicing, compression artifact reduction, and super resolution. Without any bells and whistles, our PANet (pyramid attention module with simple network backbones) can produce state-of-the-art results with superior accuracy and visual quality. Our code is available at https://github.com/SHI-Labs/Pyramid-Attention-Networks