Dataset Generation Method Research Articles

Dual-frequency phase unwrapping based on deep learning driven by simulation dataset

In fringe projection profilometry (FPP), hierarchical temporal phase unwrapping is used to reliably eliminate phase ambiguity in complex scenarios, in which triple-frequency hierarchical temporal phase unwrapping (THPU) has excellent accuracy but lower measurement speed, while dual-frequency hierarchical temporal phase unwrapping (DHPU) requires only two wrapped phases, but it is fragile to noise contamination. Some researchers have shown that deep learning techniques can be used to overcome this dilemma, but the high dataset building cost makes it difficult to apply them rapidly. In this paper, a novel simulation dataset generation method for FPP is proposed, which requires only a set of pre-collected fringe sequences on the reference plane along with some historical masks to generate a dataset, which is then used to train a neural network. The appropriately trained neural network is able to perform dual-frequency phase unwrapping in actual scenarios based on the principle of regression and segmentation respectively, and both of them show accuracy comparable to THPU, but segmentation is slightly better than regression. We verify its effectiveness through comparative experiments and demonstrate its robustness with different degrees of noise contamination. We believe that this paper can provide potential and beneficial ideas for simulation dataset driven deep learning techniques for optimizing various processes in the field of optical measurements.

Interference Suppression for an FM-Radio-Based Passive Radar via Deep Convolutional Autoencoder

Recently, the passive bistatic radar (PBR) which exploits frequency modulation (FM) radio transmitters as illuminators, has witnessed widespread usage owing to its various advantages. However, the characteristics of FM-radio-based PBR result in interference components in the range–Doppler (RD) map, which may increase false alarms. Therefore, this study proposes a method for suppressing interference components using a deep learning approach. The two main contributions of this study are as follows. First, a convolutional autoencoder model capable of effectively suppressing interference in the RD map of PBR was proposed. Second, a synthetic RD map dataset generation method that can enable the autoencoder to operate robustly in PBR in a real environment was presented. Further, a performance comparison between the proposed method and existing methods using simulated data proved that the deep learning-based method exhibited superior target detection performance. Furthermore, using the data recorded by the PBR in a real environment, the proposed autoencoder model was shown to effectively suppress interference components in a real interference environment.

Vision-based size distribution analysis of rock fragments using multi-modal deep learning and interactive annotation

Real-time size distribution analysis is of great significance to rock fragments in practical engineering. Traditional methods have struggled to strike a balance between analysis speed and precision, prompting the recent adoption of deep learning. However, prevalent approaches for estimating rock fragment sizes from RGB images (single-modality) suffer from two defects: (a)time-consuming and labour-intensive dataset annotation, (b) poor transferability between cases. To solve the above problems, a comprehensive multi-modal framework for size distribution prediction of rock fragments (SDPRF) is proposed in this paper. This framework comprises three essential components: a multi-modal image dataset generation method, a multi-modal rock surface net (Mrsnet) for fragment edge detection and a 2-step breakpoints connection algorithm. The test results indicate:(1) The generation method of SDPRF dataset greatly reduces the time required for dataset annotation, (2) Mrsnet shows better generalization ability for other cases outside the training set than traditional single-modal learning.

Deep learning for fast super-resolution ultrasound microvessel imaging

Objective. Ultrasound localization microscopy (ULM) enables microvascular reconstruction by localizing microbubbles (MBs). Although ULM can obtain microvascular images that are beyond the ultimate resolution of the ultrasound (US) diffraction limit, it requires long data processing time, and the imaging accuracy is susceptible to the density of MBs. Deep learning (DL)-based ULM is proposed to alleviate these limitations, which simulated MBs at low-resolution and mapped them to coordinates at high-resolution by centroid localization. However, traditional DL-based ULMs are imprecise and computationally complex. Also, the performance of DL is highly dependent on the training datasets, which are difficult to realistically simulate. Approach. A novel architecture called adaptive matching network (AM-Net) and a dataset generation method named multi-mapping (MMP) was proposed to overcome the above challenges. The imaging performance and processing time of the AM-Net have been assessed by simulation and in vivo experiments. Main results. Simulation results show that at high density (20 MBs/frame), when compared to other DL-based ULM, AM-Net achieves higher localization accuracy in the lateral/axial direction. In vivo experiment results show that the AM-Net can reconstruct ∼24.3 μm diameter micro-vessels and separate two ∼28.3 μm diameter micro-vessels. Furthermore, when processing a 128 × 128 pixels image in simulation experiments and an 896 × 1280 pixels image in vivo experiment, the processing time of AM-Net is ∼13 s and ∼33 s, respectively, which are 0.3–0.4 orders of magnitude faster than other DL-based ULM. Significance. We proposes a promising solution for ULM with low computing costs and high imaging performance.

A finite element-convolutional neural network model (FE-CNN) for stress field analysis around arbitrary inclusions

This study presents a data-driven finite element-machine learning surrogate model for predicting the end-to-end full-field stress distribution and stress concentration around an arbitrary-shaped inclusion. This is important because the model’s capacity to handle large datasets, consider variations in size and shape, and accurately replicate stress fields makes it a valuable tool for studying how inclusion characteristics affect material performance. An automatized dataset generation method using finite element simulation is proposed, validated, and used for attaining a dataset with one thousand inclusion shapes motivated by experimental observations and their corresponding spatially-varying stress distributions. A U-Net-based convolutional neural network (CNN) is trained using the dataset, and its performance is evaluated through quantitative and qualitative comparisons. The dataset, consisting of these stress data arrays, is directly fed into the CNN model for training and evaluation. This approach bypasses the need for converting the stress data into image format, allowing for a more direct and efficient input representation for the CNN. The model was evaluated through a series of sensitivity analyses, focusing on the impact of dataset size and model resolution on accuracy and performance. The results demonstrated that increasing the dataset size significantly improved the model’s prediction accuracy, as indicated by the correlation values. Additionally, the investigation into the effect of model resolution revealed that higher resolutions led to better stress field predictions and reduced error. Overall, the surrogate model proved effective in accurately predicting the effective stress concentration in inclusions, showcasing its potential in practical applications requiring stress analysis such as structural engineering, material design, failure analysis, and multi-scale modeling.

Quasi-real dataset generation and network improvement in defect detection based on shearography

Shearography combined with deep learning is increasingly becoming a commonly used approach in non-destructive testing (NDT). However, the validity of deep learning depends on the reliability of the training dataset, and the acquisition of real datasets is costly. In this study, the use of a shearography dataset that contains real noise-generating method is proposed. This dataset can generate random shearing direction and shearing amount defects only after one electronic speckle pattern interference experiment and annotation. Moreover, this study improves the existing semantic segmentation network and identify the defect location for the defect phase map containing noise accurately. Real NDT experiments using composite materials validate the effectiveness of the dataset generation method and semantic segmentation network proposed in this study.

Development of a synthetic dataset generation method for deep learning of real urban landscapes using a 3D model of a non-existing realistic city

In the urban landscaping field, training datasets for instance segmentation in the detection of building facades are needed for complex analysis and simulation based on data. Manual dataset generation methods are costly, so automatic generation is preferred. However, previous methods are costly and are more reliant on 3D real-city models than LOD2. The objective of this research is to propose a method for automatically generating synthetic datasets for instance segmentation using an image translation technique and procedural modeling. The image translation technique is used to generate street view images for training data, and procedural modeling can automatically specify the composition of generated street view images and generate annotation data, as well as a variety of 3D models. The proposed method generates various kinds of synthetic data at a rate of 4.45 s per set, and the model trained on the generated dataset can detect real buildings for each instance.

Infrared Dim and Small Target Sequence Dataset Generation Method Based on Generative Adversarial Networks

With the development of infrared technology, infrared dim and small target detection plays a vital role in precision guidance applications. To address the problems of insufficient dataset coverage and huge actual shooting costs in infrared dim and small target detection methods, this paper proposes a method for generating infrared dim and small target sequence datasets based on generative adversarial networks (GANs). Specifically, first, the improved deep convolutional generative adversarial network (DCGAN) model is used to generate clear images of the infrared sky background. Then, target–background sequence images are constructed using multi-scale feature extraction and improved conditional generative adversarial networks. This method fully considers the infrared characteristics of the target and the background, which can achieve effective expansion of the image data and provide a test set for the infrared small target detection and recognition algorithm. In addition, the classifier’s performance can be improved by expanding the training set, which enhances the accuracy and effect of infrared dim and small target detection based on deep learning. After experimental evaluation, the dataset generated by this method is similar to the real infrared dataset, and the model detection accuracy can be improved after training with the latest deep learning model.

SNR Enhancement for BOTDR With Spatial-Adaptive Image Denoising Method

A spatial-adaptive image denoising (SAID) method is proposed to enhance the signal-to-noise ratio (SNR) of Brillouin optical time domain reflectometer (BOTDR) without deterioration of spatial resolution. Through analyzing the principle of denoising, the spatial-adaptive denoising network structure is constructed, which can effectively reduce noise and avoid over-smoothing. Moreover, a high-similarity dataset generation method is designed to further enhance the denoising effect. The SAID method is experimentally implemented on two-dimension Brillouin gain spectrum matrix over a 25.1 km-long optical fiber with the spatial resolution of 2 m. The experimental results show that the SNR enhancement of SAID method is 21.92 dB, and the data processing time is only 2.48 s. Whereas the SNR enhancement of conventional 10000 times trace averaging method is merely 9.98 dB, but the data processing time is increased to 174.52 s. Therefore, it is demonstrated that the SAID method has obvious advantages in SNR enhancement and data processing speed without deteriorating the spatial resolution of BOTDR.

Synthetic Datasets for Rebar Instance Segmentation Using Mask R-CNN

The construction and inspection of reinforcement rebar currently rely entirely on manual work, which leads to problems such as high labor requirements and labor costs. Rebar image detection using deep learning algorithms can be employed in construction quality inspection and intelligent construction; it can check the number, spacing, and diameter of rebar on a construction site, and guide robots to complete rebar tying. However, the application of deep learning algorithms relies on a large number of datasets to train models, while manual data collection and annotation are time-consuming and laborious. In contrast, using synthetic datasets can achieve a high degree of automation of annotation. In this study, using rebar as an example, we proposed a mask annotation methodology based on BIM software and rendering software, which can establish a large and diverse training set for instance segmentation, without manual labeling. The Mask R-CNN trained using both real and synthetic datasets demonstrated a better performance than the models trained using only real datasets or synthetic datasets. This synthetic dataset generation method could be widely used for various image segmentation tasks and provides a reference for other computer vision engineering tasks and deep learning tasks in related fields.

DPPS: A deep-learning based point-light photometric stereo method for 3D reconstruction of metallic surfaces

Three-dimensional (3D) measurement provides essential geometric information for quality control and process monitoring in many manufacturing applications. Photometric stereo is one of the potential solutions for in-process metrology and active geometry compensation, which takes multiple images of an object under different illuminations as inputs and recovers its surface normal map based on a reflectance model. Deep learning approaches have shown their potential in solving the highly nonlinear problem for photometric stereo, but the main challenge preventing their practical application in process metrology lies in the difficulties in the generation of a comprehensive dataset for training the deep learning model. This paper presents a new Deep-learning based Point-light Photometric Stereo method, DPPS, which utilizes a multi-channel deep convolutional neural network (CNN) to achieve end-to-end prediction for both the surface normal and height maps in a semi-calibrated fashion. The key contribution is a new dataset generation method combining both physics-based and data-driven approaches, which minimizes the training cost and enables DPPS to handle reflective metal surfaces with unknown surface roughness. Even trained only with fully synthetic and high-fidelity dataset, our DPPS surpasses the state-of-the-art with an accuracy better than 0.15 cm over a 10 cm × 10 cm area and its real-life experimental results are on par with commercial 3D scanners. The demonstrated results provide guidance on improving the generalizability and robustness of deep-learning based computer vision metrology with minimized training cost as well as show the potential for in-process 3D metrology in advanced manufacturing processes.

An Efficient Deep Learning-Based Troposphere ZTD Dataset Generation Method for Massive GNSS CORS Stations

Nowadays, a huge amount of GNSS continuously operating reference stations (CORS) have been established around the world, which have already been and will continue providing massive troposphere zenith total delay (ZTD) data. This paper proposes an efficient deep learning-based troposphere ZTD dataset generation method including ZTD series segmentation, addictive and innovational outlier detection, and missing data imputation. The overall standard deviation of first-order ZTD difference series between adjacent epochs of the selected CORS station in January 2018 is reduced from 0.84 to 0.26 mm with the 3-sigma rule and the wavelet decomposition strategy for outlier elimination. A dense neural network (DNN) is subsequently designed to impute missing ZTD data. Only 3.4 minutes are required for the DNN training using a NVIDIA GeForce RTX 3090Ti GPU. The complete ZTD series with missing ZTD data imputed by the well-trained DNN model has a good agreement with the ZTD series before the data imputation in terms of mean value (-0.0184 vs -0.0187 mm) and standard deviation (5.43 vs 5.26 mm). Complete ZTD series of 120 CORS stations are generated to further evaluate the computational efficiency of the proposed method. An average of 2.56/5.38/5.77/4.33 hours are necessary to generate the ZTD dataset for January/April/July/October in 2018, respectively. Our study confirms that the proposed method can efficiently generate CORS-based ZTD dataset, which could be extended to applications including troposphere temporal-spatial pattern exploration and ZTD augmentation on high-precision GNSS positioning.

What makes the unsupervised monocular depth estimation (UMDE) model training better

Current computer vision tasks based on deep learning require a huge amount of data with annotations for model training or testing, especially in some dense estimation tasks, such as optical flow segmentation and depth estimation. In practice, manual labeling for dense estimation tasks is very difficult or even impossible, and the scenes of the dataset are often restricted to a small range, which dramatically limits the development of the community. To overcome this deficiency, we propose a synthetic dataset generation method to obtain the expandable dataset without burdensome manual workforce. By this method, we construct a dataset called MineNavi containing video footages from first-perspective-view of the aircraft matched with accurate ground truth for depth estimation in aircraft navigation application. We also provide quantitative experiments to prove that pre-training via our MineNavi dataset can improve the performance of depth estimation model and speed up the convergence of the model on real scene data. Since the synthetic dataset has a similar effect to the real-world dataset in the training process of deep model, we finally conduct the experiments on MineNavi with unsupervised monocular depth estimation (UMDE) deep learning models to demonstrate the impact of various factors in our dataset such as lighting conditions and motion mode, aiming to explore what makes this kind of models training better.

Data complexity-based batch sanitization method against poison in distributed learning

The security of Federated Learning (FL)/Distributed Machine Learning (DML) is gravely threatened by data poisoning attacks, which destroy the usability of the model by contaminating training samples, so such attacks are called causative availability indiscriminate attacks. Facing the problem that existing data sanitization methods are hard to apply to real-time applications due to their tedious process and heavy computations, we propose a new supervised batch detection method for poison, which can fleetly sanitize the training dataset before the local model training. We design a training dataset generation method that helps to enhance accuracy and uses data complexity features to train a detection model, which will be used in an efficient batch hierarchical detection process. Our model stockpiles knowledge about poison, which can be expanded by retraining to adapt to new attacks. Being neither attack-specific nor scenario-specific, our method is applicable to FL/DML or other online or offline scenarios.

Dynamic Multidimensional Knapsack Problem benchmark datasets

With increasing research on solving Dynamic Optimization Problems (DOPs), many metaheuristic algorithms and their adaptations have been proposed to solve them. However, from currently existing research results, it is hard to evaluate the algorithm performance in a repeatable way for combinatorial DOPs due to the fact that each research work has created its own version of a dynamic problem dataset using stochastic methods. Up to date, there are no combinatorial DOP benchmarks with replicable qualities. This work introduces a non-stochastic consistent Dynamic Multidimensional Knapsack Problem (Dynamic MKP) dataset generation method that is also extensible to solve the research replicability problem. Using this method, generated and published 1405 Dynamic MKP benchmark datasets using existing famous static MKP benchmark instances as the initial state. Then the datasets are quantitatively and qualitatively analyzed. Furthermore, 445 datasets have the optimal result found of each state using a linear solver. The optimal results and result scores are included with published datasets.

Intelligent direct thrust control for multivariable turbofan engine based on reinforcement and deep learning methods

This paper proposes a novel intelligent direct thrust control (IDTC) architecture for a two-spool turbofan engine, in which the thrust is directly controlled by the IDTC architecture with two control variables instead of transforming thrust command to other commands. To realize thrust control, the unmeasurable thrust should be estimated first. A dynamic thrust model (DTM) is proposed to characterize the strong non-linearity of the engine. With the dynamic characteristic data set generation method, the date diversity increases and the thrust under random engine dynamics can be estimated more accurately. For the multi-variable optimal thrust controller, reinforcement learning (RL) method proximal policy optimization (PPO) algorithm is applied in the intelligent controller (IC) design process. Considering the engine characteristics, the reward system and the controller structure are redesigned to realize thrust optimal control and limit protection. The network used in both the DTM and the IC designing are the long short-term memory (LSTM) recurrent neural network (RNN), which can better process the engine time series and get better performance and accuracy. As the traditional evaluation criteria are too conservative to evaluate the overall ability of the proposed IDTC architecture in the full envelope, new criteria are proposed. Numerical simulations shows the satisfactory thrust estimation and control performance of the proposed IDTC architecture in the full envelope above the idle, and verify the superiority against other methods. Also, the strategy selection of the IC under extreme control tasks also shows the intelligence of the controller and gives new ideas for the engine control law design.

Deep learning-based image reconstruction for photonic integrated interferometric imaging.

Photonic integrated interferometric imaging (PIII) is an emerging technique that uses far-field spatial coherence measurements to extract intensity information from a source to form an image. At present, low sampling rate and noise disturbance are the main factors hindering the development of this technology. This paper implements a deep learning-based method to improve image quality. Firstly, we propose a frequency-domain dataset generation method based on imaging principles. Secondly, spatial-frequency dual-domain fusion networks (SFDF-Nets) are presented for image reconstruction. We utilize normalized amplitude and phase to train networks, which reduces the difficulty of network training using complex data. SFDF-Nets can fuse multi-frame data captured by rotation sampling to increase the sampling rate and generate high-quality spatial images through dual-domain supervised learning and frequency domain fusion. Furthermore, we propose an inverse fast Fourier transform loss (IFFT loss) for network training in the frequency domain. Extensive experiments show that our method improves PSNR and SSIM by 5.64 dB and 0.20, respectively. Our method effectively improves the reconstructed image quality and opens a new dimension in interferometric imaging.

French translation of a dialogue dataset and text-based emotion detection

Chatbots allow computer programs to interact naturally with a user. However, they remain limited due to their lack of sensitivity to the user’s state of mind and emotions. This sensitivity will allow the chatbots to provide more accurate answers. Text-based emotion detection has already been explored for the english language (Chatterjee et al., 2019), yet no satisfying french dataset is available. We propose to translate the emotion corpus of multi-party conversation EmotionLines, which is based on the Friends TV show, by exploiting its french broadcasting. Our translation-based dataset generation method is adaptable to any dataset deriving from foreign movies, or TV shows broadcasted in french. Using this translated dataset, we propose a classifier based on BERT, able to detect the user’s emotion from text. It takes into account the context of the discussion to improve its inferences.

DNN-based optical performance monitoring and its application for soft failure localization by multipoint estimation

In current optical communication networks, it is becoming increasingly important to use optical performance monitoring (OPM), which monitors the transmission characteristics of optical signals, to detect early degradation of features that may affect communication quality. This paper describes a method to realize OPM by training a deep neural network (DNN) with data obtained from digital coherent transceivers. In addition, we propose a dataset generation method, Data Augmentation using Differential Image (DADI), to enable the DNN to train with fewer data. We demonstrate that this method simultaneously classifies the optical signal to noise ratio and polarization mode dispersion with high accuracy. Furthermore, we apply DNN-based OPM to detect and localize soft failure in a multi-span environment. We develop a method to identify the location of soft failure by collecting information from DNN-based OPM estimated at multiple points in multi-span optical transmission links. We demonstrate that the method correctly localizes the soft failure with more than 90% accuracy on average (min, 85%; max, 98%) under single-channel experimental conditions.

Separation of aliasing signals from inductive oil debris monitors based on fully convolutional neural networks

Inductive oil debris monitors can detect wear debris in lubricating oil in real-time, which has great potential for monitoring the working conditions of mechanical systems. However, the superimposition of the induced voltages when multiple debris particles pass through a sensor at a close distance may lead to an erroneous estimation of the peak-to-peak value of the wear debris waveforms. A complete implementation framework is proposed to separate the aliasing signals based on fully convolutional neural networks, which includes a segmented fractional calculus filtering technique and a semi-simulated training dataset generation method. The results of physical experiments indicate that the proposed method can reduce the average error rate of the peak-to-peak value from 15.36% to 3.96% and the maximum error rate from 56.33% to 9.27% compared with those before separation. The stability and computing time of this method are also evaluated through physical experiments.