Generate Training Datasets Research Articles

Texture-Less Shiny Objects Grasping in a Single RGB Image Using Synthetic Training Data

In the industrial domain, estimating the pose of texture-less shiny parts is challenging but worthwhile. In this study, it is impractical to utilize texture information to obtain the pose because the features are likely to be affected by the surrounding objects. In addition, the colors of the metal parts are similar, making object segmentation challenging. This study proposes dividing the entire process into three steps: object detection, feature extraction, and pose estimation. We use the Mask-RCNN to detect objects and HRNet to extract the corresponding features. For metal parts of different shapes, different keypoints were chosen accordingly. Conventional contour-based methods are inapplicable to parts containing planar surfaces because the objects occlude each other in clustered environments. In this case, we used dense discrete points along the edges as semantic keypoints for metal parts containing planar elements. We chose skeleton points as semantic keypoints for parts containing cylindrical components. Subsequently, we combined the localization of semantic keypoints and the corresponding CAD model information to estimate the 6D pose of an individual object in sight. The implementation of deep learning approaches requires massive training datasets and intensive labeling. Thus, we propose a method to generate training datasets and automatically label them. Experiments show that the algorithm based on synthetic data performs well in a natural environment, despite not utilizing real scenario images for training.

APPLICATION OF MACHINE LEARNING FOR OBJECT DETECTION IN OBLIQUE AERIAL IMAGES

Abstract. At the time of continuous development of all technologies, deep machine learning (more precisely, convolutional neural networks), which is one of the branches of artificial intelligence (AI), has found wide application in many fields, including photogrammetry and remote sensing. One of the areas where a lot of research is conducted using these methods is the recognition of objects in aerial and satellite imagery. Through the application of deep learning algorithms and neural networks, it is possible to automate labour-intensive processes. However, while object detection in images using machine learning is popular for natural scenes and in recent years also for nadir aerial and satellite imagery, for aerial oblique imagery at the moment of this research there were relatively few publications on the subject. The challengeable task in object detection is the time-consuming generation of training datasets when access is limited or non-existent. This study proposed the methodology to automate this process with use of existing resources for transferring of references to new databases for training models for detect objects on aerial oblique images. The object detection was performed using the YOLOv3 neural network. Experiment results tested on two datasets have shown that the proposed method could realize the task of object detection in oblique aerial images.

Efficient modelling of permanent magnet field distribution for deep learning applications

Permanent magnets are essential components in a range of applications from robotics to energy harvesting devices. Computing the field distribution of permanent magnets is crucial for designing and optimising these devices, but currently existing techniques, such as the Finite Element Method (FEM), are very time-consuming. Deep learning (DL) has been widely utilised to solve regression and classification problems and could enable an efficient computation of permanent magnet field distributions. However, deep learning requires large amounts of training data, which is difficult to obtain with current methods. In this work we represented the axial, azimuthal and radial components of the magnetic field created by permanent magnets and uniform magnetisations in semi-analytical expressions (SAE). These forms can be further simplified, and we validated these against the FEM for individual geometries such as an elliptical cylinder and a cone. The computational efficiency of the semi-analytical forms enables the generation of training datasets. Finally, we built a machine learning model using the training data generated by the SAEs and demonstrate that the machine learning model can predict the field distributions of permanent magnets in various configurations. The predicted magnetic field using the machine learning model is in good agreement with the ground-truth, with r2 for the axial, azimuthal and radial components being 0.999, 0.997 and 0.998, respectively for the magnetised elliptical cylinders, and 0.999, 0.999, 0.998 for the magnetised cones. Furthermore, the computational times of the machine learning models when executed on a CPU are more than 28 and 12 times faster than the SAEs of the elliptical cylinders and cones, respectively; the models are very fast to execute on a GPU with 22.7 s for the elliptical cylinders and 36.8 s for the cones. This work shows that it is possible to train a machine learning model to predict permanent magnet field distributions based on a training with semi-analytical expressions of various magnet geometries. The code for generating the training data, constructing the machine learning model and applying the model to new data is made openly available (https://github.com/vantainguyen/MagField_DataGen_Deep-learning).

Using CNN for solving two-player zero-sum games

We study a two-player zero-sum game (matrix game for short) with the objective of finding the saddle point and its value. We develop a novel convolutional neural network (CNN for short) approach to achieve the goal. We propose a complete training pipeline, including a specific CNN model structure to handle varying game sizes, generating training datasets, and model fitting. The experiment results show that our proposed method outperforms the traditional linear programming (LP for short) method and two regret minimization learning algorithms in terms of computational efforts.

Data-driven surrogate model with latent data assimilation: Application to wildfire forecasting

The large and catastrophic wildfires have been increasing across the globe in the recent decade, highlighting the importance of simulating and forecasting fire dynamics in near real-time. This is extremely challenging due to the complexities of physical models and geographical features. Running physics-based simulations for large wildfire events in near real-time are computationally expensive, if not infeasible. In this work, we develop and test a novel data-model integration scheme for fire progression forecasting, that combines Reduced-order modelling, recurrent neural networks (Long-Short-Term Memory), data assimilation, and error covariance tuning. The Reduced-order modelling and the machine learning surrogate model ensure the efficiency of the proposed approach while the data assimilation enables the system to adjust the simulation with observations. We applied this algorithm to simulate and forecast three recent large wildfire events in California from 2017 to 2020. The deep-learning-based surrogate model runs around 1000 times faster than the Cellular Automata simulation which is used to generate training data-sets. The daily fire perimeters derived from satellite observation are used as observation data in Latent Assimilation to adjust the fire forecasting in near real-time. An error covariance tuning algorithm is also performed in the reduced space to estimate prior simulation and observation errors. The evolution of the averaged relative root mean square error (R-RMSE) shows that data assimilation and covariance tuning reduce the RMSE by about 50% and considerably improves the forecasting accuracy. As a first attempt at a reduced order wildfire spread forecasting, our exploratory work showed the potential of data-driven machine learning models to speed up fire forecasting for various applications.

Deep learning inversion of Rayleigh-wave dispersion curves with geological constraints for near-surface investigations

SUMMARY With the emergence of massive seismic data sets, surface wave methods using deep learning (DL) can effectively obtain shear wave velocity (Vs) structure for non-invasive near-surface investigations. Previous studies on DL inversion for deep geophysical investigation have a reference model to generate the training data set, while near-surface investigations have no model. Therefore, we systematically give a set of training data set generation processes. In the process, we use both prior information and the observed data to constrain the data set so that the DL inversion model can learn the local geological characteristics of the survey area. Because the space of inverted Vs models is constrained and thus narrowed, the inversion non-uniqueness can be reduced. Furthermore, the mean squared error, which is commonly used as loss function, may cause a poor fitting accuracy of phase velocities at high frequencies in near-surface applications. To make the fitting accuracy evenly in all frequency bands, we modify the loss function into a weighted mean squared relative error. We designed a convolutional neural network (CNN) to directly invert fundamental-mode Rayleigh-wave phase velocity for 1-D Vs models. To verify the feasibility and reliability of the proposed algorithm, we tested and compared it with the Levenberg–Marquardt (L-M) inversion and neighbourhood algorithm (NA) using field data from the Lawrence experiment (USA) and the Wuwei experiment (China). In both experiments, the inverted Vs models by CNN are consistent with the borehole information and are similar to that from existing methods after fine tuning of model parameters. The average root mean squares errors (RMSEs) of the CNN, NA and L-M methods are also similar, except in the Lawrence experiment, the RMSE of CNN is 17.33 m s−1 lower than previous studies using the L-M method. Moreover, the comparison of different loss functions for the Wuwei experiment indicates that the modified loss function can achieve higher accuracy than the traditional one. The proposed CNN is therefore ideally suited for rapid, repeated near-surface subsurface imaging and monitoring under similar geological settings.

Model predictive control of long Transfer-line cooling process based on Back-Propagation neural network

• A model predictive control (MPC) method based on back-propagation (BP) neural network as a surrogate inversion model was designed to improve the automation of the long transfer-line pre-cooling process. • A one-dimensional simulation model based on unsteady flow and heat transfer process was used to generate training datasets for BP neural networks, and reasonable cleaning rules are the key to obtain better training results. • Simulation and test results show that the MPC method can be applied to the automatic control of nonlinear large hysteresis dynamical systems such as the long transfer-line pre-cooling process. As the scale of large cryogenic systems continues to expand, the thermal inertia and nonlinear characteristics of the pre-cooling process of long-distance cryogenic transfer-line become obvious, and the traditional control methods are less effective in controlling such nonlinear large hysteresis time-varying systems. To improve the automation of the pre-cooling process, a Model Predictive Control (MPC) method based on Back-Propagation (BP) neural network as a surrogate inversion model was designed and deployed on a large helium cryogenic system of the Platform of Advanced Photon Source (PAPS). Simulation and test results show that the MPC method can be applied to the automatic control of nonlinear large hysteresis dynamical systems; the BP neural network as a surrogate model can invert the one-dimensional flow heat transfer model better. The actual test results on the PAPS cryogenic system show that the method can realize the automatic pre-cooling of long transfer-line, and the overall cooling effect is stable and efficient, with the maximum absolute temperature difference of no more than 3.2 K and the maximum relative temperature difference of no more than 2.1% from the ideal cooling line.

Quantitative phase imaging based on model transfer learning.

Convolutional neural networks have been widely used in optical information processing and the generalization ability of the network depends greatly on the scale and diversity of the datasets, however, the acquisition of mass datasets and later annotation have become a common problem that hinders its further progress. In this study, a model transfer-based quantitative phase imaging (QPI) method is proposed, which fine-tunes the network parameters through loading pre-training base model and transfer learning, enable the network with good generalization ability. Most importantly, a feature fusion method based on moment reconstruction is proposed for training dataset generation, which can construct rich enough datasets that can cover most situations and accurately annotated, it fundamentally solves the problem from the scale and representational ability of the datasets. Besides, a feature distribution distance scoring (FDDS) rule is proposed to evaluate the rationality of the constructed datasets. The experimental results show that this method is suitable for different types of samples to achieve fast and high-accuracy phase imaging, which greatly relieves the pressure of data, tagging and generalization ability in the data-driven method.

Improving needle visibility in LED-based photoacoustic imaging using deep learning with semi-synthetic datasets

Photoacoustic imaging has shown great potential for guiding minimally invasive procedures by accurate identification of critical tissue targets and invasive medical devices (such as metallic needles). The use of light emitting diodes (LEDs) as the excitation light sources accelerates its clinical translation owing to its high affordability and portability. However, needle visibility in LED-based photoacoustic imaging is compromised primarily due to its low optical fluence. In this work, we propose a deep learning framework based on U-Net to improve the visibility of clinical metallic needles with a LED-based photoacoustic and ultrasound imaging system. To address the complexity of capturing ground truth for real data and the poor realism of purely simulated data, this framework included the generation of semi-synthetic training datasets combining both simulated data to represent features from the needles and in vivo measurements for tissue background. Evaluation of the trained neural network was performed with needle insertions into blood-vessel-mimicking phantoms, pork joint tissue ex vivo and measurements on human volunteers. This deep learning-based framework substantially improved the needle visibility in photoacoustic imaging in vivo compared to conventional reconstruction by suppressing background noise and image artefacts, achieving 5.8 and 4.5 times improvements in terms of signal-to-noise ratio and the modified Hausdorff distance, respectively. Thus, the proposed framework could be helpful for reducing complications during percutaneous needle insertions by accurate identification of clinical needles in photoacoustic imaging.

SOLVING POISSON EQUATION WITH CONVOLUTIONAL NEURAL NETWORKS

Context. The Poisson equation is the one of fundamental differential equations, which used to simulate complex physical processes, such as fluid motion, heat transfer problems, electrodynamics, etc. Existing methods for solving boundary value problems based on the Poisson equation require an increase in computational time to achieve high accuracy. The proposed method allows solving the boundary value problem with significant acceleration under the condition of acceptable loss of accuracy.
 Objective. The aim of our work is to develop artificial neural network architecture for solving a boundary value problem based on the Poisson equation with arbitrary Dirichlet and Neumann boundary conditions.
 Method. The method of solving boundary value problems based on the Poisson equation using convolutional neural network is proposed. The network architecture, structure of input and output data are developed. In addition, the method of training dataset generation is described.
 Results. The performance of the developed artificial neural network is compared with the performance of the numerical finite difference method for solving the boundary value problem. The results showed an acceleration of the computational speed in x10–700 times depending on the number of sampling nodes.
 Conclusions. The proposed method significantly accelerated speed of solving a boundary value problem based on the Poisson equation in comparison with the numerical method. In addition, the developed approach to the design of neural network architecture allows to improve the proposed method to achieve higher accuracy in modeling the process of pressure distribution in areas of arbitrary size.

The moving vibration source perception using bionic lateral line system and data-driven method

In recent years, the application scope of artificial lateral line sensor (ALLS) systems is mainly limited to the perception of water flow environment and dipole source. In order to expand the application scope, an ALLS based on a pressure sensor array is proposed in this work to recognize the shape and size parameters of moving vibration sources. First of all, the pressure sensor array is appropriately installed in the underwater vehicle with the help of sensor topology optimization. Then, a numerical simulation model of the ALLS is established and its validation is verified by experimental tests. Subsequently, the information of the moving vibration sources with different parameters is collected by the simulations and experiments to generate training data sets. Lastly, two typical data-driven methods (i.e., the random forest algorithm and support vector machine) are trained to identify the shape and size parameters of the moving vibration sources. The analysis results demonstrate that the developed ALLS is effective in vibration source perception, which enhances the function of underwater vehicles.

Integration of Machine Learning and Coarse-Grained Molecular Simulations for Polymer Materials: Physical Understandings and Molecular Design.

In recent years, the synthesis of monomer sequence-defined polymers has expanded into broad-spectrum applications in biomedical, chemical, and materials science fields. Pursuing the characterization and inverse design of these polymer systems requires our fundamental understanding not only at the individual monomer level, but also considering the chain scales, such as polymer configuration, self-assembly, and phase separation. However, our accessibility to this field is still rudimentary due to the limitations of traditional design approaches, the complexity of chemical space along with the burdened cost and time issues that prevent us from unveiling the underlying monomer sequence-structure-property relationships. Fortunately, thanks to the recent advancements in molecular dynamics simulations and machine learning (ML) algorithms, the bottlenecks in the tasks of establishing the structure-function correlation of the polymer chains can be overcome. In this review, we will discuss the applications of the integration between ML techniques and coarse-grained molecular dynamics (CGMD) simulations to solve the current issues in polymer science at the chain level. In particular, we focus on the case studies in three important topics—polymeric configuration characterization, feed-forward property prediction, and inverse design—in which CGMD simulations are leveraged to generate training datasets to develop ML-based surrogate models for specific polymer systems and designs. By doing so, this computational hybridization allows us to well establish the monomer sequence-functional behavior relationship of the polymers as well as guide us toward the best polymer chain candidates for the inverse design in undiscovered chemical space with reasonable computational cost and time. Even though there are still limitations and challenges ahead in this field, we finally conclude that this CGMD/ML integration is very promising, not only in the attempt of bridging the monomeric and macroscopic characterizations of polymer materials, but also enabling further tailored designs for sequence-specific polymers with superior properties in many practical applications.

Correcting Arabic Soft Spelling Mistakes using BiLSTM-based Machine Learning

Soft spelling mistakes are a class of mistakes that is widespread among native Arabic speakers and foreign learners alike. Some of these mistakes are typographical in nature. They occur due to orthographic variations of some Arabic letters and the complex rules that dictate their correct usage. Many people forgo these rules, and given the identical phonetic sounds, they often confuse such letters. In this paper, we investigate how to use machine learning to correct such mistakes given that there are no sufficient datasets to train the correction models. Soft errors detection and correction is an active field in Arabic natural language processing. We generate training datasets using proposed transformed input approach and stochastic error injec-tion approach. These approaches are applied to two acclaimed datasets that represent Classical Arabic and Modern Standard Arabic. We treat the problem as character-level, one-to-one sequence transcription problem. This one-to-one transcription of mistakes that include omissions and deletions is possible with adopted simple transformations. This approach permits using bidirectional long short-term memory (BiLSTM) models that are more effective to train compared to other alternatives such as encoder-decoder models. Based on investigating multiple alternatives, we recommend a configuration that has two BiLSTM layers, and is trained using the stochastic error injection approach with error injection rate of 40%. The best model corrects 96.4%of the injected errors and achieves a low character error rate of 1.28% on a real test set of soft spelling mistakes.

Automatic Label Injection Into Local Infrastructure LiDAR Point Cloud for Training Data Set Generation

The representation of objects in LiDAR point clouds is changed as the height of the mounting position of sensor devices gets increased. Most of the available open datasets for training machine learning based object detectors are generated with vehicle top mounted sensors, thus the detectors trained on such datasets perform weaker when the sensor is observing the scene from a significantly higher viewpoint (e.g. infrastructure sensor). In this paper a novel Automatic Label Injection method is proposed to label the objects in the point cloud of the high-mounted infrastructure LiDAR sensor based on the output of a well performing “trainer” detector deployed at optimal height while considering the uncertainties caused by various factors described in detail throughout the paper. The proposed automatic labeling approach has been validated on a small scale sensor setup in a real-world traffic scenario where accurate differential GNSS reference data where also available for each test vehicle. Furthermore, the concept of a distributed multi-sensor system covering a larger area aimed for automatic dataset generation is also presented. It is shown that a machine learning based detector trained on differential GNSS-based training dataset performs very similarly to the detector retrained on a dataset generated by the proposed Automatic Label Injection technique. According to our results a significant increase in the maximum detection range can be achieved by retraining the detector on viewpoint specific data generated fully automatically by the proposed label injection technique compared to a detector trained on vehicle top mounted sensor data.

Image-Bot: Generating Synthetic Object Detection Datasets for Small and Medium-Sized Manufacturing Companies

Training datasets for image recognition are poorly available for small and medium-sized manufacturing companies, due to the specialized products they work with, and the disproportionate investment to generate their own ones. Thus, we investigate a new approach: The Image-Bot consists of a physical apparatus and a processing pipeline to generate training datasets from real-world objects easily. It takes pictures of the objects in front of a green screen and blends them with random backgrounds. The approach was tested with 23 objects and a YOLOv5 algorithm. It creates a state-of-the-art training dataset with about 2,000 images per object in under 45 min.

Reduced-order modeling for turbulent wake of a finite wall-mounted square cylinder based on artificial neural network

This study presents an artificial neural network and a proper orthogonal decomposition (POD)-based reduced-order model (ROM) of turbulent flow around a finite wall-mounted square cylinder. The proposed model is suitable for turbulent wake control applications, because it can predict the dynamics of the main features of the flow field with low computational cost. A long short-term memory neural network (LSTM NN) and a bidirectional long short-term memory neural network (BLSTM NN) are used to predict the temporal evolution of POD time coefficients at different sections along the height of the obstacle. The improved delayed detached-eddy simulation is performed to generate training datasets. Transfer learning is utilized in the training process by using the weights of the LSTM/BLSTM NN that are used to predict the POD time coefficients of the planes at lower elevations to initialize the weights of the networks at higher elevations along the height of the obstacle. The use of transfer learning results in a remarkable improvement in the prediction capability of LSTM/BLSTM NN compared with the one when the network is initialized with random weights. The BLSTM NN shows better results compared with the LSTM NN in terms of training and prediction error, indicating that the BLSTM-POD model is more suitable to be used as a ROM for predicting the turbulent wake. Furthermore, the temporal evolution of the time coefficients is carefully examined using the phase space plots and Poincaré sections. The results of using different lengths of the prediction time window show that the prediction error of the POD time coefficients increases as the prediction time window increases and the error increasing rate decreases with ranking of the POD modes.

Building Marginal Pattern Library With Unbiased Training Dataset for Enhancing Model-Free Load-ED Mapping

Input-output mapping for a given power system problem, such as loads versus economic dispatch (ED) results, has been demonstrated to be learnable through artificial intelligence (AI) techniques, including neural networks. However, the process of identifying and constructing a comprehensive dataset for the training of such input-output mapping remains a challenge to be solved. Conventionally, load samples are generated by a pre-defined distribution, and then ED is solved based on those load samples to form training datasets, but this paper demonstrates that such dataset generation is biased regarding load-ED mapping. The marginal unit and line congestion (i.e., marginal pattern) exhibit a unique characteristic called “step change” in which the marginal pattern changes when the load goes from one critical loading level (CLL) to another, and there is no change of marginal units within the interval of the two adjacent CLLs. Those loading intervals differ significantly in size. The randomly generated training dataset overfills intervals with large sizes and underfits intervals with small sizes, so it is biased. In this paper, three algorithms are proposed to construct a marginal pattern library to examine this bias according to different computational needs, and an enhancement algorithm is proposed to eliminate the bias in the load-ED training dataset generation. Three illustrative test cases demonstrate the proposed algorithms, and comparative studies are constructed to show the superiority of the enhanced, unbiased training dataset.

Enhanced Secured Model for Smart Homes Using Human Activity Recognition and User Behaviour Analysis

This paper proffers a secured and cost effective solution for capturing data, translating this data into pre-labeled activities and then further categorizing these activities into behaviors which are either normal or abnormal based on a generated training dataset. In this research, a wooden smart home prototype was constructed with sensors and actuators in order to capture activities in the home. The sensors and actuators are interfaced with the Human Activity Recognition model which was developed using Gaussian Naïve Bayes algorithm. Output from the HAR model is passed as input into a second model which we refer to as the User Behaviour Analyser (UBA). The UBA was developed using the Support Vector Machine algorithm. This second model makes the final prediction of whether the home is safe or not based on its training. The solution is built around the raspberry Pi 4 computer board running the linux-based raspbian O.S. Experiments were performed on the developed system which gave an Accuracy of 95%, Precision 100%, Recall 92% and F1_Score of 96%. Keywords; Activity Recognition, User Behaviour Analysis, Smart Home, Internet of Things, Security.

Spatially oriented convolutional neural network for spatial relation extraction from natural language texts

AbstractSpatial relation extraction (e.g., topological relations, directional relations, and distance relations) from natural language descriptions is a fundamental but challenging task in several practical applications. Current state‐of‐the‐art methods rely on rule‐based metrics, either those specifically developed for extracting spatial relations or those integrated in methods that combine multiple metrics. However, these methods all rely on developed rules and do not effectively capture the characteristics of natural language spatial relations because the descriptions may be heterogeneous and vague and may be context sparse. In this article, we present a spatially oriented piecewise convolutional neural network (SP‐CNN) that is specifically designed with these linguistic issues in mind. Our method extends a general piecewise convolutional neural network with a set of improvements designed to tackle the task of spatial relation extraction. We also propose an automated workflow for generating training datasets by integrating new sentences with those in a knowledge base, based on string similarity and semantic similarity, and then transforming the sentences into training data. We exploit a spatially oriented channel that uses prior human knowledge to automatically match words and understand the linguistic clues to spatial relations, finally leading to an extraction decision. We present both the qualitative and quantitative performance of the proposed methodology using a large dataset collected from Wikipedia. The experimental results demonstrate that the SP‐CNN, with its supervised machine learning, can significantly outperform current state‐of‐the‐art methods on constructed datasets.

3D fluorescence microscopy data synthesis for segmentation and benchmarking.

Automated image processing approaches are indispensable for many biomedical experiments and help to cope with the increasing amount of microscopy image data in a fast and reproducible way. Especially state-of-the-art deep learning-based approaches most often require large amounts of annotated training data to produce accurate and generalist outputs, but they are often compromised by the general lack of those annotated data sets. In this work, we propose how conditional generative adversarial networks can be utilized to generate realistic image data for 3D fluorescence microscopy from annotation masks of 3D cellular structures. In combination with mask simulation approaches, we demonstrate the generation of fully-annotated 3D microscopy data sets that we make publicly available for training or benchmarking. An additional positional conditioning of the cellular structures enables the reconstruction of position-dependent intensity characteristics and allows to generate image data of different quality levels. A patch-wise working principle and a subsequent full-size reassemble strategy is used to generate image data of arbitrary size and different organisms. We present this as a proof-of-concept for the automated generation of fully-annotated training data sets requiring only a minimum of manual interaction to alleviate the need of manual annotations.