Require Training Data Sets Research Articles

A data-driven ductile fracture criterion for high-speed impact

Data-driven methods based on machine learning (ML) models offer new approaches for characterizing the fracture behavior of advanced elastoplastic materials. In this paper, a ML-based data-driven ductile fracture criterion is proposed to characterize the fracture behavior of elastoplastic materials under high-speed impact loading conditions. To reduce the required training dataset and enhance the predictability capability, several assumptions are used. Firstly, utilizing the decoupled assumption, two separate artificial neural network (ANN) models are employed to establish the fundamental fracture model and characterize the strain rate effect of ductile fracture behavior, respectively. In addition, the enhanced method with a logarithmic function is introduced to improve predictability capability of the proposed data-driven criterion under unknown high strain rates. To establish a complete numerical implementation framework, an enhanced rate-dependent data-driven constitutive model and a compatible numerical implementation algorithm are additionally introduced. Eventually, to assess the applicability of the proposed data-driven fracture criterion, numerical simulations of notched specimens and ballistic impact conditions of Ti-6Al-4V material are conducted, respectively. These investigation results demonstrate the effectiveness of the proposed data-driven ductile fracture criterion.

Gearbox fault identification using auto-encoder without training data from the damaged machine

Deep learning methods work well in machine diagnostics where operating conditions affect diagnostic signals. Classifiers are often used for fault identification, but these methods require training data sets measured for each fault. The solution to the lack of data is autoencoder-based network models, but these models can only detect, not identify faults.This article presents a new fault identification method based on auto-encoders (AE-based Fault Identification Technique AE-FIT) that does not require training data from the damaged machine. This method diagnoses pinion gearboxes operating under variable conditions (variable load, load-induced rotational speed, and temperature). The result of the technique is an interpretable diagnostic spectrum (AE-based Interpretable Order Spectrum AE-IOS).The method has been tested on two laboratory benches to detect misalignment, unbalance, and gearbox degradation. The damages introduced were used to validate a technique based on an auto-encoder trained only with data from undamaged machines.

Parameterized physics-informed neural networks (P-PINNs) solution of uniform flow over an arbitrarily spinning spherical particle

Neural network-based approaches have emerged as alternatives to conventional computational fluid dynamics (CFD) in solving multiphase flow problems. However, most of these approaches are based on data-driven machine learning methods that require training datasets prepared beforehand through CFD simulations or experiments and cannot be used independently. This study presents an unsupervised parameterized physics-informed neural networks (P-PINNs) approach for obtaining the full flow field information over a multi-dimensional parametric space. This approach is then applied to solve the three-dimensional flow around an arbitrarily rotating sphere subjected to a cross-flow. This solution is obtained for a continuous range of three parameters: the Reynolds number based on cross-flow (Re∈[1,400]), non-dimensional streamwise and spanwise angular velocity components (Ωx∗,Ωy∗∈[−3,3]). The predictions from the P-PINNs model are compared against particle-resolved direct numerical simulation (PR-DNS) results, demonstrating that the neural network model predicts all flow variables (velocity, pressure, and their spatial derivatives) accurately with R2≳0.9. The force and torque on the particle obtained from the P-PINNs model are also compared well against the corresponding PR-DNS results with R2>0.94. The key observation is that the computational cost of the unsupervised parameterized neural network model training and deployment is substantially lower than performing numerous conventional CFD simulations to systematically vary the parameters. Furthermore, once trained, the resultant neural network model is substantially more compact than the huge database of discretized numerical solutions. The P-PINNs model is then used to reveal several interesting flow features and phenomena about the rotating sphere, including flow symmetry, secondary drag and lift, critical Reynolds number, flow bifurcation, and flow separation. This study presents P-PINNs as an efficient alternative for solving parameterized flow problems, demonstrating its practical usage in flow physics discovery with the example of flow past an arbitrarily rotating sphere.

Single patient learning for adaptive radiotherapy dose prediction.

Throughout a patient's course of radiation therapy, maintaining accuracy of their initial treatment plan over time is challenging due to anatomical changes-for example, stemming from patient weight loss or tumor shrinkage. Online adaptation of their RT plan to these changes is crucial, but hindered by manual and time-consuming processes. While deep learning (DL) based solutions have shown promise in streamlining adaptive radiation therapy (ART) workflows, they often require large and extensive datasets to train population-based models. This study extends our prior research by introducing a minimalist approach to patient-specific adaptive dose prediction. In contrast to our prior method, which involved fine-tuning a pre-trained population model, this new method trains a model from scratch using only a patient's initial treatment data. This patient-specific dose predictor aims to enhance clinical accessibility, thereby empowering physicians and treatment planners to make more informed, quantitative decisions in ART. We hypothesize that patient-specific DL models will provide more accurate adaptive dose predictions for their respective patients compared to a population-based DL model. We selected 33 patients to train an adaptive population-based (AP) model. Ten additional patients were selected, and their respective initial RT data served as single samples for training patient-specific (PS) models. These 10 patients contained an additional 26 ART plans that were withheld as the test dataset to evaluate AP versus PS model dose prediction performance. We assessed model performance using Mean Absolute Percent Error (MAPE) by comparing predicted doses to the originally delivered ground truth doses. We used the Wilcoxon signed-rank test to determine statistically significant differences in terms of MAPE between the AP and PS model results across the test dataset. Furthermore, we calculated differences between predicted and ground truth mean doses for segmented structures and determined statistical significance in the differences for each of them. The average MAPE across AP and PS model dose predictions was 5.759% and 4.069%, respectively. The Wilcoxon signed-rank test yielded two-tailed p-value= , indicating that the MAPE differences between the AP and PS model dose predictions are statistically significant, and 95% confidence interval=[-2.1610, -1.0130], indicating 95% confidence that the MAPE difference between the AP and PS models for a population lies in this range. Out of 24 total segmented structures, the comparison of mean dose differences for 12 structures indicated statistical significance with two-tailed p-values<0.05. Our study demonstrates the potential of patient-specific deep learning models in application to ART. Notably, our method streamlines the training process by minimizing the size of the required training dataset, as only a single patient's initial treatment data is required. External institutions considering the implementation of such a technology could package such a model so that it only requires the upload of a reference treatment plan for model training and deployment. Our single patient learning strategy demonstrates promise in ART due to its minimal dataset requirement and its utility in personalization of cancer treatment.

Detection of backdoor attacks using targeted universal adversarial perturbations for deep neural networks

Backdoor attacks on deep neural networks (DNNs) using targeted universal adversarial perturbations (TUAPs) do not require training datasets and model tampering, and triggers based on TUAPs can make DNNs output any class the adversary wants. Retraining DNNs using adversarial training for security is time-consuming and does not apply to DNNs in runtime. We want to detect backdoors using a black-box testing approach. We observe that after superimposing random noise on the input of a backdoor attack, the output still tends to remain the same, so we propose Sequential Analysis method based on the Metamorphosis Testing (SAMT). We designed two metamorphic relations for test case generation. Using sequential sampling, we calculate the label stability rate (LSR) and infer whether the image to be verified contains a trigger based on the sequential probability ratio change. The experimental results show that our method has a higher backdoor detection success rate (dsr) than the state-of-the-art detection algorithms. Moreover, our method does not need to use the model structure of DNNs, which has more adaptability and generalization ability. Based on our proposed method, we can simply add a backdoor detection layer to detect backdoors as early as possible, which can eventually alleviate the harm of such backdoors.

Physics-informed Bayesian inference of external potentials in classical density-functional theory.

The swift progression and expansion of machine learning (ML) have not gone unnoticed within the realm of statistical mechanics. In particular, ML techniques have attracted attention by the classical density-functional theory (DFT) community, as they enable automatic discovery of free-energy functionals to determine the equilibrium-density profile of a many-particle system. Within classical DFT, the external potential accounts for the interaction of the many-particle system with an external field, thus, affecting the density distribution. In this context, we introduce a statistical-learning framework to infer the external potential exerted on a classical many-particle system. We combine a Bayesian inference approach with the classical DFT apparatus to reconstruct the external potential, yielding a probabilistic description of the external-potential functional form with inherent uncertainty quantification. Our framework is exemplified with a grand-canonical one-dimensional classical particle ensemble with excluded volume interactions in a confined geometry. The required training dataset is generated using a Monte Carlo (MC) simulation where the external potential is applied to the grand-canonical ensemble. The resulting particle coordinates from the MC simulation are fed into the learning framework to uncover the external potential. This eventually allows us to characterize the equilibrium density profile of the system by using the tools of DFT. Our approach benchmarks the inferred density against the exact one calculated through the DFT formulation with the true external potential. The proposed Bayesian procedure accurately infers the external potential and the density profile. We also highlight the external-potential uncertainty quantification conditioned on the amount of available simulated data. The seemingly simple case study introduced in this work might serve as a prototype for studying a wide variety of applications, including adsorption, wetting, and capillarity, to name a few.

Deep learning and deep transfer learning-based OPM for FMF systems

Optical performance monitoring (OPM) is indispensable to guarantee stable and reliable operation in few-mode fiber (FMF)-based transmission. OPM consists of measuring optical phenomena such as generalized signal-to-noise ratio (GSNR) based on analytical models. GSNR comprises nonlinear interference (NLI) noise which can be calculated either by exact analytical models e.g., enhanced Gaussian noise (EGN) model which is accurate but computationally complex, or asymptotic analytical models e.g., closed-form EGN model which are approximate but computationally fast. In this paper, we employ deep learning (DL) as an accurate and fast alternative for OPM in FMF-based transmission. However, DL-based OPM requires a large dataset to achieve proper performance whilst it is very difficult and time-consuming to obtain a large field or synthetic dataset. Regarding this issue, we develop deep transfer learning (DTL) for OPM in FMF to realize a fast response requiring a small training dataset and a few training epochs despite various changes in system/link parameters such as launched power, fiber type, and the number of modes. Results show root mean squared error of GSNR estimation is less than 0.02 dB for DL and DTL-based OPM methods. Compared to DL-based OPM, DTL-based OPM records 3 and 5 times reduction in required training dataset size and the number of epochs, respectively which is beneficial for real-time applications

Effect of Dataset Size and Medical Image Modality on Convolutional Neural Network Model Performance for Automated Segmentation: A CT and MR Renal Tumor Imaging Study.

The aim of this study is to investigate the use of an exponential-plateau model to determine the required training dataset size that yields the maximum medical image segmentation performance. CT and MR images of patients with renal tumors acquired between 1997 and 2017 were retrospectively collected from our nephrectomy registry. Modality-based datasets of 50, 100, 150, 200, 250, and 300 images were assembled to train models with an 80-20 training-validation split evaluated against 50 randomly held out test set images. A third experiment using the KiTS21 dataset was also used to explore the effects of different model architectures. Exponential-plateau models were used to establish the relationship of dataset size to model generalizability performance. For segmenting non-neoplastic kidney regions on CT and MR imaging, our model yielded test Dice score plateaus of [Formula: see text] and [Formula: see text] with the number of training-validation images needed to reach the plateaus of 54 and 122, respectively. For segmenting CT and MR tumor regions, we modeled a test Dice score plateau of [Formula: see text] and [Formula: see text], with 125 and 389 training-validation images needed to reach the plateaus. For the KiTS21 dataset, the best Dice score plateaus for nn-UNet 2D and 3D architectures were [Formula: see text] and [Formula: see text] with number to reach performance plateau of 177 and 440. Our research validates that differing imaging modalities, target structures, and model architectures all affect the amount of training images required to reach a performance plateau. The modeling approach we developed will help future researchers determine for their experiments when additional training-validation images will likely not further improve model performance.

How Many Data Does Machine Learning in Human–Computer Interaction Need?: Re-Estimating the Dataset Size for Convolutional Neural Network-Based Models of Visual Perception

Artificial intelligence (AI)-based user-interface (UI) design and evaluation are currently constrained by the scarcity of human-generated training data. Correspondingly, choosing appropriate neural network (NN) architecture and carefully planning the sample size is essential for building accurate machine learning models. Previously, we have estimated that for a convolutional NN to produce better mean square errors (MSEs) than feature-based models, the required training dataset size should be approximately 3000. Our current validation study with roughly 4000 web UIs and 233 subjects suggests that the estimation should be closer to 17,000. We propose corrected regression models suggesting that the dataset size effect is better described using a logarithmic function. We also report significant differences in MSEs between the employed perception dimensions, with Aesthetics models having an MSE 21.5% worse than Complexity and 12.1% worse than Orderliness.

Question answering with deep neural networks for semi-structured heterogeneous genealogical knowledge graphs

With the rising popularity of user-generated genealogical family trees, new genealogical information systems have been developed. State-of-the-art natural question answering algorithms use deep neural network (DNN) architecture based on self-attention networks. However, some of these models use sequence-based inputs and are not suitable to work with graph-based structure, while graph-based DNN models rely on high levels of comprehensiveness of knowledge graphs that is nonexistent in the genealogical domain. Moreover, these supervised DNN models require training datasets that are absent in the genealogical domain. This study proposes an end-to-end approach for question answering using genealogical family trees by: (1) representing genealogical data as knowledge graphs, (2) converting them to texts, (3) combining them with unstructured texts, and (4) training a transformer-based question answering model. To evaluate the need for a dedicated approach, a comparison between the fine-tuned model (Uncle-BERT) trained on the auto-generated genealogical dataset and state-of-the-art question-answering models was performed. The findings indicate that there are significant differences between answering genealogical questions and open-domain questions. Moreover, the proposed methodology reduces complexity while increasing accuracy and may have practical implications for genealogical research and real-world projects, making genealogical data accessible to experts as well as the general public.

Deep Prior Approach for Room Impulse Response Reconstruction

In this paper, we propose a data-driven approach for the reconstruction of unknown room impulse responses (RIRs) based on the deep prior paradigm. We formulate RIR reconstruction as an inverse problem. More specifically, a convolutional neural network (CNN) is employed prior, in order to obtain a regularized solution to the RIR reconstruction problem for uniform linear arrays. This approach allows us to avoid assumptions on sound wave propagation, acoustic environment, or measuring setting made in state-of-the-art RIR reconstruction algorithms. Moreover, differently from classical deep learning solutions in the literature, the deep prior approach employs a per-element training. Therefore, the proposed method does not require training data sets, and it can be applied to RIRs independently from available data or environments. Results on simulated data demonstrate that the proposed technique is able to provide accurate results in a wide range of scenarios, including variable direction of arrival of the source, room , and SNR at the sensors. The devised technique is also applied to real measurements, resulting in accurate RIR reconstruction and robustness to noise compared to state-of-the-art solutions.

Contour, a semi-automated segmentation and quantitation tool for cryo-soft-X-ray tomography.

Cryo-soft-X-ray tomography is being increasingly used in biological research to study the morphology of cellular compartments and how they change in response to different stimuli, such as viral infections. Segmentation of these compartments is limited by time-consuming manual tools or machine learning algorithms that require extensive time and effort to train. Here we describe Contour, a new, easy-to-use, highly automated segmentation tool that enables accelerated segmentation of tomograms to delineate distinct cellular compartments. Using Contour, cellular structures can be segmented based on their projection intensity and geometrical width by applying a threshold range to the image and excluding noise smaller in width than the cellular compartments of interest. This method is less laborious and less prone to errors from human judgement than current tools that require features to be manually traced, and does not require training datasets as would machine-learning driven segmentation. We show that high-contrast compartments such as mitochondria, lipid droplets, and features at the cell surface can be easily segmented with this technique in the context of investigating herpes simplex virus 1 infection. Contour can extract geometric measurements from 3D segmented volumes, providing a new method to quantitate cryo-soft-X-ray tomography data. Contour can be freely downloaded at github.com/kamallouisnahas/Contour.

An Energy-Fraud Detection-System Capable of Distinguishing Frauds from Other Energy Flow Anomalies in an Urban Environment

Energy fraud detection bears significantly on urban ecology. Reduced losses and power consumption would affect carbon dioxide emissions and reduce thermal pollution. Fraud detection also provides another layer of urban socio-economic correlation heatmapping and improves city energy distribution. This paper describes a novel algorithm of energy fraud detection, utilizing energy and energy consumption specialized knowledge poured into AI front-end. The proposed algorithm improves fraud detection’s accuracy and reduces the false positive rate, as well as reducing the preliminary required training dataset. The paper also introduces a holistic algorithm, specifying the major phenomena that disguises as energy fraud or affects it. Consequently, a mathematical foundation for energy fraud detection for the proposed algorithm is presented. The results show that a unique pattern is obtained during fraud, which is independent of a reference non-fraud pattern of the same customer. The theory is implemented on real data taken from smart metering systems and validated in real life scenarios.

Replacement of In-Orbit Extended Kalman Filter for Spacecraft Orbit Estimation via Neural Networks

Spacecraft orbit estimation on-board a spacecraft could be utilized in order to minimize measurement and process noise effects. This could be done through an estimation algorithm such as Kalman Filter (KF) or Extended Kalman Filter (EKF). EKF utilizes an orbital motion model of the spacecraft. The model is characterized by high degree of complexity, nonlinearity, and coupling between system states. This complexity increases the computational load of the EKF and may represent a serious problem for real-time application over the spacecraft on-board computer. This is due to limited spacecraft on-board computer computational resources. In order to overcome this problem, an algorithm based on Neural Networks is developed. The required training data set has been obtained via EKF. Various disturbance forces and moments are considered, such as earth’s oblatenss effect till (J4), aerodynamic drag, solar radiation pressure, gravity gradient moment, and magnetic disturbance moments. The developed neural networks algorithm is applied to a verified test case spacecraft, which utilizes a GPS receiver in order to obtain spacecraft position measurements. The developed neural networks algorithm has the same accuracy of the EKF with an average execution time equal to 82% of EKF. This indicates better applicability for real-time application.

Deep dose plugin: towards real-time Monte Carlo dose calculation through a deep learning-based denoising algorithm

Monte Carlo (MC) simulation is considered the gold standard method for radiotherapy dose calculation. However, achieving high precision requires a large number of simulation histories, which is time-consuming. The use of computer graphics processing units (GPUs) has greatly accelerated MC simulation and allows dose calculation within a few minutes for a typical radiotherapy treatment plan. However, some clinical applications demand real-time efficiency for MC dose calculation. To tackle this problem, we have developed a real-time, deep learning (DL)-based dose denoiser that can be plugged into a current GPU-based MC dose engine to enable real-time MC dose calculation. We used two different acceleration strategies to achieve this goal: (1) we applied voxel unshuffle and voxel shuffle operators to decrease the input and output sizes without any information loss, and (2) we decoupled the 3D volumetric convolution into a 2D axial convolution and a 1D slice convolution. In addition, we used a weakly supervised learning framework to train the network, which greatly reduces the size of the required training dataset and thus enables fast fine-tuning-based adaptation of the trained model to different radiation beams. Experimental results show that the proposed denoiser can run in as little as 39 ms, which is 11.6 times faster than the baseline model. As a result, the whole MC dose calculation pipeline can be finished within ∼ 0.15 s, including both GPU MC dose calculation and DL-based denoising, achieving the real-time efficiency needed for some radiotherapy applications, such as online adaptive radiotherapy.

Understanding the synergies of deep learning and data fusion of multispectral and panchromatic high resolution commercial satellite imagery for automated ice-wedge polygon detection

The utility of sheer volumes of very high spatial resolution (VHSR) commercial imagery in mapping the Arctic region is new and actively evolving. Commercial satellite sensors typically record image data in low-resolution multispectral (MS) and high-resolution panchromatic (PAN) mode. Spatial resolution is needed to accurately describe feature shapes and textural patterns, such as ice-wedge polygons (IWPs) that are rapidly transforming surface features due to degrading permafrost, while spectral resolution allows capturing of land-use and land-cover types. Data fusion, the process of combining PAN and MS images with complementary characteristics often serves as an integral component of remote sensing mapping workflows. The fusion process generates spectral and spatial artifacts that may affect the classification accuracies of subsequent automated image analysis algorithms, such as deep learning (DL) convolutional neural nets (CNN). We employed a detailed multidimensional assessment to understand the performances of an array of eight application-oriented data fusion algorithms when applied to VHSR image scenes for DLCNN-based mapping of ice-wedge polygons. Our findings revealed the scene dependency of data fusion algorithms and emphasized the need for careful selection of the proper algorithm. Results suggested that the fusion algorithms that preserve spatial character of original PAN imagery favor the DLCNN model performances. The choice of fusion approach needs to be considered of equal importance to the required training dataset for successful applications using DLCNN on VHRS imagery in order to enable an accurate mapping effort of permafrost thaw across the Arctic region.

From Satellite to Supply Chain: New Approaches Connect Earth Observation to Economic Decisions

Global supply chains shift environmental and social impacts of consumption to remote locations. This opacity challenges many sustainability goals. To help businesses and governments realize more sustainable supply chains, new approaches are using spatial data and machine-learning techniques to connect Earth observation data to conventional economic tools.

Solder Joint Reliability Modeling by Sequential Artificial Neural Network for Glass Wafer Level Chip Scale Package

This article combines the sequential artificial neural network (NN) machine learning with finite element (FE) modeling to assess the solder joint thermal cycling performance. A glass wafer-level chip-scale package (G-WLCSP) is used for this study. This article investigates the network structure that can achieve prediction capability both inside and outside the design domain with the minimal required training dataset. First, a detailed FE model for G-WLCSP is developed to obtain the accumulated plastic strain per cycle for thermal-cycling loading. Three critical input parameters are defined to generate a dataset based on finite element analysis. Then, applying the supervised machine learning procedure, both the recurrent neural network (RNN) and the gate-network long short-term memory (LSTM) architecture are used to train the obtained dataset. The network complexity of the sequential NN model is carefully controlled to prevent numerical overfitting. Among the total 81 FE generated data pairs, only 27 data pairs have been applied to the sequential NN learning. These 27 data pairs are carefully selected to evenly distributed among the design domain. The average error norms after the learning are 1.213 · 10 -4 and 1.190 · 10 -4 of RNN and LSTM, respectively. The prediction capability of the well-trained sequential NN model against the rest 54 data pairs has been tested and a similar scale has been obtained. Furthermore, the prediction capability is tested against the parameters outside the design domain. Approximately one order average error norm increased for both the well-trained RNN and LSTM model.

GI-Cluster: Detecting genomic islands via consensus clustering on multiple features.

The accurate detection of genomic islands (GIs) in microbial genomes is important for both evolutionary study and medical research, because GIs may promote genome evolution and contain genes involved in pathogenesis. Various computational methods have been developed to predict GIs over the years. However, most of them cannot make full use of GI-associated features to achieve desirable performance. Additionally, many methods cannot be directly applied to newly sequenced genomes. We develop a new method called GI-Cluster, which provides an effective way to integrate multiple GI-related features via consensus clustering. GI-Cluster does not require training datasets or existing genome annotations, but it can still achieve comparable or better performance than supervised learning methods in comprehensive evaluations. Moreover, GI-Cluster is widely applicable, either to complete and incomplete genomes or to initial GI predictions from other programs. GI-Cluster also provides plots to visualize the distribution of predicted GIs and related features. GI-Cluster is available at https://github.com/icelu/GI_Cluster.

Unsupervised 3D Prostate Segmentation Based on Diffusion-Weighted Imaging MRI Using Active Contour Models with a Shape Prior

Accurate estimation of the prostate location and volume fromin vivoimages plays a crucial role in various clinical applications. Recently, magnetic resonance imaging (MRI) is proposed as a promising modality to detect and monitor prostate-related diseases. In this paper, we propose an unsupervised algorithm to segment prostate with 3D apparent diffusion coefficient (ADC) images derived from diffusion-weighted imaging (DWI) MRI without the need of a training dataset, whereas previous methods for this purpose require training datasets. We first apply a coarse segmentation to extract the shape information. Then, the shape prior is incorporated into the active contour model. Finally, morphological operations are applied to refine the segmentation results. We apply our method to an MR dataset obtained from three patients and provide segmentation results obtained by our method and an expert. Our experimental results show that the performance of the proposed method is quite successful.