Small Set Of Experimental Data Research Articles

Neural network–based transfer learning to improve stiffness modeling of industrial robots with small experimental data sets

AbstractStiffness modeling is an essential subject for the composition of robot control. Accurate stiffness modeling is helpful for improving the control accuracy of industrial robots, particularly under dynamic load circumstances. The classic virtual joint modeling (VJM) method is challenging in predicting the deformation of the end-effector throughout the full workspace due to the nonlinear deformation of the robot joint and its serial articulated structure. This paper proposes a full-space stiffness modeling method for robots based on the integration of a multi-layer perceptual (MLP) model and VJM. To provide enough training data for the MLP model, VJM is used to build a stiffness model with a small set of experimental data to generate 106,400 training data. A model-based transfer learning approach is proposed to improve the model’s accuracy and generalization regarding the difference between generated training data and actual experimental data. The VJM stiffness model is compared with the MLP stiffness model and the existing CNN-based transfer learning model based on the same experimental data. Considering the deformation prediction in the three directions in Cartesian space, the mean absolute error, standard deviation, and maximum error of the MLP model are decreased by at least 24.90%, 14.20%, and 8.50%, respectively, than the VJM. These prediction results demonstrate that the proposed modeling technique can significantly increase the accuracy of robot stiffness modeling, which is essential for position compensation in precise motion control of robots under dynamic load.

Massive experimental quantification of amyloid nucleation allows interpretable deep learning of protein aggregation.

Protein aggregation is a pathological hallmark of more than fifty human diseases and a major problem for biotechnology. Methods have been proposed to predict aggregation from sequence, but these have been trained and evaluated on small and biased experimental datasets. Here we directly address this data shortage by experimentally quantifying the amyloid nucleation of >100,000 protein sequences. This unprecedented dataset reveals the limited performance of existing computational methods and allows us to train CANYA, a convolution-attention hybrid neural network that accurately predicts amyloid nucleation from sequence. We adapt genomic neural network interpretability analyses to reveal CANYA's decision-making process and learned grammar. Our results illustrate the power of massive experimental analysis of random sequence-spaces and provide an interpretable and robust neural network model to predict amyloid nucleation.

A novel deep learning strategy to optimize Al2O3–SiO2 porous ceramics with phosphate tailings as raw material

The Al2O3–SiO2 porous ceramics (ASPC) prepared from phosphorus tailings usually showed a large fluctuation in material properties due to the complex composition of the raw material, which required extensive experimental exploration to optimize their properties. To solve the above problem, a novel deep learning strategy for automatically adjust hyperparameters had been proposed to predict the property of ASPC based on the small experimental datasets in this work. Initially, phosphate tailings and bauxite were used as raw materials. Active carbon, ranging from 0 to 15 wt%, was added as a pore-forming agent. After mixing the raw materials, they were shaped under a pressure of 5 MPa and sintered in the temperature range of 1000–1200 °C to produce ASPC, the density and compressive strength of the ASPC were measured to train the artificial neural network (ANN). An exhaustive search was conducted to identify the optimal combination of hyperparameters suitable for small datasets, then the influence of each hyperparameter on the training performance of ANN was analyzed. Subsequently, the trained model was used to predict the feature space of ASPC, and the key processes for regulating ASPC performance were explored, thereby the experimental schemes that meet target performance (density and compressive strength) were identified. Three groups of experimental schemes were predicted, whose properties were compared with the experimental ones. The results showed that the mean squared errors (MSE) of 0.03 and 0.85 for density and compressive strength, respectively. Furthermore, the ASPC exhibited excellent comprehensive properties when 70 wt% phosphorus tailings and 30 wt% bauxite were added, whose apparent porosity was 59.11 %, density was 1.16 g/cm3 and cold compressive strength was 6.49 MPa. And the predicted data trend influence of the pore-forming agent content and sintering temperature on the performance of ASPC is the same as the experimental data. This research provides valuable guidance for addressing the limited controllability of traditional experimental design processes and accelerate the preparation of ASPC from solid waste.

Enhanced Sequence-Activity Mapping and Evolution of Artificial Metalloenzymes by Active Learning.

Tailored enzymes are crucial for the transition to a sustainable bioeconomy. However, enzyme engineering is laborious and failure-prone due to its reliance on serendipity. The efficiency and success rates of engineering campaigns may be improved by applying machine learning to map the sequence-activity landscape based on small experimental data sets. Yet, it often proves challenging to reliably model large sequence spaces while keeping the experimental effort tractable. To address this challenge, we present an integrated pipeline combining large-scale screening with active machine learning, which we applied to engineer an artificial metalloenzyme (ArM) catalyzing a new-to-nature hydroamination reaction. Combining lab automation and next-generation sequencing, we acquired sequence-activity data for several thousand ArM variants. We then used Gaussian process regression to model the activity landscape and guide further screening rounds. Critical characteristics of our pipeline include the cost-effective generation of information-rich data sets, the integration of an explorative round to improve the model's performance, and the inclusion of experimental noise. Our approach led to an order-of-magnitude boost in the hit rate while making efficient use of experimental resources. Search strategies like this should find broad utility in enzyme engineering and accelerate the development of novel biocatalysts.

Accurate Prediction of Punching Shear Strength of Steel Fiber-Reinforced Concrete Slabs: A Machine Learning Approach with Data Augmentation and Explainability

Reinforced concrete slabs are widely used in building structures due to their economic, durable, and aesthetic advantages. The determination of their ultimate strength often hinges on punching shear strength. Presently, methods such as closed hoops, steel bending, and fiber reinforcement are employed to enhance punching shear strength, with fiber reinforcement gaining popularity due to its ease of implementation and efficacy in improving concrete durability. This study introduces a novel approach employing six machine learning algorithms rooted in decision trees and decision tree-based ensemble learning to predict punching shear strength in steel fiber-reinforced concrete slabs. To overcome experimental data limitations, a data augmentation approach based on the Gaussian mixture model is employed. The validation of the data augmentation is conducted through “synthetic training—real testing” and “real training—real testing”. Additionally, the best machine learning model is analyzed for explainability using Shapley Additive exPlanation (SHAP). Results demonstrate that the proposed data augmentation method effectively captures the original data distribution, enhancing the robustness and accuracy of the machine learning model. Moreover, SHAP provides better insights into the features influencing punching shear strength. Thus, the proposed data enhancement model offers a reliable approach for modeling small experimental datasets in structural engineering.

WVEHDD: Weighted Voting based Ensemble System for Heart Disease Detection

Although several machine learning (ML) based algorithms are proposed by various researchers for Heart Disease detection (HDD), most of these works considered a very small experimental dataset to justify the efficiency of ML techniques in HDD. Moreover, despite of the low correlation of the features with the target, all the features were used for HDD. Considering the limitations of these existing systems, current study emphasizes on the designing of a Weighted Voting based Ensemble (WVE) Classifier for HDD from a sufficiently large dataset comprising of 1296 instances. Although there are 13 features, only 4 features are found to be statistically significant in HDD. For designing an efficient WVE classifier for HDD, the weighted votes of five efficient classifiers are combined to get the final decision. The experimental result shows that the proposed WVEHDD system outperforms the existing systems by providing the highest train accuracy of 96.15% and test accuracy of 95.64%

Resolution-enhanced multi-core fiber imaging learned on a digital twin for cancer diagnosis.

Deep learning enables label-free all-optical biopsies and automated tissue classification. Endoscopic systems provide intraoperative diagnostics to deep tissue and speed up treatment without harmful tissue removal. However, conventional multi-core fiber (MCF) endoscopes suffer from low resolution and artifacts, which hinder tumor diagnostics. We introduce a method to enable unpixelated, high-resolution tumor imaging through a given MCF with a diameter of around 0.65mm and arbitrary core arrangement and inhomogeneous transmissivity. Image reconstruction is based on deep learning and the digital twin concept of the single-reference-based simulation with inhomogeneous optical properties of MCF and transfer learning on a small experimental dataset of biological tissue. The reference provided physical information about the MCF during the training processes. For the simulated data, hallucination caused by the MCF inhomogeneity was eliminated, and the averaged peak signal-to-noise ratio and structural similarity were increased from 11.2dB and 0.20 to 23.4dB and 0.74, respectively. By transfer learning, the metrics of independent test images experimentally acquired on glioblastoma tissue ex vivo can reach up to 31.6dB and 0.97 with 14fps computing speed. With the proposed approach, a single reference image was required in the pre-training stage and laborious acquisition of training data was bypassed. Validation on glioblastoma cryosections with transfer learning on only 50 image pairs showed the capability for high-resolution deep tissue retrieval and high clinical feasibility.

Extrapolative prediction of small-data molecular property using quantum mechanics-assisted machine learning

Data-driven materials science has realized a new paradigm by integrating materials domain knowledge and machine-learning (ML) techniques. However, ML-based research has often overlooked the inherent limitation in predicting unknown data: extrapolative performance, especially when dealing with small-scale experimental datasets. Here, we present a comprehensive benchmark for assessing extrapolative performance across 12 organic molecular properties. Our large-scale benchmark reveals that conventional ML models exhibit remarkable performance degradation beyond the training distribution of property range and molecular structures, particularly for small-data properties. To address this challenge, we introduce a quantum-mechanical (QM) descriptor dataset, called QMex, and an interactive linear regression (ILR), which incorporates interaction terms between QM descriptors and categorical information pertaining to molecular structures. The QMex-based ILR achieved state-of-the-art extrapolative performance while preserving its interpretability. Our benchmark results, QMex dataset, and proposed model serve as valuable assets for improving extrapolative predictions with small experimental datasets and for the discovery of novel materials/molecules that surpass existing candidates.

Kinetics-constrained neural ordinary differential equations: Artificial neural network models tailored for small data to boost kinetic model development

Artificial neural networks (ANNs) are powerful tools for solving a wide range of tasks in fundamental and applied science. However, training and building reliable ANN models requires a lot of data which so far hinders their wider application in kinetic modelling where typically only small (experimental) datasets are available. In the present work we propose a method to design ANN models for kinetic modelling that can be trained even with small data sets as are typically available. The key idea is to constrain the architecture of the ANN models by integrating kinetic and thermodynamic knowledge leading to what we call Kinetics-Constrained Neural Ordinary Differential Equations (KCNODE). The feasibility and effectiveness of the approach is first demonstrated in a numerical experiment using the catalytic hydrogenation of CO2 to methane as example. Next, we demonstrate the approach for real experimental data of a more complex reaction, the hydrogenation of CO2 to higher hydrocarbons (CO2-FT). Finally, the ANN trained for CO2-FT is used to derive an improved mechanistic model for the reverse water gas shift reaction which is a key reaction in the CO2-FT reaction network. This last step exemplifies how the opportunity to obtain reliable ANN models from small data opens new ways to approach kinetic model development.

Assessing Weak Adhesion in Single Lap Joints Using Lamb Waves and Machine Learning Methods for Structural Health Monitoring

The use of adhesive joints has become increasingly popular in various industries due to their many benefits, such as low weight and good mechanical performance. However, adhesive joints can suffer from defects, one of them being weak adhesion. This defect poses a significant risk to structural integrity and can lead to premature failure, but is hard to detect using existing nondestructive testing methods. Therefore, there is a need for an effective technique that can detect weak adhesion in single-lap joints (SLJ) to prevent failure and assist in maintenance, namely in the framework of structural health monitoring. This paper presents a novel approach utilizing machine learning and Lamb Waves (LW) to determine the level of weak adhesion. Firstly, a numerical model of SLJs with different levels of weak adhesion is created and an original approach is proposed for its validation with data from real samples so that reliable LW data can further be easily generated to train and test any other data-driven algorithm for tackling damage. Secondly, a damage detection method is proposed, based on artificial neural networks and fed with simulated data, to determine the level of damage in SLJs, independent of their location. The results show that the simulation model can be validated with a small set of experimental data, being capable of replicating real damage in SLJs. Additionally, the use of simulated data in the training algorithm can increase the accuracy of the simulation model up to 26% when compared to only considering experimental data. The adopted artificial neural network for detecting weak adhesion emerges as a promising approach, yielding a precision of over 95%. Thus, machine learning and LW data can be used to improve the reliability and accuracy of adhesive bonding quality control, as well function as a technique for structural health monitoring, which can enhance the safety and durability of bonded structures.

A versatile strategy for hybridizing small experimental and large simulation data: A case for ceramic tape-casting process

In manufacturing industry, finding optimal design parameters for targeted properties has traditionally been guided by trial and error. However, limited data availability to few hundreds sets of experimental data in typical materials processes, the machine-learning capabilities and other data-driven modeling (DDM) techniques are too far from it to be practical. In this study, we show how a versatile design strategy, tightly coupled with physics-based modeling (PBM) data, can be applied to small set of experimental data to improve the optimization of process parameters. Our strategy uses PBM to achieve augmented data that includes essential physics: in other words, the PBM data allows the inverse design model to ‘learn’ physics, indirectly. We demonstrated the accuracy of both forward-prediction and inverse-optimization have been dramatically improved with the help of PBM data, which are relatively cheap and abundant. Furthermore, we found that the inverse model with augmented data can accurately optimize process parameters, even for ones those were not considered in the simulation. Such versatile strategy can be helpful for processes/experiments for the cases where the number of collectable data is limited, which is most of the case in industries.

A Comparative Study on Physics-Informed Machine Learning for Battery Degradation Diagnostics

Quantifying the extent of degradation in lithium-ion battery cells using non-destructive approaches can provide valuable insights into the cell’s state of health and remaining useful life. However, physics-based degradation diagnostic methods typically require collecting long-term aging data and are computationally expensive to deploy locally on a device. This work investigates combining physics-based modeling and machine learning to retain high diagnostic accuracy while mitigating the need to collect long-term degradation data. To transfer knowledge from a physics-based model to a machine learning model, we develop two new approaches: co-kriging and physics-informed neural networks. We aim to effectively diagnose cell health in the late aging stage without needing long-term degradation data. Using a small set of early-life experimental data collected from an aging experiment, we train a co-kriging machine learning model to learn the difference between the experimental data and a set of simulation data generated from a physics-based half-cell model. Additionally, we train a neural network model using a physics-informed loss function with a new term that penalizes discrepancies between neural network model prediction and physics-based model prediction. The trained models can then be used to diagnose cell capacity and three primary component-level degradation modes at a future time. The proposed methods and two existing physics-informed machine learning methods named data augmentation and delta learning are comprehensively evaluated using data from a long-term (4+ years) cycle aging experiment of 24 implantable-grade Li-ion cells cycled under two different temperatures and C-rates. Cross-validated results show that the proposed physics-informed machine learning models can significantly improve the estimation accuracy of cell capacity and three primary degradation modes compared to a purely data-driven approach. Furthermore, this work comprehensively analyzes the trends of the degradation modes and provides insight into the long-term performance of high-quality implantable-grade lithium-ion cells.

Optimization of biocementation responses by artificial neural network and random forest in comparison to response surface methodology.

In this article, the optimization of the specific urease activity (SUA) and the calcium carbonate (CaCO3) using microbially induced calcite precipitation (MICP) was compared to optimization using three algorithms based on machine learning: random forest regressor, artificial neural networks (ANNs), and multivariate linear regression. This study applied the techniques in two existing response surface method (RSM) experiments involving MICP technique. Random forest-based models and artificial neural network-based models were submitted through the optimization of hyperparameters via cross-validation technique and grid search, to select the best-optimized model. For this study, the random forest-based algorithm is aimed at having the best performance of 0.9381 and 0.9463 in comparison to the original r2 of 0.9021 and 0.8530, respectively. This study is aimed at exploring the capability of using machine learning-based models in small datasets for the purpose of optimization of experimental variables in MICP technique and the meaningfulness of the models by their specificities in the small experimental datasets applied to experimental designs. This study is aimed at exploring the capability of using machine learning-based models in small datasets for experimental variable optimization in MICP technique. The use of these techniques can create prerogatives to scale and mitigate costs in future experiments associated to the field.

Physics‐informed Transfer Learning for Out‐of‐sample Vapor Pressure Predictions

AbstractRecent advances have enabled machine learning methodologies developed for large datasets to be applied to the small experimental datasets typically available for chemical systems. Such advances typically involve a data‐based approach to transfer learning, where a portion of the experimental data for the property of interest is used to fine‐tune a model that is pre‐trained on computationally generated data. This transfer learning approach does not work for very small experimental datasets, where there are only enough data for model validation. Here, we develop a physics‐informed transfer learning strategy to train a directed‐message passing neural network (D‐MPNN) model, enabling extrapolation outside of the training domain. We demonstrate this approach by training a D‐MPNN model on interpolated vapor pressures and validate the model on an out‐of‐sample test set of energetic molecule vapor pressures, achieving accuracy comparable to those of experiments.

Bayesian optimization of ball milling conditions to obtain highly crystal-oriented Sm-Fe-N powder

The intrinsic magnetic properties of Sm2Fe17N3 are comparable or superior to those of Nd2Fe14B alloys. However, these properties have not been successfully demonstrated to date. Ball milling is commonly used to process Sm2Fe17N3 alloys because it can refine the particle size to increase their coercivity. Surfactant-assisted ball milling (SABM) produces crystal-oriented flake-like particles. Increasing the milling time and speed reduces the particle size, whereas increasing the oleic acid content changes the particle morphology from spherical to flaky. However, ball milling decomposes the Sm2Fe17N3 phase into the magnetically soft α-Fe phase, which is detrimental to remanence. Therefore, we applied the Bayesian optimization method to optimize the process conditions. Experimental process systems in laboratories are not fully established; therefore, experimental data sometime include outliers, which makes acquiring models from learning datasets difficult. In the case of a small experimental dataset, identifying an outlier is difficult. We proposed a Bayesian optimization with kernel function selection and a data screening method for a learning dataset that includes unexpected outliers. We examined the proposed system for the optimization of the SABM of Sm2Fe17N3 using actual experimental results, including outliers. The combination of kernel function selection and data screening was effective in allowing outliers to be identified in small datasets. Knowledge of the process was incorporated during data screening to optimize the process conditions, allowing the formation of a highly crystal-oriented powder.

Rapid prediction of full spin systems using uncertainty-aware machine learning.

Accurate simulation of solution NMR spectra requires knowledge of all chemical shift and scalar coupling parameters, traditionally accomplished by heuristic-based techniques or ab initio computational chemistry methods. Here we present a novel machine learning technique which combines uncertainty-aware deep learning with rapid estimates of conformational geometries to generate Full Spin System Predictions with UnCertainty (FullSSPrUCe). We improve on previous state of the art in accuracy on chemical shift values, predicting protons to within 0.209 ppm and carbons to within 1.213 ppm. Further, we are able to predict all scalar coupling values, unlike previous GNN models, achieving 3JHH accuracies between 0.838 Hz and 1.392 Hz on small experimental datasets. Our uncertainty quantification shows a strong, useful correlation with accuracy, with the most confident predictions having significantly reduced error, including our top-80% most confident proton shift predictions having an average error of only 0.140 ppm. We also properly handle stereoisomerism and intelligently augment experimental data with ab initio data through disagreement regularization to account for deficiencies in training data.

Deep learning strategies for addressing issues with small datasets in 2D materials research: Microbial Corrosion.

Protective coatings based on two dimensional materials such as graphene have gained traction for diverse applications. Their impermeability, inertness, excellent bonding with metals, and amenability to functionalization renders them as promising coatings for both abiotic and microbiologically influenced corrosion (MIC). Owing to the success of graphene coatings, the whole family of 2D materials, including hexagonal boron nitride and molybdenum disulphide are being screened to obtain other promising coatings. AI-based data-driven models can accelerate virtual screening of 2D coatings with desirable physical and chemical properties. However, lack of large experimental datasets renders training of classifiers difficult and often results in over-fitting. Generate large datasets for MIC resistance of 2D coatings is both complex and laborious. Deep learning data augmentation methods can alleviate this issue by generating synthetic electrochemical data that resembles the training data classes. Here, we investigated two different deep generative models, namely variation autoencoder (VAE) and generative adversarial network (GAN) for generating synthetic data for expanding small experimental datasets. Our model experimental system included few layered graphene over copper surfaces. The synthetic data generated using GAN displayed a greater neural network system performance (83-85% accuracy) than VAE generated synthetic data (78-80% accuracy). However, VAE data performed better (90% accuracy) than GAN data (84%-85% accuracy) when using XGBoost. Finally, we show that synthetic data based on VAE and GAN models can drive machine learning models for developing MIC resistant 2D coatings.

Method for predicting tool condition using kriging analysis of normalised frequency shifts of spindle acceleration signature and validation in trimming operations of CFRPs

ABSTRACT This work presents a method that deals with the problem of determining tool wear, surface roughness, and cutter temperate in the machining of a carbon fibre-reinforced polymer (CFRP) workpiece without the need for removing the tool for inspection. The spindle acceleration signature is dominated by the cutter passing frequency and its first harmonic. Cutting parameters and tool conditions strongly correlate with shifts in the latter two frequencies. The kriging method is utilised to construct prediction models of tool conditions using cutting parameters and spectral shifts. The presented Kriging-based method is unique in its ability to generate acceptable predictions of tool conditions using a small set of experimental data to construct the prediction models. Validation experiments have shown that the proposed method predicted tool wear, surface roughness, and cutter temperature with accuracies of 91%, 86%, and 84%, respectively. Benchmarking of the proposed method against another prediction method has shown acceptable performance.

Athermal Microwave Photonic Sensor Based on Single Microring Resonance Assisted by Machine Learning

We propose a machine learning (ML) assisted athermal microwave photonic (MWP) sensing scheme with high resolution based on a single microring resonance. The immunity of temperature interference of the high-resolution sensing is achieved by employing MWP sideband processing based interrogation, and supervised machine learning based on support vector regression (SVR) and neural tangent kernel (NTK) that are effective on small datasets. The MWP sideband processing transforms the variation of the target measurand into the shift of an ultra-deep notch in the radio frequency (RF) spectrum relieving the fabrication requirements on the microresonators, while ML accurately predicts the measurand by using the modulator bias voltage or RF passband transmission together with the RF notch position. The proposed sensor is demonstrated for relative humidity (RH) measurements using a silicon-on-insulator microring coated with polymethyl methacrylate. About 50 dB high RF rejection ratio is achieved over the sensing process, indicating a high sensing resolution. Despite the small experimental datasets, the established SVR and NTK models consistently exhibit lower mean absolute errors (MAEs) than the linear regression model in the RH prediction in the presence of temperature drifts. The NTK models achieve the lowest MAEs of 1.01% RH and 1.03% RH when the RF passband transmission and modulator bias are selected as the model input, respectively. The equivalent performances of the RF passband transmission and modulator bias voltage further demonstrate the feasibility of athermal sensing based solely on the MWP interrogation results of a single microring resonance, which simplifies the design and reduces the complexity.

Building Chemical Property Models for Energetic Materials from Small Datasets Using a Transfer Learning Approach.

For many experimentally measured chemical properties that cannot be directly computed from first-principles, the existing physics-based models do not extrapolate well to out-of-sample molecules, and experimental datasets themselves are too small for traditional machine learning (ML) approaches. To overcome these limitations, we apply a transfer learning approach, whereby we simultaneously train a multi-target regression model on a small number of molecules with experimentally measured values and a large number of molecules with related computed properties. We demonstrate this methodology on predicting the experimentally measured impact sensitivity of energetic crystals, finding that both characteristics of the computed dataset and model architecture are important to prediction accuracy of the small experimental dataset. Our directed-message passing neural network (D-MPNN) ML model using transfer learning outperforms direct-ML and physics-based models on a diverse test set, and the new methods described here are widely applicable to modeling many other structure-property relationships.