Small Data Research Articles

Improving small sample prediction performance via novel nonlinear interpolation virtual sample generation with self-supervised learning

In the early stages of production processes, due to expensive experimental costs and demanding experimental conditions, the amounts of collected samples are relatively limited. Information and knowledge extracted from minimal experimental data may be distorted and unreliable. Tackling this task, some researchers have recommended virtual sample generation (VSG), aiming at improving prediction accuracy of forecasting models for small sample sets by creating informative artificial samples. To more effectively capture nonlinear feature representations in small data, in this paper, we propose a promising VSG method based on self-supervised learning architecture to generate optimal, feasible virtual samples. The suggested technology considers distilling nonlinear feature representations using a manifold algorithm. Based on these representations, we integrated Newton’s divided difference and Chebyshev interpolation to create nonlinear interpolation points. We feed those interpolation points into a U-net model and an encoder-decoder net model to produce virtual sample inputs and output, respectively. Additionally, in this paper, we developed a Siamese network model to select virtual samples resembling original data. In the experiments, two industrial datasets were utilized to demonstrate efficacy of the proposed method. These experimental results show that the proposed method can more successfully boost prediction accuracy compared to the other three cutting-edge VSG methods.

Cuckoo search-artificial neural network aided the composition design in Al–Cr–Co–Fe–Ni high entropy alloys

In this work, we proposed a data-driven approach combining machine learning (ML), experiments and molecular dynamic (MD) simulation to accelerate the composition design and exploration of wear behavior in Al–Cr–Co–Fe–Ni based high entropy alloys. A Cuckoo Search-Artificial Neural Network (CS-ANN) model was proposed to deal with small sample data and showed fairly strong learning ability and generalization capabilities. Elemental analysis revealed that the microhardness can be enhanced by adding Al or reducing the proportion of Fe and Ni. Then the Al1.1CrCoFe0.9Ni0.9 HEAs were designed and prepared for experimental verification. The microstructure, phase composition, hardness, and wear resistance properties of the alloy were further investigated. Al1.1CrCoFe0.9Ni0.9 HEAs were composed of BCC+B2 phases, exhibited a microhardness of 580.6 HV. The wear rate was measured as 6.04×10−6 mm3/(Nm) by the reciprocating dry wear tests, the average COF was estimated as ∼0.5. The main wear mechanisms involved abrasive wear and oxidation wear. MD simulations are used to analyze the atomic evolution and deformation mechanism during the nano scratch process. The obtained information would help either facilitate developing novel HEAs or utilize the HEAs effectively to resist wear.

Fatigue Life Data Fusion Method of Different Stress Ratios Based on Strain Energy Density.

To accurately evaluate the probabilistic characteristics of the fatigue properties of materials with small sample data under different stress ratios, a data fusion method for torsional fatigue life under different stress ratios is proposed based on the energy method. A finite element numerical modeling method is used to calculate the fatigue strain energy density during fatigue damage. Torsional fatigue tests under different stresses and stress ratios are carried out to obtain a database for research. Based on the test data, the Wt-Nf curves under a single stress ratio and different stress ratios are calculated. The reliability of the models is illustrated by the scatter band diagram. More than 85% of points are within ±2 scatter bands, indicating that the fatigue life under different stress ratios can be represented by the same Wt-Nf curve. Furthermore, P-Wt-Nf prediction models are established to consider the probability characteristics. According to the homogeneity of the Wt-Nf model under different stress ratios, we can fuse the fatigue life data under different stress ratios and different strain energy densities. This data fusion method can expand the small sample test data and reduce the dispersion of the test data between different stress ratios. Compared with the pre-fusion data, the standard deviations of the post-fusion data are reduced by a maximum of 21.5% for the smooth specimens and 38.5% for the notched specimens. And more accurate P-Wt-Nf curves can be obtained to respond to the probabilistic properties of the data.

Energy efficiency optimization of NB-IoT using integrated Proxy & ERAI technique

The 3rd Generation Partnership Project (3GPP) standardized the NarrowBand Internet of Things (NB-IoT) technology in Release-13, aiming to support applications requiring minimal device power consumption, efficient small data transfer, and tolerance for delays. This paper provides two major contributions: first, it characterizes the NB-IoT architecture and its enhancements in subsequent 3GPP releases; second, it introduces an energy-saving technique for NB-IoT user equipment based on semi-Markov processes. Integrating a Proxy state and an enhanced release assistant indication (ERAI), the proposed technique improves upon the traditional Discontinuous Reception (DRX) method in the NB-IoT standard. This novel approach reduces energy consumption during low data communication, distinguishing it from other power-saving methodologies. The effectiveness of this technique is assessed through the power-saving factor (PSF) under various conditions, such as different eDRX durations (T_eDRX), PSM durations (T_PSM), DRX cycle periods (T_ACTIVE+T_SLEEP), and DRX active periods (T_ACTIVE). Our findings confirm that the proposed Proxy & ERAI technique significantly boosts the power-saving capabilities of NB-IoT devices, especially for small data packet transmissions. Theoretical and simulation results indicate that the PSF can be enhanced by up to 99.4% with optimized eDRX durations (T_eDRX) and up to 99.9% with optimized PSM (T_PSM) durations. These results underscore the significant energy-saving potential of our proposed technique, making it highly effective for NB-IoT applications with low data transmission needs.

High Glass Transition Temperature Fluorinated Polymers Based on Transfer Learning with Small Experimental Data.

Machine learning can be used to predict the properties of polymers and explore vast chemical spaces. However, the limited number of available experimental datasets hinders the enhancement of the predictive performance of a model. This study proposes a machine learning approach that leverages transfer learning and ensemble modeling to efficiently predict the glass transition temperature (Tg) of fluorinated polymers and guide the design of high Tg copolymers. Initially, the quantum machine 9 (QM9) dataset is employed for model pretraining, thus providing robust molecular representations for the subsequent fine-tuning of a specialized copolymer dataset. Ensemble modeling is used to further enhance prediction robustness and reliability, effectively addressing the problems owing to the limited and unevenly distributed nature of the copolymer dataset. Finally, a fine-tuned ensemble model is used to navigate a vast chemical space comprising 61 monomers and identify promising candidates for high Tg fluorinated polymers. The model predicts 247 entries capable of achieving a Tg over 390 K, of which 14 are experimentally validated. This study demonstrates the potential of machine learning in material design and discovery, highlighting the effectiveness of transfer learning and ensemble modeling strategies for overcoming the challenges posed by small datasets in complex copolymer systems.

Comparison of Synthetic Data Generation Techniques for Control Group Survival Data in Oncology Clinical Trials: Simulation Study.

Synthetic patient data (SPD) generation for survival analysis in oncology trials holds significant potential for accelerating clinical development. Various machine learning methods, including classification and regression trees (CART), random forest (RF), Bayesian network (BN), and conditional tabular generative adversarial network (CTGAN), have been used for this purpose, but their performance in reflecting actual patient survival data remains under investigation. The aim of this study was to determine the most suitable SPD generation method for oncology trials, specifically focusing on both progression-free survival (PFS) and overall survival (OS), which are the primary evaluation end points in oncology trials. To achieve this goal, we conducted a comparative simulation of 4 generation methods, including CART, RF, BN, and the CTGAN, and the performance of each method was evaluated. Using multiple clinical trial data sets, 1000 data sets were generated by using each method for each clinical trial data set and evaluated as follows: (1) median survival time (MST) of PFS and OS; (2) hazard ratio distance (HRD), which indicates the similarity between the actual survival function and a synthetic survival function; and (3) visual analysis of Kaplan-Meier (KM) plots. Each method's ability to mimic the statistical properties of real patient data was evaluated from these multiple angles. In most simulation cases, CART demonstrated the high percentages of MSTs for synthetic data falling within the 95% CI range of the MST of the actual data. These percentages ranged from 88.8% to 98.0% for PFS and from 60.8% to 96.1% for OS. In the evaluation of HRD, CART revealed that HRD values were concentrated at approximately 0.9. Conversely, for the other methods, no consistent trend was observed for either PFS or OS. CART demonstrated better similarity than RF, in that CART caused overfitting and RF (a kind of ensemble learning approach) prevented it. In SPD generation, the statistical properties close to the actual data should be the focus, not a well-generalized prediction model. Both the BN and CTGAN methods cannot accurately reflect the statistical properties of the actual data because small data sets are not suitable. As a method for generating SPD for survival data from small data sets, such as clinical trial data, CART demonstrated to be the most effective method compared to RF, BN, and CTGAN. Additionally, it is possible to improve CART-based generation methods by incorporating feature engineering and other methods in future work.

Evolutionary Strategies AI Addresses Multiple Technical Challenges in Deep Learning Deployment: Proof-of-Principle Demonstration for Neuroblastoma Brain Metastasis Detection.

Two significant obstacles hinder the advancement of Radiology AI. The first is the challenge of overfitting, where small training data sets can result in unreliable outcomes. The second challenge is the need for more generalizability, the lack of which creates difficulties in implementing the technology across various institutions and practices. A recent innovation, deep neuroevolution (DNE), has been introduced to tackle the overfitting issue by training on small data sets and producing accurate predictions. However, the generalizability of DNE has yet to be proven. This paper strives to overcome this barrier by demonstrating that DNE can achieve satisfactory results in diverse external validation sets. The main innovation of the work is thus showing that DNE can generalize to varied outside data. Our example use case is predicting brain metastasis from neuroblastoma, emphasizing the importance of AI with limited data sets. Despite image collection and labeling advancements, rare diseases will always constrain data availability. We optimized a convolutional neural network (CNN) with DNE to demonstrate generalizability. We trained the CNN with 60 MRI images and tested it on a separate diverse collection of images from over 50 institutions. For comparison, we also trained with the more traditional stochastic gradient descent (SGD) method, with the two variants of (1) training from scratch and (2) transfer learning. Our results show that DNE demonstrates excellent generalizability with 97% accuracy on the heterogeneous testing set, while neither form of SGD could reach 60% accuracy. DNE's ability to generalize from small training sets to external and diverse testing sets suggests that it or similar approaches may play an integral role in improving the clinical performance of AI.

Arabic text classification based on analogical proportions

AbstractText classification is the process of labelling a given set of text documents with predefined classes or categories. Existing Arabic text classifiers are either applying classic Machine Learning algorithms such as k‐NN and SVM or using modern deep learning techniques. The former are assessed using small text collections and their accuracy is still subject to improvement while the latter are efficient in classifying big data collections and show limited effectiveness in classifying small corpora with a large number of categories. This paper proposes a new approach to Arabic text classification to treat small and large data collections while improving the classification rates of existing classifiers. We first demonstrate the ability of analogical proportions (AP) (statements of the form ‘x is to as is to ’), which have recently been shown to be effective in classifying ‘structured’ data, to classify ‘unstructured’ text documents requiring preprocessing. We design an analogical model to express the relationship between text documents and their real categories. Next, based on this principle, we develop two new analogical Arabic text classifiers. These rely on the idea that the category of a new document can be predicted from the categories of three others, in the training set, in case the four documents build together a ‘valid’ analogical proportion on all or on a large number of components extracted from each of them. The two proposed classifiers (denoted AATC1 and AATC2) differ mainly in terms of the keywords extracted for classification. To evaluate the proposed classifiers, we perform an extensive experimental study using five benchmark Arabic text collections with small or large sizes, namely ANT (Arabic News Texts) v2.1 and v1.1, BBC‐Arabic, CNN‐Arabic and AlKhaleej‐2004. We also compare analogical classifiers with both classical ML‐based and Deep Learning‐based classifiers. Results show that AATC2 has the best average accuracy (78.78%) over all other classifiers and the best average precision (0.77) ranked first followed by AATC1 (0.73), NB (0.73) and SVM (0.72) for the ANT corpus v2.1. Besides, AATC1 shows the best average precisions (0.88) and (0.92), respectively for the BBC‐Arabic corpus and AlKhaleej‐2004, and the best average accuracy (85.64%) for CNN‐Arabic over all other classifiers. Results demonstrate the utility of analogical proportions for text classification. In particular, the proposed analogical classifiers are shown to significantly outperform a number of existing Arabic classifiers, and in many cases, compare favourably to the robust SVM classifier.

A novel method for settlement imputation and monitoring of earth-rockfill dams subjected to large-scale missing data

Various missing values inevitably exist in the settlement monitoring data of prolonged operation of earth-rockfill dams, which delays or even interrupts the dam structure analytical procedure. This study aims to develop a reliable method for addressing the missing data problem and monitoring earth-rockfill dam settlement behavior. First, an imputation model, support vector regression based on the finite element method and time input (FEMTSVR), is proposed and offers superior performance when capturing complex nonlinear mappings from environmental variables to settlement on small sample data. Herein, an improved particle swarm optimization algorithm (IPSO) is developed, realizing the nonlinear adjustment of weights and parameter reduction during hyperparameter optimization. Then, this study establishes a sequential prediction model based on gate recurrent unit (GRU) networks to monitor dam behavior on the imputed and complete dataset. Eventually, the proposed method is evaluated using real-world earth-rockfill dam monitoring data with the help of statistical indicators, demonstrating its efficiency in monitoring settlement data subjected to large-scale missingness. This study provides a robust database for dam structural health monitoring, while also providing a promising framework for the safety assessment of other civil or hydraulic engineering based on raw monitoring data.

Local Contrast Learning for One-Shot Learning

Learning a deep model from small data is an opening and challenging problem. In high-dimensional spaces, few samples only occupy an extremely small portion of the space, often exhibiting sparsity issues. Classifying in this globally sparse sample space poses significant challenges. However, by using a single sample category as a reference object for comparing and recognizing other samples, it is possible to construct a local space. Conducting contrastive learning in this local space can overcome the sparsity issue of a few samples. Based on this insight, we proposed a novel deep learning approach named Local Contrast Learning (LCL). This is analogous to a key insight into human cognitive behavior, where humans identify the objects in a specific context by contrasting them with the objects in that context or from their memory. LCL is used to train a deep model that can contrast the recognized sample with a couple of contrastive samples that are randomly drawn and shuffled. On a one-shot classification task on Omniglot, the deep model-based LCL with 86 layers and 1.94 million parameters, which was trained on a tiny dataset with only 60 classes and 20 samples per class, achieved an accuracy of 98.95%. Furthermore, it achieved an accuracy of 99.24% at 156 classes and 20 samples per class. LCL is a fundamental idea that can be applied to alleviate the parametric model’s overfitting resulting from a lack of training samples.

Predicting Tumor Dynamics Post-Staged GKRS: Machine Learning Models in Brain Metastases Prognosis.

This study assesses the predictive performance of six machine learning models and a 1D Convolutional Neural Network (CNN) in forecasting tumor dynamics within three months following Gamma Knife radiosurgery (GKRS) in 77 brain metastasis (BM) patients. The analysis meticulously evaluates each model before and after hyperparameter tuning, utilizing accuracy, AUC, and other metrics derived from confusion matrices. The CNN model showcased notable performance with an accuracy of 98% and an AUC of 0.97, effectively complementing the broader model analysis. Initial findings highlighted that XGBoost significantly outperformed other models with an accuracy of 0.95 and an AUC of 0.95 before tuning. Post-tuning, the Support Vector Machine (SVM) demonstrated the most substantial improvement, achieving an accuracy of 0.98 and an AUC of 0.98. Conversely, XGBoost showed a decline in performance after tuning, indicating potential overfitting. The study also explores feature importance across models, noting that features like "control at one year", "age of the patient", and "beam-on time for volume V1 treated" were consistently influential across various models, albeit their impacts were interpreted differently depending on the model's underlying mechanics. This comprehensive evaluation not only underscores the importance of model selection and hyperparameter tuning but also highlights the practical implications in medical diagnostic scenarios, where the accuracy of positive predictions can be crucial. Our research explores the effects of staged Gamma Knife radiosurgery (GKRS) on larger tumors, revealing no significant outcome differences across protocols. It uniquely considers the impact of beam-on time and fraction intervals on treatment efficacy. However, the investigation is limited by a small patient cohort and data from a single institution, suggesting the need for future multicenter research.

CPMI: comprehensive neighborhood-based perturbed mutual information for identifying critical states of complex biological processes

BackgroundThere exists a critical transition or tipping point during the complex biological process. Such critical transition is usually accompanied by the catastrophic consequences. Therefore, hunting for the tipping point or critical state is of significant importance to prevent or delay the occurrence of catastrophic consequences. However, predicting critical state based on the high-dimensional small sample data is a difficult problem, especially for single-cell expression data.ResultsIn this study, we propose the comprehensive neighbourhood-based perturbed mutual information (CPMI) method to detect the critical states of complex biological processes. The CPMI method takes into account the relationship between genes and neighbours, so as to reduce the noise and enhance the robustness. This method is applied to a simulated dataset and six real datasets, including an influenza dataset, two single-cell expression datasets and three bulk datasets. The method can not only successfully detect the tipping points, but also identify their dynamic network biomarkers (DNBs). In addition, the discovery of transcription factors (TFs) which can regulate DNB genes and nondifferential ‘dark genes’ validates the effectiveness of our method. The numerical simulation verifies that the CPMI method is robust under different noise strengths and is superior to the existing methods on identifying the critical states.ConclusionsIn conclusion, we propose a robust computational method, i.e., CPMI, which is applicable in both the bulk and single cell datasets. The CPMI method holds great potential in providing the early warning signals for complex biological processes and enabling early disease diagnosis.

Collaborative learning from distributed data with differentially private synthetic data

BackgroundConsider a setting where multiple parties holding sensitive data aim to collaboratively learn population level statistics, but pooling the sensitive data sets is not possible due to privacy concerns and parties are unable to engage in centrally coordinated joint computation. We study the feasibility of combining privacy preserving synthetic data sets in place of the original data for collaborative learning on real-world health data from the UK Biobank.MethodsWe perform an empirical evaluation based on an existing prospective cohort study from the literature. Multiple parties were simulated by splitting the UK Biobank cohort along assessment centers, for which we generate synthetic data using differentially private generative modelling techniques. We then apply the original study’s Poisson regression analysis on the combined synthetic data sets and evaluate the effects of 1) the size of local data set, 2) the number of participating parties, and 3) local shifts in distributions, on the obtained likelihood scores.ResultsWe discover that parties engaging in the collaborative learning via shared synthetic data obtain more accurate estimates of the regression parameters compared to using only their local data. This finding extends to the difficult case of small heterogeneous data sets. Furthermore, the more parties participate, the larger and more consistent the improvements become up to a certain limit. Finally, we find that data sharing can especially help parties whose data contain underrepresented groups to perform better-adjusted analysis for said groups.ConclusionsBased on our results we conclude that sharing of synthetic data is a viable method for enabling learning from sensitive data without violating privacy constraints even if individual data sets are small or do not represent the overall population well. Lack of access to distributed sensitive data is often a bottleneck in biomedical research, which our study shows can be alleviated with privacy-preserving collaborative learning methods.

PSO-optimized SSLMS adaptive filter for signal denoising of rolling bearings under small sample condition

Abstract To address the issue that the deep learning-based denoising algorithms can hardly effectively eliminate the background noise under small sample data condition, this paper proposes a new denoising method based on spectral subtraction (SS) and least mean square (LMS) adaptive filtering algorithms. To achieve the adaptive selection for the parameters of SS and LMS algorithms, particle swarm optimization (PSO) approach is employed to search and optimize the parameters in the two algorithms, which is helpful for the two algorithms to play an important role in eliminating the noise components with the different properties. Subsequently, the SS algorithm and the LMS algorithm are appropriately combined, and the SS-processed signal is input into the LMS algorithm as a desired signal to actualize the LMS adaptive filtering function. In this way, the denoising performance of both algorithms can be maximally utilized, which achieves effective noise reduction in vibration signal. The effectiveness and superiority of the proposed method are validated through simulation data and rolling bearing experiment data, respectively. The results demonstrate that the proposed method significantly diminishes noise components and retains precise and reliable fault features under small sample data condition, which provides an effective denoising method for rolling bearing vibration signals under small sample data condition in practical engineering scenarios.

Endothelial cell Piezo1 promotes vascular smooth muscle cell differentiation on large arteries.

Vascular stabilization is a mechanosensitive process, in part driven by blood flow. Here, we demonstrate the involvement of the mechanosensitive ion channel, Piezo1, in promoting arterial accumulation of vascular smooth muscle cells (vSMCs) during zebrafish development. Using a series of small molecule antagonists or agonists to temporally regulate Piezo1 activity, we identified a role for the Piezo1 channel in regulating klf2a levels and altered targeting of vSMCs between arteries and veins. Increasing Piezo1 activity suppressed klf2a and increased vSMC association with the cardinal vein, while inhibition of Piezo1 activity increased klf2a levels and decreased vSMC association with arteries. We supported the small molecule data with in vivo genetic suppression of piezo1 and 2 in zebrafish, resulting in loss of transgelin+ vSMCs on the dorsal aorta. Further, endothelial cell (EC)-specific Piezo1 knockout in mice was sufficient to decrease vSMC accumulation along the descending dorsal aorta during development, thus phenocopying our zebrafish data, and supporting functional conservation of Piezo1 in mammals. To determine mechanism, we used in vitro modeling assays to demonstrate that differential sensing of pulsatile versus laminar flow forces across endothelial cells changes the expression of mural cell differentiation genes. Together, our findings suggest a crucial role for EC Piezo1 in sensing force within large arteries to mediate mural cell differentiation and stabilization of the arterial vasculature.

Research on semantic segmentation algorithm of high latitude urban river ice based on deep transfer learning

ABSTRACT Automated observation methods for monitoring river ice in high-latitude urban areas are crucial for resource utilization, risk assessment, and navigation. However, current research lacks actual-scale river ice classification, such as low-altitude surveys. This study established a dataset of river ice in the Songhua River near Harbin, Northeast China, using UAV aerial photography and applied the RININet semantic segmentation algorithm for precise classification of different ice types in low-altitude aerial remote sensing images. To address environmental challenges, a feature extraction method integrating channel and spatial attention mechanisms was adopted, along with an improved pyramid pool structure to enhance feature recognition. Additionally, a two-stage transfer learning method established an ice recognition database, overcoming issues like small data volume and high annotation costs. Comparative evaluation metrics demonstrated the high accuracy of the semantic segmentation framework. Furthermore, a method for estimating ice blockage risk was proposed, applicable to various urban river ice management tasks, with practical significance.

A deep learning paradigm for medical imaging data

We present a novel deep learning framework to address several challenges in medical imaging analysis. Especially for functional magnetic resonance imaging (fMRI) data, we develop a statistical sampling algorithm to augment small data for deep learning modeling. Also, the ComBat harmonization is utilized for multi-source batch effect correction. Finally, we implement the prediction uncertainty in deep learning through a dropout approximation of the deep Gaussian process. The proposed approach offers effective statistical tools to solve persistent problems in deep learning modeling for medical imaging data. To assess the performance of the proposed approach and demonstrate its effectiveness for various types of data, we apply it to autism fMRI and Pneumonia imaging datasets.

Slope multi-step excavation displacement prediction surrogate model based on a long short-term memory neural network: for small sample data and multi-feature multi-task learning

ABSTRACT Machine learning-based surrogate models have become the preferred approach for large-scale and frequent simulation tasks due to its significant improvement in computational efficiency. In order to overcome the potential effects of learning with small sample data and the challenges of multi-feature multi-task learning, we developed a novel deep learning long short-term memory (LSTM) model. Taking slope excavation displacement prediction as a case study, we employed the Latin hypercube sampling method to generate a synthetic dataset for training LSTM and other mainstream models. Experimental results demonstrate that the model's prediction accuracy decreases with a reduction in sample size, while support vector regression (SVR), back propagation neural network (BPNN), LSTM and Gaussian process regression (GPR) demonstrate a stronger resistance. It is feasible to utilise excavation features as model inputs to establish a unified multi-step excavation model, but the accuracy of the SVR model decreased by 32.5% after supplementing excavation features. Even when the sample size is less than 50, both LSTM and GPR exhibit excellent performance, achieving model R-squared and RMSE surpassing 0.99 and 0.07 mm. However, when addressing multi-output learning tasks, LSTM stands out as the optimal choice. This study will assist researchers or engineers in swiftly selecting appropriate surrogate models.

Machine learning for predicting compressive strength of sustainable cement paste incorporating copper mine tailings as supplementary cementitious materials

Copper mining produces significant amounts of copper mine tailings (CMT), necessitating appropriate waste handling and disposal practices. By substituting a portion of cement with CMT as supplementary cementitious materials (SCMs), we aim to address two environmental issues simultaneously: reducing copper mine waste in landfills and decreasing embodied carbon by using less cement. The exploration of CMT recycling as a cement replacement requires evaluation of its impact on material performance, such as compressive strength. In this paper, we address this by machine learning that features data fusion of large public data with our own small data on compressive strength of CMT-incorporated cement. We developed and critically evaluated three machine learning models: a simple linear model, Gaussian process, and random forest that predict the compressive strength of CMT-incorporated cement pastes with different mix designs (e.g., varying amounts of CMT and water-binder ratios) and curing ages. Hyperparameters in the random forest model were tuned using Bayesian optimization. Following a comprehensive evaluation of the models, we find that the random forest model can accurately estimate the compressive strength of cement paste across the mix designs. Furthermore, results from SHapley Additive exPlanation (SHAP), Individual Conditional Expectation (ICE), and Partial Dependence Plots (PDP) revealed that cement, ground-granulated blast furnace slag, superplasticizers, and curing ages positively influence compressive strength. This study contributes to acceleration of sustainable material technology to obtain the best mix design and desired compressive strength.

Add-Vit: CNN-Transformer Hybrid Architecture for Small Data Paradigm Processing

The vision transformer(ViT), pre-trained on large datasets, outperforms convolutional neural networks (CNN) in computer vision(CV). However, if not pre-trained, the transformer architecture doesn’t work well on small datasets and is surpassed by CNN. Through analysis, we found that:(1) the division and processing of tokens in the ViT discard the marginalized information between token. (2) the isolated multi-head self-attention (MSA) lacks prior knowledge. (3) the local inductive bias capability of stacked transformer block is much inferior to that of CNN. We propose a novel architecture for small data paradigms without pre-training, named Add-Vit, which uses progressive tokenization with feature supplementation in patch embedding. The model’s representational ability is enhanced by using a convolutional prediction module shortcut to connect MSA and capture local features as additional representations of the token. Without the need for pre-training on large datasets, our best model achieved 81.25%\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\%$$\\end{document} accuracy when trained from scratch on the CIFAR-100.