Sparse Training Research Articles

Harnessing Manycore Processors with Distributed Memory for Accelerated Training of Sparse and Recurrent Models

Current AI training infrastructure is dominated by single instruction multiple data (SIMD) and systolic array architectures, such as Graphics Processing Units (GPUs) and Tensor Processing Units (TPUs), that excel at accelerating parallel workloads and dense vector matrix multiplications. Potentially more efficient neural network models utilizing sparsity and recurrence cannot leverage the full power of SIMD processor and are thus at a severe disadvantage compared to today's prominent parallel architectures like Transformers and CNNs, thereby hindering the path towards more sustainable AI. To overcome this limitation, we explore sparse and recurrent model training on a massively parallel multiple instruction multiple data (MIMD) architecture with distributed local memory. We implement a training routine based on backpropagation though time (BPTT) for the brain-inspired class of Spiking Neural Networks (SNNs) that feature binary sparse activations. We observe a massive advantage in using sparse activation tensors with a MIMD processor, the Intelligence Processing Unit (IPU) compared to GPUs. On training workloads, our results demonstrate 5-10x throughput gains compared to A100 GPUs and up to 38x gains for higher levels of activation sparsity, without a significant slowdown in training convergence or reduction in final model performance. Furthermore, our results show highly promising trends for both single and multi IPU configurations as we scale up to larger model sizes. Our work paves the way towards more efficient, non-standard models via AI training hardware beyond GPUs, and competitive large scale SNN models.

Translational eigenstates of He@C60 from four-dimensional abinitio potential energy surfaces interpolated using Gaussian process regression.

We investigate the endofullerene system 3He@C60 with a four-dimensional potential energy surface (PES) to include the three He translational degrees of freedom and C60 cage radius. We compare second order Møller-Plesset perturbation theory (MP2), spin component scaled-MP2, scaled opposite spin-MP2, random phase approximation (RPA)@Perdew, Burke, and Ernzerhof (PBE), and corrected Hartree-Fock-RPA to calibrate and gain confidence in the choice of electronic structure method. Due to the high cost of these calculations, the PES is interpolated using Gaussian Process Regression (GPR), owing to its effectiveness with sparse training data. The PES is split into a two-dimensional radial surface, to which corrections are applied to achieve an overall four-dimensional surface. The nuclear Hamiltonian is diagonalized to generate the in-cage translational/vibrational eigenstates. The degeneracy of the three-dimensional harmonic oscillator energies with principal quantum number n is lifted due to the anharmonicity in the radial potential. The (2l + 1)-fold degeneracy of the angular momentum states is also weakly lifted, due to the angular dependence in the potential. We calculate the fundamental frequency to range between 96 and 110 cm-1 depending on the electronic structure method used. Error bars of the eigenstate energies were calculated from the GPR and are on the order of ∼±1.5 cm-1. Wavefunctions are also compared by considering their overlap and Hellinger distance to the one-dimensional empirical potential. As with the energies, the two abinitio methods MP2 and RPA@PBE show the best agreement. While MP2 has better agreement than RPA@PBE, due to its higher computational efficiency and comparable performance, we recommend RPA as an alternative electronic structure method of choice to MP2 for these systems.

Portability of genomic predictions trained on sparse factorial designs across two maize silage breeding cycles

Key messageWe validated the efficiency of genomic predictions calibrated on sparse factorial training sets to predict the next generation of hybrids and tested different strategies for updating predictions along generations.Genomic selection offers new prospects for revisiting hybrid breeding schemes by replacing extensive phenotyping of individuals with genomic predictions. Finding the ideal design for training genomic prediction models is still an open question. Previous studies have shown promising predictive abilities using sparse factorial instead of tester-based training sets to predict single-cross hybrids from the same generation. This study aims to further investigate the use of factorials and their optimization to predict line general combining abilities (GCAs) and hybrid values across breeding cycles. It relies on two breeding cycles of a maize reciprocal genomic selection scheme involving multiparental connected reciprocal populations from flint and dent complementary heterotic groups selected for silage performances. Selection based on genomic predictions trained on a factorial design resulted in a significant genetic gain for dry matter yield in the new generation. Results confirmed the efficiency of sparse factorial training sets to predict candidate line GCAs and hybrid values across breeding cycles. Compared to a previous study based on the first generation, the advantage of factorial over tester training sets appeared lower across generations. Updating factorial training sets by adding single-cross hybrids between selected lines from the previous generation or a random subset of hybrids from the new generation both improved predictive abilities. The CDmean criterion helped determine the set of single-crosses to phenotype to update the training set efficiently. Our results validated the efficiency of sparse factorial designs for calibrating hybrid genomic prediction experimentally and showed the benefit of updating it along generations.

Physics-Informed Graphical Learning and Bayesian Averaging for Robust Distribution State Estimation

This paper proposes a robust topology change-aware distribution system state estimation (DSSE) method based on a physics-informed graph neural network and Bayesian Probability Weighted Averaging (BPWA). A general state estimator is first built utilizing a graph attention network to learn the nonlinear mapping functions under different distribution network topologies. During this stage, the topology information is embedded in the neural network and the attention mechanism is employed to capture collaborative signals and discriminate the importance of neighboring buses. Then, the BPWA method allows assigning proper weights for the state estimation results under different topologies, which finally yields a single consensus solution via the sparse training samples under the new topology. The physics-informed mechanism enables the proposed method to embed the topology knowledge in the neural network while fully exploiting the value of historical data. Robustness to anomalous measurements is achieved through the embedding of physics knowledge. The application of the BPWA method further allows the proposed method to achieve faster adaptation to topology change and quantification of the estimation uncertainties by measurement errors. MATLAB and Python are used to carry out the comparative tests to evaluate the performance of the proposed method.

Deep energy-pressure regression for a thermodynamically consistent EOS model

In this paper, we aim to explore novel machine learning (ML) techniques to facilitate and accelerate the construction of universal equation-Of-State (EOS) models with a high accuracy while ensuring important thermodynamic consistency. When applying ML to fit a universal EOS model, there are two key requirements: (1) a high prediction accuracy to ensure precise estimation of relevant physics properties and (2) physical interpretability to support important physics-related downstream applications. We first identify a set of fundamental challenges from the accuracy perspective, including an extremely wide range of input/output space and highly sparse training data. We demonstrate that while a neural network (NN) model may fit the EOS data well, the black-box nature makes it difficult to provide physically interpretable results, leading to weak accountability of prediction results outside the training range and lack of guarantee to meet important thermodynamic consistency constraints. To this end, we propose a principled deep regression model that can be trained following a meta-learning style to predict the desired quantities with a high accuracy using scarce training data. We further introduce a uniquely designed kernel-based regularizer for accurate uncertainty quantification. An ensemble technique is leveraged to battle model overfitting with improved prediction stability. Auto-differentiation is conducted to verify that necessary thermodynamic consistency conditions are maintained. Our evaluation results show an excellent fit of the EOS table and the predicted values are ready to use for important physics-related tasks.

Permanent magnet synchronous motor inter-turn short circuit diagnosis based on physical-data dual model under oil-drilling environment

Inter-turn Short Circuit (ITSC) faults in Permanent Magnet Synchronous Motor (PMSM) have gained significant attention due to the growing demand for enhanced reliability and safety in actuation systems. This paper presents a adaptive fault diagnosis approach specifically designed for ITSC faults in PMSM used in oil-drilling applications, where sparse actual data and varying speed conditions pose considerable sparse training data and ITSC feature extraction challenges respectively. To address these challenges, we first construct a physical fault model of PMSM-ITSC and establish a simulation experiment platform to replicate downhole environments with high temperatures and rapidly changing PMSM speeds, ensuring a reliable data source for analysis. Subsequently, we propose a novel adaptive peak-to-peak self-finding method (APPS) that leverages frequency-domain prior knowledge to adaptively extract ITSC fault characteristics, even amidst drastic changes in PMSM speed. Furthermore, we introduce the time-sequence efficient moving window self-attention network (EMWSAN) data model for inferring the stator phase winding state, incorporating Half-sandwich and Cascaded windows group attention operations. This approach significantly reduces computational complexity and network parameters compared to traditional self-attention mechanisms. To expedite the fitting process, we apply transfer learning (TL) theory, transferring knowledge from the physical knowledge to the data model, enabling EMWSAN to be trained more efficiently. As a result, our proposed ITSC fault diagnosis scheme achieves an impressive 96.72% classification accuracy, achieving existing state-of-the-art methods.

Towards provably efficient quantum algorithms for large-scale machine-learning models

Large machine learning models are revolutionary technologies of artificial intelligence whose bottlenecks include huge computational expenses, power, and time used both in the pre-training and fine-tuning process. In this work, we show that fault-tolerant quantum computing could possibly provide provably efficient resolutions for generic (stochastic) gradient descent algorithms, scaling as O(T2×polylog(n))\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${{{{{{{\\mathcal{O}}}}}}}}({T}^{2}\ imes {{{{{{{\\rm{polylog}}}}}}}}(n))$$\\end{document}, where n is the size of the models and T is the number of iterations in the training, as long as the models are both sufficiently dissipative and sparse, with small learning rates. Based on earlier efficient quantum algorithms for dissipative differential equations, we find and prove that similar algorithms work for (stochastic) gradient descent, the primary algorithm for machine learning. In practice, we benchmark instances of large machine learning models from 7 million to 103 million parameters. We find that, in the context of sparse training, a quantum enhancement is possible at the early stage of learning after model pruning, motivating a sparse parameter download and re-upload scheme. Our work shows solidly that fault-tolerant quantum algorithms could potentially contribute to most state-of-the-art, large-scale machine-learning problems.

Research on the Construction of an Efficient and Lightweight Online Detection Method for Tiny Surface Defects through Model Compression and Knowledge Distillation

In response to the current issues of poor real-time performance, high computational costs, and excessive memory usage of object detection algorithms based on deep convolutional neural networks in embedded devices, a method for improving deep convolutional neural networks based on model compression and knowledge distillation is proposed. Firstly, data augmentation is employed in the preprocessing stage to increase the diversity of training samples, thereby improving the model’s robustness and generalization capability. The K-means++ clustering algorithm generates candidate bounding boxes, adapting to defects of different sizes and selecting finer features earlier. Secondly, the cross stage partial (CSP) Darknet53 network and spatial pyramid pooling (SPP) module extract features from the input raw images, enhancing the accuracy of defect location detection and recognition in YOLO. Finally, the concept of model compression is integrated, utilizing scaling factors in the batch normalization (BN) layer, and introducing sparse factors to perform sparse training on the network. Channel pruning and layer pruning are applied to the sparse model, and post-processing methods using knowledge distillation are used to effectively reduce the model size and forward inference time while maintaining model accuracy. The improved model size decreases from 244 M to 4.19 M, the detection speed increases from 32.8 f/s to 68 f/s, and mAP reaches 97.41. Experimental results demonstrate that this method is conducive to deploying network models on embedded devices with limited GPU computing and storage resources. It can be applied in distributed service architectures for edge computing, providing new technological references for deploying deep learning models in the industrial sector.

DSFedCon: Dynamic Sparse Federated Contrastive Learning for Data-Driven Intelligent Systems.

Federated learning (FL) makes it possible for multiple clients to collaboratively train a machine-learning model through communicating models instead of data, reducing privacy risk. Thus, FL is more suitable for processing data security and privacy for intelligent systems and applications. Unfortunately, there are several challenges in FL, such as the low training accuracy for nonindependent and identically distributed (non-IID) data and the high cost of computation and communication. Considering these, we propose a novel FL framework named dynamic sparse federated contrastive learning (DSFedCon). DSFedCon combines FL with dynamic sparse (DSR) training of network pruning and contrastive learning to improve model performance and reduce computation costs and communication costs. We analyze DSFedCon from the perspective of accuracy, communication, and security, demonstrating it is communication-efficient and safe. To give a practical evaluation for non-IID data training, we perform experiments and comparisons on the MNIST, CIFAR-10, and CIFAR-100 datasets with different parameters of Dirichlet distribution. Results indicate that DSFedCon can get higher accuracy and better communication cost than other state-of-the-art methods in these two datasets. More precisely, we show that DSFedCon has a 4.67-time speedup of communication rounds in MNIST, a 7.5-time speedup of communication rounds in CIFAR-10, and an 18.33-time speedup of communication rounds in CIFAR-100 dataset while achieving the same training accuracy.

Research on rapid detection of cross-scale defects in surface based on deep learning

The complex and diverse forms of surface defects in metal cutting, as well as their large scale span, present new challenges for deep learning algorithms. In addition, the existing defect detection models are generally characterized by high computational amount and complex structures, which is contrary to the high real-time performance and limited computing resources required in industrial applications. Based on this, this paper proposes a rapid detection method for cross-scale defects in surfaces based on deep learning. Firstly, a dataset of defect surfaces is collected and constructed through cutting experiments. Then, the experimental analysis reveals the insufficiency of the You Only Look Once version-5 s (YOLOv5s) network model for the detection of cross-scale defects on the surface. As a result, a RepVGG-Coordinate Attention-YOLOv5s (Rep-CA-YOLOv5s) network model, suitable for cross-scale defect detection, is proposed. This model optimizes the YOLOv5s network model from three perspectives, enhancing its ability to extract and fuse features for cross-scale defects. Finally, this paper investigates methods to improve detection speed while ensuring model accuracy. Two optimal Rep-CA-YOLOv5s sparse models are obtained through sparse training based on the γ scaling factor of the Batch Normalization (BN) layer and the filter weight, respectively. The relationship between detection accuracy, parameter quantity, and inference speed of these two sparse models under different pruning rates is explored. Experimental results indicate that the filter pruning method significantly improves model inference speed. At a 50 % pruning rate, optimal detection results can be achieved. Compared with the unpruning model, the pruned model reduced the inference speed by 55.67 %, while the mean Average Precision (mAP) only decreased by 0.1 %.

Assessing the potential of synthetic and ex situ airborne laser scanning and ground plot data to train forest biomass models

Abstract Airborne laser scanning data are increasingly used to predict forest biomass over large areas. Biomass information cannot be derived directly from airborne laser scanning data; therefore, field measurements of forest plots are required to build regression models. We tested whether simulated laser scanning data of virtual forest plots could be used to train biomass models and thereby reduce the amount of field measurements required. We compared the performance of models that were trained with (i) simulated data only, (ii) a combination of simulated and real data, (iii) real data collected from different study sites, and (iv) real data collected from the same study site the model was applied to. We additionally investigated whether using a subset of the simulated data instead of using all simulated data improved model performance. The best matching subset of the simulated data was sampled by selecting the simulated forest plot with the highest correlation of the return height distribution profile for each real forest plot. For comparison, a randomly selected subset was evaluated. Models were tested on four forest sites located in Poland, the Czech Republic, and Canada. Model performance was assessed by root mean squared error (RMSE), squared Pearson correlation coefficient (r$^{2}$), and mean error (ME) of observed and predicted biomass. We found that models trained solely with simulated data did not achieve the accuracy of models trained with real data (RMSE increase of 52–122 %, r$^{2}$ decrease of 4–18 %). However, model performance improved when only a subset of the simulated data was used (RMSE increase of 21–118 %, r$^{2}$ decrease of 5–14 % compared to the real data model), albeit differences in model performance when using the best matching subset compared to using a randomly selected subset were small. Using simulated data for model training always resulted in a strong underprediction of biomass. Extending sparse real training datasets with simulated data decreased RMSE and increased r$^{2}$, as long as no more than 12–346 real training samples were available, depending on the study site. For three of the four study sites, models trained with real data collected from other sites outperformed models trained with simulated data and RMSE and r$^{2}$ were similar to models trained with data from the respective sites. Our results indicate that simulated data cannot yet replace real data but they can be helpful in some sites to extend training datasets when only a limited amount of real data is available.

YOLO sparse training and model pruning for street view house numbers recognition

This paper proposes a YOLO (You Only Look Once) sparse training and model pruning technique for recognizing house numbers in street view images. YOLO is a popular object detection algorithm that has achieved state-of-the-art performance in various computer vision tasks. However, its large model size and computational complexity limit its deployment on resource-constrained devices such as smartphones and embedded systems. To address this issue, we use a sparse training technique that trains YOLO with L1 norm regularization to encourage the network to learn sparse representations. This results in a significant reduction in the number of parameters and computation without sacrificing accuracy. Furthermore, we apply a model pruning technique to the sparse-trained model to reduce the model size and computation. We evaluate our proposed method on the SVHN (Street View House Numbers) dataset. We show that it performs comparably to the original YOLO model while reducing the model size by 5% and the computation time by 7%. Overall, our proposed YOLO sparse training and model pruning technique provides an effective solution for deploying YOLO-based object detection models on resource-constrained devices.

Studying turbulent flows with physics-informed neural networks and sparse data

Physics-informed neural networks (PINNs) have recently become a viable modelling method for the scientific machine-learning community. The appeal of this network architecture lies in the coupling of a deep neural network (DNN) with partial differential equations (PDEs): the DNN can be considered a universal function approximator, and the physical knowledge from the embedded PDEs regularises the network during training. This regularisation improves robustness of the network and means the network only requires sparse training data in the domain. We apply PINNs with embedded Reynolds-averaged Navier–Stokes (RANS) equations to a spatially developing adverse-pressure-gradient (APG) boundary layer 1000<Reθ<3000, and also to the periodic hill problem 5600≤Reb≤37,000. We do not use a turbulence model to close the RANS equations, rather the network infers the Reynolds-stress fields during training as a means to satisfy closure. As PINNs are able to produce robust predictions with sparse training data, this paper demonstrates how PINNs can be trained with data typical of experimental campaigns. For the APG boundary layer, we seek to quantify the amount and location of data required to model a time-averaged turbulent flow with PINNs, and show that PINNs can accurately model wall shear stress and wall pressure. We demonstrate how PINNs perform when trained without information of a critical feature of the flow (such as a separation bubble), and we highlight how the cost of training the network does not increase with Reynolds number. Finally, we undertake a series of benchmarks to demonstrate how the method scales with network size and the number of residual points in the problem. This work seeks to establish the quantity and type of data required to successfully apply PINNs to time-averaged flow without a closure model, and seeks to set expectations on the prediction quality of the network when trained on an available data set.

A4Development and evaluation of a chest cavity simulation model for teaching surgical chest drain insertion

Surgical chest drain insertion is indicated in pneumothorax or haemothorax secondary to thoracic trauma. It is a mandatory emergency procedure that is incorporated as a part of the core medical training curriculum [1]. However, sparse training opportunities result in low clinician competency and increased risk of complications. While simulation training can offer a solution, the affordability of commercial models and hygiene and ethical implications of animal carcasses are significant limiting factors. The aim of this project is to build a reusable, high-fidelity, low-cost human chest cavity model excluding animal use for simulation-based teaching of surgical chest drain insertion. To construct the model, plaster gauze, metal wires and u-channel rubber trims were used to build a ribcage. Soy-protein-based sausage casing was used to create the pleural layers, and the muscles and subcutaneous fat were represented with ADAMgel (Aqueous Dietary fibre Antifreeze Mix gel), a novel material with tactility and dissection sensation similar to human soft tissue [2]. Synthetic chamois leather was used to represent skin. The resulting model allowed locating the safe triangle using anatomical landmarks, blunt dissection of muscles, pleural puncturing and advancement and suturing of the chest tube. The model cost less than £130, and it could be repaired after over 20 uses with less than £15. Verbal consent on study participation was obtained from all participants who performed chest drain insertion on the model and evaluated its fidelity and educational value using an anonymized Likert scale questionnaire. All questionnaire responses were converted to numerical values for data quantification, as shown in Median and range of questionnaire responses collected from study participants Sixteen senior clinicians with multiple experience on chest drain insertion and 11 junior clinicians with limited experience took part in the study. Anatomical and haptic fidelity of the model was evaluated very highly amongst experienced clinicians. Junior doctors stated increased confidence in performing the procedure, overall assessing the model as an appropriate learning tool. Twenty participants with previous training experience compared the quality of this model to other commercial or animal-based models, and 18 rated this model to be of the same or superior quality. This chest cavity model is suitable for simulation training of chest drain insertion. Importantly, the model excluded the use of animals under the principle of replacing, refining and reducing animal use in research [3]. Further training opportunities that utilize this model can increase clinician competence in the procedure, which can improve clinical practice and reduce patient mortality. Authors confirm that all relevant ethical standards for research conduct and dissemination have been met. The submitting author confirms that relevant ethical approval was granted, if applicable.

Hardware-aware approach to deep neural network optimization

Deep neural networks (DNNs) have been a pivotal technology in a myriad of fields, boasting remarkable achievements. Nevertheless, their substantial workload and inherent redundancies pose ongoing challenges for both practitioners and academia. While numerous researchers endeavor to optimize DNNs, the inherent parallelism features of hardware are generally underutilized, resulting in inefficient use of hardware resources. To address this deficit, the paper unveils a hardware-aware mechanism, IHSOpti, which incorporates hardware characteristics with software algorithms for DNN optimization. IHSOpti endeavors to exploit the full potential of modern hardware parallelism, with significant emphasis on pipelining mechanisms. Specifically, IHSOpti formulates an advanced sparse training algorithm Polar_HSPG which incorporates the newly-proposed layer-wise refined polarization regularizer (LWPolar), grounded on the half-space project gradient (HSPG). Subsequently, IHSOpti pioneeringly introduces the residual strategy for optimizing the layer-level redundancies of neural networks, capitalizing on the pipelining attributes inherent in current hardware. Experimental findings demonstrate that IHSOpti attains outstanding pruning ratios in both parameters and FLOPs. Specifically, IHSOpti achieves up to 96.90% and 82.73% pruning ratios with the accuracy of 93.34% for VGGBN, 97.69% and 95.24% pruning ratios with the accuracy of 94.69% for ResNet, 98.07% and 97.80% pruning ratios with the accuracy of 95.73% for the cutting-edge network RegNet, respectively. Notably, the running efficiency exhibits remarkable improvements with accelerations ranging from 3.63× to 8.20× for CPUs and 1.22× to 2.25× for GPUs, respectively. These outcomes surpass the latest advances in the field. Through the incorporation of specific hardware characteristics, IHSOpti provides a comprehensive and effective approach to harness the intrinsic parallelism of contemporary hardware platforms for DNNs.

Two‐stage personalized federated learning based on sparse pretraining

AbstractAiming at solving the performance degradation of federated learning (FL) under heterogeneous data distribution, personalized FL (PFL) was proposed. It is designed to produce a dedicated model for each client. However, the existing PFL solution only focuses on the performance of personalized model, ignoring the performance of global model, which will affect the willingness of new clients to participate. In order to solve this problem, this paper proposes a new PFL solution, a two‐stage PFL based on sparse pretraining, which can not only train a sparse personalized model for each client, but also obtain a sparse global model. The whole training process is divided into sparse pretraining and sparse personalized training, which focus on the performance of global model and personalized model respectively. Also, we propose a mask sparse aggregation technique to maintain the sparsity of the global model in the sparse personalized training stage. Experimental results show that compared with existing algorithms, our proposed algorithm can improve the accuracy of the global model while maintaining advanced personalized model accuracy, and has higher communication efficiency.

A real-time detection model for smoke in grain bins with edge devices

The global food crisis is becoming increasingly severe, and frequent grain bins fires can also lead to significant food losses at the same time. Accordingly, this paper proposes a model-compressed technique for promptly detecting small and thin smoke at the early stages of fire in grain bins. The proposed technique involves three key stages: (1) conducting smoke experiments in a back-up bin to acquire a dataset; (2) proposing a real-time detection model based on YOLO v5s with sparse training, channel pruning and model fine-tuning, and (3) the proposed model is subsequently deployed on different current edge devices. The experimental results indicate the proposed model can detect the smoke in grain bins effectively, with mAP and detection speed are 94.90% and 109.89 FPS respectively, and model size reduced by 5.11 MB. Furthermore, the proposed model is deployed on the edge device and achieved the detection speed of 49.26 FPS, thus allowing for real-time detection.

Exploring Regularization Methods for Domain Generalization in Accelerometer-Based Human Activity Recognition.

The study of Domain Generalization (DG) has gained considerable momentum in the Machine Learning (ML) field. Human Activity Recognition (HAR) inherently encompasses diverse domains (e.g., users, devices, or datasets), rendering it an ideal testbed for exploring Domain Generalization. Building upon recent work, this paper investigates the application of regularization methods to bridge the generalization gap between traditional models based on handcrafted features and deep neural networks. We apply various regularizers, including sparse training, Mixup, Distributionally Robust Optimization (DRO), and Sharpness-Aware Minimization (SAM), to deep learning models and assess their performance in Out-of-Distribution (OOD) settings across multiple domains using homogenized public datasets. Our results show that Mixup and SAM are the best-performing regularizers. However, they are unable to match the performance of models based on handcrafted features. This suggests that while regularization techniques can improve OOD robustness to some extent, handcrafted features remain superior for domain generalization in HAR tasks.

Jaya-ICSM: A rapid inverse method driven by monitoring data for concrete-faced rockfill dams static displacement simulation

This paper presents a rapid static parameter inverse method for concrete-faced rockfill dams based on displacement data using an improved cloud surrogate model and the Jaya optimization algorithm. The improved cloud surrogate model (ICSM), which evolved from the uncertainty cloud theory, is used rapidly to establish a nonlinear relationship between static parameters and displacement. The Jaya optimization algorithm, which is proficient in recognizing the extreme value of the objective function, is selected for parameter identification. To verify the performance of the proposed method, an inverse analysis of two numerical models and a real engineering project is considered, and a comparative study with the BP neural network as the surrogate model is conducted. Study results demonstrate that ICSM has an extremely high prediction accuracy, with a mean relative error that is always under 1 %. The prediction accuracy does not increase too much even under complex nonlinear conditions or when dealing with sparse training data. Moreover, Jaya-ICSM maintains high computational efficiency throughout the entire inverse analysis, saving more than 98 % of the time compared to another inverse method and proving to be much better than direct inversion based on the finite element method. This enables real-time inverse analysis and state feedback of complex engineering. The results also illustrate that the Jaya-ICSM is promising in parameter recognition and superior in supporting the displacement evaluation of earth-rock dams. Therefore, the proposed method can be an effective tool for engineers to evaluate the safety of earth-rockfill dams and ensure their long-term stability.

Interpretable AI-Based Large-Scale 3D Pathloss Prediction Model for Enabling Emerging Self-Driving Networks

In modern wireless communication systems, radio propagation modeling to estimate pathloss has always been a fundamental task in system design and optimization. The state-of-the-art empirical propagation models are based on measurements in specific environments and limited in their ability to capture idiosyncrasies of various propagation environments. To cope with this problem, ray-tracing based solutions are used in commercial planning tools, but they tend to be extremely time-consuming and expensive. We propose a Machine Learning (ML)-based model that leverages novel key predictors for estimating pathloss. By quantitatively evaluating the ability of various ML algorithms in terms of predictive, generalization and computational performance, our results show that Light Gradient Boosting Machine (LightGBM) algorithm overall outperforms others, even with sparse training data, by providing a 65% increase in prediction accuracy as compared to empirical models and 13x decrease in prediction time as compared to ray-tracing. To address the interpretability challenge that thwarts the adoption of most ML-based models, we perform extensive secondary analysis using SHapley Additive exPlanations (SHAP) method, yielding many practically useful insights that can be leveraged for intelligently tuning the network configuration, selective enrichment of training data in real networks and for building lighter ML-based propagation model to enable low-latency use-cases.