Gradient-based Method Research Articles

Learning variational autoencoders via MCMC speed measures

Variational autoencoders (VAEs) are popular likelihood-based generative models which can be efficiently trained by maximising an evidence lower bound. There has been much progress in improving the expressiveness of the variational distribution to obtain tighter variational bounds and increased generative performance. Whilst previous work has leveraged Markov chain Monte Carlo methods for constructing variational densities, gradient-based methods for adapting the proposal distributions for deep latent variable models have received less attention. This work suggests an entropy-based adaptation for a short-run metropolis-adjusted Langevin or Hamiltonian Monte Carlo (HMC) chain while optimising a tighter variational bound to the log-evidence. Experiments show that this approach yields higher held-out log-likelihoods as well as improved generative metrics. Our implicit variational density can adapt to complicated posterior geometries of latent hierarchical representations arising in hierarchical VAEs.

Shape and size optimization framework of stiffened shell using isogeometric analysis

A shape and size optimization framework is developed to improve the structural performance of the stiffened shell using isogeometric analysis. To accurately model the stiffened shell, the Lagrange multiplier method is employed to establish the coupling relationship of isogeometric degenerated shell elements and isogeometric degenerated beam elements. Due to the advantages of the non-uniform rational B-spline (NURBS), the unit normal direction vectors of the shell can be defined analytically, which guarantees stiffeners perpendicular to the skin. Additionally, the stiffeners can be easily generated based on the mapping relationship of the parametric space and the physical space of the shell. Analytical sensitivity is derived in detail, and the gradient-based optimization method can be used to solve the minimum compliance optimization problem owing to the high order continuity of NURBS. Three numerical illustrative examples are presented to verify the effectiveness of the proposed framework for the shape and size optimization of stiffened shells, consisting of a cylindrical shell patch, a hyperbolic shell, and a complex engineering segment. It is worth noting that the proposed framework is especially suitable for optimization problems of the stiffened shell, eliminating the need of complex feature extraction.

Energy efficiency optimization for 6G multi-IRS multi-cell NOMA vehicle-to-infrastructure communication networks

Intelligent Reflecting Surfaces (IRS), software-controlled metasurfaces, have emerged as an upcoming sixth-generation (6G) wireless communication technology. IRS intelligently manipulates and optimizes signal propagation using a large-scale array of intelligent elements, enhancing signal coverage, increasing capacity, mitigating path loss, and combating multipath fading This work provides a new energy-efficiency model for multi-IRS-assisted multi-cell non-orthogonal multiple access (NOMA) vehicular to infrastructure communication networks. The objective is the joint optimization of the total power budget at the roadside unit (RSU), NOMA power allocation for the user equipment, and designing phase shifts for IRS in each cell to maximize the achievable energy efficiency of the system. Due to non-convexity, the original non-convex problem is first decoupled and transformed using block coordinate descent and successive convex approximation methods. Then, an efficient solution is achieved using Gradient-based and interior-point methods. We also consider two benchmark schemes: (1) NOMA power optimization at RSU with random phase shift design at IRS and (2) orthogonal multiple access power allocation with optimal phase shift design at IRS. Numerical results show the superiority of the proposed solution compared to the benchmark schemes. The proposed solution outperforms the benchmarks, demonstrating a 59.57% and 151.21% improvement over the NOMA and orthogonal schemes, respectively, at pct=2 dBm. Additionally, it shows up to a 10.43% better performance than OMA at 10 IRS elements.

Adaptive Perturbation for Adversarial Attack.

In recent years, the security of deep learning models achieves more and more attentions with the rapid development of neural networks, which are vulnerable to adversarial examples. Almost all existing gradient-based attack methods use the sign function in the generation to meet the requirement of perturbation budget on L∞ norm. However, we find that the sign function may be improper for generating adversarial examples since it modifies the exact gradient direction. Instead of using the sign function, we propose to directly utilize the exact gradient direction with a scaling factor for generating adversarial perturbations, which improves the attack success rates of adversarial examples even with fewer perturbations. At the same time, we also theoretically prove that this method can achieve better black-box transferability. Moreover, considering that the best scaling factor varies across different images, we propose an adaptive scaling factor generator to seek an appropriate scaling factor for each image, which avoids the computational cost for manually searching the scaling factor. Our method can be integrated with almost all existing gradient-based attack methods to further improve their attack success rates. Extensive experiments on the CIFAR10 and ImageNet datasets show that our method exhibits higher transferability and outperforms the state-of-the-art methods.

Backdoor attacks on unsupervised graph representation learning

Unsupervised graph learning techniques have garnered increasing interest among researchers. These methods employ the technique of maximizing mutual information to generate representations of nodes and graphs. We show that these methods are susceptible to backdoor attacks, wherein the adversary can poison a small portion of unlabeled graph data (e.g., node features and graph structure) by introducing triggers into the graph. This tampering disrupts the representations and increases the risk to various downstream applications. Previous backdoor attacks in supervised learning primarily operate directly on the label space and may not be suitable for unlabeled graph data. To tackle this challenge, we introduce GRBA,11https://github.com/fbd3/GRBA.git. a gradient-based first-order backdoor attack method. To the best of our knowledge, this constitutes a pioneering endeavor in investigating backdoor attacks within the domain of unsupervised graph learning. The initiation of this method doesn’t necessitate prior knowledge of downstream tasks, as it directly focuses on representations. Furthermore, it is versatile and can be applied to various downstream tasks, including node classification, node clustering and graph classification. We evaluate GRBA on state-of-the-art unsupervised learning models, and the experimental results substantiate the effectiveness and evasiveness of GRBA in both node-level and graph-level tasks.

Enhancing source separation quality via optimal sensor placement in noisy environments

The paper aims to bridge a part of the gap between source separation and sensor placement studies by addressing a novel problem: “Predicting optimal sensor placement in noisy environments to improve source separation quality”. The structural information required for optimal sensor placement is modeled as the spatial distribution of source signal gains and the spatial correlation of noise. The sensor positions are predicted by optimizing two criteria as measures of separation quality, and a gradient-based global optimization method is developed to efficiently address this optimization problem. Numerical results exhibit superior performance when compared with classical sensor placement methodologies based on mutual information, underscoring the critical role of sensor placement in source separation with noisy sensor measurements. The proposed method is applied to actual electroencephalography (EEG) data to separate the P300 source components in a brain-computer interface (BCI) application. The results show that when the sensor positions are chosen using the proposed method, to reach a certain level of spelling accuracy, fewer sensors are required compared with standard sensor locations.

Bayesian Optimization for State and Parameter Estimation of Dynamic Networks with Binary Space.

This paper focuses on joint state and parameter estimation in partially observed Boolean dynamical systems (POBDS), a hidden Markov model tailored for modeling complex networks with binary state variables. The majority of current techniques for parameter estimation rely on computationally expensive gradient-based methods, which become intractable in most practical applications with large size of networks. We propose a gradient-free approach that uses Gaussian processes to model the expensive log-likelihood function and utilizes Bayesian optimization for efficient likelihood search over parameter space. Joint state estimation is also achieved alongside parameter estimation using the Boolean Kalman filter. The performance of the proposed method is demonstrated using gene regulatory networks observed through synthetic gene-expression data. The numerical results demonstrate the scalability and effectiveness of the proposed method in the joint estimation of the model parameters and genes' states.

Spectral induced polarization tomography inversion: Hybridizing homotopic continuation with Bayesian inversion

Induced polarization tomography offers the potential to better characterize the subsurface structures by considering spectral content from data acquisition over a broad frequency range. Spectral induced polarization (SIP) tomography is generally defined as a nonlinear inverse problem commonly solved through deterministic gradient-based methods. To this end, the spectral parameters, i.e., direct current resistivity, chargeability, relaxation time, and frequency exponent, are resolved by individually or simultaneously inverting all frequency data, followed by fitting a generalized Cole-Cole model to the inverted complex resistivities. Due to the high correlation between the Cole-Cole model parameters and a lack of knowledge about the initial approximation of the spectral parameters, using the classical least-squares methods may lead to inaccurate solutions and impede reliable uncertainty analysis. To cope with these limitations, we introduce a new approach based on a hybrid application of a globally convergent homotopic continuation method and a Bayesian inference to reconstruct the distribution of the subsurface spectral parameters. The homotopic optimization, owing to its fast and global convergence, is first implemented to invert multifrequency SIP data sets aimed at retrieving the complex-valued resistivity. Then, Bayesian inversion based on a Markov chain Monte Carlo (MCMC) sampling method, along with a priori information including the lower and upper bounds of the prior distributions, is used to invert the complex resistivity for the Cole-Cole model parameters. By applying the MCMC inversion algorithm, a full nonlinear uncertainty appraisal can be provided. We numerically evaluate the performance of our method using synthetic and real data examples in the presence of topographical effects. Numerical results prove that the homotopic continuation method outperforms the classic, smooth inversion algorithm in the sense of approximation accuracy and computational efficiency. In addition, we determine that our hybrid inversion strategy provides reliable representations of the main features and structure of the earth’s subsurface in terms of the spectral parameters.

The Role of Mathematics in Artificial Intelligence and Machine Learning

Mathematics serves as the foundational backbone of artificial intelligence (AI) and machine learning (ML), providing the essential tools and frameworks for developing sophisticated algorithms and models. the pivotal role of various mathematical disciplines, including linear algebra, calculus, probability theory, and optimization, in advancing AI and ML technologies. We begin by examining how linear algebra facilitates the manipulation and transformation of high-dimensional data, which is crucial for techniques such as principal component analysis (PCA) and singular value decomposition (SVD). Next, we delve into the applications of calculus in training neural networks through gradient-based optimization methods, highlighting the importance of differentiation and integration in backpropagation and loss function minimization. the role of probability theory in handling uncertainty and making predictions, emphasizing its application in Bayesian networks, Markov decision processes, and probabilistic graphical models. Additionally, we discuss optimization techniques, both convex and non-convex, that are fundamental to finding optimal solutions in machine learning tasks, including support vector machines (SVMs) and deep learning architectures.

Multi-objective meta-learning

Meta-learning has arisen as a powerful tool for many machine learning problems. With multiple factors to be considered when designing learning models for real-world applications, meta-learning with multiple objectives has attracted much attention recently. However, existing works either linearly combine multiple objectives into one objective or adopt evolutionary algorithms to handle it, where the former approach needs to pay high computational cost to tune the combination coefficients while the latter approach is computationally heavy and incapable to be integrated into gradient-based optimization. To alleviate those limitations, in this paper, we aim to propose a generic gradient-based Multi-Objective Meta-Learning (MOML) framework with applications in many machine learning problems. Specifically, the MOML framework formulates the objective function of meta-learning with multiple objectives as a Multi-Objective Bi-Level optimization Problem (MOBLP) where the upper-level subproblem is to solve several possibly conflicting objectives for the meta-learner. Different from those existing works, in this paper, we propose a gradient-based algorithm to solve the MOBLP. Specifically, we devise the first gradient-based optimization algorithm by alternately solving the lower-level and upper-level subproblems via the gradient descent method and the gradient-based multi-objective optimization method, respectively. Theoretically, we prove the convergence property and provide a non-asymptotic analysis of the proposed gradient-based optimization algorithm. Empirically, extensive experiments justify our theoretical results and demonstrate the superiority of the proposed MOML framework for different learning problems, including few-shot learning, domain adaptation, multi-task learning, neural architecture search, and reinforcement learning. The source code of MOML is available at https://github.com/Baijiong-Lin/MOML.

Uncertainty quantification via localized gradients for deep learning-based medical image assessments

Objective. Deep learning models that aid in medical image assessment tasks must be both accurate and reliable to be deployed within clinical settings. While deep learning models have been shown to be highly accurate across a variety of tasks, measures that indicate the reliability of these models are less established. Increasingly, uncertainty quantification (UQ) methods are being introduced to inform users on the reliability of model outputs. However, most existing methods cannot be augmented to previously validated models because they are not post hoc, and they change a model’s output. In this work, we overcome these limitations by introducing a novel post hoc UQ method, termed Local Gradients UQ, and demonstrate its utility for deep learning-based metastatic disease delineation. Approach. This method leverages a trained model’s localized gradient space to assess sensitivities to trained model parameters. We compared the Local Gradients UQ method to non-gradient measures defined using model probability outputs. The performance of each uncertainty measure was assessed in four clinically relevant experiments: (1) response to artificially degraded image quality, (2) comparison between matched high- and low-quality clinical images, (3) false positive (FP) filtering, and (4) correspondence with physician-rated disease likelihood. Main results. (1) Response to artificially degraded image quality was enhanced by the Local Gradients UQ method, where the median percent difference between matching lesions in non-degraded and most degraded images was consistently higher for the Local Gradients uncertainty measure than the non-gradient uncertainty measures (e.g. 62.35% vs. 2.16% for additive Gaussian noise). (2) The Local Gradients UQ measure responded better to high- and low-quality clinical images (p < 0.05 vs p > 0.1 for both non-gradient uncertainty measures). (3) FP filtering performance was enhanced by the Local Gradients UQ method when compared to the non-gradient methods, increasing the area under the receiver operating characteristic curve (ROC AUC) by 20.1% and decreasing the false positive rate by 26%. (4) The Local Gradients UQ method also showed more favorable correspondence with physician-rated likelihood for malignant lesions by increasing ROC AUC for correspondence with physician-rated disease likelihood by 16.2%. Significance. In summary, this work introduces and validates a novel gradient-based UQ method for deep learning-based medical image assessments to enhance user trust when using deployed clinical models.

Causality-based counterfactual explanation for classification models

Counterfactual explanation is one branch of interpretable machine learning that produces a perturbation sample to change the model’s original decision. The generated samples can act as a recommendation for end-users to achieve their desired outputs. Most of the current counterfactual explanation approaches are the gradient-based method, which can only optimize the differentiable loss functions with continuous variables. Accordingly, the gradient-free methods are proposed to handle the categorical variables, which however have several major limitations: (1) causal relationships among features are typically ignored when generating the counterfactuals, possibly resulting in impractical guidelines for decision-makers; (2) the counterfactual explanation algorithm requires a great deal of effort into parameter tuning for determining the optimal weight for each loss functions which must be conducted repeatedly for different datasets and settings. In this work, to address the above limitations, we propose a prototype-based counterfactual explanation framework (ProCE). ProCE is capable of preserving the causal relationship underlying the features of the counterfactual data. In addition, we design a novel gradient-free optimization based on the multi-objective genetic algorithm that generates the counterfactual explanations for the mixed-type of continuous and categorical features. Numerical experiments demonstrate that our method compares favorably with state-of-the-art methods and therefore is applicable to existing prediction models. All the source codes and data are available at https://github.com/tridungduong16/multiobj-scm-cf.

An efficient solution for large offshore wind farm power optimization with the Porté-Agel wake model: Optimality and efficiency

The wake effects may cause significant energy loss in the large offshore wind farm without proper coordination. Therefore, cooperative yaw and induction control is needed to mitigate wake effects and increase power output. This study reveals that conventional methods to solve the power optimization problem may trap into local optimum and have slow computation speed for large offshore wind farm. In this paper, the Porté-Agel wake model is adopted to precisely capture the dynamics and a data-driven method is proposed to calibrate its parameters only using measurements. Then, we propose a distributed hybrid search-BFGS(DHSB) algorithm to solve the power optimization problem. In this algorithm, firstly the yaw angles of some specific turbines are searched coarsely and distributionally to provide a good initial point and escape from the local optimum. Then, a distributed BFGS is developed to solve the optimization problem with very high efficiency, in which the gradient and objective value are updated in distributed manner. Simulation results show that our proposed DHSB can always achieve global optimum with much high efficiency compared to traditional gradient-based methods.

Life-cycle production optimization with nonlinear constraints using a least-squares support-vector regression proxy

When nonlinear constraints such as field liquid or water production rate, injection pressures, etc., as functions of time need to be honored in addition to linear ones, the life-cycle production optimization problem, a component of a closed loop reservoir management, becomes challenging and computationally expensive to perform using a high-fidelity reservoir simulator with the existing gradient-based methods using the adjoint or stochastic approximate gradient methods. Therefore, the objective of this study is to present computationally efficient methods for deterministic production optimization under nonlinear constraints using a kernel-based machine learning method where the cost function is the net present value (NPV). We use the least-squares support-vector regression (LSSVR) to approximate the NPV function. To achieve computational efficiency, we generate a set of output values of the NPV and nonlinear constraint functions, which are field liquid production rate (FLPR) and water production rate (FWPR) in this study, by running the high-fidelity simulator for a broad set of input design variables (well controls) and then using the collection of input/output data to train LSSVR proxy models to replace the high-fidelity simulator to compute NPV and nonlinear state constraint functions during iterations of sequential quadratic programming (SQP). To obtain improved (higher) estimated optimal NPV values, we use the existing so-called iterative sampling refinement (ISR) method to update the LSSVR proxy so that the updated proxy remains predictive toward promising regions of search space during the optimization. Direct and indirect ways of constructing LSSVR-based NPVs as well as different combinations of input data, including nonlinear state constraints and/or the bottomhole pressures (BHPs) and water injection rates, are tested as alternative feature spaces. The results obtained from our proposed LSSVR-based optimization methods are compared with those obtained from our in-house stochastic simplex approximate gradient (StoSAG)-based line-search SQP programming (LS-SQP-StoSAG) algorithm that uses directly a high-fidelity simulator to compute the gradients of the objective function and the nonlinear state functions with StoSAG for the Brugge reservoir model. The results show that nonlinear constrained optimization with the LSSVR ISR with SQP is computationally 3.25 fold more efficient than LS-SQP-StoSAG. In addition, the results show that constructing NPV indirectly using the field liquid and water rates for a waterflooding problem where inputs come from LSSVR proxies of the nonlinear state constraints requires significantly fewer training samples than the method constructing NPV directly from the NPVs computed from a high-fidelity simulator. LSSVR has advantages for its computational efficiency which is the main goal in our research, and robustness against overfitting especially for the cases where we have limited data. However, DNNs and random forest require large training size and require much more computational resources and longer training times. Also, Random Forests and Gradient Boosting Machines can be prone to overfitting and become computationally intensive.

A multi-stage techno-economic model for harnessing flexibility from IoT-enabled appliances and smart charging systems: Developing a competitive local flexibility market using Stackelberg game theory

The increasing integration of renewable energy sources with power systems has led to greater uncertainties, heightening the need for more flexible services. Demand-side resources can provide significant flexibility cost-effectively, making it crucial to design new business models to harness their flexible capacities. Hence, this paper introduces a three-stage techno-economic framework designed to enhance the exploitation of flexible capacities from demand-side resources such as Internet-of-Things (IoT)-Enabled Appliances (IEAs), Battery-Integrated Rooftop Solar Panels (BIRSPs), and Electric Vehicle (EV) smart charging systems in the local flexibility market. The proposed framework operates in three stages. First, smart homes with IEAs, BIRSPs, and smart charging systems assess their flexibility and report it to their Microgrid (MG) operators. Next, MG operators optimize strategies to maximize market returns and report capacities to the Distribution System Operator (DSO). Finally, the DSO manages flexibility transactions based on MG inputs. A Stackelberg game theory model using a gradient-based method updates the coupling variables between market parties, ensuring convergence in the second and third stages within a decentralized space with limited information sharing. Implemented on a modified IEEE 69-node distribution system, the model fully utilizes the potential of smart homes, including IEAs, BIRSPs, and smart charging systems in the local flexibility market. It reduces the Upstream Network (UN) share in the local flexibility market by 56.86% and decreases DSO costs for energy balancing by 22.36%.

Gradient-free training of recurrent neural networks using random perturbations.

Recurrent neural networks (RNNs) hold immense potential for computations due to their Turing completeness and sequential processing capabilities, yet existing methods for their training encounter efficiency challenges. Backpropagation through time (BPTT), the prevailing method, extends the backpropagation (BP) algorithm by unrolling the RNN over time. However, this approach suffers from significant drawbacks, including the need to interleave forward and backward phases and store exact gradient information. Furthermore, BPTT has been shown to struggle to propagate gradient information for long sequences, leading to vanishing gradients. An alternative strategy to using gradient-based methods like BPTT involves stochastically approximating gradients through perturbation-based methods. This learning approach is exceptionally simple, necessitating only forward passes in the network and a global reinforcement signal as feedback. Despite its simplicity, the random nature of its updates typically leads to inefficient optimization, limiting its effectiveness in training neural networks. In this study, we present a new approach to perturbation-based learning in RNNs whose performance is competitive with BPTT, while maintaining the inherent advantages over gradient-based learning. To this end, we extend the recently introduced activity-based node perturbation (ANP) method to operate in the time domain, leading to more efficient learning and generalization. We subsequently conduct a range of experiments to validate our approach. Our results show similar performance, convergence time and scalability when compared to BPTT, strongly outperforming standard node perturbation and weight perturbation methods. These findings suggest that perturbation-based learning methods offer a versatile alternative to gradient-based methods for training RNNs which can be ideally suited for neuromorphic computing applications.

Generation of black-box adversarial attacks using many independent objective-based algorithm for testing the robustness of deep neural networks

Deep neural networks (DNNs) have become increasingly ubiquitous in our daily lives, finding applications in areas such as image recognition, voice recognition, and natural language processing. However, a growing concern revolves around the vulnerability of DNNs to adversarial examples—malicious inputs that can compromise the safety and accuracy of their outcomes. Existing studies predominantly fall into two categories: white-box and black-box techniques. White-box techniques require detailed internal information about the model, often focusing on gradient-based methods. In contrast, black-box techniques, which emulate real-world scenarios more closely, only rely on input–output knowledge. This study focuses specifically on black-box strategies, with key contributions from Single-Objective Variant of Differential Evolution (Pixel-SOO) and Multi-Objective Variant of Differential Evolution (Pixel-MOO) algorithms. While these approaches show promise, they suffer from drawbacks like long execution times and being unable to generate instances for adversarial purposes. To address these challenges, we introduce a novel archive-based Many Independent Objective (MIO) algorithm for using the first time in this context. Our proposed algorithm identifies the most vulnerable image pixels through the MIO algorithm, enabling efficient label flip attacks with minimized attempts. Furthermore, we balance exploration and exploitation by incorporating an adaptive parameter mechanism. The effectiveness of the proposed algorithm is assessed by employing VGG (VGG16 and VGG19) and ResNet (ResNet50, ResNet101, and ResNet152) architectures, both of which are convolutional neural network (CNN) models. The success criterion for the algorithms is to minimize the number of pixel changes while achieving high flip rates with a minimal number of requests to the network. A comprehensive analysis of the experimental results reveals that our algorithm consistently outperforms Pixel-SOO and Pixel-MOO, exhibiting an average speedup of three times compared to Pixel-SOO and seven times compared to Pixel-MOO. In most runs, adversarial attacks are generated with fewer pixel changes than Pixel-MOO and Pixel-SOO. In addition, our findings are openly accessible on GitHub to ensure transparency and reproducibility and encourage future research efforts.

EXACT: How to train your accuracy

Classification tasks are typically evaluated based on accuracy. However, due to the discontinuous nature of accuracy, it cannot be directly optimized using gradient-based methods. The conventional approach involves minimizing surrogate losses such as cross-entropy or hinge loss, which may result in suboptimal performance. In this paper, we introduce a novel optimization technique that incorporates stochasticity into the model’s output and focuses on optimizing the expected accuracy, defined as the accuracy of the stochastic model. Comprehensive experimental evaluations demonstrate that our proposed optimization method significantly enhances performance across various classification tasks, including SVHN, CIFAR-10, CIFAR-100, and ImageNet.

Enhanced Gradient for Differentiable Architecture Search.

In recent years, neural architecture search (NAS) methods have been proposed for the automatic generation of task-oriented network architecture in image classification. However, the architectures obtained by existing NAS approaches are optimized only for classification performance and do not adapt to devices with limited computational resources. To address this challenge, we propose a neural network architecture search algorithm aiming to simultaneously improve the network performance and reduce the network complexity. The proposed framework automatically builds the network architecture at two stages: block-level search and network-level search. At the stage of block-level search, a gradient-based relaxation method is proposed, using an enhanced gradient to design high-performance and low-complexity blocks. At the stage of network-level search, an evolutionary multiobjective algorithm is utilized to complete the automatic design from blocks to the target network. The experimental results demonstrate that our method outperforms all evaluated hand-crafted networks in image classification, with an error rate of 3.18% on Canadian Institute for Advanced Research (CIFAR10) and an error rate of 19.16% on CIFAR100, both at network parameter size less than 1 M. Obviously, compared with other NAS methods, our method offers a tremendous reduction in designed network architecture parameters.

StocIPNet: A novel probabilistic interpretable network with affine-embedded reparameterization layer for high-dimensional stochastic inverse problems

The stochastic inverse problem (StocIP), which aims to align push-forward and observed output distributions by estimating probability distributions of unknown system inputs, often faces optimization challenges and the curse of dimensionality. A novel deep network called StocIPNet which comprises an affine-embedded reparameterization subnetwork (ReparNet) and a complex system metamodeling subnetwork (MetaNet) is proposed to alleviate these issues. The ReparNet subnetwork embeds the affine transformation to convert the statistical parameters of the physical random vector into learnable weights and biases, effectively implementing the reparameterization trick by separating random and deterministic elements in the stochastic sampling operation to preserve differentiability. In parallel, the MetaNet subnetwork offers a computationally efficient alternative to time-consuming forward solvers, facilitating the generation of push-forward distributions. The entire StocIPNet utilizes the kernel maximum mean discrepancy (MMD) as a distribution-free loss function, quantifying the discrepancy between push-forward and observed output distributions. By leveraging the ReparNet’s advantage of reformulating the sampling process as a differentiable transformation and combining two subnetworks seamlessly, the StocIP is reconfigured into a pure network training paradigm preserving differentiability perfectly, which allows for direct modeling and efficient inference of uncertainty within the network using automatic differentiation, backpropagation and gradient-based optimization methods, enabling ease of scaling to high-dimensional problems. The proposed framework has been theoretically demonstrated to be equivalent to the maximum likelihood method, ensuring its solid probabilistic interpretable foundation. The proposed framework is applied to perform stochastic model updating on a numerical and an experimental structure, which effectively demonstrates the framework’s remarkable effectiveness and high efficiency in treating high-dimensional StocIPs.