Proposed Inference Method Research Articles

A failure-dependence related stochastic crack growth modeling approach of competing cracking mode

Competing crack-induced fatigue fracture has been a typical failure mode especially in the very high cycle fatigue regime. However, the stochastic crack growth modeling related to failure-dependence has not been fully investigated. Here, a failure-dependence related stochastic crack growth modeling approach for competing cracking mode is proposed to address this issue. Firstly, copulas are used to model the dependent competing relationships among multiple fatigue fracture modes. The reliability analysis of multiple fatigue fracture modes is conducted from a copula perspective. Besides, the fatigue crack growth is modeled based on a nonlinear Wiener process, with unit-to-unit variability addressed by introducing random effects. Marginal reliability expressions are derived based on the Wiener process. Furthermore, a two-stage Bayesian inference method based on Hamiltonian Monte Carlo sampling is proposed to estimate model parameters. A Monte Carlo simulation study is conducted to validate the accuracy and robustness of the proposed inference method. Finally, the effectiveness of the proposed approach is proven through a real case study. It turns out that the ignorance of the competing relationships among multiple cracking modes leads to an underestimation of overall reliability. The accuracy of the model can be further improved with random effects considered.

Optimal Inference of Hidden Markov Models Through Expert-Acquired Data.

This paper focuses on inferring a general class of hidden Markov models (HMMs) using data acquired from experts. Expert-acquired data contain decisions/actions made by humans/users for various objectives, such as navigation data reflecting drivers' behavior, cybersecurity data carrying defender decisions, and biological data containing the biologist's actions (e.g., interventions and experiments). Conventional inference methods rely on temporal changes in data without accounting for expert knowledge. This paper incorporates expert knowledge into the inference of HMMs by modeling expert behavior as an imperfect reinforcement learning agent. The proposed method optimally quantifies experts' perceptions about the system model, which, alongside the temporal changes in data, contributes to the inference process. The proposed inference method is derived through a combination of dynamic programming and optimal recursive Bayesian estimation. The applicability of this method is demonstrated to a wide range of inference criteria, such as maximum likelihood and maximum a posteriori. The performance of the proposed method is investigated through a comprehensive numerical experiment using a benchmark problem and biological networks.

Faithful Reasoning over Scientific Claims

Claim verification in scientific domains requires models that faithfully incorporate relevant knowledge from the ever-growing, vast existing literature. Unfaithful claim verifications can lead to misinformation such as those observed during the COVID-19 pandemic. Fact-checking systems often fail to capture the complex relationship between claims and evidence, especially with ambiguous claims and implicit assumptions. Relying only on current LLMs poses challenges due to hallucinations and information traceability issues. To address these challenges, our approach considers multiple viewpoints onto the scientific literature, enabling the assessment of contradictory arguments and implicit assumptions. Our proposed inference method adds faithful reasoning to large language models by distilling information from diverse, relevant scientific abstracts. This method provides a verdict label that can be weighted by the reputation of the scientific articles and an explanation that can be traced back to sources. Our findings demonstrate that humans not only perceive our explanation to be significantly superior to the off-the-shelf model, but they also evaluate it as faithfully enabling the tracing of evidence back to its original sources.

Robust Deformable Image Registration Using Cycle-Consistent Implicit Representations.

Recent works in medical image registration have proposed the use of Implicit Neural Representations, demonstrating performance that rivals state-of-the-art learning-based methods. However, these implicit representations need to be optimized for each new image pair, which is a stochastic process that may fail to converge to a global minimum. To improve robustness, we propose a deformable registration method using pairs of cycle-consistent Implicit Neural Representations: each implicit representation is linked to a second implicit representation that estimates the opposite transformation, causing each network to act as a regularizer for its paired opposite. During inference, we generate multiple deformation estimates by numerically inverting the paired backward transformation and evaluating the consensus of the optimized pair. This consensus improves registration accuracy over using a single representation and results in a robust uncertainty metric that can be used for automatic quality control. We evaluate our method with a 4D lung CT dataset. The proposed cycle-consistent optimization method reduces the optimization failure rate from 2.4% to 0.0% compared to the current state-of-the-art. The proposed inference method improves landmark accuracy by 4.5% and the proposed uncertainty metric detects all instances where the registration method fails to converge to a correct solution. We verify the generalizability of these results to other data using a centerline propagation task in abdominal 4D MRI, where our method achieves a 46% improvement in propagation consistency compared with single-INR registration and demonstrates a strong correlation between the proposed uncertainty metric and registration accuracy.

MP and MT properties of fuzzy inference with aggregation function

As the two basic fuzzy inference models, fuzzy modus ponens (FMP) and fuzzy modus tollens (FMT) have the important application in artificial intelligence. In order to solve FMP and FMT problems, Zadeh proposed a compositional rule of inference (CRI) method. This paper aims mainly to investigate the validity of A-compositional rule of inference (ACRI) method, as a generalized CRI method based on aggregation functions, from a logical view and an interpolative view, respectively. Specifically, the modus ponens (MP) and modus tollens (MT) properties of ACRI method are discussed in detail. It is shown that the aggregation functions to implement FMP and FMT problems provide more generality than the t-norms, uninorms and overlap functions as well-known the laws of T-conditionality, U-conditionality and O-conditionality, respectively. Moreover, two examples are also given to illustrate our theoretical results. Especially, Example 6.2 shows that the output B′ in FMP (FMT) problem is close to B (DC) with our proposed inference method when the fuzzy input and the antecedent of fuzzy rule are near (the fuzzy input near with the negation of the succedent in fuzzy rule).

Reliability assessment of civil structures with incomplete probability distribution information

The probability distribution of random variables is usually incomplete owing to a limited amount of data, making it a challenging problem for structural reliability assessments. This paper presents a novel reliability analysis method for civil structures with incomplete data. The probability distributions of the random variables were estimated using an improved Bayesian inference method. In the proposed inference method, the frequencies and mode shapes were selected as the observation data. The likelihood function is replaced by homogeneous chaos, and the computational efficiency of Bayesian inference is improved. The affine invariant ensemble algorithm was used to generate samplers for the posterior distribution. After estimating the probability distribution of the incomplete random variables, a reliability analysis was performed, and the failure probability of the structure was estimated. Numerical studies on a two-span, two-story reinforced concrete building and a five-span bridge are presented. Experimental studies were also performed on an eight-story steel frame structure. The results show that the probability distributions of random variables can be significantly updated using the proposed technique and modal properties. The proposed technique was more efficient than the Gibbs sampling method. The failure probabilities of the structures were estimated using the estimated probability distributions.

Data-Driven Behaviour Estimation in Parametric Games*

A central question in multi-agent strategic games deals with learning the underlying utilities driving the agents’ behaviour. Motivated by the increasing availability of large data-sets, we develop an unifying data-driven technique to estimate agents’ utility functions from their observed behaviour, irrespective of whether the observations correspond to equilibrium configurations or to temporal sequences of action profiles. Under standard assumptions on the parametrization of the utilities, the proposed inference method is computationally efficient and finds all the parameters that rationalize the observed behaviour best. We numerically validate our theoretical findings on the market share estimation problem under advertising competition, using historical data from the Coca-Cola Company and Pepsi Inc. duopoly.

Inferring State-feedback Cooperative Control of Networked Dynamical Systems

In this paper, we study the problem of inferring the state-feedback cooperative control of continuous-time NDSs from noisy state observations. To practice, we first propose a causality-based estimator to obtain the discretized system matrix, adaptive to both stable and explosive state evolution cases. Then, we derive the observation period guarantees and leverage the matrix logarithm to accurately reconstruct the continuous closed-loop matrix from the discrete one, circumventing the insufficiency of conventional sampling-recovery methods in this situation (e.g., Shannon sampling theorem). Finally, we exploit the element-wise coupling relation between the local feedback gain and the unknown interaction topology, and construct a duel-level least squares method to obtain the feedback matrix. Simulations are conducted to verify the proposed inference method.

Empirical likelihood inference for Oaxaca–Blinder decomposition

This paper proposes an empirical likelihood inference method for the Oaxaca–Blinder decompositions. In contrast to the conventional Wald statistic using the delta method, our approach circumvents the linearization errors and estimation of the variance terms. Furthermore, the shape of the resulting empirical likelihood confidence set is determined flexibly by data. Simulation results illustrate usefulness of the proposed inference method. • Empirical likelihood inference is proposed for Oaxaca–Blinder decompositions. • Our approach circumvents linearization errors and variance estimation. • Numerical examples illustrate usefulness of our method.

Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scaling

We develop a new method of online inference for a vector of parameters estimated by the Polyak-Ruppert averaging procedure of stochastic gradient descent (SGD) algorithms. We leverage insights from time series regression in econometrics and construct asymptotically pivotal statistics via random scaling. Our approach is fully operational with online data and is rigorously underpinned by a functional central limit theorem. Our proposed inference method has a couple of key advantages over the existing methods. First, the test statistic is computed in an online fashion with only SGD iterates and the critical values can be obtained without any resampling methods, thereby allowing for efficient implementation suitable for massive online data. Second, there is no need to estimate the asymptotic variance and our inference method is shown to be robust to changes in the tuning parameters for SGD algorithms in simulation experiments with synthetic data.

An Initial Residual Stress Inference Method by Incorporating Monitoring Data and Mechanism Model

Initial residual stress is the main reason causing machining deformation of the workpiece, which has been deemed as one of the most important aspects of machining quality issues. The inference of the distribution of initial residual stress inside the blank has significant meaning for machining deformation control. Due to the principle error of existing residual stress detection methods, there are still challenges in practical applications. Aiming at the detection problem of the initial residual stress field, an initial residual stress inference method by incorporating monitoring data and mechanism model is proposed in this paper. Monitoring data during machining process is used to represent the macroscopic characterization of the unbalanced residual stress, and the finite element numerical model is used as the mechanism model so as to solve the problem that the analytic mechanism model is difficult to establish; the policy gradient approach is introduced to solve the gradient descent problem of the combination of learning model and mechanism model. Finally, the initial residual stress field is obtained through iterative calculation based on the fusing method of monitoring data and mechanism model. Verification results show that the proposed inference method of initial residual stress field can accurately and effectively reflect the machining deformation in the actual machining process.

An Energy-Efficient Inference Method in Convolutional Neural Networks Based on Dynamic Adjustment of the Pruning Level

In this article, we present a low-energy inference method for convolutional neural networks in image classification applications. The lower energy consumption is achieved by using a highly pruned (lower-energy) network if the resulting network can provide a correct output. More specifically, the proposed inference method makes use of two pruned neural networks (NNs), namely mildly and aggressively pruned networks, which are both designed offline. In the system, a third NN makes use of the input data for the online selection of the appropriate pruned network. The third network, for its feature extraction, employs the same convolutional layers as those of the aggressively pruned NN, thereby reducing the overhead of the online management. There is some accuracy loss induced by the proposed method where, for a given level of accuracy, the energy gain of the proposed method is considerably larger than the case of employing any one pruning level. The proposed method is independent of both the pruning method and the network architecture. The efficacy of the proposed inference method is assessed on Eyeriss hardware accelerator platform for some of the state-of-the-art NN architectures. Our studies show that this method may provide, on average, 70% energy reduction compared to the original NN at the cost of about 3% accuracy loss on the CIFAR-10 dataset.

Scalable and efficient inference via CPE

Two primary concerns of inference for high-dimensional data are statistical accuracy and computational efficiency. Despite the appealing asymptotic properties of existing de-biasing methods, the de-biasing step is generally considered to be computationally intensive. In this article, we propose the constrained projection estimator (CPE) for deriving confidence intervals in a scalable and efficient way under high dimensions when the unknown parameters adopt an approximately sparse structure. The proposed method is implemented on the constrained projection spaces corresponding to the identifiable signals determined by a prescreening procedure, which significantly reduces the computational cost in comparison to the full de-biasing steps. Theoretically, we demonstrate that the proposed inference method enjoys equivalent asymptotic efficiency to the full de-biasing procedure in view of the lengths of confidence intervals. We demonstrate the scalability and effectiveness of the proposed method through simulation and real data studies.

Prediction of PM2.5 concentrations at unsampled points using multiscale geographically and temporally weighted regression

Numerous statistical models have established the relationship between ambient fine particulate matter (PM2.5, with an aerodynamic diameter of less than 2.5 μm) and satellite aerosol optical depth (AOD) along with other meteorological/land-related covariates. However, all the models assumed that all covariates affect the PM2.5 concentration at the same scale, and none could provide a posterior uncertainty analysis at each regression point. Therefore, a multiscale geographically and temporally weighted regression (MGTWR) model was proposed by specifying a unique bandwidth for each covariate. However, the lack of a method for predicting values at unsampled points in the MGTWR model greatly restricts its corresponding application. Thus, this study developed a method for inferring unsampled points and used the posterior uncertainty assessment value to improve the model accuracy. With the aid of the high-resolution satellite multi-angle implementation of atmospheric correction (MAIAC) AOD product, daily PM2.5 concentrations with a 1 km × 1 km resolution were generated over the Beijing-Tianjin-Hebei region between 2013 and 2019. The coefficient of determination (R2) and root mean square error (RMSE) of the fitted MGTWR results vary from 0.90 to 0.94 and from 10.66 to 25.11 μg/m3, respectively. The sample-based and site-based cross-validation R2 and RMSE vary from 0.81 to 0.89 and from 14.40 to 34.43 μg/m3 respectively, demonstrating the effectiveness of the proposed inference method at unsampled points. With the uncertainty constraint, the sample-based and site-based validated MGTWR R2 results for all years are further improved by approximately 0.02–0.04, demonstrating the effectiveness of the posterior uncertainty assessment constraint method. These results suggest that the inference method proposed in this study is promising to overcome the defects of the MGTWR model in inferring the prediction values at unsampled points and could consequently enhance the wide applications of MGTWR modeling.

Distribution inference from early-stage stationary data streams by transfer learning

Data streams are prevalent in current manufacturing and service systems where real-time data arrive progressively. A quick distribution inference from such data streams at their early stages is extremely useful for prompt decision making in many industrial applications. For example, a quality monitoring scheme can be quickly started if the process data distribution is available and the optimal inventory level can be determined early once the customer demand distribution is estimated. To this end, this article proposes a novel online recursive distribution inference method for stationary data streams that can respond as soon as the streaming data are generated and update as regularly as the data accumulate. A major challenge is that the data size might be too small to produce an accurate estimation at the early stage of data streams. To solve this, we resort to an instance-based transfer learning approach which integrates a sufficient amount of auxiliary data from similar processes or products to aid the distribution inference in our target task. Particularly, the auxiliary data are reweighted automatically by a density ratio fitting model with a prior-belief-guided regularization term to alleviate data scarcity. Our proposed distribution inference method also possesses an efficient online algorithm with recursive formulas to update upon every incoming data point. Extensive numerical simulations and real case studies verify the advantages of the proposed method.

Inferring Multilingual Domain-Specific Word Embeddings From Large Document Corpora

The use of distributed vector representations of words in Natural Language Processing has become established. To tailor general-purpose vector spaces to the context under analysis, several domain adaptation techniques have been proposed. They all require sufficiently large document corpora tailored to the target domains. However, in several cross-lingual NLP domains both large enough domain-specific document corpora and pre-trained domain-specific word vectors are hard to find for languages other than English. This paper aims at tackling the aforesaid issue. It proposes a new methodology to automatically infer aligned domain-specific word embeddings for a target language on the basis of the general-purpose and domain-specific models available for a source language (typically, English). The proposed inference method relies on a two-step process, which first automatically identifies domain-specific words and then opportunistically reuses the non-linear space transformations applied to the word vectors of the source language in order to learn how to tailor the vector space of the target language to the domain of interest. The performance of the proposed method was validated via extrinsic evaluation by addressing the established word retrieval task. To this aim, a new benchmark multilingual dataset, derived from Wikipedia, has been released. The results confirmed the effectiveness and usability of the proposed approach.

BVAR-Connect: A Variational Bayes Approach to Multi-Subject Vector Autoregressive Models for Inference on Brain Connectivity Networks.

In this paper we propose BVAR-connect, a variational inference approach to a Bayesian multi-subject vector autoregressive (VAR) model for inference on effective brain connectivity based on resting-state functional MRI data. The modeling framework uses a Bayesian variable selection approach that flexibly integrates multi-modal data, in particular structural diffusion tensor imaging (DTI) data, into the prior construction. The variational inference approach we develop allows scalability of the methods and results in the ability to estimate subject- and group-level brain connectivity networks over whole-brain parcellations of the data. We provide a brief description of a user-friendly MATLAB GUI released for public use. We assess performance on simulated data, where we show that the proposed inference method can achieve comparable accuracy to the sampling-based Markov Chain Monte Carlo approach but at a much lower computational cost. We also address the case of subject groups with imbalanced sample sizes. Finally, we illustrate the methods on resting-state functional MRI and structural DTI data on children with a history of traumatic injury.

Inferring individual network edges - with application to target identification in gene networks

The paper considers the problem of inferring individual network edges from time-series data. This is the problem faced in target identification, but also important in cases where it is of interest to learn whether two specific network nodes interact directly as well as in cases where there is insufficient information to infer the full network. The proposed inference method is based on taking a geometric perspective on a corresponding regression problem. We show that, by considering the span of individual node response vectors in sample space, it is possible to identify a given edge with a label of confidence even if the available data are not informative to infer other parts of the network. Furthermore, the method points to what further experiments are needed to infer edges for which the available response data are not sufficiently informative. We demonstrate the results on a target identification problem of a nonlinear 20-gene network and show that targets can be identified independently from a single time-series experiment using significantly fewer samples than the number of nodes in the network.

Computational urban flow predictions with Bayesian inference: Validation with field data

Urban areas are projected to expand at a rapid pace. In the context of supporting sustainable design of cities and buildings, computational fluid dynamics (CFD) can be used to provide detailed information on the urban flow field. However, the complexity and natural variability of atmospheric boundary layer flows can limit the predictive performance of CFD. In this paper, we present a validation study for a Bayesian inference method that estimates the inflow boundary conditions for Reynolds-averaged Navier-Stokes (RANS) simulations of urban flow by assimilating data from urban sensor measurements. The method employs the ensemble Kalman filter to iteratively estimate the probability density functions of the incoming wind and improve the subsequent RANS prediction. The measurements used in this study were obtained during a full-scale experimental campaign on Stanfords campus. Six sonic anemometers were deployed at roof and pedestrian level; a subset of the sensors was used for data assimilation while the remaining ones were used for validation. The accuracy of the proposed inference method is compared to the conventional approach that defines the boundary conditions based on weather station data. The hit rates increased by a factor of two when using the inference method, and the predicted mean values were ∼ 20% more likely to be within the 95% confidence interval of the experimental data. An analysis of the impact of the number of sensors and their location indicates that the assimilation approach can consistently improve the predictions, as long as the inlet flow properties are identifiable from the sensor measurements.

Estimating the Memory Order of Electrocorticography Recordings.

This paper presents a data-driven method for estimating the memory order (the average length of the statistical dependence of a given sample on previous samples) of a recorded electrocorticography (ECoG) sequence. The proposed inference method is based on the relationship between the loss in predicting the next sample in a time-series and the dependence of this sample on the previous samples. Specifically, the memory order is estimated to be the number of past samples that minimize the least squares error (LSE) in predicting the next sample. To deal with the lack of an analytical model for ECoG recordings, the proposed method combines a collection of different predictors, thereby achieving LSE at least as low as the LSE achieved by each of the different predictors. ECoG recordings from six patients with epilepsy were analyzed, and the empirical cumulative density functions (ECDFs) of the memory orders estimated from these recordings were generated, for rest as well as pre-ictal time intervals. For pre-ictal time intervals, the electrodes corresponding to the seizure-onset-zone were separately analyzed. The estimated ECDFs were different between patients and between different types of blocks. For all the analyzed patients, the estimated memory orders were on the order of tens of milliseconds (up to 100ms). The proposed method facilitates the estimation of the causal associations between ECoG recordings, as these associations strongly depend on the recordings' memory. An improved estimation of causal associations can improve the performance of algorithms that use ECoG recordings to localize the epileptogenic zone. Such algorithms can aid doctors in their pre-surgical planning for the surgery of patients with epilepsy.