Inference Model Research Articles

Advancing Climate Forecasting Accuracy Through Neurosymbolic Integration: Unveiling the Neurosymbolic Climate Inference and Prediction (NS-CIP) Model’s Approach to Predicting Extreme Weather Events

Advancing Climate Forecasting Accuracy Through Neurosymbolic Integration: Unveiling the Neurosymbolic Climate Inference and Prediction (NS-CIP) Model’s Approach to Predicting Extreme Weather Events

Adaptive fusion graph convolutional network based interpretable fault diagnosis method for HVAC systems enhanced by unlabeled data

Fault diagnosis is critical in maintaining the operational stability and reliability of Heating, Ventilation, and Air Conditioning (HVAC) systems, which are crucial for ensuring indoor environmental quality and energy efficiency in buildings. However, traditional fault diagnosis methodologies face substantial challenges in accurately capturing the dynamic interactions within these systems and effectively utilizing unlabeled data. To overcome these limitations, an innovative fault diagnosis approach utilizing the Adaptive Fusion Graph Convolution Network (AFGCN) is proposed in this paper. This method significantly enhances the model's learning and inference abilities, particularly in scenarios with limited labeled data, by adaptively integrating the associative graph features of both unlabeled and labeled data. Furthermore, to augment the transparency and trustworthiness of the diagnostic outcomes, this paper introduces an interpretability analysis module. This module is designed to quantify the contribution of each sensor node in the fault diagnosis process. Experimental evaluations using the ASHRAE RP-1043, actual building operating chillers datasets, and LBNL FDD Data Sets_SDAHU indicate that this method offers substantial performance improvements in diagnosing faults in HVAC systems.

Development of intelligent suite for malaria pathogen detection in microscopy images

The identification of malaria infection using microscope images of blood smears is considered as a ‘gold standard’. The diagnosis of malaria needs expert microscopists which are scarce in remote areas where malaria is endemic. Therefore, it is desirable to automate the repetitive task of pathogen detection in the blood samples received as microscope images. This study provides an easy to use and deploy method for implementing a malaria pathogen detection software- the Intelligent Suite. The Intelligent Suite features a graphical user interface (GUI) implemented using ‘cvui’ library to interact with the OpenVINO’s inference engine for model optimisation and deployment across several inference devices. The intelligent Suite uses a custom YOLO-mp-3l model trained on Darknet framework for detection of malaria pathogen in thick smear microscope images. Moreover, the Intelligent Suite provides user interface for inference device/mode selection, alter model parameters, and generate detection reports along with the model performance metrics. The Intelligent Suite was executed on a CPU computer with model inference running on a plug-and-play Neural Compute Stick (NCS2) and performance reported.

A unified EM framework for estimation and inference of normal ogive item response models.

Normal ogive (NO) models have contributed substantially to the advancement of item response theory (IRT) and have become popular educational and psychological measurement models. However, estimating NO models remains computationally challenging. The purpose of this paper is to propose an efficient and reliable computational method for fitting NO models. Specifically, we introduce a novel and unified expectation-maximization (EM) algorithm for estimating NO models, including two-parameter, three-parameter, and four-parameter NO models. A key improvement in our EM algorithm lies in augmenting the NO model to be a complete data model within the exponential family, thereby substantially streamlining the implementation of the EM iteration and avoiding the numerical optimization computation in the M-step. Additionally, we propose a two-step expectation procedure for implementing the E-step, which reduces the dimensionality of the integration and effectively enables numerical integration. Moreover, we develop a computing procedure for estimating the standard errors (SEs) of the estimated parameters. Simulation results demonstrate the superior performance of our algorithm in terms of its recovery accuracy, robustness, and computational efficiency. To further validate our methods, we apply them to real data from the Programme for International Student Assessment (PISA). The results affirm the reliability of the parameter estimates obtained using our method.

A Review of Common Acceleration Models in Accelerated Life Testing

The core of accelerated life testing (ALT) is the development of acceleration models. These models can be classified into three types based on their underlying methodologies: physics-based, empirical inductive, and statistical inference models. This paper primarily examines the acceleration models that are commonly applied in engineering practice and presents the relevant computational formulas to highlight the fundamental research methodologies of ALT. Additionally, the paper underscores that ALT is increasingly recognized as a vital and emerging trend, driven by its role in shortening design cycles, enhancing product reliability, and managing costs.

Bayesian inference of frequency-specific functional connectivity in MEG imaging using a spectral graph model

Abstract Understanding the relationship between structural connectivity (SC) and functional connectivity (FC) of the human brain is an important goal of neuroscience. Highly detailed mathematical models of neural masses exist that can simulate the interactions between functional activity and structural wiring. These models are often complex and require intensive computation. Most importantly, they do not provide a direct or intuitive interpretation of this structure–function relationship. In this study, we employ the emerging concepts of spectral graph theory to obtain this mapping in terms of graph harmonics, which are eigenvectors of the structural graph’s Laplacian matrix. In order to imbue these harmonics with biophysical underpinnings, we leverage recent advances in parsimonious spectral graph modeling (SGM) of brain activity. Here, we show that such a model can indeed be cast in terms of graph harmonics, and can provide a closed-form prediction of FC in an arbitrary frequency band. The model requires only three global, spatially invariant parameters, yet is capable of generating rich FC patterns in different frequency bands. Only a few harmonics are sufficient to reproduce realistic FC patterns. We applied the method to predict FC obtained from pairwise magnitude coherence of source-reconstructed resting-state magnetoencephalography (MEG) recordings of 36 healthy subjects. To enable efficient model inference, we adopted a deep neural network-based Bayesian procedure called simulation-based inference. Using this tool, we were able to speedily infer not only the single most likely model parameters, but also their full posterior distributions. We also implemented several other benchmark methods relating SC to FC, including graph diffusion and coupled neural mass models. The present method was shown to give the best performance overall. Notably, we discovered that a single biophysical parameterization is capable of fitting FCs from all relevant frequency bands simultaneously, an aspect that did not receive adequate attention in prior computational studies.

Inference in mixed models with a mixture of distributions and controlled heteroscedasticity

. In the realm of mixed models, this article explores the estimation method for incorporating a mixture of distributions and addressing controlled heteroscedasticity. By relaxing the assumptions of normality and homoscedasticity, we introduce a more flexible approach to analyzing data. The article presents estimation techniques for variance components, estimable vectors, and cumulants, while also developing prediction intervals and prediction ellipsoids for future observations. A numerical example is employed to illustrate the method and compare it with traditional ANOVA and Bayesian estimation methods. The results demonstrate the superior flexibility and broader applicability of the proposed methods in diverse contexts. By extending the analysis beyond conventional assumptions, these approaches enhance the accuracy and robustness of statistical inference in mixed models.

The Duck’s Brain

AbstractAlthough database systems perform well in data access and manipulation, their relational model hinders data scientists from formulating machine learning algorithms in SQL. Nevertheless, we argue that modern database systems perform well for machine learning algorithms expressed in relational algebra. To overcome the barrier of the relational model, this paper shows how to transform data into a coordinate relational representation for training neural networks in SQL: We first describe building blocks for data transformation, model training and inference in SQL-92 and their counterparts using an extended array data type. Then, we compare the implementation for model training and inference using array data types to the one using a coordinate relational representation in SQL-92 only. The evaluation in terms of runtime and memory consumption proves the suitability of modern database systems for matrix algebra, although specialised array data types perform better than matrices in coordinate relational representation.

Perceived Infant Discomfort Linked to Lower Maternal Oral Health Quality of Life: Results from a Cross-Sectional Study

Aim: We aimed to explore whether there is an association between maternal perceived infant discomfort due to suggestive gastrointestinal alterations and oral-health-related quality of life (OHRQoL) through a survey. Materials and Methods: The present study included two main phases involving Portuguese-speaking parents with full-term infants aged 2–12 weeks old who were not previously hospitalized in a neonatal nursery. First, the original French Infant Colic Questionnaire (ColiQ) was translated, cross-culturally adapted and validated to Portuguese (ColiQ-PT). Then, a survey was distributed, and included sociodemographics, the ColiQ-PT, an oral health value scale, OHRQoL, self-perceived periodontal status, and smoking and oral health habits. Data were analyzed through inferential, correlation and multivariate logistic models in this cross-sectional study. Results: The ColiQ-PT revealed reliability and validity. From a total of 421 responses, higher infant discomfort was correlated with less maternal professional dental care prioritization (ρ = −0.096, p < 0.05). Self-perceived periodontitis correlated with all items of OHRQoL (p < 0.001), all seven OHIP-14 domains, and with the physical (p < 0.001), psychological (p = 0.006), and social (p = 0.011) super-domains. While the infant-related score was associated with baby age (p = 0.023) and physical pain (p = 0.040) related to OHRQoL, the parent score was associated with education (p = 0.005), unemployment (p = 0.035), and physical pain (p = 0.017). The total ColiQ-PT score was significantly associated with more deteriorated social disability related to maternal OHRQoL (ρ = −0.130, p < 0.05). Conclusions: Perceived infant discomfort seems to be linked to maternal deteriorated OHRQoL. This finding highlights the importance of prioritizing oral health in postpartum care. Further research is needed to explore the mechanisms underlying this association and to develop targeted interventions.

Pruning convolutional neural networks for inductive conformal prediction

Neural network pruning is a popular approach to reducing model storage size and inference time by removing redundant parameters in the neural network. However, the uncertainty of predictions from pruned models is unexplored. In this paper we study neural network pruning in the context of conformal predictors (CP). The conformal prediction framework built on top of machine learning algorithms supplements their predictions with reliable uncertainty measure in the form of prediction sets, under the independent and identically distributed assumption on the data. Convolutional neural networks (CNNs) have complicated architectures and are widely used in various applications nowadays. Therefore, we focus on pruning CNNs and, in particular, filter-level pruning. We first propose a brute force method that estimates the contribution of a filter to the CP’s predictive efficiency and removes those with the least contribution. Given the computation inefficiency of the brute force method, we also propose the Taylor expansion to approximate the filter’s contribution. Furthermore, we improve the global pruning method by protecting the most important filters within each layer from being pruned. In addition, we explore the ConfTr loss function which is optimized to yield maximal CP efficiency in the context of neural network pruning. We have conducted extensive experimental studies and compared the results regarding the trade-offs between predictive efficiency, computational efficiency, and network sparsity. These results are instructive for deploying pruned neural networks with applications using conformal prediction where reliable predictions and reduced computational cost are relevant, such as in safety-critical applications.

A novel dual-way inference modeling method for coal coking: Predicting H2 and CH4 concentrations in coke oven gas and inferring optimal reaction conditions

Coal coking is an efficient and environmentally friendly technology for energy utilization that yields various industrial materials. However, the intricate nature of the coal coking reaction poses a challenge for chemical theories to fully capture and derive its reaction process. To address this challenge, a novel dual-way inference modeling method was proposed to simulate the coal coking process. This modeling method not only predicts H2 and CH4 concentrations, but also infers optimal operating conditions based on the desired H2 and CH4 concentrations. In this work, three representative machine learning methods (XGBoost, Random Forest, and ANN) are compared with dual-way inference modeling method, and the key factors affecting H2 and CH4 concentrations are thoroughly explored. The experimental results show that the primary factor influencing CH4 concentration is C, H2 concentration is predominantly affected by fixed carbon, while C plays the most important role for CH4 + H2. The dual-way inference modeling method achieved excellent performance in predicting H2 and CH4 concentrations and operating conditions. In the prediction task of H2, CH4, and H2 + CH4 concentrations, the coefficient of R2 achieved 0.9767, 0.9936, and 0.9851, respectively. The result for predicting coking temperature based on the desired H2 and CH4 concentrations indicate a positive correlation between coking temperature and both H2 and CH4 concentrations. In general, as coking temperature increases, the concentrations of both H2 and CH4 will increase.

Data-driven modeling of consumer product satisfaction using an orthogonal arrangement of sound attributes

Sound quality attributes have been proven to be important predictors of customer satisfaction with consumer appliances. The results of several studies indicate that level-based, tonal, and temporal aspects of sound influence the perceived quality of consumer products. The current state-of-the-art of inferring consumer satisfaction with sound attributes is based on the jury test methodology. However, listening-based jury studies fail to capture interaction effects nor study sound samples based on an orthogonal arrangement of sound quality attributes. Therefore, this study aims to develop data-driven models to mathematically relate sound attributes to consumer satisfaction. A baseline sound obtained from the operational noise generated by a fan-powered home appliance was used as a reference sound for researching the importance of sound quality attributes. An orthogonal-designed experiment was conducted to prepare sound samples based on selected sound attributes, including loudness, sharpness, roughness, and prominence ratio. Consumer studies were conducted to measure the product satisfaction of the various sound samples, which were used to train the data-driven model. The data-driven model was shown to accurately predict consumer satisfaction metrics based on sound quality attributes. Therefore, the proposed methods are valuable for evaluating new products within the model inference space without conducting further listening-based jury studies.

Lifelong Generative Adversarial Autoencoder.

Lifelong learning describes an ability that enables humans to continually acquire and learn new information without forgetting. This capability, common to humans and animals, has lately been identified as an essential function for an artificial intelligence system aiming to learn continuously from a stream of data during a certain period of time. However, modern neural networks suffer from degenerated performance when learning multiple domains sequentially and fail to recognize past learned tasks after being retrained. This corresponds to catastrophic forgetting and is ultimately induced by replacing the parameters associated with previously learned tasks with new values. One approach in lifelong learning is the generative replay mechanism (GRM) that trains a powerful generator as the generative replay network, implemented by a variational autoencoder (VAE) or a generative adversarial network (GAN). In this article, we study the forgetting behavior of GRM-based learning systems by developing a new theoretical framework in which the forgetting process is expressed as an increase in the model's risk during the training. Although many recent attempts have provided high-quality generative replay samples by using GANs, they are limited to mainly downstream tasks due to the lack of inference. Inspired by the theoretical analysis while aiming to address the drawbacks of existing approaches, we propose the lifelong generative adversarial autoencoder (LGAA). LGAA consists of a generative replay network and three inference models, each addressing the inference of a different type of latent variable. The experimental results show that LGAA learns novel visual concepts without forgetting and can be applied to a wide range of downstream tasks.

FICTURE: scalable segmentation-free analysis of submicron-resolution spatial transcriptomics.

Spatial transcriptomics (ST) technologies have advanced to enable transcriptome-wide gene expression analysis at submicron resolution over large areas. However, analysis of high-resolution ST is often challenged by complex tissue structure, where existing cell segmentation methods struggle due to the irregular cell sizes and shapes, and by the absence of segmentation-free methods scalable to whole-transcriptome analysis. Here we present FICTURE (Factor Inference of Cartographic Transcriptome at Ultra-high REsolution), a segmentation-free spatial factorization method that can handle transcriptome-wide data labeled with billions of submicron-resolution spatial coordinates and is compatible with both sequencing-based and imaging-based ST data. FICTURE uses the multilayered Dirichlet model for stochastic variational inference of pixel-level spatial factors, and is orders of magnitude more efficient than existing methods. FICTURE reveals the microscopic ST architecture for challenging tissues, such as vascular, fibrotic, muscular and lipid-laden areas in real data where previous methods failed. FICTURE's cross-platform generality, scalability and precision make it a powerful tool for exploring high-resolution ST.

Bayesian feedback in the framework of ecological sciences

In ecological studies, it is not uncommon to encounter scenarios where the same phenomenon (e.g., species occurrence, species abundance) is observed using two different types of samplers. For example, species data can be collected from scientific sampling with a completely random sample pattern, but also from opportunistic sampling (e.g., whale watching from commercial fishing vessels or bird watching from citizen science), where observers tend to look for particular species in areas where they expect to find them.Species Distribution Models (SDMs) are widely used tools for analysing this type of ecological data. In particular, two models are available for the aforementioned data: a geostatistical model (GM) for data collected where the sampling design is not directly related to the observations, and a preferential model (PM) for data obtained from opportunistic sampling.The integration of information from disparate sources can be addressed through the use of expert elicitation and integrated models. This paper focuses on a sequential Bayesian procedure for linking two models by updating prior distributions. The Bayesian paradigm is implemented together with the integrated nested Laplace approximation (INLA) methodology, which is an effective approach for making inference and predictions in spatial models with high performance and low computational cost. This sequential approach has been evaluated through the simulation of various scenarios and the subsequent comparison of the results from sharing information between models using a variety of criteria. The procedure has also been exemplified on a real dataset.The primary findings indicate that, in general, it is preferable to transfer information from the independent (with a completely random sampling) model to the preferential model rather than in the alternative direction. However, this depends on several factors, including the spatial range and the spatial arrangement of the sampling locations.

CopyMix: Mixture model based single-cell clustering and copy number profiling using variational inference

Investigating tumor heterogeneity using single-cell sequencing technologies is imperative to understand how tumors evolve since each cell subpopulation harbors a unique set of genomic features that yields a unique phenotype, which is bound to have clinical relevance. Clustering of cells based on copy number data obtained from single-cell DNA sequencing provides an opportunity to identify different tumor cell subpopulations. Accordingly, computational methods have emerged for single-cell copy number profiling and clustering; however, these two tasks have been handled sequentially by applying various ad-hoc pre- and post-processing steps; hence, a procedure vulnerable to introducing clustering artifacts. We avoid the clustering artifact issues in our method, CopyMix, a Variational Inference for a novel mixture model, by jointly inferring cell clusters and their underlying copy number profile. Our probabilistic graphical model is an improved version of the mixture of hidden Markov models, which is designed uniquely to infer single-cell copy number profiling and clustering. For the evaluation, we used likelihood-ratio test, CH index, Silhouette, V-measure, total variation scores. CopyMix performs well on both biological and simulated data. Our favorable results indicate a considerable potential to obtain clinical impact by using CopyMix in studies of cancer tumor heterogeneity.

ECTFormer: An efficient Conv-Transformer model design for image recognition

Since the success of Vision Transformers (ViTs), there has been growing interest in combining ConvNets and Transformers in the computer vision community. While the hybrid models have demonstrated state-of-the-art performance, many of these models are too large and complex to be applied to edge devices for real-world applications. To address this challenge, we propose an efficient hybrid network called ECTFormer that leverages the strengths of ConvNets and Transformers while considering both model performance and inference speed. Specifically, our approach involves: (1) optimizing the combination of convolution kernels by dynamically adjusting kernel sizes based on the scale of feature tensors; (2) revisiting existing overlapping patchify to not only reduce the model size but also propagate fine-grained patches for the performance enhancement; and (3) introducing an efficient single-head self-attention mechanism, rather than multi-head self-attention in the base Transformer, to minimize the increase in model size and boost inference speed, overcoming bottlenecks of ViTs. In experimental results on ImageNet-1K, ECTFormer not only demonstrates comparable or higher top-1 accuracy but also faster inference speed on both GPUs and edge devices compared to other efficient networks.

Dynamic collaborative learning with heterogeneous knowledge transfer for long-tailed visual recognition

Solving the long-tailed visual recognition with deep convolutional neural networks is still a challenging task. As a mainstream method, multi-experts models achieve SOTA accurary for tackling this problem, but the uncertainty in network learning and the complexity in fusion inference constrain the performance and practicality of the multi-experts models. To remedy this, we propose a novel dynamic collaborative learning with heterogeneous knowledge transfer model (DCHKT) in this paper, in which experts with different expertise collaborate to make predictions. DCHKT consists of two core components: dynamic adaptive weight adjustment and heterogeneous knowledge transfer learning. First, the dynamic adaptive weight adjustment is designed to shift the focus of model training between the global expert and domain experts via dynamic adaptive weight. By modulating the trade-off between the learning of features and classifier, the dynamic adaptive weight adjustment can enhance the discriminative ability of each expert and alleviate the uncertainty of model learning. Then, heterogeneous knowledge transfer learning, which measures the distribution differences between the fusion logits of multiple experts and the predicted logits of each expert with different specialties, can achieve message passing between experts and enhance the consistency of ensemble prediction in model training and inference to promote their collaborations. Finally, extensive experimental results on public long-tailed datasets: CIFAR-LT, ImageNet-LT, Place-LT and iNaturalist2018, demonstrate the effectiveness and superiority of our DCHKT.

Probability Modeling and Statistical Inference in Cancer Screening

Probability Modeling and Statistical Inference in Cancer Screening

An early warning method of pipeline leakage monitoring with limited leakage samples

Aiming at the problems of low computational efficiency of the real-time transient model (RTTM), and training difficulty of data-driven models with small leakage samples in existing pipeline leakage monitoring methods, this paper proposes a novel pipeline leakage detection model (Multiple Regression Modeling of Real-time Transients, MR-RTTM) based on the fusion of real-time transient mechanism model and data-driven inference. Firstly, the real-time transient mechanism model of pipelines is derived based on the Bernoulli equation, and the required eigenvalues and label values of the data-driven model are determined to achieve the integration of the mechanism and the data-driven model. Secondly, MR-RTTM is constructed by applying the data of pipelines in normal working conditions and combining with the multiple regression algorithm to fit the real-time transient mechanism model of pipelines. An optimal threshold interval selection method of the model is also proposed based on the probability of detection (POD) and the probability of false call (POFC). Experiments show that MR-RTTM can detect pipeline leakage by arranging common pressure and flow rate sensors in the pipeline inlets and outlets, and the minimum detectable leakage is 1 % with an acceptable interval of 80 %∼100 % for POD and 0 ∼ 10 % for POFC, and the leakage discrimination rate is up to 100 %.