Efficient Inference Research Articles

An efficient workflow for modelling high-dimensional spatial extremes

We develop a comprehensive methodological workflow for Bayesian modelling of high-dimensional spatial extremes that lets us describe both weakening extremal dependence at increasing levels and changes in the type of extremal dependence class as a function of the distance between locations. This is achieved with a latent Gaussian version of the spatial conditional extremes model that allows for computationally efficient inference with R-INLA. Inference is made more robust using a post hoc adjustment method that accounts for possible model misspecification. This added robustness makes it possible to extract more information from the available data during inference using a composite likelihood. The developed methodology is applied to the modelling of extreme hourly precipitation from high-resolution radar data in Norway. Inference is performed quickly, and the resulting model fit successfully captures the main trends in the extremal dependence structure of the data. The post hoc adjustment is found to further improve model performance.

Spiking neural networks with consistent mapping relations allow high-accuracy inference

Spike-based neuromorphic hardware has demonstrated substantial potential in low energy consumption and efficient inference. However, the direct training of deep spiking neural networks is challenging, and conversion-based methods still require substantial time delay owing to unresolved conversion errors. We determine that the primary source of the conversion errors stems from the inconsistency between the mapping relationship of traditional activation functions and the input-output dynamics of spike neurons. To counter this, we introduce the Consistent ANN-SNN Conversion (CASC) framework. It includes the Consistent IF (CIF) neuron model, specifically contrived to minimize the influence of the stable point's upper bound, and the wake-sleep conversion (WSC) method, synergistically ensuring the uniformity of neuron behavior. This method theoretically achieves a loss-free conversion, markedly diminishing time delays and improving inference performance in extensive classification and object detection tasks. Our approach offers a viable pathway toward more efficient and effective neuromorphic systems.

Variance-Reduced Stochastic Optimization for Efficient Inference of Hidden Markov Models

Hidden Markov models (HMMs) are popular models to identify a finite number of latent states from sequential data. However, fitting them to large datasets can be computationally demanding because most likelihood maximization techniques require iterating through the entire underlying dataset for every parameter update. We propose a novel optimization algorithm that updates the parameters of an HMM without iterating through the entire dataset. Namely, we combine a partial E step with variance-reduced stochastic optimization within the M step. We prove the algorithm converges under certain regularity conditions. We test our algorithm empirically using a simulation study as well as a case study of kinematic data collected using suction-cup attached biologgers from eight northern resident killer whales (Orcinus orca) off the western coast of Canada. In both, our algorithm converges in fewer epochs, with less computation time, and to regions of higher likelihood compared to standard numerical optimization techniques. Our algorithm allows practitioners to fit complicated HMMs to large time-series datasets more efficiently than existing baselines. Supplemental materials are available online.

Deep learning and quantization for accurate and efficient multi-target radar inference of moving targets

Real-time, radar-based human activity and target classification is useful for wide-area ground surveillance. However, the feasibility of deploying deep learning (DL) models in radar-based systems with limited computational resources remains unexplored. This paper investigated the effect of quantization on model throughput and accuracy for deployment in radar systems. A seven-layer residual network was proposed to classify ground-moving targets and achieved a test accuracy of 87.72%. The model was then quantized to 16-bit and 8-bit precision, resulting in a 3.8 times speedup in inference throughput, with less than a 0.4% drop in test and validation accuracy. The results showed that quantization can improve inference throughput with a negligible decrease in target classification accuracy. The increase in throughput and reduction in computational expense that comes with quantization promotes the feasibility of the deployment of DL models in systems with limited computational resources. The findings of this paper hold significant promise for the successful use of quantized models in modern radar systems, while adhering to stringent size, weight and power consumption constraints.

Energy Efficient and Fast CNN Inference by Exploring Weight Approximation and Computational Reuse

Energy Efficient and Fast CNN Inference by Exploring Weight Approximation and Computational Reuse

Domain Adaptation and Semantic Drawing Driven Sketch-to-Photo Retrieval using Collaborative Generative Representation Learning

Abstract: Sketch-based face recognition is an interesting task in vision and multimedia research yet it is quite challenging due to the great difference between face photos and sketches. In this paper we propose a novel approach for photo-sketch generation aiming to automatically transform face photos into detail-preserving personal sketches. Unlike the traditional models synthesizing sketches based on a dictionary of exemplars we develop a fully convolutional network to learn the end-to-end photosketch mapping. Our approach takes whole face photos as inputs and directly generates the corresponding sketch images with efficient inference and learning in which the architecture is stacked by only convolutional kernels of very small sizes. The exemplar-based method is most frequently used in face sketch synthesis because of its efficiency in representing the nonlinear mapping between face photos and sketches. However, the sketches synthesized by existing exemplar-based methods suffer from block artifacts and blur effects. In addition, most exemplar-based methods ignore the training sketches in the weight representation process. To improve synthesis performance, a novel joint training model is proposed in this paper, taking sketches into consideration. First, we construct the joint training photo and sketch by concatenating the original photo and its sketch with a high-pass filtered image of their corresponding sketch. Then, an offline random sampling strategy is adopted for each test photo patch to select the joint training photo and sketch patches in the neighboring region. Finally, a novel locality constraint is designed to calculate the reconstruction weight, allowing the synthesized sketches to have more detailed information

Adaptive data-driven subsampling for efficient neural network inference

In this paper we present a novel data-driven subsampling method that can be seamlessly integrated into any neural network architecture to identify the most informative subset of samples within the original acquisition domain for a variety of tasks that rely on deep learning inference from sampled signals. In contrast to existing methods that require signal transformation into a sparse basis, expensive signal reconstruction as an intermediate step, and that can support a single predefined sampling rate only, our approach allows the sampling inference pipeline to adapt to multiple sampling rates directly in the original signal domain. The key innovations enabling such operation are a custom subsampling layer and a novel training mechanism. Through extensive experiments with four data sets and four different network architectures, our method demonstrates a simple yet powerful sampling strategy that allows the given network to be efficiently utilized at any given sampling rate, while the inference accuracy degrades smoothly and gradually as the sampling rate is reduced. Experimental comparison with state-of-the-art sparse sensing and learning techniques demonstrates competitive inference accuracy at different sampling rates, coupled with a significant improvement in computational efficiency, and the crucial ability to operate at arbitrary sampling rates without the need for retraining.

Improved likelihood estimation for noisy gamma degradation processes via sequential Monte Carlo

The use of Gamma processes for modeling various degradation phenomena has recently gained extensive attention. In many cases, the degradation data contain measurement errors and an intractable likelihood phenomenon comes into sight. Therefore, in order to perform efficient statistical inference, one must obtain high-quality estimates of the corresponding likelihood. Our findings indicate that the crude Monte Carlo method, which is the de facto state-of-the-art method, is not adequate in practice for efficient likelihood estimation. To cope with this problem, we propose to employ the sequential Monte Carlo approach, which shows promise for improved reliability compared to the current state of the art. Our approach leads to efficient variance minimization and opens the way for effective and scalable inference procedures.

The whole and its parts: Visualizing Gaussian mixture models

Gaussian mixture models are classical but still popular machine learning models. An appealing feature of Gaussian mixture models is their tractability, that is, they can be learned efficiently and exactly from data, and also support efficient exact inference queries like soft clustering data points. Only seemingly simple, Gaussian mixture models can be hard to understand. There are at least four aspects to understanding Gaussian mixture models, namely, understanding the whole distribution, its individual parts (mixture components), the relationships between the parts, and the interplay of the whole and its parts. In a structured literature review of applications of Gaussian mixture models, we found the need for supporting all four aspects. To identify candidate visualizations that effectively aid the user needs, we structure the available design space along three different representations of Gaussian mixture models, namely as functions, sets of parameters, and sampling processes. From the design space, we implemented three design concepts that visualize the overall distribution together with its components. Finally, we assessed the practical usefulness of the design concepts with respect to the different user needs in expert interviews and an insight-based user study.

A Bayesian Joint Model of Longitudinal Kidney Disease Progression, Recurrent Cardiovascular Events, and Terminal Event in Patients with Chronic Kidney Disease

AbstractNearly 15% (37 million) of adults in the United States (US) have chronic kidney disease (CKD). The longitudinal decline of kidney function is intricately related to the development of cardiovascular disease (CVD) and eventual “terminal” event (kidney failure and mortality) in patients with CKD. Understanding the mechanism and risk factors underlying the three key outcome processes, (1) CKD progression, (2) CVD, and (3) subsequent terminal event in the CKD patient population remains incomplete. Thus, in this work, we develop a novel trivariate joint model to study the risk factors associated with the interdependent outcomes of kidney function (as measured by longitudinal estimated glomerular filtration rate), recurrent cardiovascular events, and the terminal event. Efficient estimation and inference is proposed within a Bayesian framework using Markov Chain Monte Carlo and Bayesian P-splines for hazard functions. The proposed Bayesian framework is directly generalizable beyond trivariate outcome processes to accommodate other potential modeling of complex multi-disease processes. The method is applied to study the aforementioned trivariate processes using data from the Chronic Renal Insufficiency Cohort Study, an ongoing prospective cohort study, established by the National Institute of Diabetes and Digestive and Kidney Diseases to address the rising epidemic of CKD in the US.

A stochastic world model on gravity for stability inference.

The fact that objects without proper support will fall to the ground is not only a natural phenomenon, but also common sense in mind. Previous studies suggest that humans may infer objects' stability through a world model that performs mental simulations with a priori knowledge of gravity acting upon the objects. Here we measured participants' sensitivity to gravity to investigate how the world model works. We found that the world model on gravity was not a faithful replica of the physical laws, but instead encoded gravity's vertical direction as a Gaussian distribution. The world model with this stochastic feature fit nicely with participants' subjective sense of objects' stability and explained the illusion that taller objects are perceived as more likely to fall. Furthermore, a computational model with reinforcement learning revealed that the stochastic characteristic likely originated from experience-dependent comparisons between predictions formed by internal simulations and the realities observed in the external world, which illustrated the ecological advantage of stochastic representation in balancing accuracy and speed for efficient stability inference. The stochastic world model on gravity provides an example of how a priori knowledge of the physical world is implemented in mind that helps humans operate flexibly in open-ended environments.

A stochastic world model on gravity for stability inference

The fact that objects without proper support will fall to the ground is not only a natural phenomenon, but also common sense in mind. Previous studies suggest that humans may infer objects’ stability through a world model that performs mental simulations with a priori knowledge of gravity acting upon the objects. Here we measured participants’ sensitivity to gravity to investigate how the world model works. We found that the world model on gravity was not a faithful replica of the physical laws, but instead encoded gravity’s vertical direction as a Gaussian distribution. The world model with this stochastic feature fit nicely with participants’ subjective sense of objects’ stability and explained the illusion that taller objects are perceived as more likely to fall. Furthermore, a computational model with reinforcement learning revealed that the stochastic characteristic likely originated from experience-dependent comparisons between predictions formed by internal simulations and the realities observed in the external world, which illustrated the ecological advantage of stochastic representation in balancing accuracy and speed for efficient stability inference. The stochastic world model on gravity provides an example of how a priori knowledge of the physical world is implemented in mind that helps humans operate flexibly in open-ended environments.

Partial Cross Mapping Based on Sparse Variable Selection for Direct Fault Root Cause Diagnosis for Industrial Processes.

Root cause diagnosis of process industry is of significance to ensure safe production and improve production efficiency. Conventional contribution plot methods have challenges in root cause diagnosis due to the smearing effect. Other traditional root cause diagnosis methods, such as Granger causality (GC) and transfer entropy, have unsatisfactory performance in root cause diagnosis for complex industrial processes due to the existence of indirect causality. In this work, a regularization and partial cross mapping (PCM)-based root cause diagnosis framework is proposed for efficient direct causality inference and fault propagation path tracing. First, generalized Lasso-based variable selection is performed. The Hotelling T2 statistic is formulated and the Lasso-based fault reconstruction is applied to select candidate root cause variables. Second, the root cause is diagnosed through the PCM and the propagation path is drawn out according to the diagnosis result. The proposed framework is studied in four cases to verify its rationality and effectiveness, including a numerical example, the Tennessee Eastman benchmark process, the wastewater treatment process (WWTP), and the decarburization process of high-speed wire rod spring steel.

Efficient Inference of Graph Neural Networks Using Local Sensitive Hash

Efficient Inference of Graph Neural Networks Using Local Sensitive Hash

Bayesian Estimation of Inverted Beta Mixture Models With Extended Stochastic Variational Inference for Positive Vector Classification.

The finite inverted beta mixture model (IBMM) has been proven to be efficient in modeling positive vectors. Under the traditional variational inference framework, the critical challenge in Bayesian estimation of the IBMM is that the computational cost of performing inference with large datasets is prohibitively expensive, which often limits the use of Bayesian approaches to small datasets. An efficient alternative provided by the recently proposed stochastic variational inference (SVI) framework allows for efficient inference on large datasets. Nevertheless, when using the SVI framework to address the non-Gaussian statistical models, the evidence lower bound (ELBO) cannot be explicitly calculated due to the intractable moment computation. Therefore, the algorithm under the SVI framework cannot directly use stochastic optimization to optimize the ELBO, and an analytically tractable solution cannot be derived. To address this problem, we propose an extended version of the SVI framework with more flexibility, namely, the extended SVI (ESVI) framework. This framework can be used in many non-Gaussian statistical models. First, some approximation strategies are applied to further lower the ELBO to avoid intractable moment calculations. Then, stochastic optimization with noisy natural gradients is used to optimize the lower bound. The excellent performance and effectiveness of the proposed method are verified in real data evaluation.

HyBNN: Quantifying and Optimizing Hardware Efficiency of Binary Neural Networks

Binary neural network (BNN), where both the weight and the activation values are represented with one bit, provides an attractive alternative to deploy highly efficient deep learning inference on resource-constrained edge devices. However, our investigation reveals that, to achieve satisfactory accuracy gains, state-of-the-art (SOTA) BNNs, such as FracBNN and ReActNet, usually have to incorporate various auxiliary floating-point components and increase the model size, which in turn degrades the hardware performance efficiency. In this article, we aim to quantify such hardware inefficiency in SOTA BNNs and further mitigate it with negligible accuracy loss. First, we observe that the auxiliary floating-point (AFP) components consume an average of 93% DSPs, 46% LUTs, and 62% FFs, among the entire BNN accelerator resource utilization. To mitigate such overhead, we propose a novel algorithm-hardware co-design, called FuseBNN , to fuse those AFP operators without hurting the accuracy. On average, FuseBNN reduces AFP resource utilization to 59% DSPs, 13% LUTs, and 16% FFs. Second, SOTA BNNs often use the compact MobileNetV1 as the backbone network but have to replace the lightweight 3 × 3 depth-wise convolution (DWC) with the 3 × 3 standard convolution (SC, e.g., in ReActNet and our ReActNet-adapted BaseBNN) or even more complex fractional 3 × 3 SC (e.g., in FracBNN) to bridge the accuracy gap. As a result, the model parameter size is significantly increased and becomes 2.25× larger than that of the 4-bit direct quantization with the original DWC (4-Bit-Net); the number of multiply-accumulate operations is also significantly increased so that the overall LUT resource usage of BaseBNN is almost the same as that of 4-Bit-Net. To address this issue, we propose HyBNN , where we binarize depth-wise separation convolution (DSC) blocks for the first time to decrease the model size and incorporate 4-bit DSC blocks to compensate for the accuracy loss. For the ship detection task in synthetic aperture radar imagery on the AMD-Xilinx ZCU102 FPGA, HyBNN achieves a detection accuracy of 94.8% and a detection speed of 615 frames per second (FPS), which is 6.8× faster than FuseBNN+ (94.9% accuracy) and 2.7× faster than 4-Bit-Net (95.9% accuracy). For image classification on the CIFAR-10 dataset on the AMD-Xilinx Ultra96-V2 FPGA, HyBNN achieves 1.5× speedup and 0.7% better accuracy over SOTA FracBNN.

SharePro: an accurate and efficient genetic colocalization method accounting for multiple causal signals.

Colocalization analysis is commonly used to assess whether two or more traits share the same genetic signals identified in genome-wide association studies (GWAS), and is important for prioritizing targets for functional follow-up of GWAS results. Existing colocalization methods can have suboptimal performance when there are multiple causal variants in one genomic locus. We propose SharePro to extend the COLOC framework for colocalization analysis. SharePro integrates linkage disequilibrium (LD) modeling and colocalization assessment by grouping correlated variants into effect groups. With an efficient variational inference algorithm, posterior colocalization probabilities can be accurately estimated. In simulation studies, SharePro demonstrated increased power with a well-controlled false positive rate at a low computational cost. Compared to existing methods, SharePro provided stronger and more consistent colocalization evidence for known lipid-lowering drug target proteins and their corresponding lipid traits. Through an additional challenging case of the colocalization analysis of the circulating abundance of R-spondin 3 GWAS and estimated bone mineral density GWAS, we demonstrated the utility of SharePro in identifying biologically plausible colocalized signals. SharePro for colocalization analysis is written in Python and openly available at https://github.com/zhwm/SharePro_coloc.

Improvements in viral gene annotation using large language models and soft alignments

BackgroundThe annotation of protein sequences in public databases has long posed a challenge in molecular biology. This issue is particularly acute for viral proteins, which demonstrate limited homology to known proteins when using alignment, k-mer, or profile-based homology search approaches. A novel methodology employing Large Language Models (LLMs) addresses this methodological challenge by annotating protein sequences based on embeddings.ResultsCentral to our contribution is the soft alignment algorithm, drawing from traditional protein alignment but leveraging embedding similarity at the amino acid level to bypass the need for conventional scoring matrices. This method not only surpasses pooled embedding-based models in efficiency but also in interpretability, enabling users to easily trace homologous amino acids and delve deeper into the alignments. Far from being a black box, our approach provides transparent, BLAST-like alignment visualizations, combining traditional biological research with AI advancements to elevate protein annotation through embedding-based analysis while ensuring interpretability. Tests using the Virus Orthologous Groups and ViralZone protein databases indicated that the novel soft alignment approach recognized and annotated sequences that both blastp and pooling-based methods, which are commonly used for sequence annotation, failed to detect.ConclusionThe embeddings approach shows the great potential of LLMs for enhancing protein sequence annotation, especially in viral genomics. These findings present a promising avenue for more efficient and accurate protein function inference in molecular biology.

Revisiting drug-protein interaction prediction: a novel global-local perspective.

Accurate inference of potential drug-protein interactions (DPIs) aids in understanding drug mechanisms and developing novel treatments. Existing deep learning models, however, struggle with accurate node representation in DPI prediction, limiting their performance. We propose a new computational framework that integrates global and local features of nodes in the drug-protein bipartite graph for efficient DPI inference. Initially, we employ pre-trained models to acquire fundamental knowledge of drugs and proteins and to determine their initial features. Subsequently, the MinHash and HyperLogLog algorithms are utilized to estimate the similarity and set cardinality between drug and protein subgraphs, serving as their local features. Then, an energy-constrained diffusion mechanism is integrated into the transformer architecture, capturing interdependencies between nodes in the drug-protein bipartite graph and extracting their global features. Finally, we fuse the local and global features of nodes and employ multilayer perceptrons to predict the likelihood of potential DPIs. A comprehensive and precise node representation guarantees efficient prediction of unknown DPIs by the model. Various experiments validate the accuracy and reliability of our model, with molecular docking results revealing its capability to identify potential DPIs not present in existing databases. This approach is expected to offer valuable insights for furthering drug repurposing and personalized medicine research. Our code and data are accessible at: https://github.com/ZZCrazy00/DPI.

Computationally Efficient Demographic History Inference from Allele Frequencies with Supervised Machine Learning.

Inferring past demographic history of natural populations from genomic data is of central concern in many studies across research fields. Previously, our group had developed dadi, a widely used demographic history inference method based on the allele frequency spectrum (AFS) and maximum composite-likelihood optimization. However, dadi's optimization procedure can be computationally expensive. Here, we present donni (demography optimization via neural network inference), a new inference method based on dadi that is more efficient while maintaining comparable inference accuracy. For each dadi-supported demographic model, donni simulates the expected AFS for a range of model parameters then trains a set of Mean Variance Estimation neural networks using the simulated AFS. Trained networks can then be used to instantaneously infer the model parameters from future genomic data summarized by an AFS. We demonstrate that for many demographic models, donni can infer some parameters, such as population size changes, very well and other parameters, such as migration rates and times of demographic events, fairly well. Importantly, donni provides both parameter and confidence interval estimates from input AFS with accuracy comparable to parameters inferred by dadi's likelihood optimization while bypassing its long and computationally intensive evaluation process. donni's performance demonstrates that supervised machine learning algorithms may be a promising avenue for developing more sustainable and computationally efficient demographic history inference methods.