Fast Inference Research Articles

Dynamic threshold integrate and fire neuron model for low latency spiking neural networks

Spiking Neural Networks (SNNs) operate with asynchronous discrete events which enable lower power and greater computational efficiency on event-driven hardware than Artificial Neural Networks (ANNs). Conventional ANN-to-SNN conversion methods usually employ Integrate and Fire (IF) neuron model with a fixed threshold to act as Rectified Linear Unit (ReLU). However, there is a large demand for the input spikes to reach the fixed threshold and fire, which leads to high inference latency. In this work, we propose a Dynamic Threshold Integrate and Fire (DTIF) neuron model by exploiting the biological neuron threshold variability, where the threshold is inversely related to the neuron input. The spike activity is increased by dynamically adjusting the threshold at each simulation time-step to reduce the latency. Compared to the state-of-the-art conversion methods, the ANN-to-SNN conversion using DTIF model has lower latency with competitive accuracy, which has been verified by deep architecture on image classification tasks including MNIST, CAIFAR-10, and CIFAR-100 datasets. Moreover, it achieves 7.14 × faster inference under 0.44 × energy consumption than the typical method of maximum normalization.

Deep Learning Accelerated Design of Mechanically Efficient Architected Materials

Lattice structures are known to have high performance-to-weight ratios because of their highly efficient material distribution in a given volume. However, their inherently large void fraction leads to low mechanical properties compared to the base material, high anisotropy, and brittleness. Most works to date have focused on modifying the spatial arrangement of beam elements to overcome these limitations, but only simple beam geometries are adopted due to the infinitely large design space associated with probing and varying beam shapes. Herein, we present an approach to enhance the elastic modulus, strength, and toughness of lattice structures with minimal tradeoffs by optimizing the shape of beam elements for a suite of lattice structures. A generative deep learning-based approach is employed, which leverages the fast inference of neural networks to accelerate the optimization process. Our optimized lattice structures possess superior stiffness (+59%), strength (+49%), toughness (+106%), and isotropy (+645%) compared to benchmark lattices consisting of cylindrical beams. We fabricate our lattice designs using additive manufacturing to validate the optimization approach; experimental and simulation results show good agreement. Remarkable improvement in mechanical properties is shown to be the effect of distributed stress fields and deformation modes subject to beam shape and lattice type.

Neural Importance Sampling for Rapid and Reliable Gravitational-Wave Inference.

We combine amortized neural posterior estimation with importance sampling for fast and accurate gravitational-wave inference. We first generate a rapid proposal for the Bayesian posterior using neural networks, and then attach importance weights based on the underlying likelihood and prior. This provides (1)a corrected posterior free from network inaccuracies, (2)a performance diagnostic (the sample efficiency) for assessing the proposal and identifying failure cases, and (3)an unbiased estimate of the Bayesian evidence. By establishing this independent verification and correction mechanism we address some of the most frequent criticisms against deep learning for scientific inference. We carry out a large study analyzing 42 binary black hole mergers observed by LIGO and Virgo with the SEOBNRv4PHM and IMRPhenomXPHM waveform models. This shows a median sample efficiency of ≈10% (2 orders of magnitude better than standard samplers) as well as a tenfold reduction in the statistical uncertainty in the log evidence. Given these advantages, we expect a significant impact on gravitational-wave inference, and for this approach to serve as a paradigm for harnessing deep learning methods in scientific applications.

Cardiovascular disease incidence prediction by machine learning and statistical techniques: a 16-year cohort study from eastern Mediterranean region

BackgroundCardiovascular diseases (CVD) are the predominant cause of early death worldwide. Identification of people with a high risk of being affected by CVD is consequential in CVD prevention. This study adopts Machine Learning (ML) and statistical techniques to develop classification models for predicting the future occurrence of CVD events in a large sample of Iranians.MethodsWe used multiple prediction models and ML techniques with different abilities to analyze the large dataset of 5432 healthy people at the beginning of entrance into the Isfahan Cohort Study (ICS) (1990–2017). Bayesian additive regression trees enhanced with “missingness incorporated in attributes” (BARTm) was run on the dataset with 515 variables (336 variables without and the remaining with up to 90% missing values). In the other used classification algorithms, variables with more than 10% missing values were excluded, and MissForest imputes the missing values of the remaining 49 variables. We used Recursive Feature Elimination (RFE) to select the most contributing variables. Random oversampling technique, recommended cut-point by precision-recall curve, and relevant evaluation metrics were used for handling unbalancing in the binary response variable.ResultsThis study revealed that age, systolic blood pressure, fasting blood sugar, two-hour postprandial glucose, diabetes mellitus, history of heart disease, history of high blood pressure, and history of diabetes are the most contributing factors for predicting CVD incidence in the future. The main differences between the results of classification algorithms are due to the trade-off between sensitivity and specificity. Quadratic Discriminant Analysis (QDA) algorithm presents the highest accuracy (75.50 ± 0.08) but the minimum sensitivity (49.84 ± 0.25); In contrast, decision trees provide the lowest accuracy (51.95 ± 0.69) but the top sensitivity (82.52 ± 1.22). BARTm.90% resulted in 69.48 ± 0.28 accuracy and 54.00 ± 1.66 sensitivity without any preprocessing step.ConclusionsThis study confirmed that building a prediction model for CVD in each region is valuable for screening and primary prevention strategies in that specific region. Also, results showed that using conventional statistical models alongside ML algorithms makes it possible to take advantage of both techniques. Generally, QDA can accurately predict the future occurrence of CVD events with a fast (inference speed) and stable (confidence values) procedure. The combined ML and statistical algorithm of BARTm provide a flexible approach without any need for technical knowledge about assumptions and preprocessing steps of the prediction procedure.

Causal inference based cuffless blood pressure estimation: A pilot study

Enabled by wearable sensing, e.g., photoplethysmography (PPG) and electrocardiography (ECG), and machine learning techniques, study on cuffless blood pressure (BP) measurement with data-driven methods has become popular in recent years. However, causality has been overlooked in most of current studies. In this study, we aim to examine the feasibility of causal inference for cuffless BP estimation. We first attempt to detect wearable features that are causally related, rather than correlated, to BP changes by identifying causal graphs of interested variables with fast causal inference (FCI) algorithm. With identified causal features, we then employ time-lagged link to integrate the mechanism of causal inference into the BP estimated model. The proposed method was validated on 62 subjects with their continuous ECG, PPG and BP signals being collected. We found new causal features that can better track BP changes than pulse transit time (PTT). Further, the developed causal-based estimation model achieved an estimation error of mean absolute difference (MAD) being 5.10 mmHg and 2.85 mmHg for SBP and DBP, respectively, which outperformed traditional model without consideration of causality. To the best of our knowledge, this work is the first to study the causal inference for cuffless BP estimation, which can shed light on the mechanism, method and application of cuffless BP measurement.

Panoptic SwiftNet: Pyramidal Fusion for Real-Time Panoptic Segmentation

Dense panoptic prediction is a key ingredient in many existing applications such as autonomous driving, automated warehouses, or remote sensing. Many of these applications require fast inference over large input resolutions on affordable or even embedded hardware. We proposed to achieve this goal by trading off backbone capacity for multi-scale feature extraction. In comparison with contemporaneous approaches to panoptic segmentation, the main novelties of our method are efficient scale-equivariant feature extraction, cross-scale upsampling through pyramidal fusion and boundary-aware learning of pixel-to-instance assignment. The proposed method is very well suited for remote sensing imagery due to the huge number of pixels in typical city-wide and region-wide datasets. We present panoptic experiments on Cityscapes, Vistas, COCO, and the BSB-Aerial dataset. Our models outperformed the state-of-the-art on the BSB-Aerial dataset while being able to process more than a hundred 1MPx images per second on an RTX3090 GPU with FP16 precision and TensorRT optimization.

Time Series Anomaly Detection With Adversarial Reconstruction Networks

Time series data naturally exist in many domains including medical data analysis, infrastructure sensor monitoring, and motion tracking. However, a very small portion of anomalous time series can be observed, comparing to the whole data. Most existing approaches are based on the supervised classification model requiring representative labels for anomaly class(es), which is challenging in real-world problems. So can we learn how to detect anomalous time ticks in an effective yet efficient way, given mostly normal time series data? Therefore, we propose an unsupervised reconstruction model named BeatGAN which learns to detect anomalies based on normal data, or data which majority of samples are normal. BeatGAN provides a framework to adversarially learn to reconstruct, which can cooperate with both 1-d CNN and RNN. Rarely observed anomalies can result in larger reconstruction errors, which are then detected based on extreme value theory. Moreover, data augmentation with dynamic time warping regularizes reconstruction and provides robustness. In the experiments, effectiveness and sensitivity are studied in both synthetic data and various real-world time series. BeatGAN achieves better accuracy and fast inference.

Amortized Bayesian inference on generative dynamical network models of epilepsy using deep neural density estimators

Whole-brain modeling of epilepsy combines personalized anatomical data with dynamical models of abnormal activities to generate spatio-temporal seizure patterns as observed in brain imaging data. Such a parametric simulator is equipped with a stochastic generative process, which itself provides the basis for inference and prediction of the local and global brain dynamics affected by disorders. However, the calculation of likelihood function at whole-brain scale is often intractable. Thus, likelihood-free algorithms are required to efficiently estimate the parameters pertaining to the hypothetical areas, ideally including the uncertainty. In this study, we introduce the simulation-based inference for the virtual epileptic patient model (SBI-VEP), enabling us to amortize the approximate posterior of the generative process from a low-dimensional representation of whole-brain epileptic patterns. The state-of-the-art deep learning algorithms for conditional density estimation are used to readily retrieve the statistical relationships between parameters and observations through a sequence of invertible transformations. We show that the SBI-VEP is able to efficiently estimate the posterior distribution of parameters linked to the extent of the epileptogenic and propagation zones from sparse intracranial electroencephalography recordings. The presented Bayesian methodology can deal with non-linear latent dynamics and parameter degeneracy, paving the way for fast and reliable inference on brain disorders from neuroimaging modalities.

Fast and precise inference on diffusivity in interacting particle systems

Particle systems made up of interacting agents is a popular model used in a vast array of applications, not the least in biology where the agents can represent everything from single cells to animals in a herd. Usually, the particles are assumed to undergo some type of random movements, and a popular way to model this is by using Brownian motion. The magnitude of random motion is often quantified using mean squared displacement, which provides a simple estimate of the diffusion coefficient. However, this method often fails when data is sparse or interactions between agents frequent. In order to address this, we derive a conjugate relationship in the diffusion term for large interacting particle systems undergoing isotropic diffusion, giving us an efficient inference method. The method accurately accounts for emerging effects such as anomalous diffusion stemming from mechanical interactions. We apply our method to an agent-based model with a large number of interacting particles, and the results are contrasted with a naive mean square displacement-based approach. We find a significant improvement in performance when using the higher-order method over the naive approach. This method can be applied to any system where agents undergo Brownian motion and will lead to improved estimates of diffusion coefficients compared to existing methods.

Correlation-based sparse inverse Cholesky factorization for fast Gaussian-process inference

Gaussian processes are widely used as priors for unknown functions in statistics and machine learning. To achieve computationally feasible inference for large datasets, a popular approach is the Vecchia approximation, which is an ordered conditional approximation of the data vector that implies a sparse Cholesky factor of the precision matrix. The ordering and sparsity pattern are typically determined based on Euclidean distance of the inputs or locations corresponding to the data points. Here, we propose instead to use a correlation-based distance metric, which implicitly applies the Vecchia approximation in a suitable transformed input space. The correlation-based algorithm can be carried out in quasilinear time in the size of the dataset, and so it can be applied even for iterative inference on unknown parameters in the correlation structure. The correlation-based approach has two advantages for complex settings: It can result in more accurate approximations, and it offers a simple, automatic strategy that can be applied to any covariance, even when Euclidean distance is not applicable. We demonstrate these advantages in several settings, including anisotropic, nonstationary, multivariate, and spatio-temporal processes. We also illustrate our method on multivariate spatio-temporal temperature fields produced by a regional climate model.

BTAD: A binary transformer deep neural network model for anomaly detection in multivariate time series data

In the context of big data, if the task of multivariate time series data anomaly detection cannot be performed efficiently and accurately, it will bring great security risks to industrial systems. However, fast model inference requirements, unlabeled datasets and excessively long time series make it a challenging problem to build an accurate and fast anomaly detection model. In this paper, we propose an unsupervised Bi-Transformer anomaly detection method (BTAD) for multivariate time series data, which uses Bi-Transformer structure to extract dataset association features, and uses an improved adaptive multi-head attention mechanism to infer trends in each meta-dimension of multivariate time series data in parallel. The modified Decoder structure prevents the reconstructed output of BTAD from being disturbed by the input information. Self-conditioning mechanism could enhance the robustness to noisy data, and improve model’s generalization ability. Experiments show that BTAD could outperform other models in detection performance and training efficiency. Taking NAB dataset as an example, the AUC and F1 of BTAD are increased by more than 4.78% and 1.40% separately. Finally, we look forward to the future development trend of BTAD, and put forward the corresponding improvement ideas.

Reduced-Parameter YOLO-like Object Detector Oriented to Resource-Constrained Platform

Deep learning-based target detectors are in demand for a wide range of applications, often in areas such as robotics and the automotive industry. The high computational requirements of deep learning severely limit its ability to be deployed on resource-constrained and energy-first devices. To address this problem, we propose a class YOLO target detection algorithm and deploy it to an FPGA platform. Based on the FPGA platform, we can make full use of its computational features of parallel computing, and the computational units such as convolution, pooling and Concat layers in the model can be accelerated for inference.To enable our algorithm to run efficiently on FPGAs, we quantized the model and wrote the corresponding hardware operators based on the model units. The proposed object detection accelerator has been implemented and verified on the Xilinx ZYNQ platform. Experimental results show that the detection accuracy of the algorithm model is comparable to that of common algorithms, and the power consumption is much lower than that of the CPU and GPU. After deployment, the accelerator has a fast inference speed and is suitable for deployment on mobile devices to detect the surrounding environment.

Fast Bayesian inference of block Nearest Neighbor Gaussian models for large data

Fast Bayesian inference of block Nearest Neighbor Gaussian models for large data

TriHorn-Net: A model for accurate depth-based 3D hand pose estimation

3D hand pose estimation methods have made significant progress recently. However, estimation accuracy is often far from sufficient for specific real-world applications, and thus there is significant room for improvement. This paper proposes TriHorn-Net, a novel model that uses specific innovations to improve hand pose estimation accuracy on depth images. The first innovation is decomposition of the 3D hand pose estimation into the estimation of 2D joint locations in the depth image space (UV), and the estimation of their corresponding depths aided by two complementary attention maps. This decomposition prevents depth estimation, which is a more difficult task, from interfering with the UV estimations at both the prediction and feature levels. The second innovation is PixDropout, which is, to the best of our knowledge, the first appearance-based data augmentation method for hand depth images. Experimental results demonstrate that the proposed model outperforms the state-of-the-art methods on three public benchmark datasets, achieving 5.73 mm, 7.13 mm, and 7.68 mm on the ICVL, MSRA, and NYU datasets, respectively. The proposed model achieves this performance with a relatively fast average inference speed of 9.25 ms per frame on an NVIDIA 1080Ti GPU, which, as discussed in the paper, places the proposed model among the fastest-performing 3D hand pose estimation models. Our implementation is available at https://github.com/mrezaei92/TriHorn-Net.

AdaPop: Bayesian inference of dependent population dynamics in coalescent models.

The coalescent is a powerful statistical framework that allows us to infer past population dynamics leveraging the ancestral relationships reconstructed from sampled molecular sequence data. In many biomedical applications, such as in the study of infectious diseases, cell development, and tumorgenesis, several distinct populations share evolutionary history and therefore become dependent. The inference of such dependence is a highly important, yet a challenging problem. With advances in sequencing technologies, we are well positioned to exploit the wealth of high-resolution biological data for tackling this problem. Here, we present adaPop, a probabilistic model to estimate past population dynamics of dependent populations and to quantify their degree of dependence. An essential feature of our approach is the ability to track the time-varying association between the populations while making minimal assumptions on their functional shapes via Markov random field priors. We provide nonparametric estimators, extensions of our base model that integrate multiple data sources, and fast scalable inference algorithms. We test our method using simulated data under various dependent population histories and demonstrate the utility of our model in shedding light on evolutionary histories of different variants of SARS-CoV-2.

EEFDet: an efficient and effective face detector for lightweight, fast, and accurate face detection

In practical applications, it is urgent to develop lightweight, fast, and accurate face detectors. Although the performance of face detection has made tremendous progress with the rapid development of deep learning, they still struggle to meet the requirements of both efficiency and effectiveness. This is because current face detectors either use large networks or cumbersome processes oriented for accuracy but leading to heavy model complexity and slow inference speed or use various simple designs oriented for efficiency but greatly sacrificing accuracy or not actually efficient. In this paper, we conduct an in-depth study of the network structure and anchor setting and present an efficient and effective face detector named EEFDet, which can detect faces with lightweight model, fast inference speed, and excellent accuracy. First, a lightweight, fast, and robust network (LFRNet) is constructed by carefully designing the backbone network, detection neck, and detection head of the network. LFRNet not only has small model complexity and low latency but also possesses robust feature extraction ability. Then, a sparse anchor strategy is proposed by rationally setting anchors associated with the detection head to avoid using too many redundant anchors and get the most effective anchors. This strategy enables the detector to process multiple anchor-related parts at a fast speed while still guaranteeing detection accuracy. Extensive experiments and detailed ablation analysis are conducted on several widely used face detection benchmarks based on multiple metrics and different platforms. The results show that our EEFDet is lightweight (1.195 M parameters and 0.719 G FLOPs), fast (38.780 ms on GPU and 141.787 ms on CPU), and accurate [90.8%/89.6% (easy), 88.1%/87.2% (medium), 77.9%/77.4% (hard) APs on Val/Test set of WIDER FACE, 96.3% discrete score and 79.7% continuous score on face detection data set and benchmark (FDDB), 97.7% AP on PASCAL FACE, and 98.2% AP on annotated face in-the-wild (AFW)], achieving an optimal trade-off between efficiency and effectiveness compared to state-of-the-art face detectors.

Email Spam Filtering Methods: Comparison and Analysis

Email is a common way of communication due to its cheap cost, efficacy, and efficiency. With the emerging of deep learning and machine learning methods, spam filter classification achieves boosting performance with fast inference speed. However, individuals and email servers are affected by spam, which causes issues with wasted time and computer storage space, as well as adverse effects on bandwidth. Even worse, email users are susceptible to scams and fraud that may result in financial loss. Therefore, it is essential to discover an efficient approach to filter spam from the entire number of emails. The purpose of this study is to evaluate and contrast the five most popular machine learning based spam filtering techniques, including Naive Bayes, Supported Vector Machine K-Nearest Neighbor, and XGBoost. We evaluate them based on their performance and efficacy. We hope this paper will help to conclude the current condition and help the researchers to improve better algorithms with higher accuracy.

Robust and Realtime Large Deformation Ultrasound Registration Using End-to-End Differentiable Displacement Optimisation

Image registration for temporal ultrasound sequences can be very beneficial for image-guided diagnostics and interventions. Cooperative human-machine systems that enable seamless assistance for both inexperienced and expert users during ultrasound examinations rely on robust, realtime motion estimation. Yet rapid and irregular motion patterns, varying image contrast and domain shifts in imaging devices pose a severe challenge to conventional realtime registration approaches. While learning-based registration networks have the promise of abstracting relevant features and delivering very fast inference times, they come at the potential risk of limited generalisation and robustness for unseen data; in particular, when trained with limited supervision. In this work, we demonstrate that these issues can be overcome by using end-to-end differentiable displacement optimisation. Our method involves a trainable feature backbone, a correlation layer that evaluates a large range of displacement options simultaneously and a differentiable regularisation module that ensures smooth and plausible deformation. In extensive experiments on public and private ultrasound datasets with very sparse ground truth annotation the method showed better generalisation abilities and overall accuracy than a VoxelMorph network with the same feature backbone, while being two times faster at inference.

Machine learning-based signal quality assessment for cardiac volume monitoring in electrical impedance tomography

Owing to recent advances in thoracic electrical impedance tomography (EIT), a patient’s hemodynamic function can be noninvasively and continuously estimated in real-time by surveilling a cardiac volume signal (CVS) associated with stroke volume and cardiac output. In clinical applications, however, a CVS is often of low quality, mainly because of the patient’s deliberate movements or inevitable motions during clinical interventions. This study aims to develop a signal quality indexing method that assesses the influence of motion artifacts on transient CVSs. The assessment is performed on each cardiac cycle to take advantage of the periodicity and regularity in cardiac volume changes. Time intervals are identified using the synchronized electrocardiography system. We apply divergent machine-learning methods, which can be sorted into discriminative-model and manifold-learning approaches. The use of machine-learning could be suitable for our real-time monitoring application that requires fast inference and automation as well as high accuracy. In the clinical environment, the proposed method can be utilized to provide immediate warnings so that clinicians can minimize confusion regarding patients’ conditions, reduce clinical resource utilization, and improve the confidence level of the monitoring system. Numerous experiments using actual EIT data validate the capability of CVSs degraded by motion artifacts to be accurately and automatically assessed in real-time by machine learning. The best model achieved an accuracy of 0.95, positive and negative predictive values of 0.96 and 0.86, sensitivity of 0.98, specificity of 0.77, and AUC of 0.96.

A gradient optimization and manifold preserving based binary neural network for point cloud

With significant progress of deep learning on 3D point cloud, the demand for deployment of point cloud neural network on the edge devices is growing. Binary neural network, a type of quantization compression method, with extreme low bit and fast inference speed, attracts more attention. It is more challenging, but has greater potentiality. Most of the researches on binary networks focus on images rather than point cloud. Considering the particularity of point cloud neural network, this paper presents a novel binarization framework, which includes two main contributions. Firstly, a gradient optimization method is proposed to overcome the shortcomings of Straight Through Estimator (STE) commonly used in the back propagation of binary network training. Secondly, based on the analysis of manifold distortion caused by the binary convolution and pooling operations, we propose an optimized scaling recovery method to restore manifold for the convoluted feature, and also, a pooling correction method to improve the pooled feature's fidelity. Manifold distortion leads to the severe feature homogeneity problem, which brings trouble in generating features with sufficient discrimination for classification and segmentation. The manifold preserving optimizations are designed to introduce minimum extra parameters to balance the accuracy with the computation and storage consumption. Experiments show that the proposed method outperforms state-of-the-art in accuracy with ignored overhead, and also has good scalability.