Fast Inference Research Articles

I3Net: Inter-Intra-slice Interpolation Network for Medical Slice Synthesis.

Medical imaging is limited by acquisition time and scanning equipment. CT and MR volumes, reconstructed with thicker slices, are anisotropic with high in-plane resolution and low through-plane resolution. We reveal an intriguing phenomenon that due to the mentioned nature of data, performing slice-wise interpolation from the axial view can yield greater benefits than performing super-resolution from other views. Based on this observation, we propose an Inter-Intra-slice Interpolation Network (I3Net), which fully explores information from high in-plane resolution and compensates for low through-plane resolution. The through-plane branch supplements the limited information contained in low through-plane resolution from high in-plane resolution and enables continual and diverse feature learning. In-plane branch transforms features to the frequency domain and enforces an equal learning opportunity for all frequency bands in a global context learning paradigm. We further propose a cross-view block to take advantage of the information from all three views online. Extensive experiments on two public datasets demonstrate the effectiveness of I3Net, and noticeably outperforms state-of-the-art super-resolution, video frame interpolation and slice interpolation methods by a large margin. We achieve 43.90dB in PSNR, with at least 1.14dB improvement under the upscale factor of ×2 on MSD dataset with faster inference.

Fregata: Fast Private Inference With Unified Secure Two-Party Protocols

Fregata: Fast Private Inference With Unified Secure Two-Party Protocols

Hierarchical path planning from speech instructions with spatial concept-based topometric semantic mapping.

Assisting individuals in their daily activities through autonomous mobile robots is a significant concern, especially for users without specialized knowledge. Specifically, the capability of a robot to navigate to destinations based on human speech instructions is crucial. Although robots can take different paths toward the same objective, the shortest path is not always the most suitable. A preferred approach would be to accommodate waypoint specifications flexibly for planning an improved alternative path even with detours. Furthermore, robots require real-time inference capabilities. In this sense, spatial representations include semantic, topological, and metric-level representations, each capturing different aspects of the environment. This study aimed to realize a hierarchical spatial representation using a topometric semantic map and path planning with speech instructions by including waypoints. Thus, we present a hierarchical path planning method called spatial concept-based topometric semantic mapping for hierarchical path planning (SpCoTMHP), which integrates place connectivity. This approach provides a novel integrated probabilistic generative model and fast approximate inferences with interactions among the hierarchy levels. A formulation based on "control as probabilistic inference" theoretically supports the proposed path planning algorithm. We conducted experiments in a home environment using the Toyota human support robot on the SIGVerse simulator and in a lab-office environment with the real robot Albert. Here, the user issues speech commands that specify the waypoint and goal, such as "Go to the bedroom via the corridor." Navigation experiments were performed using speech instructions with a waypoint to demonstrate the performance improvement of the SpCoTMHP over the baseline hierarchical path planning method with heuristic path costs (HPP-I) in terms of the weighted success rate at which the robot reaches the closest target (0.590) and passes the correct waypoints. The computation time was significantly improved by 7.14 s with the SpCoTMHP than the baseline HPP-I in advanced tasks. Thus, hierarchical spatial representations provide mutually understandable instruction forms for both humans and robots, thus enabling language-based navigation.

An Ensemble Approach for Robust Automated Crack Detection and Segmentation in Concrete Structures.

To prevent potential instability the early detection of cracks is imperative due to the prevalent use of concrete in critical infrastructure. Automated techniques leveraging artificial intelligence, machine learning, and deep learning as the traditional manual inspection methods are time-consuming. The existing automated concrete crack detection algorithms, despite recent advancements, face challenges in robustness, particularly in precise crack detection amidst complex backgrounds and visual distractions, while also maintaining low inference times. Therefore, this paper introduces a novel ensemble mechanism based on multiple quantized You Only Look Once version 8 (YOLOv8) models for the detection and segmentation of cracks in concrete structures. The proposed model is tested on different concrete crack datasets yielding enhanced segmentation results with at least 89.62% precision and intersection over a union score of 0.88. Moreover, the inference time per image is reduced to 27 milliseconds which is at least a 5% improvement over other models in the comparison. This is achieved by amalgamating the predictions of the trained models to calculate the final segmentation mask. The noteworthy contributions of this work encompass the creation of a model with low inference time, an ensemble mechanism for robust crack segmentation, and the enhancement of the learning capabilities of crack detection models. The fast inference time of the model renders it appropriate for real-time applications, effectively tackling challenges in infrastructure maintenance and safety.

Data mining techniques for estimating instantaneous NOX emissions from Euro 6 diesel passenger cars

Nitrogen oxides (NOX) emissions that are caused by road traffic diesel engines affects public health. The existing instantaneous emissions models are often imprecise due to the lack of knowledge of highly non-linear processes behind real-world emissions and they do not include meteorological and driving volatility variables. This paper applied data mining techniques based on the Cross Industry Standard Process for Data Mining (CRISP-DM) method to a dataset of four diesel Euro 6 passenger cars tested in real-world driving conditions to: a) model stabilised hot NOX emissions based on kinematic (speed), internal engine (engine coolant temperature, engine load, engine speed, intake air temperature, manifold absolute pressure and mass air flow), meteorological (humidity) and driving volatility (acceleration and vehicular jerk); b) compare the performance of different machine learning (ML) techniques in predicting NOX emission rates, namely: Artificial Neural Networks (ANN), Random Forest (RF), and Gradient-Boosted Trees (GBT). The model that utilizes a set of detailed variables, particularly engine coolant temperature, engine load, engine speed, intake air temperature, humidity, acceleration and vehicular jerk, and using ANN technique was better able to deal with variability in emission data than models based on a single set of these variables. It was also found that models produced high Root Mean Square Error due to their inability in predicting high peaks in measured emission data. The presented models rely on fast inference times and can therefore be deployed for engine control units to inform drivers about their NOX emissions during driving.

Adaptive Ensemble Self-Distillation With Consistent Gradients for Fast Inference of Pretrained Language Models

Adaptive Ensemble Self-Distillation With Consistent Gradients for Fast Inference of Pretrained Language Models

Multimodality Self-distillation for Fast Inference of Vision and Language Pretrained Models

Multimodality Self-distillation for Fast Inference of Vision and Language Pretrained Models

A Feasible Study of a Deep Learning Model Supporting Human–Machine Collaborative Learning of Object-Oriented Programming

Due to the development of deep learning technology, its application in education has received increasing attention from researchers. Intelligent agents based on deep learning technology can perform higher-order intellectual tasks than ever. However, the high deployment cost of deep learning models has hindered their widespread application in education. In addition, there needs to be more research on applying deep learning technology in education. In this study, we develop an intelligent agent using a Performer-based encoder-decoder neural model to classify object-oriented programming (OOP) errors in student code and generate hint feedback in natural language to help students correct the code. This study investigates the feasibility of deploying this agent in an educational setting to support the learning of OOP. This study first examines the low-speed inference problem of the deep learning model. A fast inference algorithm is proposed for the model, which achieves a speedup of eighty times. This study further explores integrating a human-machine collaborative learning process with the deep learning agent. Students were surveyed about their perceptions of the agent in supporting learning. Student responses are interpreted within the learning partnerships model (LPM) framework to show how the agent's technical automation and autonomy features support student-agent learning partnerships. Finally, implications and suggestions for educational application and research of deep learning technology are presented.

CTFNet: Long-Sequence Time-Series Forecasting Based on Convolution and Time-Frequency Analysis.

Although current time-series forecasting methods have significantly improved the state-of-the-art (SOTA) results for long-sequence time-series forecasting (LSTF), they still have difficulty in capturing and extracting the features and dependencies of long-term sequences and suffer from information utilization bottlenecks and high-computational complexity. To address these issues, a lightweight single-hidden layer feedforward neural network (SLFN) combining convolution mapping and time-frequency decomposition called CTFNet is proposed with three distinctive characteristics. First, time-domain (TD) feature mining-in this article, a method for extracting the long-term correlation of horizontal TD features based on matrix factorization is proposed, which can effectively capture the interdependence among different sample points of a long time series. Second, multitask frequency-domain (FD) feature mining-this can effectively extract different frequency feature information of time-series data from the FD and minimize the loss of data features. Integrating multiscale dilated convolutions, simultaneously focusing on both global and local context feature dependencies at the sequence level, and mining the long-term dependencies of the multiscale frequency information and the spatial dependencies among the different scale frequency information, break the bottleneck of data utilization, and ensure the integrity of feature extraction. Third, highly efficient-the CTFNet model has a short training time and fast inference speed. Our empirical studies with nine benchmark datasets show that compared with state-of-the-art methods, CTFNet can reduce prediction error by 64.7% and 53.7% for multivariate and univariate time series, respectively.

RepSGG: Novel Representations of Entities and Relationships for Scene Graph Generation.

Scene Graph Generation (SGG) has achieved significant progress recently. However, most previous works rely heavily on fixed-size entity representations based on bounding box proposals, anchors, or learnable queries. As each representation's cardinality has different trade-offs between performance and computation overhead, extracting highly representative features efficiently and dynamically is both challenging and crucial for SGG. In this work, a novel architecture called RepSGG is proposed to address the aforementioned challenges, formulating a subject as queries, an object as keys, and their relationship as the maximum attention weight between pairwise queries and keys. With more fine-grained and flexible representation power for entities and relationships, RepSGG learns to sample semantically discriminative and representative points for relationship inference. Moreover, the long-tailed distribution also poses a significant challenge for generalization of SGG. A run-time performance-guided logit adjustment (PGLA) strategy is proposed such that the relationship logits are modified via affine transformations based on run-time performance during training. This strategy encourages a more balanced performance between dominant and rare classes. Experimental results show that RepSGG achieves the state-of-the-art or comparable performance on the Visual Genome and Open Images V6 datasets with fast inference speed, demonstrating the efficacy and efficiency of the proposed methods.

SMILE: Siamese Multi-scale Interactive-representation LEarning for Hierarchical Diffeomorphic Deformable image registration

Deformable medical image registration plays an important role in many clinical applications. It aims to find a dense deformation field to establish point-wise correspondences between a pair of fixed and moving images. Recently, unsupervised deep learning-based registration methods have drawn more and more attention because of fast inference at testing stage. Despite remarkable progress, existing deep learning-based methods suffer from several limitations including: (a) they often overlook the explicit modeling of feature correspondences due to limited receptive fields; (b) the performance on image pairs with large spatial displacements is still limited since the dense deformation field is regressed from features learned by local convolutions; and (c) desirable properties, including topology-preservation and the invertibility of transformation, are often ignored. To address above limitations, we propose a novel Convolutional Neural Network (CNN) consisting of a Siamese Multi-scale Interactive-representation LEarning (SMILE) encoder and a Hierarchical Diffeomorphic Deformation (HDD) decoder. Specifically, the SMILE encoder aims for effective feature representation learning and spatial correspondence establishing while the HDD decoder seeks to regress the dense deformation field in a coarse-to-fine manner. We additionally propose a novel Local Invertible Loss (LIL) to encourage topology-preservation and local invertibility of the regressed transformation while keeping high registration accuracy. Extensive experiments conducted on two publicly available brain image datasets demonstrate the superiority of our method over the state-of-the-art (SOTA) approaches. Specifically, on the Neurite-OASIS dataset, our method achieved an average DSC of 0.815 and an average ASSD of 0.633 mm.

Vecchia Likelihood Approximation for Accurate and Fast Inference with Intractable Spatial Max-Stable Models

Max-stable processes are the most popular models for high-impact spatial extreme events, as they arise as the only possible limits of spatially-indexed block maxima. However, likelihood inference for such models suffers severely from the curse of dimensionality, since the likelihood function involves a combinatorially exploding number of terms. In this article, we propose using the Vecchia approximation, which conveniently decomposes the full joint density into a linear number of low-dimensional conditional density terms based on well-chosen conditioning sets designed to improve and accelerate inference in high dimensions. Theoretical asymptotic relative efficiencies in the Gaussian setting and simulation experiments in the max-stable setting show significant efficiency gains and computational savings using the Vecchia likelihood approximation method compared to traditional composite likelihoods. Our application to extreme sea surface temperature data at more than a thousand sites across the entire Red Sea further demonstrates the superiority of the Vecchia likelihood approximation for fitting complex models with intractable likelihoods, delivering significantly better results than traditional composite likelihoods, and accurately capturing the extremal dependence structure at lower computational cost. Supplementary materials for this article are available online.

Attention-aided lightweight networks friendly to smart weeding robot hardware resources for crops and weeds semantic segmentation.

Weed control is a global issue of great concern, and smart weeding robots equipped with advanced vision algorithms can perform efficient and precise weed control. Furthermore, the application of smart weeding robots has great potential for building environmentally friendly agriculture and saving human and material resources. However, most networks used in intelligent weeding robots tend to solely prioritize enhancing segmentation accuracy, disregarding the hardware constraints of embedded devices. Moreover, generalized lightweight networks are unsuitable for crop and weed segmentation tasks. Therefore, we propose an Attention-aided lightweight network for crop and weed semantic segmentation. The proposed network has a parameter count of 0.11M, Floating-point Operations count of 0.24G. Our network is based on an encoder and decoder structure, incorporating attention module to ensures both fast inference speed and accurate segmentation while utilizing fewer hardware resources. The dual attention block is employed to explore the potential relationships within the dataset, providing powerful regularization and enhancing the generalization ability of the attention mechanism, it also facilitates information integration between channels. To enhance the local and global semantic information acquisition and interaction, we utilize the refinement dilated conv block instead of 2D convolution within the deep network. This substitution effectively reduces the number and complexity of network parameters and improves the computation rate. To preserve spatial information, we introduce the spatial connectivity attention block. This block not only acquires more precise spatial information but also utilizes shared weight convolution to handle multi-stage feature maps, thereby further reducing network complexity. The segmentation performance of the proposed network is evaluated on three publicly available datasets: the BoniRob dataset, the Rice Seeding dataset, and the WeedMap dataset. Additionally, we measure the inference time and Frame Per Second on the NVIDIA Jetson Xavier NX embedded system, the results are 18.14 msec and 55.1 FPS. Experimental results demonstrate that our network maintains better inference speed on resource-constrained embedded systems and has competitive segmentation performance.

Exploring Hybrid Models For Short-Term Local Weather Forecasting in IoT Environment

This paper explores using and hybridizing simple prediction models to maximize the accuracy of local weather prediction while maintaining low computational effort and the need to process and acquire large volumes of data. A hybrid RF-LSTM model is proposed and evaluated in this research paper for the task of short-term local weather forecasting. The local weather stations are built within an acceptable radius of the measured area and are designed to provide a short period of forecasting - usually within one hour. The lack of local weather data might be problematic for an accurate short-term valuable prediction in sustainable applications like agriculture, transportation, energy management, and daily life. Weather forecasting is not trivial because of the non-linear nature of time series. Thus, traditional forecasting methods cannot predict the weather accurately. The advantage of the ARIMA model lies in forecasting the linear part, while the SVR model indicates the non-linear characteristic of the weather data. Both non-linear and linear approaches can represent the combined model. The hybrid ARIMA-SVR model strengthens the matched points of the ARIMA model and the SVR model in weather forecasting. The LSTM and random forest are both popular algorithms used for regression problems. LSTM is more suitable for tasks involving sequential data with long-term dependencies. Random Forest leverages the wisdom of crowds by combining multiple decision trees, providing robust predictions, and reducing overfitting. Hybrid Random forest-LSTM potentially leverages the robustness and feature importance of Random Forest along with the ability of LSTM to capture sequential dependencies. The comparison results show that the hybrid RF-LSTM model reduces the forecasting errors in metrics of MAE, R-squared, and RMSE. The proposed hybrid model can also capture the actual temperature trend in its prediction performance, which makes it even more relevant for many other possible decision-making steps in sustainable applications. Furthermore, this paper also proposes the design of a weather station based on a real-time edge IoT system. The RF-LSTM leverages the parallelized characteristics of each decision tree in the forest to accelerate the training process and faster inferences. Thus, the hybrid RF-LSTM model offers advantages in terms of faster execution speed and computational efficiency in both PC and Raspberry Pi boards. However, the RF-LSTM consumes the highest peak memory usage due to being a combination of two different models.

Fitting latent non-Gaussian models using variational Bayes and Laplace approximations

–Latent Gaussian models (LGMs) are perhaps the most commonly used class of models in statistical applications. Nevertheless, in areas ranging from longitudinal studies in biostatistics to geostatistics, it is easy to find datasets that contain inherently non-Gaussian features, such as sudden jumps or spikes, that adversely affect the inferences and predictions made using an LGM. These datasets require more general latent non-Gaussian models (LnGMs) that can handle automatically these non-Gaussian features. However, fast implementation and easy-to-use software are lacking, preventing the broad applicability of LnGMs. In this paper, we derive variational Bayes algorithms for fast and scalable inference of LnGMs. The approximation leads to an LGM that downweights extreme events in the latent process, reducing their influence and leading to more robust inferences. It can be applied to a wide range of models, such as autoregressive processes for time series, simultaneous autoregressive models for areal data, and spatial Matérn models. To facilitate Bayesian inference, we introduce the ngvb package, where LGMs implemented in R-INLA can be easily extended to LnGMs by adding a single line of code.

DSE-NN: Deeply Supervised Efficient Neural Network for Real-Time Remote Photoplethysmography.

Non-contact remote photoplethysmography can be used in a variety of medical and healthcare fields by measuring vital signs continuously and unobtrusively. Recently, end-to-end deep learning methods have been proposed to replace the existing handcrafted features. However, since the existing deep learning methods are known as black box models, the problem of interpretability has been raised, and the same problem exists in the remote photoplethysmography (rPPG) network. In this study, we propose a method to visualize temporal and spectral representations for hidden layers, deeply supervise the spectral representation of intermediate layers through the depth of networks and optimize it for a lightweight model. The optimized network improves performance and enables fast training and inference times. The proposed spectral deep supervision helps to achieve not only high performance but also fast convergence speed through the regularization of the intermediate layers. The effect of the proposed methods was confirmed through a thorough ablation study on public datasets. As a result, similar or outperforming results were obtained in comparison to state-of-the-art models. In particular, our model achieved an RMSE of 1 bpm on the PURE dataset, demonstrating its high accuracy. Moreover, it excelled on the V4V dataset with an impressive RMSE of 6.65 bpm, outperforming other methods. We observe that our model began converging from the very first epoch, a significant improvement over other models in terms of learning efficiency. Our approach is expected to be generally applicable to models that learn spectral domain information as well as to the applications of regression that require the representations of periodicity.

Efficient reduced order quadrature construction algorithms for fast gravitational wave inference

Efficient reduced order quadrature construction algorithms for fast gravitational wave inference

Fast Bayesian Inference of Phylogenies from Multiple Continuous Characters.

Time-scaled phylogenetic trees are an ultimate goal of evolutionary biology and a necessary ingredient in comparative studies. The accumulation of genomic data has resolved the tree of life to a great extent, yet timing evolutionary events remain challenging if not impossible without external information such as fossil ages and morphological characters. Methods for incorporating morphology in tree estimation have lagged behind their molecular counterparts, especially in the case of continuous characters. Despite recent advances, such tools are still direly needed as we approach the limits of what molecules can teach us. Here, we implement a suite of state-of-the-art methods for leveraging continuous morphology in phylogenetics, and by conducting extensive simulation studies we thoroughly validate and explore our methods' properties. While retaining model generality and scalability, we make it possible to estimate absolute and relative divergence times from multiple continuous characters while accounting for uncertainty. We compile and analyze one of the most data-type diverse data sets to date, comprised of contemporaneous and ancient molecular sequences, and discrete and continuous morphological characters from living and extinct Carnivora taxa. We conclude by synthesizing lessons about our method's behavior, and suggest future research venues.

RepECN: Making ConvNets Better Again for Efficient Image Super-Resolution

RepECN: Making ConvNets Better Again for Efficient Image Super-Resolution

JDT-NAS: Designing Efficient Multi-Object Tracking Architectures for Non-GPU Computers

Recent years have witnessed online multiple object tracking (MOT) revealing its enormous potential in numerous fields. However, existing MOT models are not suitable for deployment and application on non-GPU platforms (e.g. CPU-based computers), due to their inefficient inference. In this paper, we offer an automated approach for designing CPU-efficient multi-object tracking architectures, called JDT-NAS. Specifically, we first design several lightweight basic units as candidate operations to construct a heterogeneous search space for multi-object tracking. Then, based on the predefined space, we develop a four-stage hardware-aware search strategy to achieve the automated design of MOT models. Benefiting from our heterogeneous search space and special search strategy, our JDT-NAS not only allows flexible adjustment of the network scale (including depth, width and resolution dimensions), but also better meets the needs of efficient deployment. Finally, on publicly available standard datasets (e.g. MOT17), we apply JDT-NAS to search for multi-object trackers for fast inference on non-GPU computers. Comprehensive experiments show that these searched MOT architectures achieve superior speed and comparable accuracy compared to their state-of-the-art counterparts. This also indicates that our proposed JDT-NAS is a practical and feasible solution for efficient MOT architecture design.