Inference Phase Research Articles

Attention-based object pose estimation with feature fusion and geometry enhancement

Purpose The 6D pose estimation is a crucial branch of robot vision. However, the authors find that due to the failure to make full use of the complementarity of the appearance and geometry information of the object, the failure to deeply explore the contributions of the features from different regions to the pose estimation, and the failure to take advantage of the invariance of the geometric structure of keypoints, the performances of the most existing methods are not satisfactory. This paper aims to design a high-precision 6D pose estimation method based on above insights. Design/methodology/approach First, a multi-scale cross-attention-based feature fusion module (MCFF) is designed to aggregate the appearance and geometry information by exploring the correlations between appearance features and geometry features in the various regions. Second, the authors build a multi-query regional-attention-based feature differentiation module (MRFD) to learn the contribution of each region to each keypoint. Finally, a geometric enhancement mechanism (GEM) is designed to use structure information to predict keypoints and optimize both pose and keypoints in the inference phase. Findings Experiments on several benchmarks and real robot show that the proposed method performs better than existing methods. Ablation studies illustrate the effectiveness of each module of the authors’ method. Originality/value A high-precision 6D pose estimation method is proposed by studying the relationship between the appearance and geometry from different object parts and the geometric invariance of the keypoints, which is of great significance for various robot applications.

Supporting vision-language model few-shot inference with confounder-pruned knowledge prompt.

Vision-language models are pre-trained by aligning image-text pairs in a common space to deal with open-set visual concepts. Recent works adopt fixed or learnable prompts, i.e., classification weights are synthesized from natural language descriptions of task-relevant categories, to reduce the gap between tasks during the pre-training and inference phases. However, how and what prompts can improve inference performance remains unclear. In this paper, we explicitly clarify the importance of incorporating semantic information into prompts, while existing prompting methods generate prompts without sufficiently exploring the semantic information of textual labels. Manually constructing prompts with rich semantics requires domain expertise and is extremely time-consuming. To cope with this issue, we propose a knowledge-aware prompt learning method, namely Confounder-pruned Knowledge Prompt (CPKP), which retrieves an ontology knowledge graph by treating the textual label as a query to extract task-relevant semantic information. CPKP further introduces a double-tier confounder-pruning procedure to refine the derived semantic information. Adhering to the individual causal effect principle, the graph-tier confounders are gradually identified and phased out. The feature-tier confounders are eliminated by following the maximum entropy principle in information theory. Empirically, the evaluations demonstrate the effectiveness of CPKP in few-shot inference, e.g., with only two shots, CPKP outperforms the manual-prompt method by 4.64% and the learnable-prompt method by 1.09% on average.

Reservoir direct feedback alignment: deep learning by physical dynamics

The rapid advancement of deep learning has motivated various analog computing devices for energy-efficient non-von Neuman computing. While recent demonstrations have shown their excellent performance, particularly in the inference phase, computation of training using analog hardware is still challenging due to the complexity of training algorithms such as backpropagation. Here, we present an alternative training algorithm that combines two emerging concepts: reservoir computing (RC) and biologically inspired training. Instead of backpropagated errors, the proposed method computes the error projection using nonlinear dynamics (i.e., reservoir), which is highly suitable for physical implementation because it only requires a single passive dynamical system with a smaller number of nodes. Numerical simulation with Lyapunov analysis showed some interesting features of our proposed algorithm itself: the reservoir basically should be selected to satisfy the echo-state-property; but even chaotic dynamics can be used for the training when its time scale is below the Lyapunov time; and the performance is maximized near the edge of chaos, which is similar to standard RC framework. Furthermore, we experimentally demonstrated the training of feedforward neural networks by using an optoelectronic reservoir computer. Our approach provides an alternative solution for deep learning computation and its physical acceleration.

An Effective Scheme to Accelerate NeRF for Web Applications Using Hash-based Caching and Precomputed Features

In recent years, 3D reconstruction and rendering technologies have become increasingly important in various web-based applications within the field of web technology. In particular, with the emergence of technologies such as WebGL and WebGPU, which enable real-time 3D content rendering in web browsers, immersive experiences and interactions on the web have been significantly enhanced. These technologies are widely used in applications such as 3D visualization of virtual products or 3D exploration of building interiors on real estate websites. Through these advancements, users can experience 3D content directly in their browsers without the need to install additional software, greatly expanding the possibilities of the web. Amidst this trend, the neural radiance field (NeRF) has garnered attention as a cutting-edge technology that improves the accuracy of 3D reconstruction and rendering. NeRF is a technique widely used in computer vision and graphics for reconstructing 3D spaces from 2D images taken from multiple viewpoints. By predicting the color and density of each pixel, NeRF captures the complex 3D structure and optical properties of a scene, enabling highly accurate 3D reconstructions. However, NeRF’s primary limitation is the time-consuming nature of both the training and inference processes. Research efforts to address this issue have focused on two key areas: optimizing network architectures and training procedures to accelerate scene learning, and improving inference speed for faster rendering. While progress has been made in enhancing training speed, challenges remain in improving the inference process. To address these limitations, we propose a two-step approach to significantly improve NeRF’s performance. First, we optimize the training phase through a multi-resolution hash encoding technique, reducing the computational complexity and speeding up the learning process. Second, we accelerate the inference phase by caching the input data of the NeRF MLP, which allows for faster rendering without sacrificing quality. Our experimental results demonstrate that this approach reduces training time by 68.42% and increases inference speed by 98.18%.

Gallbladder cancer detection via ultrasound image analysis: An end‐to‐end hierarchical feature‐fused model

AbstractGallbladder cancer is a fatal disease, and its early diagnosis can significantly impact patient treatment. Ultrasound imaging is often the initial diagnostic test for gallbladder cancer, making the enhancement of cancer detection accuracy from these images crucial. Despite the promising results of artificial intelligence techniques in disease diagnosis, their black‐box nature hinders the reliability of their results and their practical application. Therefore, it is essential not to rely solely on a single model's output and to further investigate for more reliable outcomes. This study presents a step‐by‐step structural investigation of forming an end‐to‐end model, a conjunction of two convolutional neural network based methods, for detecting gallbladder conditions. The final model, leveraging feature fusions and hierarchical classification, achieved a high accuracy of 92.62% for detecting normal, benign, and malignant gallbladders. It also achieved a remarkable accuracy of 98.36% for classifying normal and non‐normal instances and 92.22% for classifying benign and malignant cases. Finally, comprehensive post‐processing investigations, including cross‐validation, temperature scaling, and uncertainty estimation, along with error analysis, are conducted to gain more insights into the model's output. Among these insights, the model demonstrated resilience of its results to active dropout and augmentation at the inference phase. Furthermore, when applied with test‐time data augmentation, uncertainty estimation methods have better distinguishability between the uncertainties of correctly and incorrectly classified instances, which provides additional information about the model's output. The source code of experiments conducted in this study is available at https://github.com/SaraDadjouy/GBCRet.

Feature analysis and ensemble-based fault detection techniques for nonlinear systems

AbstractMachine learning approaches play a crucial role in nonlinear system modeling across diverse domains, finding applications in system monitoring, anomaly/fault detection, control, and various other areas. With technological advancements, today such systems might include hundreds or thousands of sensors that generate large amounts of multivariate data streams. This inevitably results in increased model complexity. In response, feature selection techniques are widely employed as a means to reduce complexity, avoid the curse of high dimensionality, decrease training and inference times, and eliminate redundant features. This paper introduces a sensitivity-inspired feature analysis technique for regression tasks. Leveraging the energy distance on the model prediction errors, this approach performs both feature ranking and selection. Additionally, this paper introduces an ensemble-based unsupervised fault detection methodology that incorporates homogeneous units, specifically long short-term memory (LSTM) predictors and cumulative sum-based detectors. The proposed predictors utilize a variant of the teacher forcing (TF) algorithm during both the training and inference phases. Additionally, predictors are used to model the normal behavior of the system, whereas detectors are used to identify deviations from normality. The detector decisions are aggregated using a majority voting scheme. The validity of the proposed approach is illustrated on the two representative datasets, where numerous experiments are performed for feature selection and fault detection evaluation. Experimental assessment reveals promising results, even compared to well-established techniques. Nevertheless, the results also demonstrate the need to perform additional experiments with datasets originating from both simulators and real systems. Further possible refinements of the detection ensemble include the addition of heterogeneous units and other decision fusion techniques.

Optimizing model pruning for energy-efficient deep learning

Deep learning models have become increasingly complex, leading to high energy consumption during training and inference phases. This paper investigates the use of model pruning techniques to significantly reduce the size of deep learning models while preserving predictive accuracy. Our study focuses on optimizing the trade-off between model performance and energy consumption by systematically applying various pruning strategies—magnitude-based, structured, and random pruning—on popular neural network architectures. Using the CIFAR-10 dataset, we evaluate the models' energy consumption during training and inference phases. The results provide insights into achieving substantial energy savings with minimal loss of accuracy, contributing to the development of more sustainable and efficient AI systems.

Multi-LoRA continual learning based instruction tuning framework for universal information extraction

Universal information extraction (Universal IE) aims to develop one model capable of solving multiple IE target tasks. Previous works have enhanced extraction performance of target tasks through auxiliary tasks. However, there are still limitations in terms of learning strategies. From one aspect, joint learning-based universal IE approaches, which simply mix auxiliary tasks with target tasks, fail to enable the model to master basic knowledge from auxiliary tasks before learning target tasks. From another aspect, continual learning-based universal IE approaches, which sequentially update all the model parameters on auxiliary tasks and target tasks, tend to cause catastrophic forgetting. In this study, we design a multi-LoRA continual learning-based instruction fine-tuning framework for universal IE. Specifically, we design unique LoRA modules for learning auxiliary tasks and target tasks. We first freeze pre-trained weights and update additional parameters on auxiliary tasks through one LoRA module. Subsequently, we keep the weights frozen and further adjust parameters through another LoRA module to adapt the model to the target tasks. Finally, we merge the frozen weights with learned weights, thereby enabling the model to better leverage the acquired abilities during the inference phase. Therefore, our model masters basic extraction abilities before learning target tasks and does not forget this basic knowledge during the target learning process. Moreover, we regard extraction, classification, and recognition as basic abilities and further design auxiliary tasks based on these basic abilities. Experimental results on 37 datasets across 3 tasks show that our approach reaches state-of-the-art performance.

Empirical Evidence Regarding Few-Shot Learning for Scene Classification in Remote Sensing Images

Few-shot learning (FSL) is a learning paradigm which aims to address the issue of machine/deep learning techniques which traditionally need huge amounts of labelled data to work out. The remote sensing (RS) community has explored this paradigm with numerous published studies to date. Nevertheless, there is still a need for clear pieces of evidence on FSL-related issues in the RS context, such as which of the inference approaches is more suitable: inductive or transductive? Moreover, how does the number of epochs used during training, based on the meta-training (base) dataset, relate to the number of unseen classes during inference? This study aims to address these and other relevant questions in the context of FSL for scene classification in RS images. A comprehensive evaluation was conducted considering eight FSL approaches (three inductive and five transductive) and six scene classification databases. Some conclusions of this research are as follows: (1) transductive approaches are better than inductive ones. In particular, the transductive technique Transductive Information Maximisation (TIM) presented the best overall performance, where in 20 cases it got the first place; (2) a larger number of training epochs is more beneficial when there are more unseen classes during the inference phase. The most impressive gains occurred particularly considering the AID (6-way) and RESISC-45 (9-way) datasets. Notably, in the AID dataset, a remarkable 58.412% improvement was achieved in 1-shot tasks going from 10 to 200 epochs; (3) using five samples in the support set is statistically significantly better than using only one; and (4) a higher similarity between unseen classes (during inference) and some of the training classes does not lead to an improved performance. These findings can guide RS researchers and practitioners in selecting optimal solutions/strategies for developing their applications demanding few labelled samples.

Improving Scene Text Retrieval via Stylized Middle Modality

Scene text retrieval addresses the challenge of localizing and searching for all text instances within scene images based on a query text. This cross-modal task has significant applications in various domains, such as intelligent transportation systems and social media analysis. In practice, ensuring consistency of the same content between two modalities is crucial in improving retrieval accuracy. This article addresses the issue by introducing a stylized middle modality, which fuses the graphical query text with the style of the extracted text proposal. To this end, we propose a stylized middle modality learning (SM 2 L) framework. The proposed stylized middle modality enables the network to jointly enforce constraints on visual feature coherence and text semantic feature consistency in the optimization phase, thereby minimizing the modality gap in the retrieval space. This brings in two major advantages: (1) SM 2 L will pave the way to seamlessly benefit the scene text retrieval and (2) the proposed learning paradigm enables the machine to avoid adding redundant computing resources in the inference phase. Substantial experiments demonstrate that the proposed method outperforms the state-of-the-art retrieval performance considerably.

On the feasibility of an ensemble multi-fidelity neural network for fast data assimilation for subsurface flow in porous media

Uncertainty quantification (UQ) of the reservoir heterogeneity is essential to predict fluid flow behavior in subsurface formations accurately, and the task is often accomplished by integrating high-fidelity forward physics simulators with iterative data assimilation methods, and such workflows are usually computationally expensive due to the iterative nature and the prohibitive cost of physics simulations.In this work, we develop a new Ensemble Multi-Fidelity Neural Network (EMF-Net) to mitigate the efficiency bottleneck of UQ. EMF-Net directly infers the uncertain variables (e.g., permeability) via the sparse observation of the state variables (e.g., pressure). By leveraging the regression capability of deep neural networks, EMF-Net directly learns the nonlinear mapping from the innovation vector to the inferred update vector without the hypothesis of a linear mapping. At the training phase, high-fidelity data computed by a physics simulator (fh) and low-fidelity data computed by a forward proxy model (fl) are used to train the EMF-Net at two different stages, respectively. At the inference phase, we first adopt the 1-stage EMF-Net to infer the update vector for the initial ensembles with a global tuning and then efficiently update the innovation vector by fl. As an option, we further refine the inference, via a 2-stage EMF-Net, which captures the locality and enhances consistency with the observation data. We demonstrate that EMF-Net can reach equivalent inference accuracy compared to classic data assimilation algorithms like ES-MDA, in addition to decreasing the CPU time. Therefore, the accuracy and efficiency make it an attractive alternative for scalable real-time history-matching tasks in field-scale subsurface engineering applications.

STCO: Enhancing Training Efficiency via Structured Sparse Tensor Compilation Optimization

Network sparsification serves as an effective technique to accelerate Deep Neural Network (DNN) inference. However, existing sparsification techniques often rely on structured sparsity, which yields limited benefits. This is primarily due to the significant memory and computational overhead introduced by numerous sparse storage formats during address generation and gradient updates. Additionally, many of these solutions are tailored solely for the inference phase, neglecting the crucial training phase. In this article, we introduce STCO, a novel Sparse Tensor Compilation Optimization technique that significantly enhances training efficiency through structured sparse tensor compilation. Central to STCO is the Tensorization-aware Index Entity (TIE) format, which effectively represents structured sparse tensors by eliminating redundant indices and minimizing storage overhead. The TIE format plays a pivotal role in the Address-carry flow (AC flow) pass, which optimizes the data layout at the computational graph level. This pass leverages the TIE format to enhance the efficiency of tensor representations, enabling more compact and efficient sparse tensor storage. Meanwhile, a shape inference pass utilizes the AC flow to derive optimized tensor shapes, further refining the performance of sparse tensor operations. Moreover, the Address-Carry TIE Flow dynamically tracks nonzero addresses, extending the benefits of sparse optimization to both forward and backward propagation. This seamless integration into the training pipeline enables a smooth transition to sparse tensor compilation without significant modifications to existing codebases. To further boost training performance, we implement an operator-level AC flow optimization pass tailored for structured sparse tensors. This pass generates efficient addresses, ensuring minimal computational overhead during sparse tensor operations. The flexibility of STCO allows it to be efficiently integrated into various frameworks or compilers, providing a robust solution for enhancing training efficiency with structured sparse tensors. Experiments demonstrated that STCO achieved impressive speedups of 3.64×, 5.43×, 4.89×, and 3.91× when compared to state-of-the-art sparse formats on VGG16, ResNet-18, MobileNetV1, and MobileNetV2, respectively. These findings underscore the efficiency and superiority of our proposed approach in leveraging unstructured sparsity for DNN inference acceleration.

Multi-task deep learning for quantifying methane emissions from 2-D plume imagery with Low Signal-to-Noise Ratio

ABSTRACT Methane is the second most significant greenhouse gas after carbon dioxide. As global warming intensifies, the quantification of methane point sources is becoming increasingly crucial. However, retrieving high-quality methane signals from remote sensing data remains challenging due to various factors, including surface reflectance, atmospheric interference, sensor noise, wind speed, and sensor sensitivity. In real-world scenarios, methane remote sensing quantification often encounters unfavourable conditions that lead to low signal-to-noise ratio (SNR) signals, resulting in reduced quantification accuracy. To address these challenges, we introduce a novel multi-task learning-based approach. Specifically, we incorporate a denoising auxiliary task into the quantification network by introducing an additional denoising branch that recovers clean plume column concentration maps. The network is trained with supervision using both denoising and quantification losses, which enables it to acquire robust feature representations to noise and benefits the quantification of low SNR images. During the inference phase, the denoising branch is removed, resulting in an efficient and robust single quantification network with reduced inferring time, supporting onboard computation. We construct a low SNR database based on the AVIRIS-NG sensor and evaluate the generalization ability of our method. DQNet achieves the highest RMSE, MAPE, and R of 16.478 kg/h, 15.296%, and 95.167% respectively on the synthetic dataset. Across the entire range of SNR, DQNet exhibits a greater relative advantage as SNR decreases. However, our approach is not limited to AVIRIS-NG and can be easily extended to various multispectral and hyperspectral satellites.

UniKDD: A Unified Generative model for Knowledge-driven Dialogue

knowledge-driven dialogue (KDD) is to introduce an external knowledge base, generating an informative and fluent response. However, previous works employ different models to conduct the sub-tasks of KDD, ignoring the connection between sub-tasks and resulting in a difficulty of training and inference. To solve those issues above, we propose the UniKDD, a unified generative model for KDD, which models all sub-tasks into a generation task, enhancing the connection between tasks and facilitating the training and inference. Specifically, UniKDD simplifies the complex KDD tasks into three main sub-tasks, i.e., entity prediction, attribute prediction, and dialogue generation. These tasks are transformed into a text generation task and trained by an end-to-end way. In the inference phase, UniKDD first predicts a set of entities used for current turn dialogue according to the dialogue history. Then, for each predicted entity, UniKDD predicts the corresponding attributes by the dialogue history. Finally, UniKDD generates a high-quality and informative response using the dialogue history and predicted knowledge triplets. The experimental results show that our proposed UniKDD can perform KDD task well and outperform the baseline on the evaluation of knowledge selection and response generation. The code is available at https://github.com/qianandfei/UniKDD.git.

Efficient Biomedical Text Summarization With QuantizedLLaMA2: Enhancing Memory Usage and Inference on Low Powered Devices

ABSTRACTThe deployment of large language models (LLMs) on edge devices and non‐server environments presents significant challenges, primarily due to constraints in memory usage, computational power, and inference time. This article investigates the feasibility of running LLMs across such devices by focusing on optimising memory usage, employing quantization techniques, and reducing inference time. Specifically, we utilise LLaMA 2 for biomedical text summarization and implement low‐rank adaptation (LoRA) quantization to compress the model size to compress the model size and fine‐tune it using limited resources. Our study systematically evaluates memory consumption during both training and inference phases, demonstrating substantial reductions through efficient LoRA quantization. Our results indicate that with careful optimization, it is feasible to deploy sophisticated LLMs like LLaMA 2 on low powered devices, thereby broadening the scope of their application in resource‐constrained environments.

PPDNN-CRP: privacy-preserving deep neural network processing for credit risk prediction in cloud: a homomorphic encryption-based approach

This study proposes a Privacy-Preserving Deep Neural Network for Credit Risk Prediction (PPDNN-CRP) framework that leverages homomorphic encryption (HE) to ensure data privacy throughout the credit risk prediction process. The PPDNN-CRP framework employs the Paillier homomorphic encryption scheme to secure sensitive loan application data during both the training and inference phases. Implemented using TensorFlow for deep neural network operations and TenSEAL for HE, the framework uses the Kaggle loan dataset to evaluate its performance. The results show that PPDNN-CRP achieved an accuracy of 80.48%, demonstrating competitive performance compared to Privacy-Preserving Logistic Regression (PPLR) at 77.23% and a slight decrease from the non-private DNN-CRP model at 86.18%. The model exhibited strong metrics with a precision of 0.84, recall of 0.91, F1-score of 0.87, and an AUC of 0.74. Security analysis confirms that PPDNN-CRP effectively defends against various privacy attacks, including poisoning, evasion, membership inference, model inversion, and model extraction, through robust encryption techniques and privacy measures. This framework offers a promising approach for achieving high-quality credit risk prediction while maintaining data privacy and complying with legal and ethical standards.

Time-Reversion Fast-Sampling Score-Based Model for Limited-Angle CT Reconstruction.

The score-based generative model (SGM) has received significant attention in the field of medical imaging, particularly in the context of limited-angle computed tomography (LACT). Traditional SGM approaches achieved robust reconstruction performance by incorporating a substantial number of sampling steps during the inference phase. However, these established SGM-based methods require large computational cost to reconstruct one case. The main challenge lies in achieving high-quality images with rapid sampling while preserving sharp edges and small features. In this study, we propose an innovative rapid-sampling strategy for SGM, which we have aptly named the time-reversion fast-sampling (TIFA) score-based model for LACT reconstruction. The entire sampling procedure adheres steadfastly to the principles of robust optimization theory and is firmly grounded in a comprehensive mathematical model. TIFA's rapid-sampling mechanism comprises several essential components, including jump sampling, time-reversion with re-sampling, and compressed sampling. In the initial jump sampling stage, multiple sampling steps are bypassed to expedite the attainment of preliminary results. Subsequently, during the time-reversion process, the initial results undergo controlled corruption by introducing small-scale noise. The re-sampling process then diligently refines the initially corrupted results. Finally, compressed sampling fine-tunes the refinement outcomes by imposing regularization term. Quantitative and qualitative assessments conducted on numerical simulations, real physical phantom, and clinical cardiac datasets, unequivocally demonstrate that TIFA method (using 200 steps) outperforms other state-of-the-art methods (using 2000 steps) from available [0°, 90°] and [0°, 60°]. Furthermore, experimental results underscore that our TIFA method continues to reconstruct high-quality images even with 10 steps. Our code at https://github.com/tianzhijiaoziA/TIFADiffusion.

Reference-Based OCT Angiogram Super-Resolution With Learnable Texture Generation.

Optical coherence tomography angiography (OCTA) can visualize retinal microvasculature and is important to qualitatively and quantitatively identify potential biomarkers for different retinal diseases. However, the resolution of optical coherence tomography (OCT) angiograms inevitably decreases when increasing the field-of-view (FOV) given a fixed acquisition time. To address this issue, we propose a novel reference-based super-resolution (RefSR) framework to preserve the resolution of the OCT angiograms while increasing the scanning area. Specifically, textures from the normal RefSR pipeline are used to train a learnable texture generator (LTG), which is designed to generate textures according to the input. The key difference between the proposed method and traditional RefSR models is that the textures used during inference are generated by the LTG instead of being searched from a single reference (Ref) image. Since the LTG is optimized throughout the whole training process, the available texture space is significantly enlarged and no longer limited to a single Ref image, but extends to all textures contained in the training samples. Moreover, our proposed LTGNet does not require an Ref image at the inference phase, thereby becoming invulnerable to the selection of the Ref image. Both experimental and visual results show that LTGNet has competitive performance and robustness over state-of-the-art methods, indicating good reliability and promise in real-life deployment. The source code is available at https://github.com/RYY0722/LTGNet.

A Training-Free Latent Diffusion Style Transfer Method

Diffusion models have attracted considerable scholarly interest for their outstanding performance in generative tasks. However, current style transfer techniques based on diffusion models still rely on fine-tuning during the inference phase to optimize the generated results. This approach is not merely laborious and resource-demanding but also fails to fully harness the creative potential of expansive diffusion models. To overcome this limitation, this paper introduces an innovative solution that utilizes a pretrained diffusion model, thereby obviating the necessity for additional training steps. The scheme proposes a Feature Normalization Mapping Module with Cross-Attention Mechanism (INN-FMM) based on the dual-path diffusion model. This module employs soft attention to extract style features and integrate them with content features. Additionally, a parameter-free Similarity Attention Mechanism (SimAM) is employed within the image feature space to facilitate the transfer of style image textures and colors, while simultaneously minimizing the loss of structural content information. The fusion of these dual attention mechanisms enables us to achieve style transfer in texture and color without sacrificing content integrity. The experimental results indicate that our approach exceeds existing methods in several evaluation metrics.

A Novel Topology Adaptation Strategy for Dynamic Sparse Training in Deep Reinforcement Learning.

Deep reinforcement learning (DRL) has been widely adopted in various applications, yet it faces practical limitations due to high storage and computational demands. Dynamic sparse training (DST) has recently emerged as a prominent approach to reduce these demands during training and inference phases, but existing DST methods achieve high sparsity levels by sacrificing policy performance as they rely on the absolute magnitude of connections for pruning and randomly generating connections. Addressing this, our study presents a generic method that can be seamlessly integrated into existing DST methods in DRL to enhance their policy performance while preserving their sparsity levels. Specifically, we develop a novel method for calculating the importance of connections within the model. Subsequently, we dynamically adjust the sparse network topology by dropping existing connections and introducing new connections based on their respective importance values. Through validation on eight widely used simulation tasks, our method improves two state-of-the-art (SOTA) DST approaches by up to 70% in episode return and average return across all episodes under various sparsity levels.