Memory Savings Research Articles

Investigating the Limits of Familiarity-Based Navigation.

Insect-inspired navigation strategies have the potential to unlock robotic navigation in power-constrained scenarios, as they can function effectively with limited computational resources. One such strategy, familiarity-based navigation, has successfully navigated a robot along routes of up to 60 m using a single-layer neural network trained with an Infomax learning rule. Given the small size of the network that effectively encodes the route, here we investigate the limits of this method, challenging it to navigate longer routes, investigating the relationship between performance, view acquisition rate and dimension, network size, and robustness to noise. Our goal is both to determine the parameters at which this method operates effectively and to explore the profile with which it fails, both to inform theories of insect navigation and to improve robotic deployments. We show that effective memorization of familiar views is possible for longer routes than previously attempted, but that this length decreases for reduced input view dimensions. We also show that the ideal view acquisition rate must be increased with route length for consistent performance. We further demonstrate that computational and memory savings may be made with equivalent performance by reducing the network size-an important consideration for applicability to small, lower-power robots-and investigate the profile of memory failure, demonstrating increased confusion across the route as it extends in length. In this extension to previous work, we also investigate the form taken by the network weights as training extends and the areas of the image on which visual familiarity-based navigation most relies. Additionally, we investigate the robustness of familiarity-based navigation to view variation caused by noise.

M-adapter: Multi-level image-to-video adaptation for video action recognition

With the growing size of visual foundation models, training video models from scratch has become costly and challenging. Recent attempts focus on transferring frozen pre-trained Image Models (PIMs) to video fields by tuning inserted learnable parameters such as adapters and prompts. However, these methods require saving PIM activations for gradient calculations, leading to limited savings of GPU memory. In this paper, we propose a novel parallel branch that adapts the multi-level outputs of the frozen PIM for action recognition. It avoids passing gradients through the PIMs, thus naturally owning much lower GPU memory footprints. The proposed adaptation branch consists of hierarchically combined multi-level output adapters (M-adapters), comprising a fusion module and a temporal module. This design digests the existing discrepancies between the pre-training task and the target task with lower training costs. We show that when using larger models or on scenarios with higher demands for temporal modelling, the proposed method performs better than those with the full-parameter tuning manner. Finally, despite only tuning fewer parameters, our method achieves superior or comparable performance against current state-of-the-art methods.

Multi-Stream Scheduling of Inference Pipelines on Edge Devices - a DRL Approach

Low-power edge devices equipped with Graphics Processing Units (GPUs) are a popular target platform for real-time scheduling of inference pipelines. Such application-architecture combinations are popular in Advanced Driver-assistance Systems for aiding in the real-time decision-making of automotive controllers. However, the real-time throughput sustainable by such inference pipelines is limited by resource constraints of the target edge devices. Modern GPUs, both in edge devices and workstation variants, support the facility of concurrent execution of computation kernels and data transfers using the primitive of streams , also allowing for the assignment of priority to these streams. This opens up the possibility of executing computation layers of inference pipelines within a multi-priority, multi-stream environment on the GPU. However, manually co-scheduling such applications while satisfying their throughput requirement and platform memory budget may require an unmanageable number of profiling runs. In this work, we propose a Deep Reinforcement Learning (DRL)-based method for deciding the start time of various operations in each pipeline layer while optimizing the latency of execution of inference pipelines as well as memory consumption. Experimental results demonstrate the promising efficacy of the proposed DRL approach in comparison with the baseline methods, particularly in terms of real-time performance enhancements, schedulability ratio, and memory savings. We have additionally assessed the effectiveness of the proposed DRL approach using a real-time traffic simulation tool IPG CarMaker.

Linear Sources and Anisotropic Scattering in the Random Ray Method

An enhanced implementation of the random ray method (TRRM), a stochastic adaptation of the method of characteristics, is presented. This implementation generalizes from a traditional flat source (FS) approximation to a linear source (LS) approximation, and from the standard isotropic scattering approximation to an anisotropic approximation, up to P3. On the two-dimensional C5G7 benchmark, LS alone enables a 1.4× and 1.7× enhancement in run time for parallel and serial executions, respectively, without compromising accuracy. Further testing on the three-dimensional C5G7 Rodded B benchmark demonstrates that LS enables significant axial coarsening and reduction in ray populations. This results in a 5.3× speedup in run time while still offering comparable accuracy. On the Babcock and Wilcox 1484 Core II benchmark, incorporating anisotropic scattering in TRRM up to P3 with a FS decreased keff errors (~110 pcm) and maximum and average pin power errors. LS with anisotropic scattering (LSPN), showed similarly improved keff errors while benefiting from a coarser mesh. The linear isotropic flat anisotropic (LIFA) method, relative to LSPN, offers further run time and memory savings of 4.1× and 1.6×, respectively, for P3, while maintaining comparable accuracy to anisotropic FS on more refined meshes.

Nested Neural Networks: A Novel Approach to Flexible and Deep Learning Architectures

In this paper, we introduce a novel neural network architecture termed Nested Neural Networks (NNN), which incorporates nested layers within a more complex overarching structure. The architecture leverages the concept of internal and external conditions, akin to linguistic structures like Future Perfect and Future Perfect Progressive in English grammar, to create a flexible and deep model. This architecture allows the network to handle complex data patterns with enhanced computational efficiency and memory savings. We demonstrate the effectiveness of NNNs through experiments on benchmark datasets, showcasing their ability to outperform traditional models in terms of accuracy, training efficiency, and adaptability.

Hard‐state Protocol Independent Multicast—Source‐Specific Multicast (HPIM‐SSM)

AbstractSource‐specific multicast is a key technology for multicast services such as IPTV broadcasting, which relies on IGMPv3/MLDv2 for source‐group membership signalling and multicast routing protocols such as PIM‐SSM for building and maintaining receiver‐initiated source‐based distribution trees across the network. The authors propose the Hard‐state Protocol Independent Multicast—Source‐Specific Multicast (HPIM‐SSM), a novel multicast routing protocol that keeps the design principles of PIM‐SSM but overcomes its limitations, such as slow convergence and the possibility of creating suboptimal trees. The state machines of HPIM‐SSM were designed to react promptly to all network events susceptible to reconfiguring the multicast trees, avoiding the need for soft‐state maintenance through the periodic transmission of control messages. Moreover, the authors eliminated the need for designated routers, which led to suboptimal trees, and introduced a control‐driven assert protocol that operates per source, allowing for considerable memory savings. Finally, the protocol enables the coexistence of multiple unicast routing protocols. HPIM‐SSM was implemented in Python, and its correctness was extensively validated through model‐checking techniques. Furthermore, a comparison between HPIM‐SSM and PIM‐SSM was conducted, encompassing both theoretical analysis and experimental evaluation of convergence times. The results demonstrate clearly that HPIM‐SSM outperforms PIM‐SSM, exhibiting significantly faster convergence times and completely avoiding suboptimal trees.

DELTA: Memory-Efficient Training via Dynamic Fine-Grained Recomputation and Swapping

To accommodate the increasingly large-scale models within limited-capacity GPU memory, various coarse-grained techniques, such as recomputation and swapping, have been proposed to optimize memory usage. However, these methods have encountered limitations, either in terms of inefficient memory reduction or diminished training performance. In response to this, our paper introduces DELTA, an innovative approach for memory-efficient large-scale model training that combines fine-grained memory optimization and prefetching technology to reduce memory usage while maintaining high training throughput concurrently. Initially, we formulate the problem of memory-throughput joint optimization as an easy-solving 0/1 Knapsack problem. Leveraging this formalization, we use an improving polynomial complexity heuristic algorithm to address the problem effectively. Furthermore, we introduce a novel bidirectional prefetching technology into dynamic memory management, which significantly accelerates the model training when compared to relying solely on recomputation or swapping. Finally, DELTA offers users an automated training execution library, eliminating the need for manual configuration or specialized expertise. Experimental results demonstrate the effectiveness of DELTA in reducing GPU memory consumption. Compared to state-of-the-art methods, DELTA achieves substantial memory savings ranging from 40% to 72%, while maintaining comparable convergence performance for various models, including ResNet-50, ResNet-101, and BERT-Large. Notably, DELTA enables the training of GPT2-Large and GPT2-XL with batch sizes increased by 5.5 × and 6 ×, respectively, showcasing its versatility and practicality in enabling large-scale model training on GPU hardware.

Towards dynamical low-rank approximation for neutrino kinetic equations. Part I: Analysis of an idealized relaxation model

Dynamical low-rank approximation (DLRA) is an emerging tool for reducing computational costs and provides memory savings when solving high-dimensional problems. In this work, we propose and analyze a semi-implicit dynamical low-rank discontinuous Galerkin (DLR-DG) method for the space homogeneous kinetic equation with a relaxation operator, modeling the emission and absorption of particles by a background medium. Both DLRA and the discontinuous Galerkin (DG) scheme can be formulated as Galerkin equations. To ensure their consistency, a weighted DLRA is introduced so that the resulting DLR-DG solution is a solution to the fully discrete DG scheme in a subspace of the standard DG solution space. Similar to the standard DG method, we show that the proposed DLR-DG method is well-posed. We also identify conditions such that the DLR-DG solution converges to the equilibrium. Numerical results are presented to demonstrate the theoretical findings.

A CNN pruning approach using constrained binary particle swarm optimization with a reduced search space for image classification

Deep convolutional neural networks (CNNs) have exhibited exceptional performance in a range of computer vision tasks. However, these deep CNNs typically demand significant computational resources, which not only hinders their practical deployment but also contributes to a considerable carbon footprint. To tackle this issue, several filter pruning methods based on evolutionary algorithms have been proposed to provide significant memory and energy savings during CNN inference. However, due to the curse of high dimensionality in the structure of deep CNNs, the search space expands dramatically, presenting significant challenges for these methods. This paper proposes a novel algorithm called BPSO-FPruner for CNN filter pruning. BPSO-FPruner utilizes a constrained binary particle swarm optimization algorithm for filter pruning, incorporating a new initialization strategy based on filter weighting information and a reduced search space strategy. Extensive validation using VGG, ResNet, DenseNet, and MobileNetv2 architectures on the CIFAR-10, CIFAR-100, and Tiny ImageNet datasets demonstrates the effectiveness of BPSO-FPruner in reducing model computational costs and carbon footprint emissions while maintaining or improving performance.

PQSDC: a parallel lossless compressor for quality scores data via sequences partition and run-length prediction mapping.

The quality scores data (QSD) account for 70% in compressed FastQ files obtained from the short and long reads sequencing technologies. Designing effective compressors for QSD that counterbalance compression ratio, time cost, and memory consumption is essential in scenarios such as large-scale genomics data sharing and long-term data backup. This study presents a novel parallel lossless QSD-dedicated compression algorithm named PQSDC, which fulfills the above requirements well. PQSDC is based on two core components: a parallel sequences-partition model designed to reduce peak memory consumption and time cost during compression and decompression processes, as well as a parallel four-level run-length prediction mapping model to enhance compression ratio. Besides, the PQSDC algorithm is also designed to be highly concurrent using multicore CPU clusters. We evaluate PQSDC and four state-of-the-art compression algorithms on 27 real-world datasets, including 61.857 billion QSD characters and 632.908 million QSD sequences. (1) For short reads, compared to baselines, the maximum improvement of PQSDC reaches 7.06% in average compression ratio, and 8.01% in weighted average compression ratio. During compression and decompression, the maximum total time savings of PQSDC are 79.96% and 84.56%, respectively; the maximum average memory savings are 68.34% and 77.63%, respectively. (2) For long reads, the maximum improvement of PQSDC reaches 12.51% and 13.42% in average and weighted average compression ratio, respectively. The maximum total time savings during compression and decompression are 53.51% and 72.53%, respectively; the maximum average memory savings are 19.44% and 17.42%, respectively. (3) Furthermore, PQSDC ranks second in compression robustness among the tested algorithms, indicating that it is less affected by the probability distribution of the QSD collections. Overall, our work provides a promising solution for QSD parallel compression, which balances storage cost, time consumption, and memory occupation primely. The proposed PQSDC compressor can be downloaded from https://github.com/fahaihi/PQSDC.

GeoTaichi: A Taichi-powered high-performance numerical simulator for multiscale geophysical problems

This study introduces GeoTaichi, an open-source high-performance numerical simulator designed for addressing multiscale geophysical problems. By leveraging the power of the Taichi parallel language, GeoTaichi maximizes the utilization of modern computer resources on multicore CPU and GPU architectures. It offers robust and reliable modules for the discrete element method (DEM), material point method (MPM), and coupled material point-discrete element method (MPDEM). These modules enable efficient solving of large-scale problems while being implemented in pure Python. The design philosophy of GeoTaichi focuses on creating a framework that is readable, extensible, and user-friendly. This paper highlights the coupling procedure of MPDEM, the code structures, and the most important features of GeoTaichi. Rigorous benchmark tests have been conducted to verify the validity and robustness of GeoTaichi. Additionally, the performance of GeoTaichi is compared with similar software tools in the field, underscoring a notable improvement in both computational efficiency and memory savings when compared to existing alternatives. Program summaryProgram title: GeoTaichiCPC Library link to program files:https://doi.org/10.17632/858bmcf7j6.1Developer's repository link:https://github.com/Yihao-Shi/GeoTaichiLicensing provisions: GNU General Public License v3.0Programming language: PythonNature of problem: The simulations of large-deformation geophysical flows and their interaction with structures play a crucial role in the field of geophysics. To address the complexities of these nonlinear problems, the discrete element method (DEM), material point method (MPM), and their coupling (MPDEM) have proven to be highly suitable numerical schemes. However, these schemes impose substantial computational demands, necessitating the development of an efficient framework that can harness modern computer resources on multicore CPU and GPU architectures.Solution method: The open-source code GeoTaichi implements the DEM, MPM, and coupled MPDEM, encompassing a range of constitutive models and contact laws for different geologic materials. The clump particle model is also introduced in DEM to solve granular mechanics involving complex-shaped particles. One significant advantage of GeoTaichi is its utilization of the Taichi parallel language, which is designed to be user-friendly and easily extensible for customized applications.

Lighter and faster simulations on domains with symmetries

A strategy to improve the performance and reduce the memory footprint of simulations on meshes with spatial reflection symmetries is presented in this work. By using an appropriate mirrored ordering of the unknowns, discrete partial differential operators are represented by matrices with a regular block structure that allows replacing the standard sparse matrix–vector product with a specialised version of the sparse matrix-matrix product, which has a significantly higher arithmetic intensity. Consequently, matrix multiplications are accelerated, whereas their memory footprint is reduced, making massive simulations more affordable. As an example of practical application, we consider the numerical simulation of turbulent incompressible flows using a low-dissipation discretisation on unstructured collocated grids. All the required matrices are classified into three sparsity patterns that correspond to the discrete Laplacian, gradient, and divergence operators. Therefore, the above-mentioned benefits of exploiting spatial reflection symmetries are tested for these three matrices on both CPU and GPU, showing up to 5.0x speed-ups and 8.0x memory savings. Finally, a roofline performance analysis of the symmetry-aware sparse matrix–vector product is presented.

Investigation of Modified EMST Micromixing Model Performance on Lagrangian PDF Transported in Lifted Hydrogen/Air Fames

This article presents a numerical simulation of high hydrogen/air flames using the Lagrangian transported PDF method. This method enables the calculation of fluid composition changes resulting from convection and reaction without the need for modelling, while requiring modelling for molecular mixing. Consequently, the accuracy of calculations in this L-PDF method heavily relies on an accurate representation of the mixture model term. The Euclidean model, which provides a better description of physical mixing processes, is well-suited for modelling the molecular mixing term EMST. Additionally, the accuracy of this model depends on the value of the mixing constant, representing the ratio between the mechanical time scale and the scalar time scale. Two algebraic models for the mixing constant have been implemented in the computational code, employing a well-defined function to calculate this ratio for each cell. These models contribute to memory and CPU time savings. To account for turbulence and its interaction with physical phenomena, the RSM model is employed due to its ability to identify different areas of turbulent stresses. Hence, the primary objective of this study is to evaluate the capabilities of these algebraic models in predicting scalar fields within such flames. Overall, the predictions align well with experimental data, affirming the validity of these models.

ERA*: Enhanced Relaxed A* algorithm for solving the shortest path problem in regular grid maps

This paper introduces a novel algorithm for solving the point-to-point shortest path problem in a static regular 8-neighbor connectivity (G8) grid. This algorithm can be seen as a generalization of Hadlock algorithm to G8 grids, and is shown to be theoretically equivalent to the relaxed A⁎ (RA⁎) algorithm in terms of the provided solution's path length, but with substantial time and memory savings, due to a completely different computation strategy, based on defining a set of lookup matrices. Through an experimental study on grid maps of various types and sizes (1290 runs on 43 maps), it is proven to be 2.25 times faster than RA⁎ and 17 times faster than the original A⁎, in average. Moreover, it is more memory-efficient, since it does not need to store a G score matrix.

Retrieval and savings of contextual fear memories across an extended retention interval in juvenile and adult male and female rats.

Adult rodents exhibit an exceptional ability to retrieve context fear memories across lengthy retention intervals. In contrast, these memories established in younger rodents are susceptible to significant forgetting. The present study aimed to examine the persistence of contextual fear memories established in juvenile and adult Long-Evans male and female rats. Testing 1-day after conditioning, adult males exhibited evidence for greater conditioning than juvenile males, while in females, conditioning did not differ between juvenile and adult rats. In adults, males displayed greater conditioning than females, while in juveniles, males and females reached similar conditioning levels. At the 60-day retention interval, adult sex differences were maintained; however, juvenile rats failed to retrieve this remote contextual fear memory. Next, we examined whether a savings test procedure could recover these remotely established juvenile memories. Following a 60-day retention test, the now adult rats were presented with an additional context-shock pairing to assess the level of savings. While this procedure produced greater conditioning in males than females, the relative savings of this early life memory were similar in males and females. The results of these experiments indicate that adult sex differences in contextual fear memory are maintained across an extended retention interval, while in juveniles, there were no significant sex differences. A novel finding in the present study was that both male and female rats failed to retrieve an initial juvenile memory following an extended retention interval. However, these memories were recovered with a single reminder of the original juvenile experience. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Multiplier-Free Convolution Neural Network Hardware Accelerator for Real-Time Bearing Condition Detection of CNC Machinery

In various industrial domains, machinery plays a pivotal role, with bearing failure standing out as the most prevalent cause of malfunction, contributing to approximately 41% to 44% of all operational breakdowns. To address this issue, this research employs a lightweight neural network, boasting a mere 8.69 K parameters, tailored for implementation on an FPGA (field-programmable gate array). By integrating an incremental network quantization approach and fixed-point operation techniques, substantial memory savings amounting to 63.49% are realized compared to conventional 32-bit floating-point operations. Moreover, when executed on an FPGA, this work facilitates real-time bearing condition detection at an impressive rate of 48,000 samples per second while operating on a minimal power budget of just 342 mW. Remarkably, this system achieves an accuracy level of 95.12%, showcasing its effectiveness in predictive maintenance and the prevention of costly machinery failures.

Implementation Of The Swin Transformer and Its Application In Image Classification

There are big differences between the field of view of the calculator and the field of natural languages, for example, in the field of vision, the difference is in the size of the object as well as in the accuracy of the pixels in the image, and this contradicts the words in the text, and this makes the adaptation of the transformers to see somewhat difficult.Very recently a vision transformer named Swin Transformer was introduced by the Microsoft research team in Asia to achieve state-of-the-art results for machine translation. The computational complexity is linear and proportional to the size of the input image, because the processing of subjective attention is within each local window separately, and thus results in processor maps that are hierarchical and in deeper layers, and thus serve as the backbone of the calculator's vision in image classification and dense recognition applications. This work focuses on applying the Swin transformer to a demonstrated mathematical example with step-by-step analysis. Additionally, extensive experimental results were carried out on several standardized databases from CIFAR-10, CIFAR-100, and MNIST. Their results showed that the Swin Transformer can achieve flexible memory savings. Test accuracy for CIFAR-10 gave a 71.54% score, while for the CIFAR-100 dataset the accuracy was 46.1%. Similarly, when the Swin transformer was applied to the MNIST dataset, the accuracy increased in comparison with other vision transformer results.

Enabling Binary Neural Network Training on the Edge

The ever-growing computational demands of increasingly complex machine learning models frequently necessitate the use of powerful cloud-based infrastructure for their training. Binary neural networks are known to be promising candidates for on-device inference due to their extreme compute and memory savings over higher-precision alternatives. However, their existing training methods require the concurrent storage of high-precision activations for all layers, generally making learning on memory-constrained devices infeasible. In this article, we demonstrate that the backward propagation operations needed for binary neural network training are strongly robust to quantization, thereby making on-the-edge learning with modern models a practical proposition. We introduce a low-cost binary neural network training strategy exhibiting sizable memory footprint reductions while inducing little to no accuracy loss vs Courbariaux & Bengio’s standard approach. These decreases are primarily enabled through the retention of activations exclusively in binary format. Against the latter algorithm, our drop-in replacement sees memory requirement reductions of 3–5×, while reaching similar test accuracy (± 2 pp) in comparable time, across a range of small-scale models trained to classify popular datasets. We also demonstrate from-scratch ImageNet training of binarized ResNet-18, achieving a 3.78× memory reduction. Our work is open-source, and includes the Raspberry Pi-targeted prototype we used to verify our modeled memory decreases and capture the associated energy drops. Such savings will allow for unnecessary cloud offloading to be avoided, reducing latency, increasing energy efficiency, and safeguarding end-user privacy.

Accuracy Assessment of Atomistic Neural Network Potentials: The Impact of Cutoff Radius and Message Passing.

Atomistic neural network potentials have achieved great success in accelerating atomistic simulations in complicated systems in recent years. They are typically based on the atomic decomposition of total properties, truncating the interatomic correlations to a local environment within a given cutoff radius. A more recently developed message passing (MP) neural network framework can, in principle, incorporate nonlocal effects through iteratively correlating some atoms outside the cutoff sphere with atoms inside, a process referred to as MP. However, how the model accuracy depends on the cutoff radius and the MP process has rarely been discussed. In this work, we investigate this dependence using a recursively embedded atom neural network method that possesses both local and MP features, in two representative systems: liquid H2O and solid Al2O3. We focus on how these settings influence predictions for structural and vibrational properties, namely, radial distribution functions (RDFs) and vibrational density of states (VDOSs). We find that while MP lowers test errors of energy and forces in general, it may not improve the prediction for RDFs and/or VDOSs if direct interatomic correlations in the local environment are insufficiently described. A cutoff radius exceeding the first neighbor shell is necessary, beyond which involving MP quickly enhances the model accuracy until convergence. This is a potentially more efficient way to increase the model accuracy than directly increasing the cutoff radius, especially with more memory savings in the GPU implementation. Our findings also suggest that using the mean test error as the measure of the model accuracy alone is inadequate.

Multi-level Power Series Solution for Large Surface and Volume Electric Field Integral Equation

In this paper, we propose a new multi-level power series solution method for solving a large surface and volume electric field integral equation-based H-Matrix. The proposed solution method converges in a fixed number of iterations and is solved at each level of the H-Matrix computation. The solution method avoids the computation of a full matrix, as it can be solved independently at each level, starting from the leaf level. Solution at each level can be used as the final solution, thus saving the matrix computation time for full H-Matrix. The paper shows that the leaf level matrix computation and solution with power series gives as accurate results as the full H-Matrix iterative solver method. The method results in considerable savings time and memory savings compared to the H-Matrix iterative solver. Further, the proposed method retains the O(NlogN) solution complexity.