Memory Efficiency Research Articles

Memory-Efficient Prompt Tuning for Incremental Histopathology Classification

Recent studies have made remarkable progress in histopathology classification. Based on current successes, contemporary works proposed to further upgrade the model towards a more generalizable and robust direction through incrementally learning from the sequentially delivered domains. Unlike previous parameter isolation based approaches that usually demand massive computation resources during model updating, we present a memory-efficient prompt tuning framework to cultivate model generalization potential in economical memory cost. For each incoming domain, we reuse the existing parameters of the initial classification model and attach lightweight trainable prompts into it for customized tuning. Considering the domain heterogeneity, we perform decoupled prompt tuning, where we adopt a domain-specific prompt for each domain to independently investigate its distinctive characteristics, and one domain-invariant prompt shared across all domains to continually explore the common content embedding throughout time. All domain-specific prompts will be appended to the prompt bank and isolated from further changes to prevent forgetting the distinctive features of early-seen domains. While the domain-invariant prompt will be passed on and iteratively evolve by style-augmented prompt refining to improve model generalization capability over time. In specific, we construct a graph with existing prompts and build a style-augmented graph attention network to guide the domain-invariant prompt exploring the overlapped latent embedding among all delivered domains for more domain-generic representations. We have extensively evaluated our framework with two histopathology tasks, i.e., breast cancer metastasis classification and epithelium-stroma tissue classification, where our approach yielded superior performance and memory efficiency over the competing methods.

What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation

Quantization has emerged as a promising technique for improving the memory and computational efficiency of large language models (LLMs). Though the trade-off between performance and efficiency is well-known, there is still much to be learned about the relationship between quantization and LLM performance. To shed light on this relationship, we propose a new perspective on quantization, viewing it as perturbations added to the weights and activations of LLMs. We call this approach ``the lens of perturbation". Using this lens, we conduct experiments with various artificial perturbations to explore their impact on LLM performance. Our findings reveal several connections between the properties of perturbations and LLM performance, providing insights into the failure cases of uniform quantization and suggesting potential solutions to improve the robustness of LLM quantization. To demonstrate the significance of our findings, we implement a simple non-uniform quantization approach based on our insights. Our experiments show that this approach achieves minimal performance degradation on both 4-bit weight quantization and 8-bit quantization for weights and activations. These results validate the correctness of our approach and highlight its potential to improve the efficiency of LLMs without sacrificing performance.

Box2Poly: Memory-Efficient Polygon Prediction of Arbitrarily Shaped and Rotated Text

Recently, Transformer-based text detection techniques have sought to predict polygons by encoding the coordinates of individual boundary vertices using distinct query features. However, this approach incurs a significant memory overhead and struggles to effectively capture the intricate relationships between vertices belonging to the same instance. Consequently, irregular text layouts often lead to the prediction of outlined vertices, diminishing the quality of results. To address these challenges, we present an innovative approach rooted in Sparse R-CNN: a cascade decoding pipeline for polygon prediction. Our method ensures precision by iteratively refining polygon predictions, considering both the scale and location of preceding results. Leveraging this stabilized regression pipeline, even employing just a single feature vector to guide polygon instance regression yields promising detection results. Simultaneously, the leverage of instance-level feature proposal substantially enhances memory efficiency ( > 50% less vs. the SOTA method DPText-DETR) and reduces inference speed (> 40% less vs. DPText-DETR) with comparable performance on benchmarks. The code is available at https://github.com/Albertchen98/Box2Poly.git.

Embedded Feature Selection on Graph-Based Multi-View Clustering

Recently, anchor graph-based multi-view clustering has been proven to be highly efficient for large-scale data processing. However, most existing anchor graph-based clustering methods necessitate post-processing to obtain clustering labels and are unable to effectively utilize the information within anchor graphs. To solve these problems, we propose an Embedded Feature Selection on Graph-Based Multi-View Clustering (EFSGMC) approach to improve the clustering performance. Our method decomposes anchor graphs, taking advantage of memory efficiency, to obtain clustering labels in a single step without the need for post-processing. Furthermore, we introduce the l2,p-norm for graph-based feature selection, which selects the most relevant data for efficient graph factorization. Lastly, we employ the tensor Schatten p-norm as a tensor rank approximation function to capture the complementary information between different views, ensuring similarity between cluster assignment matrices. Experimental results on five real-world datasets demonstrate that our proposed method outperforms state-of-the-art approaches.

Approximate and Memorize (A&M) : Settling opposing views in replay-based continuous unsupervised domain adaptation

Continuous Unsupervised Domain Adaptation (CUDA) can alleviate deep learning models’ performance degradation on out-of-distribution data. However, low stability, the erosion of past knowledge while adapting to new domains, remains a major challenge. Traditional proposed solutions such as (1) Approximation, or (2) Memorization fail to harmonize system’s computational load, memory resources and stability. In response, we introduce Approximate and Memorize (A&M). Compared to traditional approximation methods, A&M utilizes factorized generative models to mitigate modal collapse, offering improved computational efficiency and training stability. Moreover, compared to traditional memorization approaches, A&M boosts memory efficiency by learning compressed representations of past information rather than storing it in the raw format. A&M shows 50% less forgetting for problems with constrained memory and an extended number of domains, making A&M ideal for real-world systems due to its scalable and parallel design.

Local Differentially Private Heavy Hitter Detection in Data Streams with Bounded Memory

Top-k frequent items detection is a fundamental task in data stream mining. Many promising solutions are proposed to improve memory efficiency while still maintaining high accuracy for detecting the Top-k items. Despite the memory efficiency concern, the users could suffer from privacy loss if participating in the task without proper protection, since their contributed local data streams may continually leak sensitive individual information. However, most existing works solely focus on addressing either the memory-efficiency problem or the privacy concerns but seldom jointly, which cannot achieve a satisfactory tradeoff between memory efficiency, privacy protection, and detection accuracy. In this paper, we present a novel framework HG-LDP to achieve accurate Top-k item detection at bounded memory expense, while providing rigorous local differential privacy (LDP) protection. Specifically, we identify two key challenges naturally arising in the task, which reveal that directly applying existing LDP techniques will lead to an inferior "accuracy-privacy-memory efficiency" tradeoff. Therefore, we instantiate three advanced schemes under the framework by designing novel LDP randomization methods, which address the hurdles caused by the large size of the item domain and by the limited space of the memory. We conduct comprehensive experiments on both synthetic and real-world datasets to show that the proposed advanced schemes achieve a superior "accuracy-privacy-memory efficiency" tradeoff, saving 2300× memory over baseline methods when the item domain size is 41,270. Our code is anonymously open-sourced via the link.

CAFE: Towards Compact, Adaptive, and Fast Embedding for Large-scale Recommendation Models

Recently, the growing memory demands of embedding tables in Deep Learning Recommendation Models (DLRMs) pose great challenges for model training and deployment. Existing embedding compression solutions cannot simultaneously meet three key design requirements: memory efficiency, low latency, and adaptability to dynamic data distribution. This paper presents CAFE, a Compact, Adaptive, and Fast Embedding compression framework that addresses the above requirements. The design philosophy of CAFE is to dynamically allocate more memory resources to important features (called hot features), and allocate less memory to unimportant ones. In CAFE, we propose a fast and lightweight sketch data structure, named HotSketch, to capture feature importance and report hot features in real time. For each reported hot feature, we assign it a unique embedding. For the non-hot features, we allow multiple features to share one embedding by using hash embedding technique. Guided by our design philosophy, we further propose a multi-level hash embedding framework to optimize the embedding tables of non-hot features. We theoretically analyze the accuracy of HotSketch, and analyze the model convergence against deviation. Extensive experiments show that CAFE significantly outperforms existing embedding compression methods, yielding 3.92% and 3.68% superior testing AUC on Criteo Kaggle dataset and CriteoTB dataset at a compression ratio of 10000x. The source codes of CAFE are available at GitHub.

Scalable Learning for Spatiotemporal Mean Field Games Using Physics-Informed Neural Operator

This paper proposes a scalable learning framework to solve a system of coupled forward–backward partial differential equations (PDEs) arising from mean field games (MFGs). The MFG system incorporates a forward PDE to model the propagation of population dynamics and a backward PDE for a representative agent’s optimal control. Existing work mainly focus on solving the mean field game equilibrium (MFE) of the MFG system when given fixed boundary conditions, including the initial population state and terminal cost. To obtain MFE efficiently, particularly when the initial population density and terminal cost vary, we utilize a physics-informed neural operator (PINO) to tackle the forward–backward PDEs. A learning algorithm is devised and its performance is evaluated on one application domain, which is the autonomous driving velocity control. Numerical experiments show that our method can obtain the MFE accurately when given different initial distributions of vehicles. The PINO exhibits both memory efficiency and generalization capabilities compared to physics-informed neural networks (PINNs).

Exploring the Impact of Cognitive Conflict, Meaning, and Motivation on Memory

Cognition and memory are two related but distinct psychological processes. Cognition refers to the process by which people acquire, process, and form memories from information, while memory is the process of storing and retrieving information. It should be noted that memory is not a static process, but a dynamic and complex one that can be influenced by various factors. Motivation, attention, and emotional state all play important roles in shaping people’s memory processes. This study aims to explore the impact of these factors on cognition and memory, in order to improve people's understanding of the cognitive process and provide a theoretical basis for improving learning and memory efficiency. By applying the insights gained from the research, people can develop more effective education and training plans and enhance their learning abilities in daily life.

A Memory-Efficient Neural Ordinary Differential Equation Framework Based on High-Level Adjoint Differentiation

Neural ordinary differential equations (neural ODEs) have emerged as a novel network architecture that bridges dynamical systems and deep learning. However, the gradient obtained with the continuous adjoint method in the vanilla neural ODE is not reverse-accurate. Other approaches suffer either from an excessive memory requirement due to deep computational graphs or from limited choices for the time integration scheme, hampering their application to large-scale complex dynamical systems. To achieve accurate gradients without compromising memory efficiency and flexibility, we present a new neural ODE framework, PNODE, based on high-level discrete adjoint algorithmic differentiation. By leveraging discrete adjoint time integrators and advanced checkpointing strategies tailored for these integrators, PNODE can provide a balance between memory and computational costs, while computing the gradients consistently and accurately. We provide an open-source implementation based on PyTorch and PETSc, one of the most commonly used portable, scalable scientific computing libraries. We demonstrate the performance through extensive numerical experiments on image classification and continuous normalizing flow problems. We show that PNODE achieves the highest memory efficiency when compared with other reverse-accurate methods. On the image classification problems, PNODE is up to two times faster than the vanilla neural ODE and up to 2.3 times faster than the best existing reverse-accurate method. We also show that PNODE enables the use of the implicit time integration methods that are needed for stiff dynamical systems.

Variational learning algorithms for quantum query complexity

Quantum query complexity is pivotal in the analysis of quantum algorithms, encompassing well-known examples like search and period-finding algorithms. These algorithms typically involve a sequence of unitary operations and oracle calls dependent on an input variable. In this study, we introduce a variational learning approach to explore quantum query complexity. Our method employs an efficient parameterization of the unitary operations and utilizes a loss function derived from the algorithm’s error probability. We apply this technique to various quantum query complexities, notably devising a new algorithm that resolves the 5-bit Hamming modulo problem with four queries, addressing an open question from Cornelissen et al (2021 arXiv:2112.14682). This finding is corroborated by a semidefinite programming (SDP) approach. Our numerical method exhibits superior memory efficiency compared to SDP and can identify quantum query algorithms (QQAs) that require a smaller workspace register dimension, an aspect not optimized by SDP. These advancements present a significant step forward in the practical application and understanding of QQAs.

A Systematic Evaluation of Recurrent Neural Network Models for Edge Intelligence and Human Activity Recognition Applications

The Recurrent Neural Networks (RNNs) are an essential class of supervised learning algorithms. Complex tasks like speech recognition, machine translation, sentiment classification, weather prediction, etc., are now performed by well-trained RNNs. Local or cloud-based GPU machines are used to train them. However, inference is now shifting to miniature, mobile, IoT devices and even micro-controllers. Due to their colossal memory and computing requirements, mapping RNNs directly onto resource-constrained platforms is arcane and challenging. The efficacy of edge-intelligent RNNs (EI-RNNs) must satisfy both performance and memory-fitting requirements at the same time without compromising one for the other. This study’s aim was to provide an empirical evaluation and optimization of historic as well as recent RNN architectures for high-performance and low-memory footprint goals. We focused on Human Activity Recognition (HAR) tasks based on wearable sensor data for embedded healthcare applications. We evaluated and optimized six different recurrent units, namely Vanilla RNNs, Long Short-Term Memory (LSTM) units, Gated Recurrent Units (GRUs), Fast Gated Recurrent Neural Networks (FGRNNs), Fast Recurrent Neural Networks (FRNNs), and Unitary Gated Recurrent Neural Networks (UGRNNs) on eight publicly available time-series HAR datasets. We used the hold-out and cross-validation protocols for training the RNNs. We used low-rank parameterization, iterative hard thresholding, and spare retraining compression for RNNs. We found that efficient training (i.e., dataset handling and preprocessing procedures, hyperparameter tuning, and so on, and suitable compression methods (like low-rank parameterization and iterative pruning) are critical in optimizing RNNs for performance and memory efficiency. We implemented the inference of the optimized models on Raspberry Pi.

Long Horizon Episodic Decision Making for Cognitively Inspired Robots

The Human decision-making process works by recollecting past sequences of observations and using them to decide the best possible action in the present. These past sequences of observations are stored in a derived form which only includes important information the brain thinks might be useful in the future, while forgetting the rest. we propose an architecture that tries to mimic the human brain and improve the memory efficiency of transformers by using a modified Transformer XL architecture which uses Automatic Chunking which only attends to the relevant chunks in the transformer block. On top of this, we use Forget Span which is technique to remove memories that do not contribute to learning. We also theorize the technique of Similarity based forgetting to remove repetitive memories. We test our model in various tasks that test the abilities required to perform well in a human-robot collaboration scenario.

The Reflective Mind of the Anxious in Action: Metacognitive Beliefs and Maladaptive Emotional Regulation Strategies Constrain Working Memory Efficiency.

The Attentional Control Theory (ACT) posits that, while trait anxiety may not directly impact performance, it can influence processing efficiency by prompting the use of compensatory mechanisms. The specific nature of these mechanisms, which might be reflective, is not detailed by the ACT. In a study involving 110 students (M = 20.12; SD = 2.10), surveys were administered to assess the students' metacognitive beliefs, trait anxiety, and emotion regulation strategies (ERSs). The participants engaged in two working memory exercises: the digit span task from the WAIS-IV and an emotional n-back task. The findings indicated that anxiety, metacognitive beliefs, and maladaptive ERSs did not affect task performance but were correlated with increased response times. Several regression analyses demonstrated that a lack of confidence in one's cognitive abilities and maladaptive ERSs predict higher reaction times (RT) in the n-back task. Additionally, maladaptive ERSs also predict an increased use of strategies in the digit span task. Finally, two mediation analyses revealed that anxiety increases processing efficiency, and this relation is mediated by the use of maladaptive ERSs. These results underscore the importance of the reflective level in mediating the effects of trait anxiety on efficiency. They highlight the necessity of incorporating metacognitive beliefs and maladaptive emotion regulation strategies for a thorough comprehension of the Attentional Control Theory. Recognizing these factors offers valuable perspectives for enhancing cognitive capabilities and fostering academic achievement.

SuperGuardian: Superspreader removal for cardinality estimation in data streaming

Measuring flow cardinality is one of the fundamental problems in data stream mining, where a data stream is modeled as a sequence of items from different flows and the cardinality of a flow is the number of distinct items in the flow. Many existing sketches based on estimator sharing have been proposed to deal with huge flows in data streams. However, these sketches suffer from inefficient memory usage due to allocating the same memory size for each estimator without considering the skewed cardinality distribution. To address this issue, we propose SuperGuardian to improve the memory efficiency of existing sketches. SuperGuardian intelligently separates flows with high-cardinality from the data stream, and keeps the information of these flows with the large estimator, while using existing sketches with small estimators to record low-cardinality flows. We carry out a mathematical analysis for the cardinality estimation error of SuperGuardian. To validate our proposal, we have implemented SuperGuardian and conducted experimental evaluations using real traffic traces. The experimental results show that existing sketches using SuperGuardian reduce error by 79 % - 96 % and increase the throughput by 0.3–2.3 times.

An Effective Selection of Memory Technologies for TCAM to Improve the Search Operations: Demonstration of Memory Efficiency in SDN Recovery

Ternary Content-Addressable Memory (TCAM) is used for storing the flow tables in software-defined networking (SDN)-based OpenFlow switches. However, the TCAM can store only a certain number of flow tables (8000). Moreover, when the switch flow tables need to be updated due to the link failure in the SDN, further updates may be lost due to the flow tables limit of the TCAM space. Hence, to resolve this issue, other memories need to be used in conjunction with TCAM to enhance the memory operations of TCAM. When considering which flash memory technology is to be used in conjunction with TCAM, we need to balance several factors to ensure optimal performance, speed, endurance, reliability, integration complexity, and cost-effectiveness. Hence, it leads to a multi-criteria decision-making problem regarding the selection of other memory technologies such as 3D XPoint, Magnetoresistive RAM, Resistive RAM, and Ferroelectric RAM. In this paper, we use the analytical network process (ANP) method to select the suitable technology in conjunction with TCAM, considering the features of the memory technologies for Software-Defined Internet-of-Things (SD-IoT). We provide a comprehensive numerical model leveraging the ANP to rank the memory technologies regarding their weights. The highest weights identify the most suitable technology for TCAM. We perform simulations to show the effectiveness of the mathematical model utilizing the ANP. The results show that the suggested methodology reduces the recovery delay, improves the packets received ratio (PRR), decreases the jitter, and increases the throughput.

Exploring Working Memory Capacity and Efficiency Processes to Understand Working Memory Training Outcomes in Primary School Children.

Despite the abundance of research evaluating working memory training outcomes in children, few studies have examined the underlying cognitive mechanisms. This study aimed to contribute understanding by exploring whether working memory capacity (maximum span) and/or efficiency (basic and cognitive processing speeds), two proposed cognitive mechanisms, are associated with children's working memory performance immediately and 6-months post-intervention. We used data from a previous trial in primary school children (7-11 years) who completed working memory training (n = 52) or an active control (n = 36), comprising 10 sessions (each 20-minutes) in class over two weeks. Children completed five working memory measures at baseline, immediately and 6-months post-intervention: two Backwards Span and two Following Instructions measures (same paradigms as training activities), and one n-back measure (different paradigm). Maximum span, basic and cognitive processing speeds, and performance were calculated for each measure. Associations between change in maximum span, processing speeds and change in performance on the working memory measures from baseline to immediately and 6-months post-intervention did not differ between groups (all p < .05). Maximum span, processing speeds and performance on working memory measures did not differ between groups. Findings provide little evidence that the studied capacity or efficiency processes contribute to understanding working memory training outcomes in primary school children. Furthermore, working memory training did not have benefits for children's working capacity, efficiency or performance up to 6-months post-intervention. It is of interest for future studies to explore cognitive mechanisms, including strategy use, maximum span and information processing, in datasets where training effects are observed.

Exploiting Graphics Processing Units to Speed Up Subgraph Enumeration for Efficient Graph Pattern Mining GraphDuMato

Subgraph selection involves searching an input graph for subgraphs with a certain attribute. Graph pattern mining (GPM) relies on this technique, despite its computing complexity and rapid growth. Problems with uncoalesced access to memory, separation, and strain unbalance make efficient subgraph identification on GPUs a huge problem, given GPUs' success in speeding up operations across multiple industries. It is surprising that these challenges have not received sufficient attention in earlier research. This work presents new methods for effectively building and running subgraph enumeration on GPUs. Optimization of computational resource usage is achieved by combining a warp-centric design with a Depth first search (DFS) style approach. Memory efficiency, execution divergence, and GPU activity parallelization could all be enhanced by integrating these two methods. An affordable load balancing level is also incorporated for the purpose of dispersing work among thread warps, which further decreases GPU idleness. The GraphDuMato system facilitates the utilization of GPM methods with its intuitive application programming interface (API). Testing proves that GraphDuMato can outperform state-of-the-art GPM algorithms on a regular basis and may mine subgraphs with up to twelve trees.

Maternal aerobic exercise decreases the effects of a perinatal Western diet on the short and long-term memory of CD1 mouse progeny

Maternal nutrition and physical activity during pregnancy and lactation can modify offspring development. Here, we investigated the effects of maternal aerobic exercise (AE) and Western diet (WD) on brain development, cognitive flexibility, and memory of progenies. Sixteen adult female mice were assigned to AE or sedentary groups (SED) and fed a balanced diet (BD) or WD. Offspring were categorized into four groups: WD + AE, WD + SED, BD + AE, and BD + SED. The AE group showed enhanced spontaneous alternation in the T-maze test, suggesting an improvement in working memory and tasks related to cognitive flexibility. The novel object recognition (NOR) test showed that the BD + AE pups improved their absolute discrimination and discrimination index at 24 h, which suggests a delay in memory consolidation without affecting evocation. WD + SED showed poorer discrimination and recognition memory. The pups of AE mothers had better efficiency in short-term memory, whereas WD offspring showed low performance in long-term memory. Interestingly, exercise improved tasks related to cognitive flexibility, regardless of the diet. These findings indicate that maternal diet and physical activity modify offspring development and suggest that maternal AE during pregnancy could be a beneficial intervention to counteract the adverse effects of WD by improving spatial memory and cognitive flexibility in offspring.

Storage of telecom wavelength heralded single photons in a fiber cavity quantum memory

We demonstrate the storage and retrieval of heralded single photons in a fiber-based cavity quantum memory. The photons are stored, and retrieved, from the memory using quantum frequency conversion which switches the photon into, and out of, resonance with the cavity. The photons, generated in the telecom O-band with a bandwidth of 81 GHz, are retrieved from the memory with a 1/e lifetime of 1.64 µs, or 32.8 cavity round trips. We show that non-classical photon statistics remain for 70 round trips. The internal memory efficiency after 0.5 µs of storage is 10.9 ± 0.5%; a coupling efficiency of 60% into the memory cavity yields a total efficiency of 6.0 ± 0.3%. These results mark a crucial step forward in the development of fiber-based quantum memories, and high-bandwidth memories operating at telecom wavelengths, with applications to photon source multiplexing and fiber-based quantum networking.