Memory Efficiency Research Articles

Modify Self-Attention via Skeleton Decomposition for Effective Point Cloud Transformer

Although considerable progress has been achieved regarding the transformers in recent years, the large number of parameters, quadratic computational complexity, and memory cost conditioned on long sequences make the transformers hard to train and implement, especially in edge computing configurations. In this case, a dizzying number of works have sought to make improvements around computational and memory efficiency upon the original transformer architecture. Nevertheless, many of them restrict the context in the attention to seek a trade-off between cost and performance with prior knowledge of orderly stored data. It is imperative to dig deep into an efficient feature extractor for point clouds due to their irregularity and a large number of points. In this paper, we propose a novel skeleton decomposition-based self-attention (SD-SA) which has no sequence length limit and exhibits favorable scalability in long-sequence models. Due to the numerical low-rank nature of self-attention, we approximate it by the skeleton decomposition method while maintaining its effectiveness. At this point, we have shown that the proposed method works for the proposed approach on point cloud classification, segmentation, and detection tasks on the ModelNet40, ShapeNet, and KITTI datasets, respectively. Our approach significantly improves the efficiency of the point cloud transformer and exceeds other efficient transformers on point cloud tasks in terms of the speed at comparable performance.

Multimode capacity of atomic-frequency comb quantum memories

Ensemble-based quantum memories are key to developing multiplexed quantum repeaters, able to overcome the intrinsic rate limitation imposed by finite communication times over long distances. Rare-earth ion doped crystals are main candidates for highly multimode quantum memories, where time, frequency and spatial multiplexing can be exploited to store multiple modes. In this context the atomic frequency comb (AFC) quantum memory provides large temporal multimode capacity, which can readily be combined with multiplexing in frequency and space. In this article, we derive theoretical formulas for quantifying the temporal multimode capacity of AFC-based memories, for both optical memories with fixed storage time and spin-wave memories with longer storage times and on-demand read out. The temporal multimode capacity is expressed in key memory parameters, such as AFC bandwidth, fixed-delay storage time, memory efficiency, and control field Rabi frequency. Current experiments in europium- and praseodymium-doped Y2SiO5 are analyzed within this theoretical framework, which is also tested with newly acquired data, as prospects for higher temporal capacity in these materials are considered. In addition we consider the possibility of spectral and spatial multiplexing to further increase the mode capacity, with examples given for praseodymium doped Y2SiO5.

Three-Dimensional Magnetotelluric Inversion for Triaxial Anisotropic Medium in Data Space

The interpretation of three-dimensional (3-D) magnetotelluric (MT) data is usually based on the isotropic assumption of the subsurface structures, and this assumption could lead to erroneous interpretation in the area with considerable electrical anisotropy. Although arbitrary anisotropy is much closer to the ground truth, it is generally more challenging to recover full anisotropy parameters from 3-D inversion. In this paper, we present a 3-D triaxial anisotropic inversion framework using the edge-based finite element method with a tetrahedral mesh. The 3-D inverse problem is solved by the Gauss-Newton (GN) method which shows fast convergence behavior. The computation cost of the data-space method depends on the size of data, which is usually smaller than the size of model; therefore, we transform the inversion algorithm from the model space to the data space for memory efficiency. We validate the effectiveness and applicability of the developed algorithm using several synthetic model studies.

A Two-Way Split-Step Wavelet Scheme for Tropospheric Long-Range Propagation in Various Environments

In the context of improving the dimensioning of observation and telecommunication, the characterization of the propagation canal is very important. Thus, accurate models of propagation phenomenona in their environment and above a rough surface (maritime or terrestrial) are of major interest for many applications (such as radar, communications, and teledetection). To provide solutions to this problem, in this paper, we propose a fast, memory-efficient, and accurate asymptotic method for 2D tropospheric propagation for a large band of frequency that accounts for relief, as well as ground composition and roughness. This latter is a two-way split-step wavelet scheme with an intrinsic stopping criterion. For overseas propagation, roughness effects are considered through a hybrid method. A complete theoretical comparison with SSF in terms of memory and time efficiency is proposed. Simulations in various environments (ground, sea, and snow), as well as different frequencies (UHF, S, and X-band) are performed to validate the method and highlight its advantages. To highlight the interest of the developed methodology, this latter is applied to different real-life applications, such as the prediction of radar coverage and the optimization of an antenna location.

Development and validation of the Childhood Cancer Survivor Study-Neurocognitive Questionnaire (CCSS-NCQ) Japanese version.

e22025 Background: The lives of childhood cancer survivors (CCS) may be impacted by symptoms of neurocognitive dysfunction. Decline in executive function, one component of the construct concept of neurocognitive function, may affect the ability to be independent in later life; thus,ssessment and intervention are important. In the United States, the Childhood Cancer Survivor Study-Neurocognitive Questionnaire (CCSS-NCQ) developed by St. Jude Children’s Research Hospital has been widely utilized in clinical practice. It is a self-administered scale for screening daily difficulties regarding organization, emotional regulation, memory, and task efficiency which all fall under the concept of the executive function. To expand the tools for executive function assessment in Japan, we developed the CCSS-NCQ Japanese version and performed a validation study among the adolescent and young adult (AYA)-CCS population. Methods: The Japanese version of the CCSS-NCQ was developed by conducting linguistic validation. Japanese AYA-CCS of all cancer types were recruited and asked to complete this 32-question self-administered questionnaire. Exploratory and confirmatory factor analysis (FA) with the Promax rotation were conducted. Results: In total, 106 participants responded. The mean age of participants was 20.6 ± 5.3 years, and 49.1% were male. The mean age at diagnosis was 8.7 ± 4.8 years; 46.2% of participants were diagnosed with brain tumors, 37.7% with hematological malignancy, and 12.3% with solid tumors. The FA identified the similar four-factor structure of organization, emotional regulation, memory, and task efficiency as the original CCSS-NCQ with factor loading set at 0.4. The Cronbach’s alpha was 0.84 - 0.92. Conclusions: Neurocognitive dysfunction could be a crucial late complication of childhood cancer, and maintaining executive function, in particular, is essential for sustaining the quality of life among CCS. The CCSS-NCQ Japanese version is the first internationally translated CCSS-NCQ. The successful validation in the Japanese population will expand opportunities for future international collaboration to identify race/ethnicity-based disparities, as well as scale enhancement for application to CCS worldwide.

Efficiency of voluntary auditory-speech memory in younger schoolchildren with different types of dysontogenesis

IntroductionAssessing of voluntary auditory verbal memory, mediated or non-mediated by meaning context is an important component of impaired mental or psychic development estimation in children with different types of dysontogenesis.ObjectivesInvestigation of memory in chidren with mental disordersMethodsParticipants: Children 9-12 years: ND - 25 normal development (14 boys), F70 - 31 with mild mental retardation (18 boys), F20.8 - 15 children with childhood type of schizophrenia (11 boys) and F21- 29 with schizotypal disorder (22 boys). Learning of 10 words, simple (SPA) and complex (CPA) paired associations. Assessing Parameter - auditory-speech memory efficiency in each of techniques. Mann-Whitney criterion.ResultsThere were significant differences in memory efficiency of 10 words between F21 and F70 (p≤0.01) and F70 and F20.8 (p≤0.05). Simple paired associations - no differences between all groups. Complex paired associations: F21 and F70 (p≤0.05), F21 and F20.8 (p≤0.01). ND - significant differences in memory efficiency of 10 words with all groups (F21 – p ≤0.01, F20.8 – p ≤0.05, F70 – p ≤0.01). SPA: significant differences with all groups (p≤ 0.01). CPA - significant differences words with all groups (F21 – p≤ 0.05, F20.8 – p≤ 0.01, F70 – p ≤0.01).ConclusionsСommon features of working memory in children with diseased type of development:- improved memorization with introduction of strong semantic connections for all types of dysontogenesis. For complex connections in different groups we suppose different mechanisms of destroyed memory: F70 - low level of thinking, F20.8 - decreasing in mental activity or as a result of forming defect.DisclosureNo significant relationships.

A Neuromorphic Model With Delay-Based Reservoir for Continuous Ventricular Heartbeat Detection.

There is a growing interest in neuromorphic hardware since it offers a more intuitive way to achieve bio-inspired algorithms. This paper presents a neuromorphic model for intelligently processing continuous electrocardiogram (ECG) signal. This model aims to develop a hardware-based signal processing model and avoid employing digitally intensive operations, such as signal segmentation and feature extraction, which are not desired in an analogue neuromorphic system. We apply delay-based reservoir computing as the information processing core, along with a novel training and labelling method. Different from the conventional ECG classification techniques, this computation model is a end-to-end dynamic system that mimics the real-time signal flow in neuromorphic hardware. The input is the raw ECG stream, while the amplitude of the output represents the risk factor of a ventricular ectopic heartbeat. The intrinsic memristive property of the reservoir empowers the system to retain the historical ECG information for high-dimensional mapping. This model was evaluated with the MIT-BIH database under the inter-patient paradigm and yields 81% sensitivity and 98% accuracy. Under this architecture, the minimum size of memory required in the inference process can be as low as 3.1 MegaByte(MB) because the majority of the computation takes place in the analogue domain. Such computational modelling boosts memory efficiency by simplifying the computing procedure and minimizing the required memory for future wearable devices.

Shape memory, thermal conductivity, and mechanical property of polylactic acid and natural rubber composites reinforced by an inorganic thermal conductive network

AbstractWith the development and application of thermotropic shape memory polymer materials in the biomedical field, it is very crucial for these materials to improve their thermal conductivity efficiency and shape memory recovery rate by endowing them a high thermal conductivity. In our work, a PLA/NR/T‐ZnOw/GR shape memory composite was prepared by melt blending with tetrapod zinc oxide whisker (T‐ZnOw) and graphene (GR) as the double inorganic fillers and polylactic acid (PLA) and natural rubber (NR) as the polymer matrix. The synergism of T‐ZnOw and GR constructs a three‐dimensional (3D) thermal conductive network, making the thermal conductivity of PLA/NR/5wt%T‐ZnOw/2.5wt%GR increase to 1.215 W∙(m∙K)−1. The addition of T‐ZnOw reduces the agglomeration of GR in the PLA/NR matrix and the interfacial thermal resistance between the fillers and the matrix. It also improves the heat transfer rate, resulting in an excellent shape memory of PLA/NR/T‐ZnOw/GR with a morphology recovery rate of ca. 96%, a fixed rate of 95% and high thermal stability. The presented biodegradable shape memory composite with the 3D thermal conductive network is promising in the fields of medical treatment and intelligent sensors.

Formation Control of Automated Guided Vehicles in the Presence of Packet Loss.

This paper presents the formation tracking problem for non-holonomic automated guided vehicles. Specifically, we focus on a decentralized leader–follower approach using linear quadratic regulator control. We study the impact of communication packet loss—containing the position of the leader—on the performance of the presented formation control scheme. The simulation results indicate that packet loss degrades the formation control performance. In order to improve the control performance under packet loss, we propose the use of a long short-term memory neural network to predict the position of the leader by the followers in the event of packet loss. The proposed scheme is compared with two other prediction methods, namely, memory consensus protocol and gated recurrent unit. The simulation results demonstrate the efficiency of the long short-term memory in packet loss compensation in comparison with memory consensus protocol and gated recurrent unit.

High-performance cavity-enhanced quantum memory with warm atomic cell

High-performance quantum memory for quantized states of light is a prerequisite building block of quantum information technology. Despite great progresses of optical quantum memories based on interactions of light and atoms, physical features of these memories still cannot satisfy requirements for applications in practical quantum information systems, since all of them suffer from trade-off between memory efficiency and excess noise. Here, we report a high-performance cavity-enhanced electromagnetically-induced-transparency memory with warm atomic cell in which a scheme of optimizing the spatial and temporal modes based on the time-reversal approach is applied. The memory efficiency up to 67 ± 1% is directly measured and a noise level close to quantum noise limit is simultaneously reached. It has been experimentally demonstrated that the average fidelities for a set of input coherent states with different phases and amplitudes within a Gaussian distribution have exceeded the classical benchmark fidelities. Thus the realized quantum memory platform has been capable of preserving quantized optical states, and is ready to be applied in quantum information systems, such as distributed quantum logic gates and quantum-enhanced atomic magnetometry.

Distinct Generation of Subjective Vividness and Confidence during Naturalistic Memory Retrieval in Angular Gyrus.

Our subjective experience of remembering guides and monitors the reconstruction of past and simulation of the future, which enables us to identify mistakes and adjust our behavior accordingly. However, what underlies the process of subjective mnemonic experience remains incompletely understood. Here, we combined behavior, repetitive TMS, and functional neuroimaging to probe whether vividness and confidence are generated differently during retrieval. We found that preretrieval repetitive TMS targeting the left angular gyrus (AnG) selectively attenuated the vividness efficiency compared with control stimulation while keeping metacognitive efficiency and objective memory accuracy unaffected. Using trialwise data, we showed that AnG stimulation altered the mediating role of vividness in confidence in the accuracy of memory judgment. Moreover, resting-state functional connectivity of hippocampus and AnG was specifically associated with vividness efficiency, but not metacognitive efficiency across individuals. Together, these results identify the causal involvement of AnG in gauging the vividness, but not the confidence, of memory, thereby suggesting a differentiation account of conscious assessment of memory by functionally and anatomically dissociating the monitoring of vividness from confidence.

Memory-Efficient Algorithm for Scalar Multiplications on Twisted Edwards Curves for Isogeny-Based Cryptosystems

Scalar multiplications are considered an essential aspect of implementations of isogeny-based cryptography. The efficiency of scalar multiplication depends on the equation of the underlying elliptic curves and the addition chain employed. Bos and Friedberger recently stated that, for larger scalar multiplication, addition-subtraction chains will become more useful for twisted Edwards curves because of the differential restriction on Montgomery curves in the setting of isogeny-based cryptosystem. Motivated by these comments, we attempt to increase the efficiency of scalar multiplication in twisted Edwards curves in terms of the memory of algorithms. In this paper, we present a double-base addition-subtraction chain algorithm with memory efficiency for scalar multiplication. The memory usage of this part is O log n 2 / log log n , which is better than the result of the tree-based approach, which is O log n 2 .

Slide Attack on Full-Round ULC Lightweight Block Cipher Designed for IoT

To meet the security demand for IoT devices, many new lightweight block ciphers are proposed. ULC is a lightweight block cipher designed for the IoT. It has many advantages in terms of memory use, efficiency, and security. In this paper, a slide attack on full-round ULC is proposed in a related key setting. First, two properties on ULC are discovered. The first property is presented to illustrate the property of a slid pair for ULC. The second property is introduced to construct a link between some round key bits and some master key bits. Based on these properties, a key recovery attack on ULC is proposed. In this attack, with the condition of one related key, all the 80 master key bits can be recovered with the data complexity of O(232), the memory complexity of O(232), and the time complexity of O(263). The result of this paper indicates that ULC can’t resist slide attack in related key settings.

LWRN: Light-Weight Residual Network for Edge Detection

Edge detection is one of the most fundamental fields in computer vision. With the rapid development of the combination of Convolutional Neural Network and Multi-Scale Representation of image, significant progress has been made in this field. However, most of them have a huge size, which makes it hard to apply in reality, and a huge number of parameters may lead to waste of computing resources. In this paper, we focus on qualitative analysis of the role of each part in the network, and propose a modified light-weight architecture based on our result and the study of former works. Our new architecture is composed of residual-blocks, max-pooling layers and batch normalization layers. Compared with the previous models, the new architecture performs better in memory, convergence and computation efficiency with similar model size. Moreover, the new architecture can achieve better accuracy with smaller model size. When evaluating our model on the well-known BSDS500 benchmark, we achieve ODS F-measure of 0.769 with parameters less than 0.3[Formula: see text]M, which shows a better property than the state-of-the-art result 0.766 at this level.

A Configurable and Real-Time Multi-Frequency 3D Image Signal Processor for Indirect Time-of-Flight Sensors

3D time-of-flight (ToF) cameras have recently attracted widespread attention because of their potential in wide applications. However, the lack of versatile 3D image signal processor (ISP) is still an obstacle that hinders their popularity. The main challenges lie in the management of high-throughput raw data and the complexity of 3D image processing in hardware. To address these challenges, this paper presents a top-down 3D ISP with high real-time performance and high energy efficiency for indirect ToF (IToF) imaging. First, we propose configurable cache strategies to improve the efficiency of non-streaming memory accesses and support different IToF sensors. Then, to resolve the phase ambiguity problem commonly existing in IToF sensors, we develop a hardware-optimized cascaded multi-frequency fusion method, which not only supports up to six modulation frequencies but also facilitates the pixel-wise detection of multipath interference. Moreover, we propose a scalable calculation engine for depth calculation and calibration, 3D image enhancement, and post-processing. The pipeline is prototyped on a Xilinx ZCU102 FPGA board. Results show that the 3D ISP has negligible precision loss and low latency, achieving <inline-formula> <tex-math notation="LaTeX">$53.35\times $ </tex-math></inline-formula> energy efficiency improvement compared with the implementation on Intel CPU and <inline-formula> <tex-math notation="LaTeX">$55.45\times $ </tex-math></inline-formula> improvement over the ARM CPU implementation. When running at 75 MHz, the 3D ISP can support <inline-formula> <tex-math notation="LaTeX">$1280\times 960$ </tex-math></inline-formula> depth, intensity, and point cloud output at a frame rate of 30 f/s.

A non-alternating graph hashing algorithm for large-scale image search

In the era of big data, methods for improving memory and computational efficiency have become crucial for the successful deployment of technology. Hashing is one of the most effective approaches to deal with the computational limitations associated with big data. One natural way to formulate this problem is spectral hashing, which directly incorporates an affinity to learning binary codes. However, owing to the binary constraints, the optimization becomes intractable. To mitigate this challenge, different relaxation approaches have been proposed to reduce the computational load required to obtain binary codes and still attain a good solution. The problem with all existing relaxation methods involves the use of one or more additional auxiliary variables to attain high-quality binary codes while relaxing the problem. The existence of auxiliary variables leads to the coordinate descent approach, which increases the computational complexity. We argue that the introduction of these variables is unnecessary. To this end, we propose a novel relaxed formulation for spectral hashing that adds no additional variables to the problem. Furthermore, instead of solving the problem in the original space where the number of variables is equal to the data points, we solve the problem in a much smaller space and retrieve the binary codes from this solution. This technique reduces both the memory and computational complexity simultaneously. We apply two optimization techniques, namely, the projected gradient and optimization on the manifold, to obtain the solution. Using comprehensive experiments on four public datasets, we show that the proposed efficient spectral hashing (ESH) algorithm achieves a highly competitive retrieval performance compared with the state-of-the-art algorithms at low complexity.

Low-temperature processed beta-phase In2Se3 ferroelectric semiconductor thin film transistors

As scaling becomes increasingly difficult, there is growing interest in vertical or three-dimensional stacking of transistors and especially memory. Ferroelectric semiconductor field effect transistors can be key enablers to improve energy efficiency and overall chip and memory performance. In this work, low-temperature processed, back-end-of-the-line compatible transistors were demonstrated by depositing a layered chalcogenide ferroelectric semiconductor, beta-phase In2Se3, at temperature as low as 400 °C. Top gate n-channel In2Se3 thin film transistors were fabricated with field-effect mobility ∼1 cm2 V−1 s−1, and simple polarization switching based memory results are presented.

Key Concepts, Weakness and Benchmark on Hash Table Data Structures

Most computer programs or applications need fast data structures. The performance of a data structure is necessarily influenced by the complexity of its common operations; thus, any data-structure that exhibits a theoretical complexity of amortized constant time in several of its main operations should draw a lot of attention. Such is the case of a family of data structures that is called hash tables. However, what is the real efficiency of these hash tables? That is an interesting question with no simple answer and there are some issues to be considered. Of course, there is not a unique hash table; in fact, there are several sub-groups of hash tables, and, even more, not all programming languages use the same variety of hash tables in their default hash table implementation, neither they have the same interface. Nevertheless, all hash tables do have a common issue: they have to solve hash collisions; that is a potential weakness and it also induces a classification of hash tables according to the strategy to solve collisions. In this paper, some key concepts about hash tables are exposed and some definitions about those key concepts are reviewed and clarified, especially in order to study the characteristics of the main strategies to implement hash tables and how they deal with hash collisions. Then, some benchmark cases are designed and presented to assess the performance of hash tables. The cases have been designed to be randomized, to be self-tested, to be representative of a real user cases, and to expose and analyze the impact of different factors over the performance across different hash tables and programming languages. Then, all cases have been programmed using C++, Java and Python and analyzed in terms of interfaces and efficiency (time and memory). The benchmark yields important results about the performance of these structures and its (lack of) relationship with complexity analysis.

ABL-TC: A lightweight design for network traffic classification empowered by deep learning

Network traffic classification is an increasingly significant prerequisite for network management. An accurate traffic classifier can contribute to traffic engineering, traffic intrusion detection and user behavior analysis. Recently, deep learning (DL) has attracted considerable attention for traffic classification due to its excellent learning ability without the need for handcrafted feature engineering or privacy invasion. However, most DL-based traffic classification solutions mainly focus on improving the identification performance, with no regard for the computational and memory burden caused by a large model size or data redundancy. Motivated by the above gaps, we elaborate on the preprocessing of raw traffic data and investigate a lightweight traffic classifier. Specifically, we design a preprocessing approach to convert raw traffic data into available datasets for deep learning based traffic classifiers, which tailors raw traffic data as training datasets by resolving its structure and content and pruning redundant information. Thanks to its ability to focus on key information, attention mechanism is introduced to design an attention-based long short-term memory (LSTM) model for traffic classification, termed the ABL-TC. ABL-TC can effectively identify global dependencies between the input and the output with a limited number of important raw features, which can contribute to learning key information to identify various traffic types. Extensive experiments on real-world public datasets show that the ABL-TC outperforms state-of-the-art approaches in terms of various metrics for recognizing traffic categories while remaining competitive in terms of time and memory efficiency.

REANN: A PyTorch-based end-to-end multi-functional deep neural network package for molecular, reactive, and periodic systems.

In this work, we present a general purpose deep neural network package for representing energies, forces, dipole moments, and polarizabilities of atomistic systems. This so-called recursively embedded atom neural network model takes advantages of both the physically inspired atomic descriptor based neural networks and the message-passing based neural networks. Implemented in the PyTorch framework, the training process is parallelized on both the central processing unit and the graphics processing unit with high efficiency and low memory in which all hyperparameters can be optimized automatically. We demonstrate the state-of-the-art accuracy, high efficiency, scalability, and universality of this package by learning not only energies (with or without forces) but also dipole moment vectors and polarizability tensors in various molecular, reactive, and periodic systems. An interface between a trained model and LAMMPs is provided for large scale molecular dynamics simulations. We hope that this open-source toolbox will allow for future method development and applications of machine learned potential energy surfaces and quantum-chemical properties of molecules, reactions, and materials.