Memory Banks Research Articles

Memory-Guided Multi-View Multi-Domain Fake News Detection

The wide spread of fake news is increasingly threatening both individuals and society. Great efforts have been made for automatic fake news detection on a <i>single</i> domain (e.g., politics). However, correlations exist commonly across multiple news domains, and thus it is promising to simultaneously detect fake news of <i>multiple</i> domains. Based on our analysis, we pose two challenges in multi-domain fake news detection: 1) <bold/><i>domain shift</i><bold/>, caused by the discrepancy among domains in terms of words, emotions, styles, etc. 2) <bold/><i>domain labeling incompleteness</i><bold/>, stemming from the real-world categorization that only outputs one single domain label, regardless of topic diversity of a news piece. In this paper, we propose a Memory-guided Multi-view Multi-domain Fake News Detection Framework (M <inline-formula><tex-math notation="LaTeX">$^{3}$</tex-math></inline-formula> FEND) to address these two challenges. We model news pieces from a multi-view perspective, including semantics, emotion, and style. Specifically, we propose a Domain Memory Bank to enrich domain information which could discover potential domain labels based on seen news pieces and model domain characteristics. Then, with enriched domain information as input, a Domain Adapter could adaptively aggregate discriminative information from multiple views for news in various domains. Extensive offline experiments on English and Chinese datasets demonstrate the effectiveness of M <inline-formula><tex-math notation="LaTeX">$^{3}$</tex-math></inline-formula> FEND, and online tests verify its superiority in practice. Our code is available at <uri>https://github.com/ICTMCG/M3FEND</uri>.

Toward Tightness of Scalable Neighborhood Component Analysis for Remote-Sensing Image Characterization

Deep metric learning methods have recently drawn significant attention in the field of remote sensing (RS), owing to their prominent capabilities for modeling relations among RS images based on their semantic contents. In the context of scene classification and large-scale image retrieval, one of the most prominent deep metric learning methods is the scalable neighborhood component analysis (SNCA), which has demonstrated excellent performance on the locality neighborhood structure in the metric space. However, the standard SNCA has important constraints on separating the hard positive and other negative images in the metric space, and this may become a major limitation when dealing with the large-scale variance problem inherent to RS data. To address this issue, we propose a novel deep metric learning formulation that introduces a new margin parameter to enforce the compactness of the within-class feature embeddings. Based on this innovative scheme, we propose two novel loss functions: 1) T-SNCA-c, where the parameter is based on the cosine similarity, and 2) T-SNCA-a, where the parameter is based on the angular distance. Besides, we exploit memory bank optimization to further enhance the semantic diversity during training. Our experimental results, conducted using three downstream applications ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> -NN classification, clustering, and image retrieval) and two large-scale RS benchmark datasets, demonstrate that the proposed approach can achieve superior performance when compared to current state-of-the-art deep metric learning methods. The codes of this work will be made available online ( <uri xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">https://github.com/jiankang1991/GRSL_TSNCA</uri> ).

Unsupervised person re-identification based on removal of camera bias and dynamic updating of the memory bank

In recent years, unsupervised person reidentification technology has made great strides. The technology retrieves images of interested persons under different cameras from massive repositories of unlabeled images. However, in the current research, there are some existing problems, such as the influence of pedestrians appearing across cameras and pseudo-label noise. To solve these problems, we conduct research in two ways: removing the camera bias and dynamically updating the memory model. In removing the camera bias, based on a learnable channel attention module, the features that are only related to cameras can be extracted from the feature map, thereby removing the camera bias in the global features and obtaining the features that can represent the pedestrians. In regards to dynamically updating the memory model, since the instance features do not necessarily belong to the identity represented by the pseudo-label, we adopt a method to update the memory dynamically according to the distance between the instance features and the category features so that the category features tend to be true. We combine the removal of the camera bias and the dynamic updating of the memory model to better solve problems in this field. Extensive experimentation demonstrates the superiority of our method over the state-of-the-art approaches on fully unsupervised Re-ID tasks.

Setting the Stage for Speech Production: Infants Prefer Listening to Speech Sounds With Infant Vocal Resonances.

Current models of speech development argue for an early link between speech production and perception in infants. Recent data show that young infants (at 4-6 months) preferentially attend to speech sounds (vowels) with infant vocal properties compared to those with adult vocal properties, suggesting the presence of special "memory banks" for one's own nascent speech-like productions. This study investigated whether the vocal resonances (formants) of the infant vocal tract are sufficient to elicit this preference and whether this perceptual bias changes with age and emerging vocal production skills. We selectively manipulated the fundamental frequency (f0 ) of vowels synthesized with formants specifying either an infant or adult vocal tract, and then tested the effects of those manipulations on the listening preferences of infants who were slightly older than those previously tested (at 6-8 months). Unlike findings with younger infants (at 4-6 months), slightly older infants in Experiment 1 displayed a robust preference for vowels with infant formants over adult formants when f0 was matched. The strength of this preference was also positively correlated with age among infants between 4 and 8 months. In Experiment 2, this preference favoring infant over adult formants was maintained when f0 values were modulated. Infants between 6 and 8 months of age displayed a robust and distinct preference for speech with resonances specifying a vocal tract that is similar in size and length to their own. This finding, together with data indicating that this preference is not present in younger infants and appears to increase with age, suggests that nascent knowledge of the motor schema of the vocal tract may play a role in shaping this perceptual bias, lending support to current models of speech development. https://doi.org/10.23641/asha.17131805.

Strategy of Mitigating Breakdown Interference and Yield Loss in Crossbar Memory

The fault devices induced by device breakdown (BD) are unavoidable due to defect formation during fabrication or along with device cycling stress. The BD cells not only deteriorate device yield, but they also introduce sneak current and result in cell-to-cell interference in the high density crossbar array, which further produces severe array yield loss. Although the BD interference is important, methods of alleviation is less discussed. In this work, we evaluate the strategy of mitigating the strong interference between normal cells and leaky BD cells. We show that while a sufficiently high device yield is the prerequisite, optimizing cell parameters such as resistance of the BD cell and line resistance are proven crucial. However, the yield loss induced by BD interference is not easily compensated using naive redundancy designs. Therefore, a careful optimization on array partition is thus required. We show that with a device yield of 99% and a total memory capacity of megabits, partitioning into a smaller array size (<128 x 128) provides a higher area density than that into larger array sizes. Finally, we proposed a testing bias scheme to rapidly evaluate the device and array yields with an n-time speedup in test latency for an n x n array. The proposed testing bias scheme provides an effective way of ruling out inappropriate low-yield arrays in a memory bank with multiple array blocks.

Leakage Reuse for Energy Efficient Near-Memory Computing of Heterogeneous DNN Accelerators

The exploration of custom deep neural network (DNN) accelerators for highly energy constrained edge devices with on-device intelligence is gaining traction in the research community. Despite the superior throughput and performance of custom accelerators as compared to CPUs or GPUs, the energy efficiency and versatility of state-of-the-art DNN accelerators is constrained due to a) the storage and movement of a large volume of data and b) the limited scope of monolithic architectures, where the entire accelerator executes only a single model at any given time. In this paper, a multi-voltage domain heterogeneous DNN accelerator is proposed that executes multiple models simultaneously with different power-performance operating points. The proposed architecture concurrently implements near-memory computing and leakage reuse, where the leakage current of idle memory banks within each processing element is utilized to deliver current to the adjacently placed multiply-and-accumulate (MAC) units. The proposed architecture and circuit techniques are evaluated with SPICE simulation in a 65 nm CMOS technology. The simulation results indicate that the proposed heterogeneous architecture with leakage reuse improves the energy efficiency to 3.27 tera-operations per second per watt (TOPS/W) as compared to a conventional monolithic and single voltage domain architecture that exhibits an energy efficiency of 0.0458 TOPS/W. In addition, the proposed accelerator that applies the leakage reuse technique to only half of the memory elements storing the weights reduces the power consumption of the sub-arrays of processing elements by 100 mW as compared to an accelerator that does not apply leakage reuse.

Importance and Problems of Oral Teaching in Teaching Chinese as a Foreign Language

This study reviews the importance of oral teaching in teaching Chinese as a foreign language. The oral teaching of the primary stage of Chinese as a foreign language has different characteristics and properties from other languages and other stages of learning in terms of the nature of the subject and the stage of learning. The main goal of oral teaching is to improve students’ oral communication skills. The training of second language teaching skills is generally divided into listening, speaking, reading, writing and translation. The purpose of language teaching is to cultivate students’ ability to communicate in the language they have learned. This study introduces the problems that should be paid attention to in the primary stage of oral teaching, which is helpful in teaching oral Chinese as a foreign language. Teaching Chinese as a foreign language should take the cultivation of learners’ language communication skills as the main goal, which has become a consensus among people. Among the many courses of Chinese as a foreign language, oral course can be regarded as the most flexible and directly related to the actual communicative ability of the training language. Speaking class provides students with speaking opportunities, such that students can master spoken words, spoken grammar and spoken expression patterns; fully mobilise the language information accumulated in the brain memory bank for communication; and move up from language learning as soon as possible The ‘plateau area’ in China is a problem that teachers of oral English classes need to explore. This study aims to improve the effect of oral Chinese teaching in the primary stage of teaching as a foreign language and achieve the expected teaching goals. This study also discusses this issue from the principles of specific teaching implementation.

A critical analysis of the parameters affecting the oxidation behavior of ultra‐high‐temperature diboride ceramics

AbstractIn the last few decades, research on the processing and properties of ultra‐high‐temperature ceramics (UHTCs) has generated a substantial base of knowledge and left several unanswered questions. There is a large scatter in the literature data associated with the processing of UHTC borides prompting the non‐reproducibility and non‐uniformity in the microstructure and, thus, desired properties. Herein, the data on the oxidation behavior of UHTC borides ubiquitous in the entire literature are analyzed to understand the effect of composition, sintering parameters, densification, grain size, and oxidation conditions. A conjunction of graphical methods has been utilized to converge the scattered data and correlate the effect of variables and testing environment on the oxidation behavior. It was concluded that high densification (&gt;95%), large grain size (∼8 μm), and 20–30 vol.% of Si‐containing additives (SiC, Ta5Si3, and TaSi2) could augment the oxidation resistance. The study elucidates that oxide scale thickness should be preferred over mass gain in future studies as a metric for measuring oxidation. The analysis presented here will allow the UHTC community to optimize diboride materials' design for hypersonic applications. The developed database on the oxidation performance of diborides will also transfer knowledge beyond the memory banks of the experts in the field. Furthermore, well‐structured databases such as the one developed herein could be employed in data‐driven approaches to optimize the design and manufacturing of ultra‐high‐temperature materials in an efficient scheme.

Memory-Augmented Transformer for Remote Sensing Image Semantic Segmentation

The semantic segmentation of remote sensing images requires distinguishing local regions of different classes and exploiting a uniform global representation of the same-class instances. Such requirements make it necessary for the segmentation methods to extract discriminative local features between different classes and to explore representative features for all instances of a given class. While common deep convolutional neural networks (DCNNs) can effectively focus on local features, they are limited by their receptive field to obtain consistent global information. In this paper, we propose a memory-augmented transformer (MAT) to effectively model both the local and global information. The feature extraction pipeline of the MAT is split into a memory-based global relationship guidance module and a local feature extraction module. The local feature extraction module mainly consists of a transformer, which is used to extract features from the input images. The global relationship guidance module maintains a memory bank for the consistent encoding of the global information. Global guidance is performed by memory interaction. Bidirectional information flow between the global and local branches is conducted by a memory-query module, as well as a memory-update module, respectively. Experiment results on the ISPRS Potsdam and ISPRS Vaihingen datasets demonstrated that our method can perform competitively with state-of-the-art methods.

Hardware Implementation of Hierarchical Temporal Memory Algorithm

Hierarchical temporal memory (HTM) is an un-supervised machine learning algorithm that can learn both spatial and temporal information of input. It has been successfully applied to multiple areas. In this paper, we propose a multi-level hierarchical ASIC implementation of HTM, referred to as processor core, to support both spatial and temporal pooling. To improve the unbalanced workload in HTM, the proposed design provides different mapping methods for the spatial and temporal pooling, respectively. In the proposed design, we implement a distributed memory system by assigning one dedicated memory bank to each level of hierarchy to improve the memory bandwidth utilization efficiency. Finally, the hot-spot operations are optimized using a series of customized units. Regarding scalability, we propose a ring-based network consisting of multiple processor cores to support a larger HTM network. To evaluate the performance of our proposed design, we map an HTM network that includes 2,048 columns and 65,536 cells on both the proposed design and NVIDIA Tesla K40c GPU using the KTH database as input. The latency and power of the proposed design is 6.04 ms and 4.1 W using GP 65 nm technology. Compared to the equivalent GPU implementation, the latency and power is improved 12.45× and 57.32×, respectively.

Retracing Political Dimensions: Strategies in Contemporary New Media Art

Retracing Political Dimensions: Strategies in Contemporary New Media Art

Generalizable model-agnostic semantic segmentation via target-specific normalization

Semantic segmentation in a supervised learning manner has achieved significant progress in recent years. However, its performance usually drops dramatically due to the data-distribution discrepancy between seen and unseen domains when we directly deploy the trained model to segment the images of unseen (or new coming) domains. To this end, we propose a novel domain generalization framework for the generalizable semantic segmentation task, which enhances the generalization ability of the model from two different views, including the training paradigm and the test strategy. Concretely, we exploit the model-agnostic learning to simulate the domain shift problem, which deals with the domain generalization from the training scheme perspective. Besides, considering the data-distribution discrepancy between seen source and unseen target domains, we develop the target-specific normalization scheme to enhance the generalization ability. Furthermore, when images come one by one in the test stage, we design the image-based memory bank (Image Bank in short) with style-based selection policy to select similar images to obtain more accurate statistics of normalization. Extensive experiments highlight that the proposed method produces state-of-the-art performance for the domain generalization of semantic segmentation on multiple benchmark segmentation datasets, i.e., Cityscapes, Mapillary.

Instruction Roofline: An insightful visual performance model for GPUs

AbstractThe Roofline performance model provides an intuitive approach to identify performance bottlenecks and guide performance optimization. However, the classic FLOP‐centric approach is inappropriate for the emerging applications that perform more integer operations than floating point operations. In this article, we reintroduce our Instruction Roofline Model on NVIDIA GPUs and expand our evaluation of it. The Instruction Roofline incorporates instructions and memory transactions across all memory hierarchies together, and provides more performance insights than the FLOP‐oriented Roofline Model, that is, instruction throughput, stride memory access patterns, bank conflicts, and thread predication. We use our Instruction Roofline methodology to analyze eight proxy applications: HPGMG from AMReX, Matrix Transpose benchmarks, ADEPT from MetaHipMer's sequence alignment phase, EXTENSION from MetaHipMer's local assembly phase, CUSP, cuSPARSE, cudaTensorCoreGemm, and cuBLAS. We demonstrate the ability of our methodology to understand various aspects of performance and performance bottlenecks on NVIDIA GPUs and motivate code optimizations.

Design of a portable radio-frequency-identification reader capable to reading a user memory bank for smart-building energy management

Communication is an important factor in smart-building energy management (SBEM). Many communications technologies have been applied to SBEM, including radio-frequency identification (RFID). RFID has been used not only for identification but also for carrying information, which is stored in a user memory bank attached to the tag. To access the user memory bank, an RFID reader should comply with ISO 18000-6C standards. The greatest challenge of RFID-reader technology is its short communication range, which limits the sensing area. To overcome this problem, this paper proposes a portable RFID reader built to an ISO 18000-6C standard to extend the sensing area due to its moveability. The reader is designed using low-cost devices widely available on the market for ease of duplication and assembly by researchers, educators, and startups. The proposed RFID reader can read passive tags with distances up to 12 and 5.5 m for line-of-sight (LOS) and non-line-of-sight (NLOS) communication, respectively. The minimum received-signal-strength indicators (RSSIs) for LOS and NLOS are found to be −63.75 and −59.66 dBm, respectively. These results are comparable with those of non-portable RFID readers on the market.

DLUX: A LUT-Based Near-Bank Accelerator for Data Center Deep Learning Training Workloads

The frequent data movement between the processor and the memory has become a severe performance bottleneck for deep neural network (DNN) training workloads in data centers. To solve this off-chip memory access challenge, the 3-D stacking processing-in-memory (3D-PIM) architecture provides a viable solution. However, existing 3D-PIM designs for DNN training suffer from the limited memory bandwidth in the base logic die. To overcome this obstacle, integrating the DNN related logic near each memory bank becomes a promising yet challenging solution, since naively implementing the floating-point (FP) unit and the cache in the memory die incurs a large area overhead. To address these problems, we propose DLUX, a high performance and energy-efficient 3D-PIM accelerator for DNN training using the near-bank architecture. From the hardware perspective, to support the FP multiplier with low area overhead, an in-DRAM lookup table (LUT) mechanism is invented. Then, we propose to use a small scratchpad buffer together with a lightweight transformation engine to exploit the locality and enable flexible data layout without the expensive cache. From the software aspect, we split the mapping/scheduling tasks during DNN training into intralayer and interlayer phases. During the intralayer phase, to maximize data reuse in the LUT buffer and the scratchpad buffer, achieve high concurrency, and reduce data movement among banks, a 3D-PIM customized loop tiling technique is adopted. During the interlayer phase, efficient techniques are invented to ensure the input-output data layout consistency and realize the forward-backward layout transposition. Experiment results show that DLUX can reduce FP32 multiplier area overhead by 60% against the direct implementation. Compared with a Tesla V100 GPU, end-to-end evaluations show that DLUX can provide on average 6.3× speedup and 42× energy efficiency improvement.

Large-capacity and high-speed instruction cache based on divide-by-2 memory banks

An increase in the cache capacity is usually accompanied by a decrease in access speed. To balance the capacity and performance of caches, this paper proposes an instruction cache (ICache) architecture based on divide-by-2 memory banks (D2MB-ICache). The control circuit and memory banks of D2MB-ICache work at the central processing unit (CPU) frequency and the divide-by-2 CPU frequency, respectively, so that the capacity of D2MB-ICache can be expanded without lowering its frequency. For sequential access, D2MB-ICache can output the required instruction from memory banks per CPU cycle by dividing the memory banks with a partition mechanism and employing an inversed clock technique. For non-sequential access, D2MB-ICache will fetch certain jump instructions one or two more times, so that it can catch the jump of the request address in time and send the correct instruction to the pipeline. Experimental results show that, compared with conventional ICache, D2MB-ICaches with the same and double capacities show a maximum frequency increase by an average of 14.6% and 6.8%, and a performance improvement by an average of 10.3% and 3.8%, respectively. Moreover, the energy efficiency of 64-kB D2MB-ICache is improved by 24.3%.

Type of encoded material and age modulate the relationship between episodic recall of visual perspective and autobiographical memory

ABSTRACT Episodic memory enables us to form a bank of autobiographical memories across our lifespan. The relationship between autobiographical memory and laboratory-measures of episodic memory is complicated and these processes might be differentially affected by ageing (e.g. Diamond et al., [2020]. Different patterns of recollection for matched real-world and laboratory-based episodes in younger and older adults. Cognition, 202, 104309.). Here, we examine whether the ability to recall one’s own visual perspective relates to richness of autobiographical recall, and how this relationship is affected by age. Memory of perspective at encoding, was assessed in younger (18–35 years) and older adults (65–85 years). Participants, wearing head cameras, viewed arrays of objects. Later they were asked which images represented earlier scenes, and if the image was taken from their perspective (i.e. from their camera). Performance was compared with autobiographical memory. Accuracy in identifying their own perspective correlated with autobiographical scores. Age-group was a moderating factor in this relationship. Subsequently, new participants encoded photographs of objects and were later asked whether they recognised the images. Visual perspective was manipulated in these photographs. In this task there was no relationship between performance and autobiographical memory. In younger adults only 3-D encoding of scenes relates directly to autobiographical memory but ability to complete these two tasks appears to operate independently in the older group.

Memory-Augmented Image Captioning

Current deep learning-based image captioning systems have been proven to store practical knowledge with their parameters and achieve competitive performances in the public datasets. Nevertheless, their ability to access and precisely manipulate the mastered knowledge is still limited. Besides, providing evidence for decisions and updating memory information are also important yet under explored. Towards this goal, we introduce a memory-augmented method, which extends an existing image caption model by incorporating extra explicit knowledge from a memory bank. Adequate knowledge is recalled according to the similarity distance in the embedding space of history context, and the memory bank can be constructed conveniently from any matched image-text set, e.g., the previous training data. Incorporating such non-parametric memory-augmented method to various captioning baselines, the performance of resulting captioners imporves consistently on the evaluation benchmark. More encouragingly, extensive experiments demonstrate that our approach holds the capability for efficiently adapting to larger training datasets, by simply transferring the memory bank without any additional training.

MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection

State-of-the-art video object detection methods maintain a memory structure, either a sliding window or a memory queue, to enhance the current frame using attention mechanisms. However, we argue that these memory structures are not efficient or sufficient because of two implied operations: (1) concatenating all features in memory for enhancement, leading to a heavy computational cost; (2) frame-wise memory updating, preventing the memory from capturing more temporal information. In this paper, we propose a multi-level aggregation architecture via memory bank called MAMBA. Specifically, our memory bank employs two novel operations to eliminate disadvantages of existing methods: (1) light-weight key-set construction which can significantly reduce the computational cost; (2) fine-grained feature-wise updating strategy which enables our method to utilize knowledge from the whole video. To better enhance features from complementary levels, i.e., feature maps and proposals, we further propose a generalized enhancement operation (GEO) to aggregate multi-level features in a unified manner. We conduct extensive evaluations on the challenging ImageNetVID dataset. Compared with existing state-of-the-art methods, our method achieves superior performance in terms of both speed and accuracy. More remarkably, MAMBA achieves mAP of 83.7%/84.6% at 12.6/9.1 FPS with ResNet-101.

Decoupled and Memory-Reinforced Networks: Towards Effective Feature Learning for One-Step Person Search

The goal of person search is to localize and match query persons from scene images. For high efficiency, one-step methods have been developed to jointly handle the pedestrian detection and identification sub-tasks using a single network. There are two major challenges in the current one-step approaches. One is the mutual interference between the optimization objectives of multiple sub-tasks. The other is the sub-optimal identification feature learning caused by small batch size when end-to-end training. To overcome these problems, we propose a decoupled and memory-reinforced network (DMRNet). Specifically, to reconcile the conflicts of multiple objectives, we simplify the standard tightly coupled pipelines and establish a deeply decoupled multi-task learning framework. Further, we build a memory-reinforced mechanism to boost the identification feature learning. By queuing the identification features of recently accessed instances into a memory bank, the mechanism augments the similarity pair construction for pairwise metric learning. For better encoding consistency of the stored features, a slow-moving average of the network is applied for extracting these features. In this way, the dual networks reinforce each other and converge to robust solution states. Experimentally, the proposed method obtains 93.2% and 46.9% mAP on CUHK-SYSU and PRW datasets, which exceeds all the existing one-step methods.