Long-term Memory Capabilities Research Articles

Dynamic learning of flue gas desulfurization process using deep LSTMs neural network

SO2 emissions are known to pose great harm to both human health and atmospheric air, and flue gas generated from coal-fueled power plant is the prime source of sulfur dioxide. For this reason, flue gas desulfurization (FGD) technology has found wide applications in most coal-fired power stations. Correctly describe the dynamic behavior of an FGD process is the precondition of controlling it effectively. However, FGD process modeling is by no means an easy task, as the underlying process dynamics are highly nonlinear in nature, meanwhile time-delay effect is significant therein. Long short-term memory (LSTM) network possesses remarkable long-term memory capability, hence it is anticipated to have a powerful identification capability. In this paper, the connection between deep learning and system identification is established, further a unidirectional/bidirectional LSTM deep network is designed and employed to identify a real FGD process. Simulation results clearly demonstrate the effectiveness of deep learning-based identification approach, and the superiority of deep LSTMs over other conventional identification models is also verified.

Fully Ferroelectric-FETs Reservoir Computing Network for Temporal and Random Signal Processing

Reservoir computing (RC), a derivation of recurrent neural networks (RNNs), is an energy-efficient computational framework suitable for temporal signal processing. Owing to the short-term and long-term memory capability, the ferroelectric field-effect transistor (FeFET) is regarded as a promising hardware component for implementing RC networks. This article aims to optimize the fully FeFETs RC network by evaluating the recognition accuracy in various classification tasks, which includes the operating voltage sequence, device numbers as well as connection methods. The physical random telegraph noise (RTN), working as an ideal temporal and random signal, is investigated and extended by using the optimized fully FeFET RC network, resulting in a rapid time constant extraction method. Our findings may provide the broad potential for hardware security and cyber security based on the fully FeFET RC network.

A Novel Fault Diagnosis Method Based on SWT and VGG-LSTM Model for Hydraulic Axial Piston Pump

Since the hydraulic axial piston pump is the engine that drives hydraulic transmission systems, it is widely utilized in aerospace, marine equipment, civil engineering, and mechanical engineering. Operating safely and dependably is crucial, and failure poses a major risk. Hydraulic axial piston pump malfunctions are characterized by internal concealment, challenging self-adaptive feature extraction, and blatant timing of fault signals. By completely integrating the time-frequency feature conversion capability of synchrosqueezing wavelet transform (SWT), the feature extraction capability of VGG11, as well as the feature memory capability of the long short-term memory (LSTM) model, a novel intelligent fault identification method is proposed in this paper. First, the status data are transformed into two dimensions in terms of time and frequency by using SWT. Second, the depth features of the time–frequency map are obtained and dimensionality reduction is carried out by using the deep feature mining capability of VGG11. Third, LSTM is added to provide the damage identification model for long-term memory capabilities. The Softmax layer is utilized for the intelligent evaluation of various damage patterns and health state. The proposed method is utilized to identify and diagnose five typical states, including normal state, swash plate wear, sliding slipper wear, loose slipper, and center spring failure, based on the externally observed vibration signals of a hydraulic axial piston pump. The results indicate that the average test accuracy for five typical state signals reaches 99.43%, the standard deviation is 0.0011, and the average test duration is 2.675 s. The integrated model exhibits improved all-around performance when compared to LSTM, LeNet-5, AlexNet, VGG11, and other typical models. The proposed method is validated to be efficient and accurate for the intelligent identification of common defects of hydraulic axial piston pumps.

Bioartificial Synapses for Neuromorphic Computing

As a promising solution for overcoming the bottlenecks of traditional von Neumann computers, the hardware implementation of neuromorphic computing has attracted increasing interest. High-performance artificial synapses are the basic units of brain-like chips and are important for achieving efficient neuromorphic calculations. This paper reports the fabrication of Al/chitosan (CS)/graphene oxide (GO)/indium tin oxide (ITO) artificial synapses. The electronic insulation and proton conduction properties of CS enable it to conduct electricity alone and accelerate the movement of oxygen vacancies in GO. In particular, an experimental device successfully simulated the Pavlov associative memory experiment and was made to exhibit both short-term and long-term memory capabilities by modifying the external stimuli. This device provides a possible avenue to realize neuromorphic engineering on the basis of biomemristors.

Organoid intelligence (OI) – The ultimate functionality of a brain microphysiological system

Understanding brain function remains challenging as work with human and animal models is complicated by compensatory mechanisms, while in vitro models have been too simple until now. With the advent of human stem cells and the bioengineering of brain microphysiological systems (MPS), understanding how both cognition and long-term memory arise is now coming into reach. We suggest combining cutting-edge AI with MPS research to spearhead organoid intelligence (OI) as synthetic biological intelligence. The vision is to realize cognitive functions in brain MPS and scale them to achieve relevant short- and long-term memory capabilities and basic information processing as the ultimate functional experimental models for neurodevelopment and neurological function and as cell-based assays for drug and chemical testing. By advancing the frontiers of biological computing, we aim to (a) create models of intelligence-in-a-dish to study the basis of human cognitive functions, (b) provide models to advance the search for toxicants contributing to neurological diseases and identify remedies for neurological maladies, and (c) achieve relevant biological computational capacities to complement traditional computing. Increased understanding of brain functionality, in some respects still superior to today's supercomputers, may allow to imitate this in neuromorphic computer architectures or might even open up biological computing to complement silicon computers. At the same time, this raises ethical questions such as where sentience and consciousness start and what the relationship between a stem cell donor and the respective OI system is. Such ethical discussions will be critical for the socially acceptable advance of brain organoid models of cognition.

Frequency-Dependent Synapse Weight Tuning in 1S1R with a Short-Term Plasticity TiOx-Based Exponential Selector.

Short-term plasticity (STP) is an important synaptic characteristic in the hardware implementation of artificial neural networks (ANN), as it enables the temporal information processing (TIP) capability. However, the STP feature is rather challenging to reproduce from a single nonvolatile resistive random-access memory (RRAM) element, as it requires a certain degree of volatility. In this work, a Pt/TiOx/Pt exponential selector is introduced not only to suppress the sneak current but also to enable the TIP feature in a one selector-one RRAM (1S1R) synaptic device. Our measurements reveal that the exponential selector exhibits the STP characteristic, while a Pt/HfOx/Ti RRAM enables the long-term memory capability of the synapse. Thereafter, we experimentally demonstrated pulse frequency-dependent multilevel switching in the 1S1R device, exhibiting the TIP capability of the developed 1S1R synapse. The observed STP of the selector is strongly influenced by the bottom metal-oxide interface, in which Ar plasma treatment on the bottom Pt electrode resulted in the annihilation of the STP feature in the selector. A mechanism is thus proposed to explain the observed STP, using the local electric field enhancement induced at the metal-oxide interface coupled with the drift-diffusion model of mobile O2- and Ti3+ ions. This work therefore provides a reliable means of producing the STP feature in a 1S1R device, which demonstrates the TIP capability sought after in hardware-based ANN.

TCAD Modeling of Resistive-Switching of HfO2 Memristors: Efficient Device-Circuit Co-Design for Neuromorphic Systems

In neuromorphic computing, memristors (or “memory resistors”) have been primarily studied as key elements in artificial synapse implementations, where the memristor provides a variable weight with intrinsic long-term memory capabilities, based on its modifiable resistive-switching characteristics. Here, we demonstrate an efficient methodology for simulating resistive-switching of HfO2 memristors within Synopsys TCAD Sentaurus—a well established, versatile framework for electronic device simulation, visualization and modeling. Kinetic Monte Carlo is used to model the temporal dynamics of filament formation and rupture wherein additional band-to-trap electronic transitions are included to account for polaronic effects due to strong electron-lattice coupling in HfO2. The conductive filament is modeled as oxygen vacancies which behave as electron traps as opposed to ionized donors, consistent with recent experimental data showing p-type conductivity in HfOx films having high oxygen vacancy concentrations and ab-initio calculations showing the increased thermodynamic stability of neutral and charged oxygen vacancies under conditions of electron injection. Pulsed IV characteristics are obtained by inputting the dynamic state of the system—which consists of oxygen ions, unoccupied oxygen vacancies, and occupied oxygen vacancies at various positions—into Synopsis TCAD Sentaurus for quasi-static simulations. This allows direct visualization of filament electrostatics as well as the implementation of a nonlocal, trap-assisted-tunneling model to estimate current-voltage characteristics during switching. The model utilizes effective masses and work functions of the top and bottom electrodes as additional parameters influencing filament dynamics. Together, this approach can be used to provide valuable device- and circuit-level insight, such as forming voltage, resistance levels and success rates of programming operations, as we demonstrate.

Intelligent Fault Diagnosis for Chemical Processes Using Deep Learning Multimodel Fusion.

Deep learning technology has been widely used in fault diagnosis for chemical processes. However, most deep learning technologies currently adopted only use a single network stack or a certain network stack with multilayer perceptron (MLP) behind it. Compared with traditional fault diagnosis technologies, this method has made progress in both the diagnosis accuracy and speed, but due to the limited performance of a single network, the accuracy or speed cannot meet the requirements to the greatest extent. In order to overcome such problems, this article proposes a fault diagnosis method using deep learning multimodel fusion. Different from previous deep learning diagnosis methods, this method uses long short-term memory (LSTM) and convolutional neural network (CNN) to extract features separately. The extracted features are then fused and MLP is taken as the input for further feature compression and extraction, and finally the diagnosis results will be obtained. LSTM has long-term memory capabilities, the extracted features have temporal characteristics, and CNNs have a good effect on the extraction of spatial features. The proposed method integrates these two aspects for diagnosis such that the features finally extracted by the network have both spatial and temporal characteristics, thereby improving the network's diagnostic performance. Finally, a TE chemical process and an industrial coking furnace process are taken for simulation testing. It is proved that the performance of this method is superior to existing deep learning fault diagnosis methods with simple sequential stacking for unilateral feature extraction.

Correlations Between Mutant Huntingtin Aggregates and Behavioral Changes in R6/1 Mice.

Huntington's disease (HD) is a neurodegenerative disorder caused by the expansion of the trinucleotide CAG in the HD gene. While the presence of nuclear aggregates of mutant huntingtin (mHtt) in neurons is a hallmark of HD, the reason behind its toxicity remains elusive. The present study was conducted to assess a correlation between the number of mHtt aggregates and the severity of HD symptoms in R6/1 mice. We investigated correlations between behavioral deficits and the level of nuclear mHtt aggregates in different neuroanatomical regions in 3-month-old R6/1 mice, the age at which a large variability of symptom severity between animals has been observed. R6/1 mice were deficient in instinctive and anxiety related behaviors as well as long-term memory capabilities. Significant differences were also found between the sexes; female transgenic mice displayed less severe deficits than males. While the level of mHtt aggregates was correlated with the severity of HD phenotypes in most regions of interest, an opposite relationship also was found for some other regions examined. The obtained results suggest harmful and region-specific roles of mHtt aggregates in HD symptoms.

The effect of slightly warm temperature on work performance and comfort in open-plan offices - a laboratory study.

The aim of the study was to determine the effect of a temperature of 29°C on performance in tasks involving different cognitive demands and to assess the effect on perceived performance, subjective workload, thermal comfort, perceived working conditions, cognitive fatigue, and somatic symptoms in a laboratory with realistic office environment. A comparison was made with a temperature of 23°C. Performance was measured on the basis of six different tasks that reflect different stages of cognitive performance. Thirty-three students participated in the experiment. The exposure time was 3.5h in both thermal conditions. Performance was negatively affected by slightly warm temperature in the N-back working memory task. Temperature had no effect on performance in other tasks focusing on psychomotor, working memory, attention, or long-term memory capabilities. Temperature had no effect on perceived performance. However, slightly warm temperature caused concentration difficulties. Throat symptoms were found to increase over time at 29°C, but no temporal change was seen at 23°C. No effect of temperature on other symptoms was found. As expected, the differences in thermal comfort were significant. Women perceived a temperature of 23°C colder than men.

Long-term retention of visual tasks by two species of emydid turtles, Pseudemys nelsoni and Trachemys scripta.

Long-lived species are expected to have long-term memory capabilities. In this study we tested nine Florida Red-bellied Cooters (Pseudemys nelsoni) on their retention for both a procedural food acquisition task and visual discrimination task learned in a previous experiment. The turtles were tested and retrained after two months, after another 7.5 months, and finally after 36 months of no interaction with the test apparatus during the intervening periods. Turtles retained memory for the choice task and needed little retraining throughout. Furthermore, in a different visual discrimination task, both P. nelsoni and Trachemys scripta turtles showed 100% retention after 3.5 months of no testing. Odor-controlled tests confirmed that turtles were using visual cues to solve the task. Thus, in a laboratory context turtles demonstrate long-term memory of visual discrimination tasks, which relates to apparent abilities in natural environments.

Effects of postnatal treadmill exercise on apoptotic neuronal cell death and cell proliferation of maternal-separated rat pups

Early adverse experiences resulting from maternal separation may lead to neuronal cell death and they can eventually cause memory impairment. In this study, we investigated the effects of postnatal treadmill exercise on the long-term memory capability, spatial learning ability, and depressive state, on the 5-hydroxytryptamine (5-HT, serotonin) synthesis and tryptophan hydroxylase (TPH) expression in the dorsal raphe nuclei, and on the apoptosis and cell proliferation in the hippocampal dentate gyrus of rat pups following maternal separation. The rat pups in the maternal separation groups were separated from their respective mothers on the postnatal day 14. The rat pups in the maternal separation group showed depressive state with deceased memory capability and learning ability compared to the rat pups in the maternal care group. Postnatal treadmill exercise increased memory capability and learning ability and alleviated depressive state of the rat pups in the maternal separation group. The 5-HT synthesis and TPH expression in the dorsal raphe nuclei and cell proliferation in the hippocampal dentate gyrus were significantly decreased in the maternal-separated rat pups, and postnatal treadmill exercise increased 5-HT synthesis, the TPH expression, and the cell proliferation. In contrast, apoptotic neuronal cell death in the hippocampal dentate gyrus was significantly increased in the maternal-separated rat pups, and postnatal treadmill exercise suppressed the maternal separation-induced apoptosis. The present results demonstrated that postnatal treadmill exercise alleviated maternal separation-induced depression with decrease of memory capability and learning ability, by suppressing apoptotic neuronal cell death and by enhancing cell proliferation.

The memory remains: long-term vocal recognition in Australian sea lions

The ability to recognize other individuals plays an important role in mediating social interactions. As longitudinal studies are challenging, there is only limited evidence of long-term memory of individuals and concepts in mammals. We examined the ability of six wild Australian sea lions to discriminate between the voice of their mother and another adult female, both while they were dependent on their mother and when they were independent, 2 years after weaning. Here, we show that even after a long period of independence, juveniles retain the ability to identify their mother's voice. Both when dependent and independent, animals showed stronger responses to maternal calls than to the calls of another female. This demonstration of recognition provides rare evidence of the long-term memory capabilities of wild mammals.

Category-specific organization of prefrontal response-facilitation during priming

Perceptual priming is a fundamental long-term memory capability that allows more efficient and faster responding to a stimulus as a result of prior exposure to that stimulus. The two major components of priming are facilitated response expression and improved stimulus identification. Recent fMRI studies have identified a potential neural correlate for response-facilitation, namely the repetition-related activity decrements in prefrontal cortex that are linearly correlated with improvements in reaction times. However, the neural processes underlying such response-facilitation are still unclear. They could be stimulus-selective stimulus-response mapping processes or general response-learning mechanisms. In human imaging studies, behavioral priming has been associated with decreased hemodynamic responses in prefrontal cortex and in category-specific brain regions of the ventral visual stream. Currently, it is unclear whether priming-related response decreases in prefrontal cortex are also category-specific. In this fMRI study, 16 subjects performed a repetition priming task employing category-specific identification judgments on pictures of faces (male/female judgment), scenes (indoor/outdoor judgment) and scrambled ‘noise’ pictures (simple button press). The repeated faces and scenes were identified faster than first presentations indicating priming. Hemodynamic decreases for repetitions were observed in a left inferior (near Brodman Area, BA, 44) and middle frontal (BA8) region of the prefrontal cortex, in category-specific areas of the ventral stream (bilateral fusiform face area, FFA, parahippocampal place area, PPA), and two category-specific right lateral occipital (LOC) regions. Hemodynamic increases for repetitions appeared in the caudate and cerebellum. However, the prefrontal areas were the only regions that showed a correlation between repetition-related reaction time improvement and hemodynamic decrease. Importantly, the correlations were category-specific in their relationship to reaction time improvement: in the left inferior frontal cortex the correlations were specific for scenes whereas and in left middle frontal gyrus they were specific for faces. There were no correlations between behavior and repetition suppression for both LOC regions, FFA and PPA. These data reveal that response-facilitation in prefrontal cortex is organized according to stimulus-properties, compatible with learning of stimulus-response mapping rather than response learning in general.

Early, partly anticipatory, neural oscillations during identification set the stage for priming

Perceptual priming is a fundamental long-term memory capability by which exposure to a stimulus improves later perceptual processing of that stimulus. A widespread hypothesis is that priming is the later result of perceptual learning during stimulus identification. Testing this hypothesis involves isolating priming without explicit memory, and appropriately measuring brain activity during initial experimental exposure to assess whether brain activity related to identification differs as a function of later priming. Here, we show, using magnetoencephalography (MEG), that words primed in a later test are distinguished from unprimed words at initial exposure by (a) more specific responses in perceptual brain areas, indicated by an early (within 240 ms after word onset) decrease in amplitude but increase in phase alignment of beta and gamma oscillations, and (b) improved coordination of responses across perceptual and higher brain areas in the same time window, indicated by an increase in interareal phase synchrony of alpha oscillations. The increase in interareal phase synchrony partly started already in the pre-stimulus period, approximately 60–80 ms prior to word onset, showing that the improved coordination of responses across areas was partly anticipatory. The anatomy and early timing of these patterns reveal a neural link between identification and long-term memory. The pre-stimulus findings additionally show that priming is related to the stimulus-specific anticipatory state of visual identification areas at initial exposure.

Self-Organizing Probability State Variable Parameter Search Algorithms for Systems that Must Avoid High-Penalty Operating Regions

Self-organizing probability state variable (PSV) parameter search algorithms possessing long-term memory have been formulated to cope with systems that must avoid high performance-penalty operating regions. The information gained from all previous experiments is efficiently encoded in multivariate probability distribution functions (pdf's). This long-term memory capability enables the PSV algorithms to avoid effectively future experiments in high penalty regions. The systems considered are resource-limited, and catastrophic failure may occur if parameter values lying in high penalty regions are implemented. Those cases in which the high penalty regions are not known in advance were investigated. The PSV algorithms have the capability of adaptively learning the location and hypervolume of these regions as the search proceeds. The algorithms are explicitly guided in their internal strategies as a function of the remaining system resources and the updated probability distribution functions. Clustering analysis is used both in the discovery of new operating regions and for updating the pdf's. As a by-product of this research, clustering was also investigated as a presearch scheme. It is shown that this procedure has great promise as a means of assessing the complexity of an optimization problem. Experimental results are presented to demonstrate the utility of the self-organizing PSV search algorithms.