Memory Layer Research Articles

A neural network for recognizing and classifying images at the boundaries of several classes

The neural network belongs to the computer systems of computing technology and artificial intelligence in the field of building automated systems for recognizing and classifying images at the border of several classes, which is made with the ability to perform diagnostics and image recognition at the borders of two, three or more classes due to the fact that the layer schemes for selection of one, two or more single signals from the outputs of the layer of output neurons , , ¼, , ¼, . at each moment of time remembers the number of non-zero signals at the output of neurons of the output layer, and the second layer of output neurons , , ¼, , ¼, remembers neurons of the output layer that have non-zero signals at their outputs. The technical result achieved by this neural network is an increase in the number of classes that can be recognized and the ability to recognize images that are at the same Hamming distance from two, three or more reference images stored in the connection weights of the neurons of the memory layer. haunting An experimental evaluation of the method and neural network for solving the problems of matching digital images was carried out. The neural network is implemented as a software utility Il.: 2; Bibliogr.: 15 titles.

Models for diagnosing interactive computer networks at the structural and logical level

Methods of diagnosing interactive computer networks at the structural and logical level are developed. Methods and procedures for synthesizing one-dimensional and two-dimensional networks with distributed configuration control are developed. The existing methods of diagnosing interactive computer networks at the structural and logical level have been analyzed. Diagnostics models for interactive computer networks at the structural and logical level have been created. According to the results of diagnostic experiments the conclusion about testability of the networks considered in the paper is made. 4 figures, 7 bibliographic sources. Figs.: 4. Refs.: 14 titles.

Atrial lead system for enhanced P-wave recording: A comparative study on optimal leads using gradient boosting and deep learning algorithms

Morphological characteristics of the P-wave in an Electrocardiogram (ECG) provide insights into interatrial and atrioventricular (AV) conduction pathologies. However, accurate diagnosis through visual inspection is hindered by the P-wave’s diminutive size and susceptibility to noise. Furthermore, conventional 12-lead systems often fail to capture these elusive P-waves. Consequently, our study describes a new Atrial Lead System (ALS) aimed at enhancing P-wave signal strength and ranking leads using advanced Gradient-Boosting (GB) and Deep Learning (DL) algorithms. ECG data from 75 healthy volunteers (mean age: 45 ± 17.2 years; 40 % women) were recorded using ALS and specially designed optimal bipolar leads to improve atrial activity. Employing Gradient Boosting Classifier (GBC), Extreme Gradient Boosting Classifier (XGBoost), Light Gradient Boosting Machine (LightGBM), and a synergistic One-dimensional Convolutional Neural Network (1D-CNN) with Bidirectional Long Short-Term Memory (Bi-LSTM) layers, we assessed and ranked optimal bipolar leads based on P-wave parameters, indices, and AV ratios. Notably, AL-I and AL-II exhibited significantly higher median amplitudes, RMS, and area for the recorded P-wave compared to other leads (p < 0.001). Evaluations using LightGBM, GBC, XGBoost, and 1D-CNN+Bi-LSTM models identified P-lead, AL-II, and AL-I as the top three leads, with accuracies of 0.96, 0.95, and 0.94; 0.96, 0.94, and 0.94; 0.96, 0.95, and 0.94; and 0.96, 0.94, and 0.93, respectively. These findings underscore the efficacy of ALS in elevating P-wave signal strength and AV ratios, suggesting its potential as a valuable tool for improved clinical screening and diagnosis of atrial arrhythmias.

Estimating the Health State of Lithium-Ion Batteries Using an Adaptive Gated Sequence Network and Hierarchical Feature Construction

State of health (SOH) estimation plays a vital role in ensuring the safe and stable operation of lithium-ion battery management systems (BMSs). Data-driven methods are widely used to estimate SOH; however, existing methods often suffer from fixed or excessively high feature dimensions, impacting the model’s adjustability and applicability. This study first proposed a layered knee point strategy based on the charging voltage curve, which reduced the complexity of feature extraction. Then, a new hybrid framework called the adaptive gated sequence network (AGSN) model was proposed. This model integrated independently recurrent neural network (IndRNN) layers, active state tracking long short-term memory (AST-LSTM) layers, and adaptive gating mechanism (AGM) layers. By integrating a multi-layered structure and an adaptive gating mechanism, the SOH prediction performance was significantly improved. Finally, batteries under different operating conditions were tested using the NASA battery dataset. The results show that the AGSN model demonstrated higher accuracy and robustness in battery SOH estimation, with estimation errors consistently within 1%.

Improving decryption quality of optical chaos communication using neural networks.

Optical chaos communication is a promising secure transmission technique because of the advantages of high speed and compatibility with existing fiber-optic systems. The deterioration of chaotic synchronization quality caused by fiber optic transmission impairments affects the quality of recovery of information, especially high-order modulated signals. Here, we demonstrate that the use of a convolutional neural network (CNN) with a bidirectional long short-term memory (LSTM) layer can reduce the decryption BER in an optical chaos communication system based on common-signal-induced semiconductor laser synchronization. The performance of a neural network is investigated as a function of network parameters and chaos synchronization coefficient. Experimental results show that the BER of 16-ary quadrature-amplitude-modulation (16QAM) signal after 100-km fiber transmission is decreased from 3.05 × 10-2 to below the soft-decision forward-error-correction (SD-FEC) threshold of 2.0 × 10-2.

Machine learning-based predictive control of an electrically-heated steam methane reforming process

Hydrogen plays a crucial role in improving sustainability and offering a clean and efficient energy carrier that significantly reduces greenhouse gas emissions. However, the primary method of industrial hydrogen production, steam methane reforming (SMR), relies on the combustion of hydrocarbons as the heating source for the reforming reactions, resulting in significant carbon emissions. To address this issue, an experimental setup of an electrically-heated steam methane reformer (e-SMR) has been constructed at UCLA, and a lumped first-principle dynamic process model was built based on parameters estimated from the experimental data in a previous study. Subsequently, the first-principle dynamic process model was implemented into the computational model predictive control (MPC) scheme, successfully driving the hydrogen production rate to the desired setpoint. While these works are important and pave the way for developing MPC for large-scale e-SMR processes, the first-principle process model may not accurately reflect the actual process behavior, particularly as the process behavior changes with time. Therefore, the development and establishment of an adaptive data-driven approach for implementing model predictive control in the e-SMR process is necessary. To address this need, the present work investigates the construction of recurrent neural network (RNN) models for an e-SMR process in-depth, utilizing data from an experimentally-validated first-principle model. Specifically, a long short-term memory (LSTM) layer was utilized in the RNN model to effectively capture the complex correlations present in long-term sequential data. Subsequently, this LSTM-based RNN process model was employed to design an MPC, and its performance was evaluated through comparison with proportional–integral (PI) control. To address potential disturbances and variability in a typical e-SMR process, three distinct approaches were developed: MPC with an integrator, MPC with real-time online retraining (transfer learning), and offset-free MPC. These approaches effectively eliminated the offset caused by disturbances. Overall, this study underscores the effectiveness of utilizing RNN models to capture process dynamics in an experimental e-SMR process. It also outlines strategies for employing RNN-based control and multiple approaches to address disturbances in general processes with partially infrequent and delayed measurement feedback. This approach is particularly valuable in scenarios where developing first-principle models for a new process may be challenging.

Controlled Migration of Lithium Cations by Diamine Bridges in Water-Processable Polymer-Based Solid-State Electrolyte Memory Layers for Organic Synaptic Transistors.

Synaptic transistors require sufficient retention (memory) performances of current signals to exactly mimic biological synapses. Ion migration has been proposed to achieve high retention characteristics but less attention has been paid to polymer-based solid-state electrolytes (SSEs) for organic synaptic transistors (OSTRs). Here, OSTRs with water-processable polymer-based SSEs, featuring ion migration-controllable molecular bridges, which are prepared by reactions of poly(4-styrenesulfonic acid) (PSSA), diethylenetriamine (DETA), and lithium hydroxide (LiOH) are demonstrated. The ion conductivity of PSSA:LiOH:DETA (1:0.4:X, PLiD) films is remarkably changed by the molar ratio (X) of DETA, which is attributed to the extended distances between the PSSA chains by the DETA bridges. The devices with the PLiD layers deliver noticeably changed hysteresis reaching an optimum at X = 0.2, leading to the longest retention of current signals upon single/double pulses. The long-term potentiation test confirms that the present OSTRs can gradually build up the postsynaptic current by gate pulses of -2V, while the long-term depression can be adjusted by varying the depression gate pulses (≈0.2-1.2V). The artificial neural network simulations disclose that the present OSTRs with the ion migration-controlled PLiD layers can perform synaptic processes with an accuracy of ≈96%.

Advancements in Real-Time Human Activity Recognition via Innovative Fusion of 3DCNN and ConvLSTM Models

Object detection (OD) is a computer vision procedure for locating objects in digital images. Our study examines the crucial need for robust OD algorithms in human activity recognition, a vital domain spanning human-computer interaction, sports analysis, and surveillance. Nowadays, three-dimensional convolutional neural networks (3DCNNs) are a standard method for recognizing human activity. Utilizing recent advances in Deep Learning (DL), we present a novel framework designed to create a fusion model that enhances conventional methods at integrates three-dimensional convolutional neural networks (3DCNNs) with Convolutional Long-Short-Term Memory (ConvLSTM) layers. Our proposed model focuses on utilizing the spatiotemporal features innately present in video streams. An important aspect often missed in existing OD methods. We assess the efficacy of our proposed architecture employing the UCF-50 dataset, which is well-known for its different range of human activities. In addition to designing a novel deep-learning architecture, we used data augmentation techniques that expand the dataset, improve model robustness, reduce overfitting, extend dataset size, and enhance performance on imbalanced data. The proposed model demonstrated outstanding performance through comprehensive experimentation, achieving an impressive accuracy of 98.11% in classifying human activity. Furthermore, when benchmarked against state-of-the-art methods, our system provides adequate accuracy and class average for 50 activity categories.

Deep learning model predictive control of a high-density polyethylene reactor with a physics-guided sequence-to-sequence model with memory

In chemical process industry, the input-output data of processes display complex nonlinear dynamics and strong influences of unobserved hidden states. Their behavior must be modeled using nonlinear time series with an observer-predictor structure. To address this, a sequence-to-sequence model with a memory layer was proposed for a high-density polyethylene slurry reactor. The memory layer retains the chronological contribution of the observer and predictor and effectively captures the lengthy time response. A physics-guided approach was adopted to ensure the directional consistency between input and output variables in key control loops. In this way, a deep learning model can be obtained with historical data and there is no need for plant tests. The resulting model can be used in a nonlinear model predictive control system that not only quickly navigates different grade transitions but also provides steady-state control even though key internal state variables such as catalyst activity change.

Enhancing corrosion detection in pulsed eddy current testing systems through autoencoder-based unsupervised learning

Pulsed Eddy Current Testing (PECT) stands out as an advanced method in Non-Destructive Testing due to its extensive spectrum characteristics in comparison to traditional ECT techniques, making it exceptionally suitable for identifying corrosion. Nonetheless, the analysis of PECT signals for corrosion detection poses a challenge due to the transient nature of these signals and the impact of sensor lift-off effects. As a result, conventional methods are facing hurdles in dealing with corrosion signals of poor quality. In this study, the challenge is addressed by employing unsupervised learning methods utilizing an autoencoder neural network. This autoencoder integrates Long Short-Term Memory and 1D convolutional layers, acquiring the underlying features of normal PECT signals from non-corrosive regions. Significantly, the model is trained exclusively on this normal data, thereby obviating the necessity for pre-existing corrosion information. Through learning the inherent structure of normal signals, the model can detect anomalies in unseen data, potentially indicating corrosion. The unsupervised framework presents several advantages, such as reducing reliance on prior corrosion knowledge, mitigating inherent noise, and addressing sensor lift-off effects. Experimental results were conducted to compare with traditional methods like the lift-off of intersection and lift-off compensation methods. This approach resulted in a significant improvement in SNR, ranging from 100 % to 200 %, thus facilitating more robust NDT applications employing smart PECT sensors empowered by unsupervised learning techniques.

Enhancing Big Data Performance Through Graph Coloring-Based Locality of Reference

Efficiency is a crucial factor when handling the retrieval and storage of data from vast amounts of records in a Big Data repository. These systems require a subset of data that can be accommodated within the combined physical memory of a cluster of servers. It becomes impractical to analyze all of the data if its size exceeds the available memory capacity. Retrieving data from virtual storage, primarily hard disks, is significantly slower compared to accessing data from main memory, resulting in increased access time and diminished performance. To address this, a proposed model aims to enhance performance by identifying the most suitable data locality structure within a big data set and reorganizing the data schema accordingly; by locality, it has been referred to as a particular access pattern. This allows transactions to be executed on data residing in the fastest memory layer, such as cache, main memory, or disk cache

Automobile Maintenance Prediction Using Integrated Deep Learning and Geographical Information System

In the industry, predictive maintenance, or PdM, has gained popularity as a means of lowering maintenance costs and achieving efficient operational oversight. The essence of PdM is to anticipate the occurrence of the next breakdown, allowing for the timely scheduling of maintenance activities prior to its actual manifestation. The objective of this work is to develop a Time-Between-Failure (TBF) forecasting framework using a data-driven methodology. Forecasting the Remaining Usable Life (RUL) is a vital issue in PdM. The objective of the current research is to include Geographic Information Systems (GIS) data into TBF (Time Between Failures) modeling and investigate their influence on automotive TBF via the use of deep learning techniques. Initially, an information fusion technique has been investigated to address the disparity in information category and sample rate between the maintenance information and GIS information. The Cox Proportional Hazard Model (Cox PHM) has been employed using the combined information to create the Health Index (HI). This research introduces an Integrated Deep Learning (IDL) architecture that aims to provide a unique perspective on PdM. This design consists of an input layer, a Long-Short Term Memory (LSTM) layer, a Dropout layer (DO) followed by another LSTM layer, a hidden layer, and an output layer. The Genetic Algorithm (GA) has been employed to discover the optimal number of periods and batch size for the design. The activation function is utilized after the output level and the DO ratio, and the optimization method enhances the loss function established using Grid Searching (GS). Research utilizing an extensive record of vehicle upkeep from a fleet firm demonstrated the efficiency of the suggested method and the influence of GIS parameters on the investigated automobiles.

Multi-task deep learning model for quantitative volatile organic compounds analysis by feature fusion of electronic nose sensing

In exploring pattern recognition for electronic noses via deep neural networks, traditional networks encounter key challenges, such as low training efficiency, and neglect of spatial-temporal attributes of gas sensor response sequences. In this study, an unmanned gas-sensing test system is used to generate a large dataset to ensure robust model training. The Gramian angular field-Markov transition field is utilized to convert time sequences into images. Using advanced image processing tools, the images are then subsequently compressed with data augmentation. This method fully preserves temporal and spatial features within the sequences, thus enhancing model performance. Furthermore, the proposed multi-task learning (MTL) framework competently performs simultaneous classification and regression tasks. The primary component of the MTL network, majorly constituted of convolutional neural networks, emphasizes the spatial features of the sequences. The integration of a long short-term memory layer ensures the preservation of temporal feature analysis of the input data, thereby enhancing predictive performance. When images are compressed to only 3.9 % of the original data, substantial information can still be preserved. Subsequently, the model trained by such compressed images attains an accuracy of 95.31 % and an R2 score of 0.9510 for classification and regression tasks, respectively. Our work reveals the remarkable potential of integrating temporal and spatial features in pattern recognition, promoting the potency of multi-tasking deep learning networks in electronic nose technology.

HumourHindiNet: Humour detection in Hindi web series using word embedding and convolutional neural network

Humour is a crucial aspect of human speech, and it is, therefore, imperative to create a system that can offer such detection. While data regarding humour in English speech is plentiful, the same cannot be said for a low-resource language like Hindi. Through this article, we introduce two multimodal datasets for humour detection in the Hindi web series. The dataset was collected from over 500 minutes of conversations amongst the characters of the Hindi web series Kota-Factory and Panchayat . Each dialogue is manually annotated as Humour or Non-Humour. Along with presenting a new Hindi language-based Humour detection dataset, we propose an improved framework for detecting humour in Hindi conversations. We start by preprocessing both datasets to obtain uniformity across the dialogues and datasets. The processed dialogues are then passed through the Skip-gram model for generating Hindi word embedding. The generated Hindi word embedding is then passed onto three convolutional neural network (CNN) architectures simultaneously, each having a different filter size for feature extraction. The extracted features are then passed through stacked Long Short-Term Memory (LSTM) layers for further processing and finally classifying the dialogues as Humour or Non-Humour. We conduct intensive experiments on both proposed Hindi datasets and evaluate several standard performance metrics. The performance of our proposed framework was also compared with several baselines and contemporary algorithms for Humour detection. The results demonstrate the effectiveness of our dataset to be used as a standard dataset for Humour detection in the Hindi web series. The proposed model yields an accuracy of 91.79 and 87.32 while an F1 score of 91.64 and 87.04 in percentage for the Kota-Factory and Panchayat datasets, respectively.

Multifunctional In-Memory Logics Based on a Dual-Gate Antiambipolar Transistor toward Non-von Neumann Computing Architecture.

In-memory computing may make it possible to realize non-von Neumann computing because the logic circuits are unified in the memory units. We investigated two types of in-memory logic operations, namely, two-input logic circuits and multifunctional artificial synapses. These were realized in a dual-gate antiambipolar transistor (AAT) with a ReS2/WSe2 heterojunction, in which polystyrene with a zinc phthalocyanine core (ZnPc-PS4) was incorporated as a memory layer. Here, reconfigurability is a key concept for both types of device operations and was achieved by merging the Λ-shaped transfer curve of the AAT and the nonvolatile memory effect of ZnPc-PS4. First, we achieved electrically reconfigurable two-input logic circuits. Versatile logic circuits such as AND, OR, NAND, NOR, and XOR circuits were demonstrated by taking advantage of the Λ-shaped transfer curve of the dual-gate AAT. Importantly, the nonvolatile memory function provided the electrical switching of the individual circuits between AND/OR, NAND/NOR, and XOR/NAND circuits with constant input signals. Second, the memory effect was applied to multifunctional artificial synapses. The inhibitory/excitatory and long-term potentiation/depression synaptic operations were electrically reconfigured simply by controlling one parameter (readout voltage), making three distinct responses possible even with the same presynaptic signals. These findings provide hints that may lead to the realization of new in-memory computing architectures beyond the current von Neumann computers.

ConvLSNet: A lightweight architecture based on ConvLSTM model for the classification of pulmonary conditions using multichannel lung sound recordings

Characterization of lung sounds (LS) is indispensable for diagnosing respiratory pathology. Although conventional neural networks (NNs) have been widely employed for the automatic diagnosis of lung sounds, deep neural networks can potentially be more useful than conventional NNs by allowing accurate classification without requiring preprocessing and feature extraction. Utilizing the long short-term memory (LSTM) layers to reveal the sequence-based properties of the LS time series, a novel architecture consisting of a cascade of convolutional long short-term memory (ConvLSTM) and LSTM layers, namely ConvLSNet is developed, which permits highly accurate diagnosis of pulmonary disease states. By modeling the multichannel lung sounds through the ConvLSTM layer, the proposed ConvLSNet architecture can concurrently deal with the spatial and temporal properties of the six-channel LS recordings without heavy preprocessing or data transformation. Notably, the proposed model achieves a classification accuracy of 97.4 % based on LS data corresponding to three pulmonary conditions, namely asthma, COPD, and the healthy state. Compared with architectures consisting exclusively of CNN or LSTM layers, as well as those employing a cascade integration of 2DCNN and LSTM layers, the proposed ConvLSNet architecture exhibited the highest classification accuracy, while imposing the lowest computational cost as quantified by the number of parameters, training time, and learning rate.

Comparison of Monthly Rainfall Prediction using Long Short Term Memory and Multi Layer Perceptron Methods in South Tangerang City

Rainfall is one of the meteorological and climatological parameters whose information must be disseminated to the public and related stakeholders. Rainfall information has an important role in the sectors of people's lives. In agriculture, the amount of rainfall has an important role in determining the planting season, so that this can prevent potential crop failure. On Disaster, South Tangerang City during the 2016-2021 period experienced floods, landslides, and droughts. Therefore, the importance of rainfall prediction information can improve meteorological and climatological information services in various sectors. Nevertheless, it is still difficult for the community and stakeholders to get monthly rainfall predictions with high accuracy in the long term. In this research, monthly rainfall prediction is designed using MLP (Multi Layer Perceptron) and LSTM (Long Short Term Memory). The data used is the monthly rainfall data of Climate Hazards Group InfraRed Precipitations (CHIRPS) for 42 years (period 1981-2022) with coordinate boundaries according to the research location, namely South Tangerang City, which is located between 106.625 º - 106.825 º East and 6.4 ° - 6.2 ° LS as many as 16 grids with a resolution of 0.05 ° each grid. Monthly rainfall prediction using MLP produces an RMSE value of 90.19, and a MAPE of 40.55, while the LSTM method produces an RMSE value of 88.12 and a MAPE of 40.49. Monthly rainfall prediction results using the LSTM method are better than the MLP method; this can be seen from the RMSE value of the LSTM method is smaller than MLP.

Automatic classification of normal and abnormal cell division using deep learning

In recent years, there has been a surge in the development of methods for cell segmentation and tracking, with initiatives like the Cell Tracking Challenge driving progress in the field. Most studies focus on regular cell population videos in which cells are segmented and followed, and parental relationships annotated. However, DNA damage induced by genotoxic drugs or ionizing radiation produces additional abnormal events since it leads to behaviors like abnormal cell divisions (resulting in a number of daughters different from two) and cell death. With this in mind, we developed an automatic mitosis classifier to categorize small mitosis image sequences centered around one cell as “Normal” or “Abnormal.” These mitosis sequences were extracted from videos of cell populations exposed to varying levels of radiation that affect the cell cycle’s development. We explored several deep-learning architectures and found that a network with a ResNet50 backbone and including a Long Short-Term Memory (LSTM) layer produced the best results (mean F1-score: 0.93 ± 0.06). In the future, we plan to integrate this classifier with cell segmentation and tracking to build phylogenetic trees of the population after genomic stress.

Residual and bidirectional LSTM for epileptic seizure detection.

Electroencephalogram (EEG) plays a pivotal role in the detection and analysis of epileptic seizures, which affects over 70 million people in the world. Nonetheless, the visual interpretation of EEG signals for epilepsy detection is laborious and time-consuming. To tackle this open challenge, we introduce a straightforward yet efficient hybrid deep learning approach, named ResBiLSTM, for detecting epileptic seizures using EEG signals. Firstly, a one-dimensional residual neural network (ResNet) is tailored to adeptly extract the local spatial features of EEG signals. Subsequently, the acquired features are input into a bidirectional long short-term memory (BiLSTM) layer to model temporal dependencies. These output features are further processed through two fully connected layers to achieve the final epileptic seizure detection. The performance of ResBiLSTM is assessed on the epileptic seizure datasets provided by the University of Bonn and Temple University Hospital (TUH). The ResBiLSTM model achieves epileptic seizure detection accuracy rates of 98.88-100% in binary and ternary classifications on the Bonn dataset. Experimental outcomes for seizure recognition across seven epilepsy seizure types on the TUH seizure corpus (TUSZ) dataset indicate that the ResBiLSTM model attains a classification accuracy of 95.03% and a weighted F1 score of 95.03% with 10-fold cross-validation. These findings illustrate that ResBiLSTM outperforms several recent deep learning state-of-the-art approaches.

Predict Future Transient Fire Heat Release Rates Based on Fire Imagery and Deep Learning

The fire heat release rate (HRR) is a crucial parameter for describing the combustion process and its thermal effects. In recent years, some studies have employed fire scene images and deep learning algorithms to predict real-time fire HRR, which has led to the advancement of HRR prediction in terms of both lightweightness and real-time monitoring. Nevertheless, the development of an early-stage monitoring system for fires and the ability to predict future HRR based on current moment data represents a crucial foundation for evaluating the scale of indoor fires and enhancing the capacity to prevent and control such incidents. This paper proposes a deep learning model based on continuous fire scene images (containing both flame and smoke features) and their time-series information to predict the future transient fire HRR. The model (Att-BiLSTM) comprises three bi-directional long- and short-term memory (Bi-LSTM) layers and one attention layer. The model employs a bidirectional feature extraction approach, followed by the introduction of an attention mechanism to highlight the image features that have a critical impact on the prediction results. In this paper, a large-scale dataset is constructed by collecting 27,231 fire scene images with instantaneous HRR annotations from 40 different fire trials from the NIST database. The experimental results demonstrate that Att-BiLSTM is capable of effectively utilizing fire scene image features and temporal information to accurately predict future transient HRR, including those in high-brightness fire environments and complex fire source situations. The research presented in this paper offers novel insights and methodologies for fire monitoring and emergency response.