Patterns In Sequential Data Research Articles

A cloud 15kV-HDPE insulator leakage current classification based improved particle swarm optimization and LSTM-CNN deep learning approach

Real-time insulator leakage current classification is crucial in preventing the pollution flashover phenomenon and providing appropriate maintenance schedules in high-voltage transmission towers. However, current methodologies only utilize traditional artificial neural networks, which have limitations when performing big data analysis. This research developed a novel cloud 15kV-HDPE insulator leakage current classified framework, utilizing a long short-term memory convolutional neural network (LSTM-CNN). The hybrid model structure is optimized through hyperparameter fine-tuning based on improved particle swarm optimization (IPSO), which reduces human effort and considerable time compared with PSO and random search (RS) techniques. The IPSO-LSTM-CNN model can productively identify correlations between selected weather features and target leakage current levels of 15kV-HDPE insulators. LSTM efficiently captures long-term patterns in sequential data, while CNN layers competently extract high-level dependency in time-invariant information. Four 15kV-HDPE insulators’ datasets, collected in high-voltage transmission lines in the coastal area of Taiwan for more than one year, are deployed for analyzing and comparing classified performance. Other conventional models are developed to evaluate and compare classified performance with the proposed IPSO-LSTM-CNN approach, which acquires the most significant enhancement of 48.08 % loss, 45.91 % validating loss, 52.57 % MAE, 35.47 % validating MAE, 47.34 % MSE, 27.02 % validating MSE, 9.15 % PRE, 3.40 % validating PRE, 4.76 % REC, and 6.17 % validating REC. The experiment outcomes demonstrate that the developed IPSO-LSTM-CNN model acquires improved robustness and accuracy in the leakage current classified capability of 15kV-HDPE insulators.

Time Series Forecasting of Emission Trends Using Recurrent Neural Networks

This paper explores the application of Recurrent Neural Networks (RNNs) for forecasting emission trends, a critical aspect of addressing climate change and formulating effective environmental policies. Traditional forecasting methods, such as Autoregressive Integrated Moving Average (ARIMA) and Exponential Smoothing, often fail to capture the complex nonlinear relationships and temporal dependencies inherent in emission data. In contrast, RNNs, particularly Long Short-Term Memory (LSTM) networks, are designed to recognize patterns in sequential data, making them well-suited for time series forecasting tasks. This study employs a comprehensive methodology that includes data collection from reputable sources, feature selection, RNN model design, and rigorous evaluation using various performance metrics. The results indicate that RNNs significantly outperform traditional forecasting techniques in terms of accuracy, providing valuable insights into future emission trajectories. The findings underscore the potential of RNNs as powerful tools for policymakers and researchers, facilitating more informed decision-making in the fight against climate change.

Sentiment analysis of movie review classifications using deep learning approaches

Movie reviews reflect how the public feels about a movie they have watched. However, because many reviews are posted on various websites, it is practically impossible to read each one. Summarizing all movie reviews can help people make informed decisions without reading through all of them. Previous studies have used different machine learning and deep learning techniques for sentiment analysis (SA), but few have combined comprehensive hyperparameter tuning and novel datasets for better performance. This paper presents an SA approach using deep learning models with optimized hyperparameters and a novel Rotten Tomatoes (RT) dataset to help viewers make better movie choices. SA, or opinion mining, is a computational technique to extract and analyze opinions and emotions expressed in text. We explore deep learning models such as Long Short-Term Memory (LSTM), XLNet, Convolutional Neural Networks-LSTM (CNN-LSTM), and Bidirectional Encoder Representations from Transformers (BERT). These models are known for capturing complex language patterns and context from raw text data. XLNet, a pre-trained model, effectively understands context by considering all possible permutations of the input sequence, BERT excels at using bidirectional context to understand text, LSTM retains information about long-term patterns in sequential data, and CNN-LSTM combines local and global context for reliable feature extraction. The RT dataset was pre-processed with data cleaning, spelling correction, lemmatization, and handling of informal words to improve the results. Our experiments show that XLNet performed better than other models on the Rotten Tomatoes dataset. The study demonstrates that SA of movie reviews provides insights into emotions and attitudes, allowing us to estimate a movie’s performance based on its overall sentiment.

Weather Prediction: Predicting Rain Using Weather Conditions

Weather forecasting is a major discipline that plays an important role in fields such as agriculture, transport, and emergency management, and it largely depends on accurate forecasts. Concerning this problem, this work aimed to analyze the effectiveness of recurrent neural networks, particularly the Long Short-Term Memory (LSTM), for estimating rainfall depending on precipitation, maximum temperature, minimum temperature, and wind speed. We will therefore use a large database containing recorded weather data obtained over several years to calibrate accurate predictive models designed to distinguish between drizzle, rain, sun, snow, and fog. The main idea of the work is to teach LSTM models that are capable of revealing temporal relations and patterns in sequential data, which makes them suitable to work on various time series forecasting such as weather prediction. The data is preprocessed effectively to clean it and make it ideal for our analysis to accurately compare the performance of one model against the others, we have divided the data into training, validation, and testing sets. The concurrency of the proposed LSTM model is then evaluated with the Mean Squared Error (MSE), Root Mean Squared Error (RMSE), Mean Absolute Error (MAE), and the coefficient of determination (R²) to measure the forecasting accuracy. The findings show a better predictive performance uplift whereby the best-performing LSTM model has an MSE of 8.74, RMSE of 2.96, MAE of 2.35, and R² of 0.83. Such metrics represent logical dependence between the predicted and actual weather conditions, proving thus the efficiency of the model. Also, the evaluation shows how hyper parameters’ optimization, features’ selection, and normalization, make a huge difference in the model’s performance and indicate that the precise management of weather parameters can result in better forecasts. However, the contributors of this research are not recluded to theoretical perspective; the present study can be useful for various subjects since the dependability of weather forecasts can be improved. They will be advantaged to have more precise weather data for crop growing, road networks, and other transport systems to prepare for the worst conditions, and emergency, rescue operations to be in a better position to handle certain disasters. Consequently, this study improves the academic literature on weather peculiarities with unforeseen downpours through a demonstration and explanation of the potential of LSTM networks to analyze key meteorological characteristics for rainfall prediction. Possible future study directions are outlined, proposing the expansion of features beyond those analyzed in the existing study to improve the predictive models, the usage of continuous rather than weekly data, as well as considering the mixed-ingredients approach for increasing the prediction accuracy. This inclusive strategy seeks to enhance the realistic stages in the phased meteorological prognosis and also timely resource allocation and management tactics within climate volatility.

Research on Anomaly Detection in Time Series: Exploring United States Exports and Imports Using Long Short-Term Memory

This survey aims to offer a thorough and organized overview of research on anomaly detection, which is a significant problem that has been studied in various fields and application areas. Some anomaly detection techniques have been tailored for specific domains, while others are more general. Anomaly detection involves identifying unusual patterns or events in a dataset, which is important for a wide range of applications including fraud detection and medical diagnosis. Not much research on anomaly detection techniques has been conducted in the field of economic and international trade. Therefore, this study attempts to analyse the time-series data of United Nations exports and imports for the period 1992 – 2022 using LSTM based anomaly detection algorithm. Deep learning, particularly LSTM networks, are becoming increasingly popular in anomaly detection tasks due to their ability to learn complex patterns in sequential data. This paper presents a detailed explanation of LSTM architecture, including the role of input, forget, and output gates in processing input vectors and hidden states at each timestep. The LSTM based anomaly detection approach yields promising results by modelling small-term as well as long-term temporal dependencies. 

A Multilayer Approach for Intrusion Detection with Lightweight Multilayer Perceptron and LSTM Deep Learning Models

Intrusion detection is essential in the field of cybersecurity for protecting networks and computer systems from nefarious activity. We suggest a novel multilayer strategy that combines the strength of the Lightweight Multilayer Perceptron (MLP) and Long Short-Term Memory (LSTM) deep learning models in order to improve the precision and effectiveness of intrusion detection.The initial layer for extraction of features and representation is the Lightweight MLP. Its streamlined architecture allows for quick network data processing while still maintaining competitive performance. The LSTM deep learning model, which is excellent at identifying temporal correlations and patterns in sequential data, receives the extracted features after that.Our multilayer technique successfully manages the highly dimensional and dynamic nature of data from networks by merging these two models. We undertake extensive tests on benchmark datasets, and the outcomes show that our strategy performs better than conventional single-model intrusion detection techniques.The suggested multilayer method also demonstrates outstanding efficiency, which makes it particularly ideal for real-time intrusion detection in expansive network environments. Our multilayer approach offers a strong and dependable solution for identifying and reducing intrusions, strengthening the security position of computer systems and networks as cyber threats continue to advance.

DITAN: A deep-learning domain agnostic framework for detection and interpretation of temporally-based multivariate ANomalies

We present DITAN, a novel unsupervised domain-agnostic framework for detecting and interpreting temporal-based anomalies. It is based on an encoder-decoder architecture with both implicit/explicit attention and adjustable layers/units for predicting normality as regular patterns in sequential data. A two-stage thresholding methodology with built-in pruning is used to detect anomalies, while root cause and similarities are interpreted in data and units space. Our approach is designed to intersect the 9 fundamental characteristics extracted from the union of related works. We demonstrate the DITAN modules on real-world datasets of 6 multivariate time series contaminated by point and contextual temporal-based anomalies at a varying duration. Experiments show a dominant predictability power of DITAN against the originally proposed models. DITAN is able to determine critical regions and thus identify anomalous events similarly well. Informative similarities between anomalous records are interpreted, since almost all similarities in units space are also verified in data space.

An International Study of Application of Long Short-Term Memory (LSTM) Neural Networks for the prediction of stock and forex markets

This study focuses on the use of Long Short-Term Memory (LSTM) neural networks for stock and currency market forecasting. Accurate projections are difficult to make because of the complex dynamics and non-linear interactions that characterise financial markets. A Recurrent Neural Networks (RNN) variation called LSTM is particularly good at identifying temporal dependencies and long-term patterns in sequential data. The goal of LSTM models is to discover significant insights and produce precise predictions using past price data. In addition to discussing data pretreatment methods, model creation, and assessment metrics related to stock and FX market prediction, this work examines the benefits of LSTM in capturing market dynamics. Case studies and empirical analysis are used to investigate the capabilities and constraints of LSTM models in forecasting market movements.

Predictive maintenance in pharmaceutical manufacturing lines using deep transformers

Inherent complexities in pharmaceutical manufacturing lines of modern industrial facilities make precise and timely detection of malfunction occurrences necessary. In fact, unpredicted malfunctions in a production line can often provoke a cascade of adverse effects that can occur everywhere in the production chain bringing the manufacturing line to a halt for undefined time periods. Such events can have unfortunate consequences that are not always confined to the damaged part itself but propagate throughout the production line. Nevertheless, modern production lines are equipped with a multitude of data sensors that enable the real-time and fine-grained monitoring of each constituent part of the production process providing a richness of information that can be exploited by intelligent data processing methods.In this work, we present ManuTrans, a deep learning-based model for monitoring real-time raw sensor data, deriving the condition of a pharmaceutical manufacturing line and predicting the next moment in time when a malfunction can occur. The model is further able to predict the severity of the next malfunction and can contribute adjunct information in corporate decision-making. The suggested approach exploits the capacity of deep transformer models for extracting both long- and short-term correlations as well as patterns in sequential data and, combined with a linear output layer, conducts both classification and regression. The proposed approach was tested on a real dataset comprising raw data from two manufacturing lines, and it achieved promising results.

USING A NEURAL NETWORK FOR PRICE PREDICTION OF VIRTUAL ASSETS

Due to the structure of recurrent neural networks, they are used for the prediction tasks, such as price prediction. The price prediction tasks are based on the historical data of price movements during the specified period. This data can be used for training the recurrent neural network for price prediction. Expected, that the neural network will recognize specific patterns in sequential data and will be able to predict the next trends etc. From the 2021 year, virtual assets such as Bitcoin increase their popularity all around the world. Virtual assets such as Bitcoin are classified as highly volatile assets. For this type of asset, the prediction task is so important, due to the ability to make a long and a short position several times per day, week, etc. Using the recurrent neural network, against the ARIMA methods, can help to include the other data except for the price history. For example, it can you the history of several operations for these assets per day, the price history of the other virtual assets, that can have some relations with, etc. In this article, as a first step, work was focused on properly formatting the price history data for achieving the lowest prediction error. Also, the idea is to create a framework for working with different virtual assets for prediction. The problem with using a neural network for prediction is that absolute values of virtual assets price are not stationary. And the neural network training process stuck between the minimum and maximum values of training data. It creates a problem, where the trained neural network cannot handle the data, that is bigger than the trained and it always tried to predict the value in the trained range. This work, investigated the neural network for single prediction only. Also, was compared the two possible ways to format the data to the stationary data. Key words : recurrent neural networks, prediction, virtual assets.

FastRCA-Seq: An efficient approach for extracting hierarchies of multilevel closed partially-ordered patterns

Discovering concise representations of sequential patterns in sequential data is a well-established data mining task. Recently, Nica et al. have put forward an original approach RCA-Seq for directly extracting a hierarchy of multilevel closed partially-ordered patterns (MCPO-patterns) from a sequence database within the Relational Concept Analysis (RCA) framework. RCA-Seq has been applied successfully to small (∼1,000 sequences) but interesting real hydro-ecological datasets. However, RCA-Seq only focuses on providing comprehensible results to the detriment of performance. To improve the performance of RCA-Seq, we propose a new approach FastRCA-Seq that stems from RCA-Seq, and whose contributions are beneficial for two fields: Formal Concept Analysis, namely the RCA extension, and sequential pattern mining. FastRCA-Seq spans two key steps: the exploration of sequential data based on RCA, and the extraction of MCPO-patterns by navigating the RCA result. Firstly, our approach introduces an effective RCA implementation based on bit-array representations, bitwise operations, parallel computing, and several new properties of RCA that may prevent expensive computations. In addition, we state the bottleneck of RCA. Secondly, FastRCA-Seq is a self-contained approach for directly and efficiently mining hierarchies of MCPO-patterns from sequential data. We assess FastRCA-Seq on various benchmark datasets, precisely Gazelle, Kosarak, and FIFA. The results show that FastRCA-Seq outperforms RCA-Seq in terms of execution time (in average ∼169 times faster) and memory usage (in average with ∼42% less) while preserving the benefits of interpretability and usability of results by stakeholders.

Recurrent Neural Network Model for Constructive Peptide Design.

We present a generative long short-term memory (LSTM) recurrent neural network (RNN) for combinatorial de novo peptide design. RNN models capture patterns in sequential data and generate new data instances from the learned context. Amino acid sequences represent a suitable input for these machine-learning models. Generative models trained on peptide sequences could therefore facilitate the design of bespoke peptide libraries. We trained RNNs with LSTM units on pattern recognition of helical antimicrobial peptides and used the resulting model for de novo sequence generation. Of these sequences, 82% were predicted to be active antimicrobial peptides compared to 65% of randomly sampled sequences with the same amino acid distribution as the training set. The generated sequences also lie closer to the training data than manually designed amphipathic helices. The results of this study showcase the ability of LSTM RNNs to construct new amino acid sequences within the applicability domain of the model and motivate their prospective application to peptide and protein design without the need for the exhaustive enumeration of sequence libraries.

Mining closed strict episodes

Discovering patterns in a sequence is an important aspect of data mining. One popular choice of such patterns are episodes, patterns in sequential data describing events that often occur in the vicinity of each other. Episodes also enforce in which order the events are allowed to occur. In this work we introduce a technique for discovering closed episodes. Adopting existing approaches for discovering traditional patterns, such as closed itemsets, to episodes is not straightforward. First of all, we cannot define a unique closure based on frequency because an episode may have several closed superepisodes. Moreover, to define a closedness concept for episodes we need a subset relationship between episodes, which is not trivial to define. We approach these problems by introducing strict episodes. We argue that this class is general enough, and at the same time we are able to define a natural subset relationship within it and use it efficiently. In order to mine closed episodes we define an auxiliary closure operator. We show that this closure satisfies the needed properties so that we can use the existing framework for mining closed patterns. Discovering the true closed episodes can be done as a post-processing step. We combine these observations into an efficient mining algorithm and demonstrate empirically its performance in practice.

VOGUE

We present VOGUE, a novel, variable order hidden Markov model with state durations, that combines two separate techniques for modeling complex patterns in sequential data: pattern mining and data modeling. VOGUE relies on a variable gap sequence mining method to extract frequent patterns with different lengths and gaps between elements. It then uses these mined sequences to build a variable order hidden Markov model (HMM), that explicitly models the gaps. The gaps implicitly model the order of the HMM, and they explicitly model the duration of each state. We apply VOGUE to a variety of real sequence data taken from domains such as protein sequence classification, Web usage logs, intrusion detection, and spelling correction. We show that VOGUE has superior classification accuracy compared to regular HMMs, higher-order HMMs, and even special purpose HMMs like HMMER, which is a state-of-the-art method for protein classification. The VOGUE implementation and the datasets used in this article are available as open-source. 1

Approximate entropy as a measure of irregularity for psychiatric serial metrics

The quantification of subtle patterns in sequential data, and their changes, has considerable potential utility throughout psychiatry, including the analyses of mood ratings, heart rate, respiratory, and electroencephalographic recordings. Approximate entropy (ApEn), a relatively recently developed statistic quantifying serial irregularity, has been applied in numerous studies throughout mathematics and other fields of study, especially biology. We discussed applications of ApEn, both extant and potential, of most relevance to psychiatrists. We provided a mechanistic interpretation of lowered ApEn values, and discusses the relationship between ApEn and other (both classical and complexity) measures of serial dynamics. We also briefly discussed cross-ApEn, a thematically similar quantification of two-variable asynchrony that can aid in uncovering subtle disruptions in complicated network dynamics. ApEn and cross-ApEn have significant potential to consequentially enhance present statistical methodologies of analysis of psychiatric data, in both clinical and in research settings.

Approximate entropy in cardiology

Background. The quantification of subtle patterns in sequential data, and their changes, has considerable potential utility throughout cardiology, including the analysis of heart rate rhythms. Aim of the study. Approximate entropy (ApEn), a recently developed statistic quantifying serial irregularity, has been applied in numerous studies throughout mathematics and applications, especially biology. We indicate results to date, and future direction, of interest to cardiologists. Methods. We define ApEn, indicating basic properties. We discuss typical applications of ApEn, with special focus on a representative aspect of ApEn applications to heart rate dynamics, to pre- and early life studies. Subsequently, we introduce and briefly discuss cross-ApEn, a thematically similar quantification of two-variable asynchrony. Results. ApEn consistently detects subtle shifts in heart rate rhythmicity in many studies in which mean levels and classical variability assessments fail to discriminate normative from pathophysiological subjects. Greater regularity (lower ApEn) clinically corresponds to compromised physiology in all cardiologic settings discussed herein. We provide a mechanistic interpretation of lowered ApEn values, based on mathematical analysis, yet linked to physiology. We discuss and clarify why ApEn is complementary to classical ‘moment analysis’, to chaos-related statistical measures, and to spectral and correlation measures, and oftentimes provides clearer discriminatory capability. Conclusions. Both ApEn and cross-ApEn have significant potential to consequentially enhance present statistical methodologies of analysis of cardiologic data, in both clinical and in research settings.

Assessing sequential irregularity of both endocrine and heart rate rhythms.

The quantification of subtle patterns in sequential data, and their changes, has considerable potential utility both in analysis of hormonal secretory dynamics, and of fetal heart rate rhythms. Approximate entropy, a recently developed statistic quantifying serial irregularity, has been applied in both these settings, and has yielded a number of new findings. Among endocrine applications, approximate entropy increases with increasing age for luteinizing hormone and follicle stimulating hormone, for both women and men, indicating greater irregularity (more apparently random dynamics) in the older groups; for the luteinizing hormone-testosterone axis, both irregularity and asynchrony increases accompany advancing age for men. These findings produce a means to potentially predict time until menopause onset, the efficacy of infertility drugs, and also provide firm quantitative support of a partial male menopause. In fetal monitoring, antepartum, and postnatal heart rate studies, approximate entropy consistently detected subtle shifts in heart rate rhythmicity, with greater regularity clinically corresponding to compromised physiology in all settings. Both the capability of approximate entropy to quantify rhythm changes undiscernible by auscultation, and a continuum interpretation linking per-individual antepartum, perinatal, and postnatal heart rate analyses provide considerable potential enhancement to the present clinical utility of fetal monitoring.