CNN LSTM Research Articles

Deep-Eware: spatio-temporal social event detection using a hybrid learning model

Event detection from social media aims at extracting specific or generic unusual happenings, such as, family reunions, earthquakes, and disease outbreaks, among others. This paper introduces a new perspective for the hybrid extraction and clustering of social events from big social data streams. We rely on a hybrid learning model, where supervised deep learning is used for feature extraction and topic classification, whereas unsupervised spatial clustering is employed to determine the event whereabouts. We present ‘Deep-Eware’, a scalable and efficient event-aware big data platform that integrates data stream and geospatial processing tools for the hybrid extraction and dissemination of spatio-temporal events. We introduce a pure incremental approach for event discovery, by developing unsupervised machine learning and NLP algorithms and by computing events’ lifetime and spatial spanning. The system integrates a semantic keyword generation tool using KeyBERT for dataset preparation. Event classification is performed using CNN and bidirectional LSTM, while hierarchical density-based spatial clustering was used for location-inference of events. We conduct experiments over Twitter datasets to measure the effectiveness and efficiency of our system. The results demonstrate that this hybrid approach for spatio-temporal event extraction has a major advantage for real-time spatio-temporal event detection and tracking from social media. This leads to the development of unparalleled smart city applications, such as event-enriched trip planning, epidemic disease evolution, and proactive emergency management services.

An efficient hand gesture recognition based on optimal deep embedded hybrid convolutional neural network‐long short term memory network model

AbstractHand gestures are the nonverbal communication done by individuals who cannot represent their thoughts in form of words. It is mainly used during human‐computer interaction (HCI), deaf and mute people interaction, and other robotic interface applications. Gesture recognition is a field of computer science mainly focused on improving the HCI via touch screens, cameras, and kinetic devices. The state‐of‐art systems mainly used computer vision‐based techniques that utilize both the motion sensor and camera to capture the hand gestures in real‐time and interprets them via the usage of the machine learning algorithms. Conventional machine learning algorithms often suffer from the different complexities present in the visible hand gesture images such as skin color, distance, light, hand direction, position, and background. In this article, an adaptive weighted multi‐scale resolution (AWMSR) network with a deep embedded hybrid convolutional neural network and long short term memory network (hybrid CNN‐LSTM) is proposed for identifying the different hand gesture signs with higher recognition accuracy. The proposed methodology is formulated using three steps: input preprocessing, feature extraction, and classification. To improve the complex visual effects present in the input images, a histogram equalization technique is used which improves the size of the gray level pixel in the image and also their occurrence probability. The multi‐block local binary pattern (MB‐LBP) algorithm is employed for feature extraction which extracts the crucial features present in the image such as hand shape structure feature, curvature feature, and invariant movements. The AWMSR with the deep embedded hybrid CNN–LSTM network is applied in the two‐benchmark datasets namely Jochen Triesch static hand posture and NUS hand posture dataset‐II to detect its stability in identifying different hand gestures. The weight function of the deep embedded CNN‐LSTM architecture is optimized using the puzzle optimization algorithm. The efficiency of the proposed methodology is verified in terms of different performance evaluation metrics such as accuracy, loss, confusion matrix, Intersection over the union, and execution time. The proposed methodology offers recognition accuracy of 97.86% and 98.32% for both datasets.

Estimation of concise video summaries from long sequence videos using deep learning via LSTM

Video summarization is a field of signal processing which deals with removing redundant information from a sequence of frames. In order to design an efficient video summarization model, it is imperative that frame features must be analyzed, and their similarity must be evaluated. This process would assist in aggregation of frames that have similar feature sets, and then use this information to reduce frame redundancy. Design of an efficient feature extraction and analysis model must be done in order to improve efficiency of video summarization. Models that aim at improving this efficiency, tend to improve the computational complexity of summarization, thereby limiting their applicability. In order to remove this drawback, The proposed event-based model for long video summarization uses a combination of LSTM based CNN with feature variance method for estimation of keyframes. The model works is 2-phases, wherein in the first phase various events are recognized from the video using LSTM based CNN model; while in the second phase these events are individually summarized using variance-based threshold engine. The proposed event classification-based LSTM CNN model with variance-based summarization outperforms other models, and achieves 8% improvement in terms of compression ratio.

Activity Recognition Based on Millimeter-Wave Radar by Fusing Point Cloud and Range–Doppler Information

Millimeter-wave radar has demonstrated its high efficiency in complex environments in recent years, which outperforms LiDAR and computer vision in human activity recognition in the presence of smoke, fog, and dust. In previous studies, researchers mostly analyzed either 2D (3D) point cloud or range–Doppler information from radar echo to extract activity features. In this paper, we propose a multi-model deep learning approach to fuse the features of both point clouds and range–Doppler for classifying six activities, i.e., boxing, jumping, squatting, walking, circling, and high-knee lifting, based on a millimeter-wave radar. We adopt a CNN–LSTM model to extract the time-serial features from point clouds and a CNN model to obtain the features from range–Doppler. Then we fuse the two features and input the fused feature into the full connected layer for classification. We built a dataset based on a 3D millimeter-wave radar from 17 volunteers. The evaluation result based on the dataset shows that this method has higher accuracy than utilizing the two kinds of information separately and achieves a recognition accuracy of 97.26%, which is about 1% higher than other networks with only one kind of data as input.

Human Activity Recognition System Using Smartphone Data Sensors with Python and Machine Learning

Abstract: This project depicts recognition Human activity Using data generated from user Smartphones Machine Learning repository to recognize six human activities. These activities are standing, sitting, laying, walking, upstair and walking, ddownstairs. Data is collected from embedded accelerometer, gyroscope and other sensor .Data is randomly divided into 7:3 ratios to From training and testing data set respectively. Activity Classification done using Machine Learning models Namely Random Forest. support vector machine, Neural Network and k-Nearest Neighbor. We have compared accuracy and performance of these model using confusion matrix and random simulation. Human Activity recognition(HAR) is classifying activity of person using responsive sensor that are affected from human movement. Both users and capabilities of smartphone With them. These facts makes HAR more important and Popular. This work focuses on recognition of Human activity using smartphone sensor different machine learning clssification approaches. Data retrieved from smartphones accelerometer and gyroscope sensor are classified On order to recognize human activity. Results of the approaches used compared in terms of efficiency and precision. Keywords: CNN, Accelerometer and gyroscope LSTM Model, Machine Learning, SVM etc

An Optimization-Based Diabetes Prediction Model Using CNN and Bi-Directional LSTM in Real-Time Environment

Diabetes is a long-term illness caused by the inefficient use of insulin generated by the pancreas. If diabetes is detected at an early stage, patients can live their lives healthier. Unlike previously used analytical approaches, deep learning does not need feature extraction. In order to support this viewpoint, we developed a real-time monitoring hybrid deep learning-based model to detect and predict Type 2 diabetes mellitus using the publicly available PIMA Indian diabetes database. This study contributes in four ways. First, we perform a comparative study of different deep learning models. Based on experimental findings, we next suggested merging two models, CNN-Bi-LSTM, to detect (and predict) Type 2 diabetes. These findings demonstrate that CNN-Bi-LSTM surpasses the other deep learning methods in terms of accuracy (98%), sensitivity (97%), and specificity (98%), and it is 1.1% better compared to other existing state-of-the-art algorithms. Hence, our proposed model helps clinicians obtain complete information about their patients using real-time monitoring and can check real-time statistics about their vitals.

A full-view management method based on artificial neural networks for energy and material-savings in wastewater treatment plants

Carbon neutrality has been received extensive attention in the field of wastewater treatment. The optimal management of wastewater treatment plants (WWTPs) has great significance and urgency since the serious energy and materials waste. In this study, a full-view management method based on artificial neural networks (ANNs) for energy and material savings in WWTPs was established. More than 5 years of historical operating data from two typical plants (size 40,000 t/d and 10,000 t/d) located in Chongqing, China, were obtained, and public data in the service area of each plant were systematically collected from open channels. These abundant historical and public data were used to train two ANNs (GRA–CNN–LSTM model and PCA-BPNN model) to predict the inlets/outlets wastewater quality and quantity. The overall average prediction accuracy of inlets/outlets wastewater indicators are greater than 92.60% and 93.76%, respectively. By combining the two models, more appropriate process operation strategies can be obtained 2 weeks in advance, with more than 11.20% and 16.91% reduction of energy and material costs, respectively. This proposed method can provide full-view decision support for the optimal management of WWTPs and is also expected to support carbon emission control and carbon neutrality in the field of wastewater treatment.

A protein succinylation sites prediction method based on the hybrid architecture of LSTM network and CNN.

The succinylation modification of protein participates in the regulation of a variety of cellular processes. Identification of modified substrates with precise sites is the basis for understanding the molecular mechanism and regulation of succinylation. In this work, we picked and chose five superior feature codes: CKSAAP, ACF, BLOSUM62, AAindex, and one-hot, according to their performance in the problem of succinylation sites prediction. Then, LSTM network and CNN were used to construct four models: LSTM-CNN, CNN-LSTM, LSTM, and CNN. The five selected features were, respectively, input into each of these four models for training to compare the four models. Based on the performance of each model, the optimal model among them was chosen to construct a hybrid model DeepSucc that was composed of five sub-modules for integrating heterogeneous information. Under the 10-fold cross-validation, the hybrid model DeepSucc achieves 86.26% accuracy, 84.94% specificity, 87.57% sensitivity, 0.9406 AUC, and 0.7254 MCC. When compared with other prediction tools using an independent test set, DeepSucc outperformed them in sensitivity and MCC. The datasets and source codes can be accessed at https://github.com/1835174863zd/DeepSucc.

Pyrolytic characteristics of fine materials from municipal solid waste using TG-FTIR, Py-GC/MS, and deep learning approach: Kinetics, thermodynamics, and gaseous products distribution

Fine materials (FM) from municipal solid waste (MSW) classification require disposal, and pyrolysis is a feasible method for the treatments. Hence, the behavior, kinetics, and products of FM pyrolysis were investigated in this study. A deep learning algorithm was firstly employed to predict and verify the TG data during the process of FM pyrolysis. The results showed that FM pyrolysis could be divided into drying (<138 °C), de-volatilization (138–570 °C), and decomposition stage (≥570 °C above). The de-volatilization can further be divided into stage 2 and stage 3, with values of activation energy estimated by Flynn-Wall-Ozawa and Kissinger-Akahira-Sunose methods as 123.35 and 172.95 kJ/mol, respectively. The gas products like H2O, CO2, CH4, and CO, as well as functional groups like phenols and carbonyl (CO), were all detected during the process of FM pyrolysis by thermogravimetric-fourier transform infrared spectrometry at a heating rate of 10 °C/min. The main species detected by pyrolysis-gas chromatography-mass spectrometry analyzer included acid (41.98%) and aliphatic hydrocarbon (22.44%). Finally, the 1D–CNN–LSTM algorithm demonstrated an outstanding generalization capability to predict the relationship between FM composition and temperature, with R2 reaching 93.91%. In sum, this study provided a reference for the treatment of FM from MSW classification as well as the feasibility and practicability of deep learning applied in pyrolysis.

Forecasting agricultural commodities prices using deep learning-based models: basic LSTM, bi-LSTM, stacked LSTM, CNN LSTM, and convolutional LSTM

Forecasting agricultural commodities prices using deep learning-based models: basic LSTM, bi-LSTM, stacked LSTM, CNN LSTM, and convolutional LSTM

Forecasting agricultural commodities prices using deep learning-based models: basic LSTM, bi-LSTM, stacked LSTM, CNN LSTM, and convolutional LSTM

The literature argues that an accurate price prediction of agricultural goods is quintessential to ensure the economy's good functioning worldwide. Research reveals that studies with the application of deep learning in the tasks of agricultural price forecast on short historical agricultural price data are very scarce. The gap mentioned above is removed in this study by employing five versions of LSTM deep learning techniques for five agricultural commodities prices prediction on a univariate time series dataset of rice, wheat, gram, banana, and groundnut spanning January 2000 to July 2020. The study obtained good forecasting results for all five commodities employing the five LSTM models. The study validated the results with lower values of error metrics, MAE, MAPE, MSE, and RMSE, and two paired t-tests with hypothesis and confidence levels of 95% as a measure of robustness. The study predicted one month ahead future price and compared it with actual prices using LSTM models.

Traffic Prediction-Enabled Energy-Efficient Dynamic Computing Resource Allocation in CRAN Based on Deep Learning

Due to the greatly increased bandwidth of 5G networks compared with that of 4G networks, the power consumption brought by baseband signal processing of 5G networks is much higher, which inevitably raises the operation expenditures. Cloud Radio Access Network (CRAN) is widely adopted in 5G networks, which splits the traditional base stations into Remote Radio Heads (RRHs) and Baseband Units (BBUs), which are equipped with computing resource for baseband signal processing. The number of required BBUs varies due to the fluctuation of wireless traffic of RRHs. Hence, fixed computing resource allocation might waste power. This paper investigates energy-efficient dynamic computing resource allocation in CRAN by predicting the wireless traffic of RRHs and allocating computing resource based on the prediction results aiming at using fewest BBUs to minimize power consumption. For wireless traffic prediction, a novel method based on two-dimensional CNN LSTM model with temporal aggregation is proposed. By treating the wireless traffic data as images, this model could extract spatial correlation from these data to improve accuracy. Moreover, the problem of dynamic computing resource allocation in CRAN is formulated as an offline four-constraint bin packing problem, considering both uplink and downlink baseband signal processing capacities of BBUs and Common Public Radio Interface (CPRI) bandwidths. For solving this problem, a Multi-start Simulated Annealing (MSA) algorithm is proposed. Simulation results demonstrate that the proposed method for wireless traffic prediction could outperform the state-of-the-art deep learning models. In addition, the proposed MSA algorithm could achieve lower power consumption than the state-of-the-art heuristic algorithms.

Prediction of College Employment Rate Based on Big Data Analysis

This paper uses big data technology to predict the employment rate of colleges and universities. In this paper, combined with the current rental price, daily life consumption, and college students’ personal interests and hobbies consumption and other indicators, the individual is simulated by big data, and the individual is associated by using the AI-driven edge fog computing service optimization algorithm to form a cluster, so as to realize the prediction from element to neural network cluster by using edge computing. In addition, this paper takes the employment data of colleges and universities in Hunan province from June 2020 to May 2021 as the research sample to test the prediction model and makes a comparative analysis with the CNN model and LSTM model. The edge fog computing model in this paper has more analytical indexes as tuples compared to the CNN model, so the results show that the prediction accuracy can reach 83.25%. In this case, there is little difference between the two models of data processing and predictive efficiency. Compared with the LSTM based classification prediction model, this model is edge computing, which greatly improves the data quality of model and data parameters, and the calculation efficiency can be increased by 45%–65%. Therefore, the use of big data technology can provide a reference for the research direction of higher education.

Sliding window approach with first-order differencing for very short-term solar irradiance forecasting using deep learning models

The intermittent behavior of solar is usually responsible for creating uncertainty while generating power. By implementing suitable forecasting techniques for solar irradiance (SI), we can overcome this intermittency which can be helpful in the economic load dispatch as well as to control, manage and optimize the power generation in the microgrid. This paper presents three deep learning (DL) models to forecast SI from 1 step (15-minute) to 6 steps (1 h 30 min) ahead. By implementing the sliding window technique, the input variables are converted into 12 steps lag datasets to train the model whereas outputs are transformed by first-order differencing. A total dataset of 18,277 from average 15-min interval global SI collected during January 1, 2016, to January 6, 2017, from the Asian Institute of Technology (AIT) Metrological station is used to train and evaluate the performance of the DL models. Based on the obtained results from different evaluation parameters such as maximum absolute error, confidence interval (CI), linear regression plot, mean absolute error (MAE), root mean square error (RMSE), mean absolute percentage error (MAPE), and R squared, we found that the deep hybrid model consists of convolutional neural network-long short term memory (CNN-LSTM) can outperform during multistep forecasting. The findings of the present work suggest that the proposed deep hybrid LSTM–CNN model is a reliable alternative for very short-term SI prediction due to its high predictive accuracy.

A performance compensation method for GPS/INS integrated navigation system based on CNN–LSTM during GPS outages

When the global position system (GPS) signal is unavailable, the performance of the GPS/inertial navigation system (INS) integrated navigation system degrades severely. In this article, the performance of the ultra-low cost inertial measurement unit (IMU) is studied and the objective is to enhance its performance during GPS outages. To be specific, a performance compensation method is proposed, which consists of two parts. First, to deal with the large noise and drift of the micro-electro-mechanical system (MEMS)-based inertial measurement unit (IMU), a wavelet regional correlation threshold denoising algorithm is proposed. Then, to improve the performance of traditional LSTM network when dealing with navigation data with strong coupling, a convolutional neural network-long short-term memory (CNN–LSTM) model is formulated. It employs CNN to quickly extract the features of the input, and utilizes LSTM network to output pseudo-GPS signals as the compensation object. Finally, simulation experiments and real road tests are implemented to evaluate the proposed method. Comparison experiment results show that the proposed method can effectively improve the performance of the integrated navigation system during GPS outages.

Arrhythmia Classification Based on CNN and Bidirectional LSTM

Arrhythmia Classification Based on CNN and Bidirectional LSTM

Combustion performance of fine screenings from municipal solid waste: Thermo-kinetic investigation and deep learning modeling via TG-FTIR

The combustion behavior, kinetics, thermodynamics and gas products of fine screenings (FS) classified from municipal solid waste (MSW) in an air atmosphere were explored by TG-FTIR. A deep learning model was established using 1D–CNN–LSTM algorithm to predict thermogravimetric data of FS combustion, with visualization technology (TensorBoard) applied to display the weights and biases in various cells. The thermogravimetric analysis (TG) and differential thermal gravity (DTG) curves indicated that the FS combustion process can be divided into four stages. The average activation energy (Ea) of FS combusted at different stages, exhibited different change tendencies with increasing levels of conversion (α). The highest enthalpy (ΔH) of 206.40 kJ/mol and free Gibbs energy (ΔG) of 55.03 kJ/mol emerged in stage Ⅳ, while the highest changes of entropy (ΔS) of 169.11 J/(mol·K) occurred in stage Ⅱ. The main gas products (CO2, H2O and CO) and functional groups (CO and phenols) were all detected. For the 1D–CNN–LSTM model, the optimal settings for the prediction of thermogravimetric data were a neuron number of 150, dropout of 0.003, epoch number of 200, and batch size of 25. The highest correlation coefficient (R2) of 94.41% was obtained using the optimum model parameters, achieving an excellent prediction performance.

Sliding Window and Parallel LSTM with Attention and CNN for Sentence Alignment on Low-Resource Languages

A parallel text corpus is an important resource for building a machine translation (MT) system. Existing resources such as translated documents, bilingual dictionaries, and translated subtitles are excellent resources for constructing parallel text corpus. A sentence alignment algorithm automatically aligns source sentences and target sentences because manual sentence alignment is resource-intensive. Over the years, sentence alignment approaches have improved from sentence length heuristics to statistical lexical models to deep neural networks. Solving the alignment problem as a classification problem is interesting as classification is the core of machine learning. This paper proposes a parallel long-short-term memory with attention and convolutional neural network (parallel LSTM+Attention+CNN) for classifying two sentences as parallel or non-parallel sentences. A sliding window approach is also proposed with the classifier to align sentences in the source and target languages. The proposed approach was compared with three classifiers, namely the feedforward neural network, CNN, and bi-directional LSTM. It is also compared with the BleuAlign sentence alignment system. The classification accuracy of these models was evaluated using Malay-English parallel text corpus and UN French-English parallel text corpus. The Malay-English sentence alignment performance was then evaluated using research documents and the very challenging Classical Malay-English document. The proposed classifier obtained more than 80% accuracy in categorizing parallel/non-parallel sentences with a model built using only five thousand training parallel sentences. It has a higher sentence alignment accuracy than other baseline systems.

Analysis of DNA Sequence Classification Using CNN and Hybrid Models.

In a general computational context for biomedical data analysis, DNA sequence classification is a crucial challenge. Several machine learning techniques have used to complete this task in recent years successfully. Identification and classification of viruses are essential to avoid an outbreak like COVID-19. Regardless, the feature selection process remains the most challenging aspect of the issue. The most commonly used representations worsen the case of high dimensionality, and sequences lack explicit features. It also helps in detecting the effect of viruses and drug design. In recent days, deep learning (DL) models can automatically extract the features from the input. In this work, we employed CNN, CNN-LSTM, and CNN-Bidirectional LSTM architectures using Label and K-mer encoding for DNA sequence classification. The models are evaluated on different classification metrics. From the experimental results, the CNN and CNN-Bidirectional LSTM with K-mer encoding offers high accuracy with 93.16% and 93.13%, respectively, on testing data.

A deep learning based non-intrusive household load identification for smart grid in China

Load identification have shown significant performance gains in Chinese smart grids. Most existing load identification algorithms are based on electrical characteristics of a steady or transient state, which are therefore limited by feature selection and analysing pattern. To address the above issues, this paper proposes the use of the deep neural network for load identification in a Non-Intrusive Load Monitoring (NILM) test-bed, which is set up by introducing diversified household appliances with different load characteristics, to collect the real-time power usage of appliances in a typical Chinese home. The collected load dataset are then sampled, preprocessed and input to the CNN–LSTM framework for training and features extraction. Next, according to several experiments, the structure of our CNN–LSTM network is determined with reasonable hyper-parameters initialised. Numerical results show that our model is superior to the k-NN, SVM, LSTM and CNN load identification methods, with the average recognition accuracy of 99%, across different kinds of appliances enabled in the typical power grid in China.