Time Sequence Characteristics Research Articles

A Novel Distributed Process Monitoring Framework of VAE-Enhanced with Deep Neural Network

Intelligent manufacturing process needs to adopt distributed monitoring scenario due to its massive, high-dimensional and complex data. Distributed process monitoring has been introduced into global monitoring and local monitoring to analyze the characteristic relationship between process data. However, the existing framework methods ignore or suppress the fault information and thus cannot effectively identify the local faults and the time sequence characteristics between units in the batch production system. This paper proposes a novel distributed process monitoring framework based on Girvan-Newman algorithm modular subunit partitioning and probabilistic learning model with deep neural networks. First, Girvan-Newman algorithm is used to divide the complex manufacturing system modularized to reduce the latitude of data processing. Second, variational autoencoder (VAE) is adopted to ensure the stability of local analysis, and long short-term memory is adopted to improve the VAE model to detect global multi-time scale anomalies. Finally, distributed process fault detection is carried out for each subunit in a separate and integrated manner, and the performance of the framework in distributed process monitoring is analyzed through two fault detection indicators T2 and SPE statistics. A case study of the Tennessee Eastman Process is used to demonstrate the performance and applicability of the proposed framework. Results show that the proposed VAE enhancement framework based on the DNN could accurately identify faults in distributed process monitoring and locate the specific sub-units where the fault occurs. Compared with VAE-DNN method and traditional process monitoring methods, the framework proposed in this paper has higher fault detection rate and lower false alarm rate, and the detection rate of some faults can reach 100%.

Industrial process fault diagnosis based on feature enhanced meta-learning toward domain generalization scenarios

With the rise of the Internet of Things, computing science, and artificial intelligence, data-driven fault diagnosis methods for industrial processes have attracted considerable attention. However, these methods typically assume that the distribution of the test data (target domains) is similar to that of the training data (source domains), making it difficult to deploy them in actual scenarios because the distribution of the collected data varies with operating conditions. Faced with this challenge, a novel fault diagnosis method based on feature enhanced meta-learning (FEML) is proposed to learn domain generalization knowledge effectively. It aims to jointly extract domain-invariant information that can be transferred across domains, and domain-specific information related to classification labels. First, feature extractors based on gated recurrent units (GRUs) are designed to capture domain-invariant and domain-specific representations with time-sequence characteristics from industrial process data. Second, a meta-learning strategy is introduced to enhance the domain-specific representation, thus improving the generalization ability of the model without accessing the target domains. Finally, the Tennessee Eastman process (TEP), Fed-Batch Fermentation Penicillin process (FBFP), and Three-Phase Flow process (TPFP) are used to verify the effectiveness of FEML. The results illustrate that the FEML is robust and outperforms other related methods. The diagnostic accuracies for TEP, FBFP, and TPFP are 87.11%, 95.94%, and 83.07%, respectively.

Precise detection of coal and gangue based on natural γ-ray

To address the technical limitations of automatic coal and gangue detection technology in fully mechanized top coal caving mining operations, the low radiation level radioactivity measurement method is utilized to assess the degree of coal-gangue mixture in top coal caving process. This approach is based on the distinguishing radiation characteristics of natural γ-rays between coal and gangue. This study analyzed the distribution characteristics of natural γ-rays in coal and rock layers of thick coal seams and the applicability of this method, introduced the basic principle of coal-gangue detection technology based on natural γ-ray, developed the test system about automatic coal-gangue detection, studied the radiation characteristics of coal and gangue, proposed determination model of the coal-gangue mixed degree, combined with the time sequence characteristics of the top coal’s releasing flow and the energy spectrum characteristics of different layers of rock, realized the precise coal-gangue detection technology in complex structure thick coal seam with multiple gangue. Field tests were conducted in Lilou, Xiaoyu and Tashan Coal Mine. The test results were well corroborated with the research results and achieved the expected results, which laid the foundation for the field application of intelligent coal mining.

A unified multi-step wind speed forecasting framework based on numerical weather prediction grids and wind farm monitoring data

Wind speed forecasting is the basis of wind farm operation, which provides a reference for the future operation status evaluation of wind farms. For the wind speed forecast of wind turbines in the whole wind farm, a strategy combining unified forecast and single site error correction is proposed in this paper. The unified forecast framework is composed of a unified forecast model and multiple single site error correction models, which combines the forecasted grids of numerical weather prediction (NWP) with the monitoring data of wind farms. The proposed unified forecast model is called spatiotemporal conversion deep predictive network (STC-DPN), which is composed of temporal convolution network (TCN) and 2D convolution long short-term memory network (ConvLSTM). Firstly, the NWP forecasted grids are interpolated to the fan location, and the sequence matrix is composed of the NWP data and the monitored data of each wind turbine according to the time series, which is entered into the TCN network for time sequence feature extraction. Then, the output of the TCN network is converted into a regular spatio-temporal data matrix, which is entered into the ConvLSTM network for joint learning of spatio-temporal features to obtain the wind speed sequence forecasted in the whole wind farm. Finally, an independent TCN-LSTM error correction model is added for each site. Variational modal decomposition (VMD) is used to process data series, and different processing methods are adopted in unified forecast and single site error correction. In the 96 steps forecast test of a wind farm from Jining City, China, the proposed method is superior to several baseline methods and has important practical application value.

Experimental study on extraction of organic matter from coal by liquid CO2

Liquid CO2 injection into coal seam for CH4 displacement is one of the important means to strengthen gas extraction and carbon sequestration in recent years. Liquid CO2 is a non-polar solvent, based on the principle of “similar phase solubility”, part of non-polar or weakly polar organic matter can be dissolved in liquid CO2. By controlling the key parameters of liquid CO2 in the extraction process, small molecular organic matter can be dissolved and migrated from the pore structure of coal as far as possible. In order to clarify the main factors affecting the process of liquid CO2 extraction of coal body, the experimental platform of liquid CO2 extraction of organic matter in coal body was built by ourselves, the single factor experimental design method was used to analyze the evolution law of organic matter components in the process of liquid CO2 extraction of coal body. The experimental results showed that: the macromolecular structure of coal was not significantly damaged by liquid CO2 extraction; after liquid CO2 extraction, the relative contents of aliphatic hydrocarbon and aromatic hydrocarbon are positively correlated with pressure, the dissolution of ketones, alcohols, ethers and esters has the characteristics of stage, time sequence and explosion; at the initial stage of extraction, the dissolved compounds are free in the structure of the macromolecular network. With the increase of extraction pressure and time, the small molecular compounds of hydrocarbons and non-hydrocarbons in the embedded state are dissolved.

Signal Modulation Recognition Algorithm Based on Improved Spatiotemporal Multi-Channel Network

Automatic modulation recognition (AMR) plays an essential role in modern communication systems. In recent years, various modulation recognition algorithms based on deep learning have been emerging, but the problem of low recognition accuracy has not been solved well. To solve this problem, based on the existing MCLDNN algorithm, in this paper, we proposed an improved spatiotemporal multi-channel network (IQ-related features Multi-channel Convolutional Bi-LSTM with Gaussian noise, IQGMCL). Firstly, dividing the input IQ signals into three channels, time sequence feature extraction is carried out for route I, route Q, and route IQ, respectively. For route IQ, convolution kernel (2,1) is first used to extract relevant features. Two layers of the small convolution kernel (1,3) are used to extract time sequence features further, and the three channels are used to extract features further. Then, a two-layer short-length memory network is used to extract features from time and space more effectively. Through comparison experiments, Bi-LSTM is introduced to replace one layer of LSTM, and a fully connected layer is removed to prevent overfitting. Finally, multiplicative Gaussian noise is introduced to naturally corrode the feature parameters, further improving the robustness and accuracy of the model. Experiments are carried out on three public datasets RML2016.10a, RML2016.10b, and RML2016.04C. The experiments show that the IQGMCL network has higher recognition accuracies on all datasets, especially on the RML2016.10a dataset. When the SNR is 4 dB, the recognition accuracy reaches 93.52%. When the SNR is greater than 0 dB, the average recognition accuracy reaches 92.3%, 1.31%, and 1.2% higher than the original MCLDNN network, respectively.

Optimal allocation of electric vehicle charging stations and renewable distributed generation with battery energy storage in radial distribution system considering time sequence characteristics of generation and load demand

The addition of electric vehicle (EV) charging station (EVCS)/EV battery swapping stations (EVBSSs) in radial distribution system (RDS) draws extra real power from the distribution substation. This paper proffers a novel strategy for obtaining the best location of EVCS/EVBSSs in the RDS. Further, the EV charger has been modeled as constant current load and the influence of EVCS/EVBSSs demand on the voltage profile, real power loss, total voltage deviation, energy loss cost and overall operating cost of the RDS have been investigated considering constant power (CP), industrial (IL), residential (RES) and commercial (COM) load models. In order to make the network more self-sustainable and reliable, it is obligatory to assimilate the distributed generations (DGs) of optimal size and at proper location in the RDS to diminish the impact of EVCS/EVBSS(s) load. In addition, non-dispatchable solar photovoltaic (SPV) and wind turbine (WT) units are converted into a dispactable SPV and WT units with a combination of battery energy storage (BES) (i.e., SPV-BES and WT-BES). This research work suggests a novel chaotic student psychology based optimization (SPBO) (CSPBO) algorithm to acquire the optimal size and site of SPV-BES, WT-BES and biomass in IEEE 33-bus and practical Brazil 136-bus RDS for CP, IL, RES and COM load models considering the average hourly load demand profile and the average hourly variation in generation profile of SPV and WT. The obtained results based on benchmark test problems reveals that chaotic maps are proficient to enhance the performance of the SPBO algorithm significantly in terms of local optima circumvention and faster convergence mobility. The obtained outcomes show that the attained size and site of renewable DGs in the RDS may be feasible ones. The attained outcomes of the proffered CSPBO algorithm are contrasted to SPBO and Harris hawk's optimization algorithm. The yielded results will definitely help the EV and distribution industry in improving the reliability and the efficiency of the system.

Research on PM2.5 Pollution Prediction Method in Hefei City Based on CNN-LSTM Hybrid Model

The air quality data has the characteristics of large scale and high time sequence. In the deep learning model, CNN has good feature extraction and expression ability through convolution operation. LSTM has strong time series data processing ability, which effectively solves the long-term dependence problem in the time series data that CNN cannot process. However, a single feature extraction or time series processing method will affect the accuracy of the model. This paper proposes a PM2.5 pollution prediction method based on the CNN-LSTM hybrid model, which can fuse the time series and nonlinear features of the data. By selecting the air quality and meteorological data of Hefei City in Anhui Province from 2018 to 2021 for prediction research, the experimental results show that the CNN-LSTM hybrid model is better than the LSTM model in terms of RMSE, MAE, and R2 evaluation indexes, which reflects higher prediction accuracy. It can not only be used for real-time prediction of PM2.5 pollution, but also for the prediction of other pollutants in the atmosphere. It also provides a reference for the application of CNN-LSTM model in solving other time series big data.

A collaborative scheduling method for cloud computing heterogeneous workflows based on deep reinforcement learning

Aiming at the problem of low overall service quality caused by the disordered collaboration of heterogeneous workflows and discontinuous task execution in cloud computing scenarios, this paper proposes a collaborative scheduling method for heterogeneous workflows in cloud computing based on deep reinforcement learning. The method optimizes workflow makespan, cost, fairness and continuity in cloud computing under the constraints of task execution continuity. First, the structure and time sequence features are extracted for the dynamic scheduling process, and a reasonable scheduling decision support feature set is constructed. Second, a time-step adaptive scheduling mechanism is designed to simplify redundant information in the scheduling process and enables the agent to achieve efficient learning. In addition, using equilibrium, priority and preference scheduling strategies, an immediate-lag compound reward mechanism and a scheduling-switching hybrid action are designed to achieve a unification of the agent’s learning objectives and actual scheduling requirements. Finally, by constructing a simulation platform and conducting comparative experiments with four other algorithms, the results show that the proposed method has advantages in collaborative optimization of high-dimensional objectives under task continuity constraints. Including the task loading strategy can optimize the makespan performance by 16.6% and improve the fairness index by 5.3%.

Power grid fault diagnosis model based on the time series density distribution of warning information

Fast and intelligent power grid fault diagnosis is a key means of improving the intelligence level of power grid dispatching. Currently, data-driven power grid fault diagnosis lacks a method to quantify the time series characteristics of alarm information, and it is difficult to build a deep learning model. This paper proposes a power grid fault diagnosis method based on the time series density distribution of alarm information. First, the alarm information is classified; quantized time sequence feature extraction is performed for the device, and the obtained quantized time sequence features are superimposed. After graphical coding, a time series density distribution map of the alarm information of each device and a mosaic map reflecting the overall fault event are generated, providing a unified data representation for the deep learning model. Then, a faulty equipment type classification model and equipment state judgment model are constructed based on a residual network. The faulty equipment and fault type are determined using a sequential evaluation strategy that first determines the type of faulty equipment and then determines the state of the equipment. The proposed method can meet the requirements of on-line fault diagnosis of a power grid and provide emergency decision support for dispatchers.

Research on power quality disturbance analysis and identification based on LSTM

In view of the cumbersome and inaccurate process caused by manual feature extraction in power quality disturbance classification, according to the characteristics of power quality classification and time sequence. This article presents a method of power quality disturbance analysis and identification based on LSTM. Firstly, the random single electric energy signal is spliced into a large signal to form a continuous electric energy signal sequence. Secondly, based on the existing neural network, an LSTM model suitable for PQD classification is constructed, and then the spliced large signals are used as input to train and optimize the model. The LSTM model will classify different power quality disturbances. Finally, six common power quality disturbances such as voltage sag, voltage swell, interruption, impact, oscillation and harmonic are simulated and verified respectively. The analysis results show that the high accuracy of the method is reflected, which proves the correctness and effectiveness of the proposed method, and is suitable for the power quality disturbance identification system.

Research on Named Entity Recognition Based on Model Fusion

In view of the defects of using deep learning method to identify named entities, which can’t represent polysemy and doesn’t contain context information, this paper proposes a new fusion model, which organically combines the bidirectional gated recurring neural network (BiGRU) and the dynamic graph edge evolutionary network (DGCNN) which are improved with attention mechanism. The time sequence features of the vector are captured by the improved BiGRU-ATT, and the local spatial features of the vector are enriched by DGCNN. Finally, fused output results are corrected and output by the constraint rules of CRF model, and the global optimal sequence is obtained by calculating the probability. The experimental results demonstrate that the proposed model is improved compared with the single model, which further highlights the superiority of the proposed fusion model.

Uncovering APT malware traffic using deep learning combined with time sequence and association analysis

Traditional malware detection methods based on static traffic characteristics and machine learning are hard to cope with the increasing number of APT malware variants. In order to alleviate this problem, this paper proposes a deep-learning-based malware classification approach that combines time sequence features and association rules features. This method uses the improved LSTM neural network structure named RESNET_LSTM and PARALLEL_LSTM to extract time sequence features of different protocol traffic. It also utilizes association analysis to generate quantitative rule features. Finally, we connect the time sequence feature vector and the quantization rule vector as input to deep learning models to detect malware traffic. We evaluated our proposed approach on a dataset consisting of malicious traffic generated by 57 types of malware and normal traffic. The experimental results demonstrate that the loss decline rate of PARALLEL_LSTM structure during the training phase is faster than that of the LSTM and RESNET_LSTM structures. When the RESNET_LSTM structure is used, the prediction accuracy is close to 100%, which is slightly higher than the other two structures. The accuracy of the detection methods proposed in this paper are all above 96%, while the accuracy of malware detection methods combined with static traffic characteristics and machine learning is about 85%.

Statistics of the Performance of Gridded Precipitation Datasets in Indonesia

Gridded precipitation datasets have been used as alternatives to rain gauge observations, but their applicability for a specific region should be thoroughly evaluated. This article aims at finding the most appropriate one for climatological and hydrological applications in Indonesia, by evaluating the statistics of the performance of eight different datasets (research products) having horizontal resolutions between 0.1 and 0.25 and with a time span of data availability from 2003 to 2015. The datasets are compared against the observed daily rainfall at 133 stations using 13 statistical metrics that can be classified into three groups with different characteristics of measurements, namely distribution, time sequence, and extreme value representations. By applying summation of rank (SR), it is found that MSWEP and TMPA 3B42 are the top two datasets that outperformed based on distribution and time sequence performance metric groups. The extreme performances for all datasets are still good in 75th percentiles; however, the performances decrease at more than 75th percentiles indicating still a poorly representation of daily extreme rainfall for all gridded datasets. Results of this study suggest that MSWEP (v2) is presently the best gridded precipitation datasets available for climatological and hydrological applications in Indonesia.

Deep-reinforcement-learning-based self-organization of freely undulatory swimmers.

It is fascinating that fish groups spontaneously form different formations. The collective locomotions of two and multiple undulatory self-propelled foils swimming in a fluid are numerically studied and the deep reinforcement learning (DRL) is applied to control the locomotion. We explored whether typical patterns emerge spontaneously under the driven two DRL strategies. One strategy is that only the following fish gets hydrodynamic advantages. The other is that all individuals in the group take advantage of the interaction. In the DRL strategy, we use swimming efficiency as the reward function, and the visual information is included. We also investigated the effect of involving hydrodynamic force information, which is an analogy to that detected by the lateral line of fish. Each fish can adjust its undulatory phase to achieve the goal. Under the two strategies, collective patterns with different characteristics, i.e., the staggered-following, tandem-following phalanx and compact modes emerge. They are consistent with the results in the literature. The hydrodynamic mechanism of the above high-efficiency collective traveling modes is analyzed by the vortex-body interaction and thrust. We also found that the time sequence feature and hydrodynamic information in the DRL are essential to improve the performance of collective swimming. Our research can reasonably explain the controversial issue observed in the relevant experiments. The paper may be helpful for the design of bionic fish.

Types and occurrence time of rockbursts in tunnel affected by geological conditions and drilling & blasting procedures

Rockburst is a difficult problem that restricts the development of deep rock mass engineering technology, and the warning of rockburst occurrence time remain a common challenge worldwide. This paper presents geological information and rockburst occurrence time in different drilling & blasting procedures collected in detail from a deep tunnel. A characterization method of MS (Microseismic) sequences has been designed to characterize the evolution law of microseismicity in the blasting cycle. An LSTM (Long Short-Term Memory) model is established to construct and augment the MS sequence samples, and a database was established based on information such as geology, excavation, support, MS sequence, and rockburst. Based on the information in 600 blasting cycles of the database, a multisource data mining method for analyzing the MS sequence formation mechanism is established. The relationship between the characteristics of rockburst occurrence time and MS sequence is established, and the formation mechanism of the MS sequence is used to interpret the potential rockburst type and occurrence time affected by geological conditions and drilling & blasting procedures. The results show that the number and spacing of structural planes have an important influence on the frequency, intensity and type of rockbursts. Rockburst may occur in each procedure of the blasting cycle. The frequency of rockburst during the danger elimination and support stage is the highest, during the drilling and charging blasting stage is the lowest. The MS sequence in blasting cycle is divided into four types, which have a good relationship with the rockburst occurrence time in blasting cycle procedures. Rockburst type and occurrence time in different procedures is mainly controlled by the rock mass structure, excavation footage, excavation rate and blasting time interval. The results have important reference value and guiding significance for rockburst time warning and hazard control.

Research on the Prediction of the Operational Risk Field of Intelligent Vehicles Based on Dual Multiline LiDAR

To effectively evaluate the risk situation between intelligent vehicles and surrounding traffic participants in complex scenes, a complex traffic environment perception technology based on dual multiline light detection and ranging (LiDAR) is proposed in this work. The vehicle motion state is predicted by fusing the multiview characteristics of point cloud timing and multitarget interaction information, and the risk assessment model is constructed via artificial potential field theory. The real-time point cloud information is used to obtain the time-sequence bird’s-eye view and range image. The improved VGG19 network model is used to extract the time-sequence high-level abstract combined features in the multiview scene. The constructed time-sequence feature vector is used as the input data of the attention mechanism, and the attention-bidirectional long short-term memory (Attention-BiLSTM) model is used for training to form the desired input-output mapping relationship. The motion state of the target vehicle can therefore be updated, and the static and dynamic risk fields of traffic participants surrounding the vehicle can be established based on artificial potential field theory, thereby allowing for the evaluation of the operational risk of the intelligent vehicle. The results of experiments demonstrate that the prediction effect of the target vehicle state parameters via the use of the proposed model is better than that of other compared models, and the prediction effect of the risk field of intelligent vehicle operation based on the multiview point cloud features and vehicle interaction information is good.

EEMD and GUCNN-XGBoost Joint Recognition Algorithm for Detection of Precursor Chemicals Based on Semiconductor Gas Sensor

The precursor chemical (PC) plays a crucial role in the drug manufacture. In recent years, PC intelligent recognition technology based on the analysis of semiconductor gas sensor signals has become a research hotspot due to PC’s volatility. However, in practice, the weak discrimination for commercial semiconductor gas sensor signals of PC gases is frequent, making it difficult to accurately recognize PC gases by using the existing intelligent recognition methods. To tackle this problem, this article proposes a data-driven model based on ensemble empirical mode decomposition (EEMD) and gate recurrent unit (GRU) union convolutional neural network XGBoost (GUCNN-XGBoost) methods for PC gases recognition. Firstly, the signals are collected by the commercial semiconductor gas sensor (MQ2) under the square wave temperature modulation, and then processed by the EEMD to obtain the intrinsic mode functions (IMF1-IMF7). Secondly, the GUCNN-XGBoost model is constructed to automatically extracts the binary morphological characteristics (BMC) and the time sequence characteristics (TSC) of the IMF1-IMF6; The XGBoost is utilized as the final classifier instead of common softmax to realize the high precision of 97.54%. Subsequently, we employ the IMF7-XGBoost regression model to achieve the accurate estimation of PC gases concentration, which can result in an average mean absolute error (MAE) of 4.98. Finally, a typical comparative experimental analysis validates the effectiveness of the proposed approach.

Remaining Useful Life Prediction of Machinery: A New Multiscale Temporal Convolutional Network Framework

With the rapid development of deep learning (DL) techniques, data-driven models have been increasingly used in remaining useful life (RUL) prediction, in which convolution neural network (CNN)-based RUL prognostics models have received special attention. However, there are still two main issues that need to be addressed: (1) Traditional CNN is not suitable to extract the time-sequence characteristics from the long-term historical signals. (2) The receptive field range of convolution operation is fixed, thus only learning the feature information at a specific scale, which is insufficient for complex feature extraction. To address these two issues, a multi-scale temporal convolutional network (MsTCN) which has powerful time-sequence characteristics is proposed for RUL prediction in this paper. The MsTCN adopts the temporal convolutional network (TCN) framework, which is good at extracting time-sequence information. Based on this, a new multi-scale dilated causal convolution residual block (MsDCCRB) is developed to constitute the RUL prognostics model, where multiple dilated convolutions (DCs) are based on different dilation factors are put on each layer in parallel. Furthermore, the squeeze-and-excitation (SE) unit is embedded into the MsDCCRB to adaptively recalibrate the sequence feature responses and enhance the representation learning ability. Through stacking multiple MsDCCRBs, the historical condition monitoring data can be fed directly into the proposed model to realize the high-level representations of RUL estimation. Finally, the proposed approach is validated with the accelerated whole-life degradation dataset of rolling element bearings (REBs). The experimental results exhibit the proposed MsTCN achieves a higher RUL prediction accuracy, which is superior to some state-of-the-art data-driven prognostics methods.

Seamless and intuitive control of a powered prosthetic leg using deep neural network for transfemoral amputees.

Powered prosthetic legs are becoming a promising option for amputee patients. However, developing safe, robust, and intuitive control strategies for powered legs remains one of the greatest challenges. Although a variety of control strategies have been proposed, creating and fine-tuning the system parameters is time-intensive and complicated when more activities need to be restored. In this study, we developed a deep neural network (DNN) model that facilitates seamless and intuitive gait generation and transitions across five ambulation modes: level-ground walking, ascending/descending ramps, and ascending/descending stairs. The combination of latent and time sequence features generated the desired impedance parameters within the ambulation modes and allowed seamless transitions between ambulation modes. The model was applied to the open-source bionic leg and tested on unilateral transfemoral users. It achieved the overall coefficient of determination of 0.72 with the state machine-based impedance parameters in the offline testing session. In addition, users were able to perform in-laboratory ambulation modes with an overall success rate of 96% during the online testing session. The results indicate that the DNN model is a promising candidate for subject-independent and tuning-free prosthetic leg control for transfemoral amputees.