Massive Historical Data Research Articles

A transfer learning CNN-LSTM network-based production progress prediction approach in IIoT-enabled manufacturing

In make-to-order manufacturing workshops, accurate prediction value of production progress (PP) is a significant reference index for dynamic optimisation of production process and on-time delivery of production orders. The implementation of big data and Industrial Internet of Things (IIoT) in manufacturing workshops makes it possible to obtain large amounts of production data which can affect PP. However, the particularities of massive historical order data are not fully excavated and the amount of target order data is insufficient to support the training of high-precision prediction model, which will result in bad training approximation and generalisation. To overcome these shortcomings, a PP prediction approach consisting of two models with transfer learning (TL) is proposed. TL can avoid the training of PP prediction model from scratch every time. Consequently, computational efficiency can be greatly improved. A convolutional neural network (CNN) model with TL is devised to excavate the comprehensive features from historical and current orders. Additionally, a long short-term memory network (LSTM) model with TL is constructed to fit the nonlinear relation of the features provided by CNN-TL model for PP prediction. In order to validate the performance of the proposed PP prediction approach, comparative experiments of eight algorithms are conducted in an IIoT-enabled manufacturing workshop.

Physics-informed machine learning model for battery state of health prognostics using partial charging segments

The accurate and efficient estimation of battery state-of-health (SoH) is an ever-significant issue for applications of lithium-ion batteries (LIBs). Physics-of-failure (PoF) and machine-learning (ML) approaches have shown success in the field of prognostics and health management. However, accurate prediction of these models depends on clear understandings of complicated underlying physics, or full access to massive historical usage data of the system, which are not always obtainable in the engineering applications. Studies on physics-informed ML frameworks have gained attention because they embed physics knowledge into ML algorithms; recently, this method has demonstrated the potential to enhance performance over traditional model-based techniques, particularly when handling highly non-linear complex systems, such as LIBs. In this study, an advanced physics-informed ML (PIML) framework is proposed for the LIB SoH estimation, which includes three steps. The physics-based finite element (FE) model is firstly used to calculate the influences of the dominating aging mode, i.e., the solide electrolyte interface (SEI) growth on anode particle surfaces, and on capacity loss of LIB under various operating conditions. Then, the FE results are fused with experimental data from NASA Ames Prognostics Center of Excellence to construct a multi-fidelity model. Lastly, the Gaussian Process Regression (GPR) model is trained to create the mapping between voltage curves and the corresponding SEI thickness. A case study with the partial charging voltage segment as input is introduced to validate the PIML framework. Overall, the framework demonstrates a favorable performance for SoH estimation, as well as providing a basis for future online estimation frameworks.

An online identification approach for ship domain model based on AIS data.

As an important basis of navigation safety decisions, ship domains have always been a pilot concern. In the past, model parameters were usually obtained from statistics of massive historical cumulative data, but the results were mostly historical analysis and static data, which obviously could not meet the needs of pilots who wish to master the ship domain in real time. To obtain and update the ship domain parameter online in time and meet the real-time needs of maritime applications, this paper obtains CRI as the weight coefficient-based PSO-LSSVM method and proposes to use short-term AIS data accumulation through the risk-weighted least squares method online rolling identification method, which can filter nonhazardous targets and improve the identification accuracy and real-time performance of nonlinear models in the ship domain. The experimental examples show that the method can generate the ship domain dynamically in real time. At the same time, the method can be used to study the dynamic evolution characteristics of the ship domain over the course of navigation, which provides a reference for navigation safety decisions and the analysis of ship navigation behavior.

Source Analysis of Ambient PM2.5 in Wuhan City Based on Random Forest Model

Based on the online monitoring data of fine particle(PM2.5) mass concentration, carbonaceous components, ionic constituents, and elemental components in an urban site of Wuhan from December 2019 to November 2020, the chemical characteristics of PM2.5 were analyzed. In addition, seasonal source apportionment of PM2.5 was conducted using the principal component analysis(PCA) method and random forest(RF) algorithm model. The results indicated that ρ(PM2.5) was the highest in winter[(61.33±35.32) μg·m-3] and the lowest in summer[(17.87±10.06) μg·m-3]. Furthermore, organic carbon(OC), with a concentration of(7.27±3.51) μg·m-3, accounted for the major proportion compared with that of elemental carbon(EC) in the carbonaceous component of PM2.5. NO3-, SO42-, and NH4+ had the highest proportion in ionic components, with concentrations of (11.55±3.86),(7.55±1.53), and (7.34±1.99) μg·m-3, respectively. K, Fe, and Ca were the main elements in elemental components, with concentrations of (752.80±183.98),(542.34±142.55), and (459.70±141.99) ng·m-3, respectively. Relying on main factor extraction by PCA and quantitative analysis by RF, five emission sources were ultimately confirmed. The seasonal concentration distribution of these emission sources was as follows:coal burning and secondary sources(46%, 39%, 41%, and 52% for spring, summer, autumn, and winter, respectively) made the highest contribution to PM2.5, followed by vehicle emission sources(22%, 28%, 27%, and 21%), industrial emission sources (14%, 18%, 17%, and 13%), dust sources (10%, 8%, 6%, and 6%), and biomass burning sources (8%, 7%, 9%, and 8%). The valuation of the RF model was evaluated using multiple indicators, including RMSE, MSE, and R2. The evaluation results showed that the model for winter had the best performance (R2=0.974, RMSE=3.795 μg·m-3, MAE=2.801 μg·m-3), the models for spring (R2=0.936, RMSE=3.512 μg·m-3, MAE=2.503 μg·m-3) and autumn (R2=0.937, RMSE=4.114 μg·m-3, MAE=3.034 μg·m-3) performed with moderate-fitting goodness, and the summer model showed a relatively weak-fitting performance (R2=0.866, RMSE=5.665 μg·m-3, MAE=3.889 μg·m-3). The RF model had a satisfactory performance in PM2.5 source apportionment and had excellent prospects in analyzing massive historical data of air pollutants.

A multi-sensory stimulating attention model for cities\u2019 taxi service demand prediction

Taxi demand forecasting is crucial to building an efficient transportation system in a smart city. Accurate taxi demand forecasting could help the taxi management platform to allocate taxi resources in advance, alleviate traffic congestion, and reduce passenger waiting time. Thus, more efforts in industrial and academic circles have been directed towards the cities’ taxi service demand prediction (CTSDP). However, the complex nonlinear spatio-temporal relationship in demand data makes it challenging to construct an accurate forecasting model. There remain challenges in perceiving the micro spatial characteristics and the macro periodicity characteristics from cities’ taxi service demand data. What’s more, the existing methods are significantly insufficient for exploring the potential multi-time patterns from these demand data. To meet the above challenges, and also stimulated by the human perception mechanism, we propose a Multi-Sensory Stimulus Attention (MSSA) model for CTSDP. Specifically, the MSSA model integrates a detail perception attention and a stimulus variety attention for capturing the micro and macro characteristics from massive historical demand data, respectively. The multiple time resolution modules are employed to capture multiple potential spatio-temporal periodic features from massive historical demand data. Extensive experiments on the yellow taxi trip records data in Manhattan show that the MSSA model outperforms the state-of-the-art baselines.

Self-supervised pretraining via contrast learning for intelligent incipient fault detection of bearings

Data-driven approaches for prognostic and health management (PHM) increasingly rely on massive historical data, yet annotations are expensive and time-consuming. Learning approaches that utilize semi-labeled or unlabeled data are becoming increasingly popular. In this paper, a self-supervised pre-training via contrast learning (SSPCL) is introduced to learn discriminative representations from unlabeled bearing datasets. Specifically, the SSPCL employs momentum contrast learning (MCL) to investigate the local representation in terms of instance-level discrimination contrast. Further, we propose a specific architecture for SSPCL deployment on bearing vibration signals by presenting several data augmentations for 1D sequences. On this basis, we put forward an incipient fault detection method based on SSPCL for run-to-failure cycle of rolling bearings. This approach transfers the SSPCL pre-trained model to a specific semi-supervised downstream task, effectively utilizing all unlabeled data and relying on only a little priori knowledge. A case study on FEMTO-ST datasets shows that the fine-tuned model is competent for incipient fault detection, outperforming other state-of-the-art methods. Furthermore, a supplemental case on a self-built fault datasets further demonstrate the great potential and superiority of our proposed SSPCL method in PHM.

Research on the Route Pricing Optimization Model of the Car-Free Carrier Platform Based on the BP Neural Network Algorithm

The car-free carrier platform is a product of the rapid development of the modern logistics industry and has a vital strategic value for promoting the construction of a country’s comprehensive transportation. However, due to the unreasonable platform pricing model, the industry is currently in a bottleneck period. In order to solve this problem, we established a gray correlation model to calculate the degree of correlation between each characteristic index and platform pricing based on the massive historical transaction data of a certain platform and performed K-means clustering on the results to discover the main factors affecting platform pricing. Based on the abovementioned results, we created a pricing optimization model based on the BP neural network, with the structure of 8-13-1 to predict the freight pricing of the order and test the prediction results. The test shows that the goodness of fit (R2) of the predicted value is close to 1, and the prediction error range is less than 3.7%, which proves the accuracy and effectiveness of the BP neural network model and provides an effective reference for the optimization of the pricing model of the car-free carrier platform.

A new dynamic model and transfer learning based intelligent fault diagnosis framework for rolling element bearings race faults: Solving the small sample problem

Intelligent fault diagnosis of rolling element bearings gains increasing attention in recent years due to the promising development of artificial intelligent technology. Many intelligent diagnosis methods work well requiring massive historical data of the diagnosed object. However, it is hard to get sufficient fault data in advance in real diagnosis scenario and the diagnosis model constructed on such small dataset suffers from serious overfitting and losing the ability of generalization, which is described as small sample problem in this paper. Focus on the small sample problem, this paper proposes a new intelligent fault diagnosis framework based on dynamic model and transfer learning for rolling element bearings race faults. In the proposed framework, dynamic model of bearing is utilized to generate massive and various simulation data, then the diagnosis knowledge learned from simulation data is leveraged to real scenario based on convolutional neural network (CNN) and parameter transfer strategies. The effectiveness of the proposed method is verified and discussed based on three fault diagnosis cases in detail. The results show that based on the simulation data and parameter transfer strategies in CNN, the proposed method can learn more transferable features and reduce the feature distribution discrepancy, contributing to enhancing the fault identification performance significantly.

Short-term household load forecasting based on Long- and Short-term Time-series network

Focusing on the issue of significant randomness and low latitude of short-term household electrical load data, this paper proposes a novel short-term load multi-step forecasting method based on long-and short-term time series network (LSTNet). Firstly, the time sliding window method is used to sample massive historical load data to construct feature maps as input. Secondly, convolutional neural network (CNN) and long short-term memory (LSTM) are used to capture temporal short-term local information and long-term related information respectively, and autoregressive (AR) models are used as linear components. Then, the models will be evaluated using a scheme called walk-forward validation, and the average absolute percentage error (MAPE) and root mean square error (RMSE) are used as accuracy evaluation indicators. Finally, the four-year electric load data of a family in Paris, France is used to verify the proposed method and comprehensively compare the proposed method with the three most popular load forecasting algorithms. The experimental results show that in the prediction results for the next week, the MAPE and RMSE of the prediction method proposed in the paper are smaller than those of other algorithms, which can more effectively express the time series relationship of household short-term load and have higher prediction accuracy.

Research on telemetry data compression technology based on inter frame differential adaptive run length encoding

Aiming at the problem of massive historical telemetry data storage in flight test, a lossless compression method based on adaptive interval run length encoding is proposed. Aiming at the problem of low compression efficiency of traditional run length encoding algorithm for word data, by studying the storage characteristics of telemetry data, this algorithm automatically identifies the frame format of telemetry data, and carries out longitudinal run length adaptive interval encoding for inter frame differential data to improve the compression efficiency. The test results show that the compression ratio of the improved algorithm is improved by 58.1% and 1.5% compared with the traditional run length encoding algorithm and the inter frame differential lateral run length encoding algorithm.

Bearing Faulty Prognostic Approach Based on Multiscale Feature Extraction and Attention Learning Mechanism

Recently, researches on data‐driven faulty identification have been achieving increasing attention due to the fast development of the modern conditional monitoring technology and the availability of the massive historical storage data. However, most industrial equipment is working under variable industrial operating conditions which can be a great challenge to the generalization ability of the normal data‐driven model trained by the historical storage operating data whose distribution might be different from the current operating datasets. Moreover, the traditional data‐driven faulty prognostic model trained on massive historical data can hardly meet the real‐time requirement of the practical industry. Since the hierarchical feature extraction can enhance the model generalization ability and the attention learning mechanism can promote the prediction efficiency, this paper proposes a novel bearing faulty prognostic approach combining the U‐net‐based multiscale feature extraction network and the CBAM‐ (convolutional block attention module‐) based attention learning network. First, time domain conditional monitoring signals are converted into the two‐dimensional gray‐scale image which can be applicable for the input of the CNN. Second, a CNN model based on the U‐net structure is adopted as the feature extractor to hierarchically extract the multilevel features which can be very sensitive to the faulty information contained in the converted image. Finally, the extracted multilevel features containing different representations of the raw signals are sent to the designed CBAM‐based attention learning network for high efficiency faulty classification with its unique emphasize discrimination characteristic. The effectiveness of the proposed approach is validated by two case studies offered by the CWRU (Case Western Reserved University) and the Paderborn University. The experimental result indicates that the proposed faulty prognostic approach outperforms other comparison models in terms of the generalization ability and the speed‐up properties.

Optimal frequency regulation based on characterizing the air conditioning cluster by online deep learning

The air conditioning cluster (ACC) is a potential candidate to provide frequency regulation reserves. However, the effective assessment of the ACC willing reserve capacity is often the obstacle for the existing demand response (DR) programs, influenced by incentive prices, temperatures, etc. In this paper, the complex relationship between the ACC willing reserve capacity and its key influence factors is defined as the demand response characteristic (DRC). To learn the DRC along with real-time frequency regulation, an online deep learning-based DRC (ODL-DRC) modeling methodology is designed to retrain the deep neural network-based model continuously. The ODL-DRC model trained by incoming new data does not require massive historical training data any more, which is more time-efficient. Then, the coordinate operation between ODL-DRC modeling and optimal frequency regulation (OFR) is put forward. A robust decentralized sliding mode controller (DSMC) is designed to manage the ACC response power in primary frequency regulation against the ACC response uncertainty. An ODL-DRC model-based OFR scheme is formulated by taking the learning error into consideration. So that the ODL-DRC model can be applied to minimize the total operation cost while maintaining frequency stability, without waiting for a well-trained model. The simulation cases validate the superiority of the OFR based on characterizing the ACC by online learning, which can capture the real DRC and optimize the regulation performance simultaneously with strong robustness against the ACC response uncertainty and the learning error.

Prediction of fire risk based on cloud computing

The development of cloud computing and big data analysis has given rise to various disaster prediction methods. To reduce the probability of fire accidents, it is critical to predict the fire risk by mining the massive historical data on fire. Considering the advantages of MapReduce, a cloud computing method, in parallel processing of data, this paper puts forward a novel prediction method for fire risk that mines the association rules in the time domain. Firstly, the risk of disaster-causing factors and the ability of disaster-reducing factors were evaluated. Based on the evaluation results, an evaluation index system was constructed for fire risk, and the indices were quantified through proper weighting. Facing the historical fire data, the authors designed the spatiotemporal density-based spatial clustering of applications with noise (spatiotemporal DBSCAN), and quantitatively evaluated fire risk by the association rule mining algorithm based on time domain partition (TDP). The effectiveness of our method in fire risk prediction was verified through experiments. The research results provide reference for the risk prediction of other disasters.

Joint Geosequential Preference and Distance Metric Factorization for Point-of-Interest Recommendation

Point-of-interest (POI) recommendation is a valuable service to help users discover attractive locations in location-based social networks (LBSNs). It focuses on capturing users’ movement patterns and location preferences by using massive historical check-in data. In the past decade, matrix factorization has become a mature and widely used technology in POI recommendation. However, the inner product of latent vectors adopted in matrix factorization methods does not satisfy the triangle inequality property, which may limit the expressiveness and lead to suboptimal solutions. Besides, the extreme sparsity of check-in data makes it challenging to capture users’ movement preferences accurately. In this paper, we propose a joint geosequential preference and distance metric factorization framework, called GeoSeDMF, for POI recommendation. First, we introduce a distance metric factorization method that is capable of learning users’ personalized preferences from a position and distance perspective in the metric space. Specifically, we convert the user-POI interaction matrix into a distance matrix and factorize it into user and POI dense embeddings. Additionally, we measure users’ personalized preference for the POI by using the Euclidean distance metric instead of the inner product. Then, we model the users’ geospatial preference by applying a geographic weight coefficient and model the users’ sequential preference by using the Euclidean distance of continuous check-in locations. Moreover, a pointwise loss strategy and AdaGrad algorithm are adopted to optimize the positions and relationships of users and POIs in a metric space. Finally, experimental results on three large-scale real-world datasets demonstrate the effectiveness and superiority of the proposed method.

Health status prediction of airborne systems based on transfer learning

In this paper, based on the prediction of the decay mode of the system health state, a health pattern recognition and prediction method based on transfer learning is proposed. In the context of big data, the system's healthy decline mode is summarized from the massive historical flight data, and then the research on the health status of the airborne system based on the recognition results is carried out. Firstly, this paper demonstrates the feasibility of transfer learning applied to the prediction of the health status of airborne systems. Then, a HMM-based parameter migration health state prediction method is proposed. Finally, the model is verified by the hydraulic system of a certain type of aircraft. The results show that the model can predict the time when the health state changes.

Estimating Urban Road GPS Environment Friendliness with Bus Trajectories: A City-Scale Approach †.

GPS is taken as the most prevalent positioning system in practice. However, in urban areas, as the GPS satellite signal could be blocked by buildings, the GPS positioning is not accurate due to multi-path errors. Estimating the negative impact of urban environments on GPS accuracy, that is the GPS environment friendliness (GEF) in this paper, will help to predict the GPS errors in different road segments. It enhances user experiences of location-based services and helps to determine where to deploy auxiliary assistant positioning devices. In this paper, we propose a method of processing and analysing massive historical bus GPS trajectory data to estimate the urban road GEF integrated with the contextual information of roads. First, our approach takes full advantage of the particular feature that bus routes are fixed to improve the performance of map matching. In order to estimate the GEF of all roads fairly and reasonably, the method estimates the GPS positioning error of each bus on the roads that are not covered by its route, by taking POIinformation, tag information of roads, and building layout information into account. Finally, we utilize a weighted estimation strategy to calculate the GEF of each road based on the GPS positioning performance of all buses. Based on one month of GPS trajectory data of 4835 buses within the second ring road in Chengdu, China, we estimate the GEF of 8831 different road segments and verify the rationality of the results by satellite maps, street views, and field tests.

Energy consumption forecast and charging demand alert based on operation condition clustering and control variable method

Travel anxiety of automobile owners has been aggravated because of the difficulty in accurately controlling the operation energy consumption and imperfection in charging infrastructure construction and other problems. Relying on the massive historical operation data of automobiles, it acquired the powerconsumption increasing coefficient of speed and temperature by means of clustering and control variable methods. Furthermore, the map Application Programming Interface (API) was invoked to obtain the path planning results thus realizing prediction on power consumption. The historical charging data of current automobile was used to build the mapping relations of the state of charge (SOC) and the state of energy (SOE). Combining with the prediction value of energy consumption it calculated the needed charge capacity and judge whether to issue the charging demand alert. Indicated by the application results, the proposed algorithm of energy-consumption forecast is more accurate than traditional average energy-consumption forecast algorithm. Accordingly, the charging demand alert function can effectively relive the travel anxiety of automobile owners.

Multi-objective combustion optimization based on data-driven hybrid strategy

In order to reduce pollutant discharge and improve boiler efficiency, data-driven hybrid strategy is proposed to solve the problem of multi-objective combustion optimization. First, massive historical operation data of a coal-fired power station are preprocessed (including resampling, steady-state detection, data cleaning and cluster analysis), so as to divide the whole boiler working condition into different partitions. Next, combustion association rules based multi-objective optimal strategy is applied to extract a combustion optimal rule from every partition, and a combustion optimal rule-base is built up by merging all the rules, so that the preliminary combustion optimization can be quickly finished based on the combustion optimal rule-base. Meanwhile, combustion mathematical model based multi-objective optimal strategy is applied to develop the LSSVR (least square support vector regression) model of boiler combustion process for every partition, and a combustion optimal model-base, which contains all the LSSVR models, is built up. After that, an improved multi-objective particle swarm optimization algorithm is presented to calculate Pareto optimal solutions depend on the corresponding LSSVR model with the constraint of real-time boiler working condition. To achieve further combustion optimization, the method of multiple attribute decision making is used to determine the unique solution from all the Pareto optimal solutions. Data-driven hybrid strategy is to combine the above two strategies. Simulation experiments verified the validity and feasibility of data-driven hybrid strategy on multi-objective test function ZDT1. Taking advantage of data-driven hybrid strategy, the comprehensive average of NOx emissions dropped by 29.63% and the comprehensive average of boiler efficiency increased by 0.69% in the application experiments with historical operation data under some boiler working conditions. Data-driven hybrid strategy based multi-objective combustion optimization makes the integration of instantaneity and effectiveness, so that it is suitable for online application.

Prediction of blood glucose concentration for type 1 diabetes based on echo state networks embedded with incremental learning

Valid prediction of blood glucose concentration can help people to manage diabetes mellitus, alert hypoglycemia/hyperglycemia, exploit artificial pancreas, and plan a treatment program. Along the development of continuous glucose monitoring system (CGMS), the massive historical data require a new modeling framework based on a data-driven perspective. Studies indicate that the glucose time series (i.e., CGMS readings) involve chaotic properties; therefore, echo state networks (ESN) and its improved variants are proposed to establish subject-specific prediction models owing to their superiority in processing chaotic systems. This study mainly has two innovations: (1) a novel combination of incremental learning and ESN is developed to obtain a suitable network structure through partial optimization of parameters; (2) a feedback ESN is proposed to excavate the relationship of different predictions. These methods are assessed on ten patients with diabetes mellitus. Experimental results substantiate that the proposed methods achieve superior prediction performance in terms of four evaluation metrics compared with three conventional methods.

Special Issue on Artificial Intelligence and Machine Learning for Networking and Communications

Research in large-scale networking systems has been shaped and will continue to be guided by specific characteristics of applications and the underlying platforms and infrastructures. On the one hand, applications are growing at an accelerated pace, which is fundamentally unpredictable in both breadth and depth. On the other hand, the underlying networking has been the focus of a huge transformation enabled by new models resulting from virtualization and cloud computing. This has led to a number of novel architectures supported by emerging technologies such as Software-Defined Networking (SDN), Network Function Virtualization (NFV), and more recently, edge cloud and fog networking, or network slicing [1] , [2] . This evolution towards enhanced design opportunities along with increasing complexity in networking and its applications has fueled the need for improved network automation in agile infrastructures. At the same time, their complexity has dramatically increased. The networking dynamics have had the effect of making it even more important and challenging to design scalable network measurement and analysis techniques and associated tools. Critical applications such as resource allocation, network monitoring, security enforcement, or dynamic network management require real-time mechanisms for online analysis as well as efficient techniques for offline deep analysis of massive historical data.