Sufficient Historical Data Research Articles

Model predictive control of switched nonlinear systems using online machine learning

This work introduces an online learning-based model predictive control (MPC) approach for the modeling and control of switched nonlinear systems with scheduled mode transitions. Initially, recurrent neural network (RNN) models are constructed offline, utilizing sufficient historical operational data to capture the nominal system dynamics for each mode. Subsequently, we employ real-time process data to develop online learning RNN models, aiming to approximate the dynamics of switched nonlinear systems in the presence of of bounded disturbances. In cases where the initial RNN model is unavailable for a specific switching mode due to very limited historical data, we use real-time data from closed-loop operations under a proportional–integral (PI) controller to build online learning RNN models. To evaluate the predictive performance of online learning RNNs, a theoretical analysis on their generalization error bound is developed using statistical machine learning theory. Additionally, considering the presence or absence of initial RNN models, two MPC schemes are developed. These schemes employ RNNs as prediction models to stabilize switched nonlinear systems, ensuring closed-loop stability by accounting for the generalization error bound derived for online learning RNNs. Finally, the effectiveness of the proposed MPC schemes is demonstrated through a nonlinear process example with two switching modes.

A deep transfer learning model for the deformation of braced excavations with limited monitoring data

The current deep learning models for braced excavation cannot predict deformation from the beginning of excavation due to the need for a substantial corpus of sufficient historical data for training purposes. To address this issue, this study proposes a transfer learning model based on a sequence-to-sequence two-dimensional (2D) convolutional long short-term memory neural network (S2SCL2D). The model can use the existing data from other adjacent similar excavations to achieve wall deflection prediction once a limited amount of monitoring data from the target excavation has been recorded. In the absence of adjacent excavation data, numerical simulation data from the target project can be employed instead. A weight update strategy is proposed to improve the prediction accuracy by integrating the stochastic gradient masking with an early stopping mechanism. To illustrate the proposed methodology, an excavation project in Hangzhou, China is adopted. The proposed deep transfer learning model, which uses either adjacent excavation data or numerical simulation data as the source domain, shows a significant improvement in performance when compared to the non-transfer learning model. Using the simulation data from the target project even leads to better prediction performance than using the actual monitoring data from other adjacent excavations. The results demonstrate that the proposed model can reasonably predict the deformation with limited data from the target project.

Fault diagnosis of landing gear retraction system with bond graph under uncertain conditions.

Given the lack of sufficient historical data for aircraft landing gear retractor systems, a model-based fault diagnosis approach is needed to overcome this data deficiency. Meanwhile, inherent uncertainties are inevitable in engineering practice, and it is a great challenge to construct a model that accurately reflects the complexity of the actual system under uncertain conditions. Due to the urgent need for reliable model-based diagnostic methods and the need to cope with inherent uncertainties, this paper proposes an improved fault diagnostic method aimed at increasing the diagnostic efficiency of the landing gear retractor system, a critical component in aircraft take-off and landing operations. Due to a lack of historical data, the model-based fault diagnosis method can solve the problem of lack of data. The proposed uncertainty method addresses the challenge of multiple sources of uncertainty by using subsystems to reduce complexity. Fault diagnosis is achieved by comparing residuals with thresholds derived from a diagnostic bond graph (DBG) model. To address the problem of limited fault data, we modeled and simulated the landing gear retractor system using AMESim®. In addition, the linear fractional transform (LFT) approach has been used to resolve parametric uncertainties, but is unable to resolve system structural uncertainties. Therefore, we also analyzed the comparative fault diagnosis results derived from the linear fractional transformation-DBG (LFT-DBG) and the subsystem-DBG approaches. The experimental results support the effectiveness of the subsystem approach in improving fault diagnosis accuracy and reliability, highlighting its potential as a viable diagnostic strategy in aerospace engineering applications.

Seagrass recovery trajectories and recovery potential in relation to nutrient reduction

Abstract Seagrass recovery has been reported across the globe where previously eutrophied waters have become less nutrient‐rich. In the European Wadden Sea, different recovery trajectories were found after riverine nutrient loads decreased, namely full, temporary and no recovery. We compiled intertidal seagrass presence (Zostera noltei and Z. marina) and eutrophication data for 1930–2020, to relate the seagrass trajectories and regional eutrophication differences to riverine nutrient loads, and inferred prospects for seagrass recovery. Seagrass fully recovered in the less eutrophic North Frisian region. The recovery trajectory was tightly coupled to riverine nutrient load reduction. Relative seagrass area (meadow area/region area) dropped from 10% prior to eutrophication to 2% during the eutrophication peak, increased to 7% during the nutrient reduction period and subsequently expanded to 13%. Colonization of marginal habitats was observed, indicating propagule spillover from neighbouring meadows. The more eutrophic southern regions showed no or only temporary seagrass recovery. Prospects for (limited) recovery are good in only two out of four southern regions, provided that riverine nutrient loads are further reduced by ~40% (reference: 2010–2017). Without this reduction, seagrasses may only temporarily recover and will remain vulnerable to erratic disturbances like macroalgae accumulation or storms. Historical evidence and application of habitat suitability models suggest that the potential relative seagrass area in the southern regions is low: less than 0.2% in the western Dutch region and maximum 2.4% in the Ems‐Jade region. Synthesis and applications. Within a large seascape (15,000 km2) the least eutrophicated region showed seagrass recovery upon nutrient reduction. We translated the critical riverine nutrient loads for this recovery, via regional eutrophication indicators, to loads that may enable a sustained recovery in the other regions. This technique is applicable in other complex systems, provided sufficient historical data are available. Propagule spillover exerts a positive feedback at metapopulation scale leading to acceleration of recovery. Occupied and potential seagrass habitat (e.g. assessed by the maximum recorded area in the past) are thus important landscape selection criteria for restoration, particularly when eutrophication is not yet sufficiently reduced.

MTTLA‐DLW: Multi‐task TCN‐Bi‐LSTM transfer learning approach with dynamic loss weights based on feature correlations of the training samples for short‐term wind power prediction

AbstractWind power prediction for newly built wind farms is usually faced with the problem of no sufficient historical data. To efficiently extract the useful features from related wind farms, a novel transfer learning method based on temporal convolutional network (TCN)‐Bi‐long short‐term memory (LSTM) with dynamic loss weights is proposed. Firstly, a novel multi‐task TCN‐Bi‐LSTM model is designed to extract common features. The separate TCNs, and common Bi‐LSTM layers of the proposed model are designed to extract the temporal features from related wind farms. Secondly, in the pre‐training stage, to optimize the training process of the neural networks, a dynamic loss‐weighting strategy is proposed for multi‐task learning (MTL) to select the most related features, which increase the prediction accuracy by providing a suitable optimization object. Thirdly, the multi‐task TCN‐Bi‐LSTM model is re‐trained based on the samples from the target wind farm. Finally, a dataset of seven wind farms was employed to evaluate the efficiency of the proposed MTL structure and the dynamic loss‐weighting strategy. The result shows that the root mean squared error of the 12‐h short‐term prediction can be decreased by 4.19% compared with the traditional single‐task learning model, which verifies the validity of the proposed multi‐task transfer learning method.

Near-Optimal Scheduling for IC Packaging Operations Considering Processing-Time Variations and Factory Practices

Due to the short life cycles of electronic products, trial run lots of new products are crucial in IC packaging for production verification and engineering adjustments. The processing time of trial run lots may differ significantly from production lots due to engineering adjustments and is difficult to predict without sufficient historical data. The purpose of this paper is to develop a framework for near-optimal scheduling of IC packaging considering processing time variations and factory practices. Using an orthogonal greedy algorithm and recurrent neural network, this framework extracts key features and predicts the processing times of trial run lots and production lots for each operation using these algorithms. After formulating the scheduling problem in an integer linear programming form, an ordinal optimization method is embedded within the decomposition and coordination framework to obtain dynamic and near-optimal schedules in a computationally efficient manner. This approach can improve the efficiency of IC packaging and potentially other operations by optimizing lot scheduling, reducing production tardiness, and increasing productivity in the factory, paving the way for self-optimizing factories in the future.

Analyzing Cryptocurrency Prices through Artificial Intelligence

Cryptocurrency is reshaping the financial landscape, gaining popularity and acceptance among merchants. Despite the increasing investments in cryptocurrency, its dynamic features, uncertainties, and predictability remain largely unknown, posing significant investment risks. This study aims to understand the factors influencing cryptocurrency value formation. Leveraging advanced artificial intelligence frameworks like fully connected Artificial Neural Network (ANN) and Long Short-Term Memory (LSTM) Recurrent Neural Network, we analyze the price dynamics of Bitcoin, Ethereum, and Ripple. Our findings indicate that ANN relies more on long-term history, while LSTM focuses on short-term dynamics, suggesting LSTM's superior efficiency in utilizing historical information. However, with sufficient historical data, ANN can achieve comparable accuracy to LSTM. This study underscores the predictability of cryptocurrency market prices, though the explanation may vary depending on the machine-learning model employed.

Traffic light optimization with low penetration rate vehicle trajectory data

Traffic light optimization is known to be a cost-effective method for reducing congestion and energy consumption in urban areas without changing physical road infrastructure. However, due to the high installation and maintenance costs of vehicle detectors, most intersections are controlled by fixed-time traffic signals that are not regularly optimized. To alleviate traffic congestion at intersections, we present a large-scale traffic signal re-timing system that uses a small percentage of vehicle trajectories as the only input without reliance on any detectors. We develop the probabilistic time-space diagram, which establishes the connection between a stochastic point-queue model and vehicle trajectories under the proposed Newellian coordinates. This model enables us to reconstruct the recurrent spatial-temporal traffic state by aggregating sufficient historical data. Optimization algorithms are then developed to update traffic signal parameters for intersections with optimality gaps. A real-world citywide test of the system was conducted in Birmingham, Michigan, and demonstrated that it decreased the delay and number of stops at signalized intersections by up to 20% and 30%, respectively. This system provides a scalable, sustainable, and efficient solution to traffic light optimization and can potentially be applied to every fixed-time signalized intersection in the world.

IC‐GraF: An Improved Clustering with Graph‐Embedding‐Based Features for Software Defect Prediction

Software defect prediction (SDP) has been a prominent area of research in software engineering. Previous SDP methods often struggled in industrial applications, primarily due to the need for sufficient historical data. Thus, clustering‐based unsupervised defect prediction (CUDP) and cross‐project defect prediction (CPDP) emerged to address this challenge. However, the former exhibited limitations in capturing semantic and structural features, while the latter encountered constraints due to differences in data distribution across projects. Therefore, we introduce a novel framework called improved clustering with graph‐embedding‐based features (IC‐GraF) for SDP without the reliance on historical data. First, a preprocessing operation is performed to extract program dependence graphs (PDGs) and mark distinct dependency relationships within them. Second, the improved deep graph infomax (IDGI) model, an extension of the DGI model specifically for SDP, is designed to generate graph‐level representations of PDGs. Finally, a heuristic‐based k‐means clustering algorithm is employed to classify the features generated by IDGI. To validate the efficacy of IC‐GraF, we conduct experiments based on 24 releases of the PROMISE dataset, using F‐measure and G‐measure as evaluation criteria. The findings indicate that IC‐GraF achieves 5.0%−42.7% higher F‐measure, 5%−39.4% higher G‐measure, and 2.5%−11.4% higher AUC over existing CUDP methods. Even when compared with eight supervised learning‐based SDP methods, IC‐GraF maintains a superior competitive edge.

Summer monsoon promotes the energy transfer efficiency of the zooplankton community in northern South China sea

The summer monsoon shows a fundamental influence on the pelagic ecosystem of the South China Sea. Zooplankton are a major link for energy transfer between primary producers and upper trophic levels. Therefore, evaluating the energy transfer efficiency (ETE) of zooplankton is crucial to understand the function of pelagic ecosystem under the influence of monsoon. In this study, field surveys were conducted during May (intermonsoon) and August 2021 (summer monsoon) focusing on the variation of zooplankton size and trophic structures across the shelf and slope. The result showed that the summer monsoon reinforced the gradient of abundance, biovolume, and biomass from slope to shelf, and greatly intensified the role of environmental factors in driving spatial variation in most taxa. Both the results of size and trophic structures indicated that the ETE of zooplankton decreased from slope to shelf. The size structure also indicated that the ETE of zooplankton significantly increased under the influence of summer monsoon. These results were consistent with previous studies by different methods, suggesting that these approaches of size and trophic structures had important potential value in assessing changes in the function of marine pelagic ecosystem, especially when compared with sufficient historical data or reanalyzing historical samples.

On the use of machine learning to generate in-silico data for batch process monitoring under small-data scenarios

Batch process monitoring using principal component analysis requires sufficient historical manufacturing data to model the normal operating conditions of the process. However, when a new product is to be manufactured for the first time in a given facility, very limited historical data are available, thus entailing a small-data scenario. We thoroughly investigate and improve a data-driven methodology, previously reported in the literature (Tulsyan, Garvin & Ündey (2019). J. Process Control,77, 114–133), that enables batch process monitoring under such type of scenarios. The methodology exploits machine learning algorithms (based on Gaussian process state-space models) to generate in-silico batch trajectory data from the few available historical ones, and then uses the overall pool of real and in-silico data to build a process monitoring model. We develop automatic procedures to tune the values of several parameters of this machine-learning framework, in such a way that the generation of consistent in-silico batch trajectory data can be streamlined, thus facilitating the deployment of the framework at an industrial level. Furthermore, we develop indicators and a metric to assist the in-silico data generation activity from a process monitoring-relevant perspective. Finally, using datasets from a benchmark simulated semi-batch process for the manufacturing of penicillin, we thoroughly investigate the appropriateness of the in-silico generated data for the purpose of process monitoring.

An adjustable Predictive&Prescriptive method for the RO-based optimal power flow problem

Traditionally before solving the optimal power flow considering uncertainty (OPF–U) problem, the predicted value of uncertainty parameters, such as wind power, e.g., is derived from data using a statistics approach or machine learning. Based on the predicted uncertainty parameters, the solution to the OPF-U problem can be obtained by the prescriptive analytics technique, such as robust optimization (RO). However, it is unclarified how the prediction error in predictive analytics affects solving the OPF-U problem in prescriptive analytics. We propose an adjustable framework method combining machine learning and RO for the OPF-U problem. The k-nearest neighbor is applied to obtain k samples around the predicted value from sufficient historical data. And the optimization results from a minimum volume ellipsoid set containing the k samples are applied to construct KMV set. Then a robust fluctuation region with an adjustable budget level is gained from the KMV set by a two-term exponential formula, which can be embedded into a two-stage RO model. Computational experiments under test cases of different uncertainty scales show the robustness and adjustability of the proposed fluctuation region are better than the state-of-the-art box and ellipsoidal sets. The solution of the proposed two-stage RO model is more economical than the state-of-the-art RO model. The out-of-sample simulation also demonstrates the proposed adjustable Predictive&Prescriptive method can reduce the computational burden as the scale of the system increases when predictive and prescriptive analytics are separated.

Robust vehicle routing with drones under uncertain demands and truck travel times in humanitarian logistics

Resource transport in the aftermath of disasters is critical, yet in the absence of sufficient historical data or accurate forecasting approaches, the development of resource transport strategies often faces the challenge of dealing with uncertainty, especially uncertainties in demand and travel time. In this paper we investigate the vehicle routing problem with drones under uncertain demands and truck travel times. Specifically, there is a set of trucks and drones (each truck is associated with a drone) collaborating to transport relief resources to the affected areas, where a drone can be launched from its associated truck at a node, independently transporting relief resources to one or more of the affected areas, and returning to the truck at another node along the truck route. For this problem, we present a tailored robust optimization model based on the well-known budgeted uncertainty set, and develop an enhanced branch-and-price-and-cut algorithm incorporating a bounded bidirectional labelling algorithm to solve the pricing problem, which can be modelled as a robust resource-constrained vehicle and drone synthetic shortest path problem. To enhance the performance of the algorithm, we employ subset-row inequalities to tighten the lower bound and incorporate some enhancement strategies to quickly solve the pricing problem. We perform extensive numerical studies to assess the performance of the developed algorithm, discuss the benefits of considering uncertainty and robustness, and analyse the impacts of key model parameters on the optimal solution. We also evaluate the benefits of the truck–drone collaborative transport mode over the truck-only transport mode through a real case study of the 2008 earthquake in Wenchuan, China.

Operational adjustment modeling approach based on Bayesian network transfer learning for new flotation process under scarce data

Establishing an intelligent operational adjustment model is significant for optimizing flotation performance and realizing the stable control of the flotation process. When sufficient historical data is available, the Bayesian network (BN) has been used to achieve this goal due to its powerful uncertainty modeling and reasoning capabilities. However, for a new flotation process in the early stage with scarce data, an accurate BN model can hardly be established to make reliable decisions. In this scenario, it is an effective way to transfer data or knowledge from other similar flotation processes to the new flotation process to help complete the target BN modeling task. Therefore, the operational adjustment modeling problem is transformed into the BN transfer learning problem. Moreover, this article proposes a new BN transfer learning approach, including structure and parameters transfer learning techniques. For structure transfer learning, a majority voting mechanism is proposed to get an integrated result of conditional independence tests and generates the network structure based on them. For parameters transfer learning, the sources are judged whether to be used to learn the parameters first, and then the suitable sources are used to learn the network parameters based on the weight determined by the similarity. The Asia network first assesses the proposed approach and then applies it to real-world flotation operational adjustment decisions. Experimental results on the Asia network show that the proposed approach can learn more accurate structure and parameters according to F1 score, Hamming distance, and KL divergence indicators compared to existing advanced methods. Furthermore, the experimental results of the flotation process show that the inference accuracy is improved from 57.1% to 85.7% compared with the traditional modeling method.

Adaptive transfer learning for multimode process monitoring and unsupervised anomaly detection in steam turbines

Through condition-based maintenance strategy, engineers can monitor the health states of equipment and take actions based on the sensor data. Limited by the low failure frequency and high monitoring costs, it is difficult to obtain sufficient historical data of all fault types for condition monitoring (CM). In the steam turbine operation, environmental factors, varying power consumption and manual adjustments can lead to a multimode process, which consists of multiple normal and abnormal conditions. This paper proposes a framework for online unsupervised CM and anomaly detection, not relying on expert knowledge or labeled historical data. Since there are often few monitoring data at the beginning of a new incoming operating mode, an adaptive self-transfer learning algorithm based on Gaussian processes is developed to model the monitoring data with uncertainty information, and to capture the cross-correlations between the different normal modes. A two-hierarchical identification criterion based on the predicted posterior intervals is introduced to first identify the change-points in the observations, and second to decide whether it is an anomaly or a transition between normal modes. The proposed framework is tested on a real steam turbine. The results illustrate its high effectiveness.

Prediction of photovoltaic power generation based on LSTM and transfer learning digital twin

This research offers a digital twin model for solar power production power prediction based on long short term memory network (LSTM), and then applies this model to other models with limited operational time and inadequate data through transfer learning. The prediction for the solar system’s electrical output. Due to the effect of sun irradiation, temperature, and other random elements, photovoltaic power output is very intermittent and fluctuating, making it impossible to anticipate photovoltaic power with precision. Synchronization and real-time updating of physical entities, thereby obtaining more accurate forecasting results than traditional forecasting methods, while utilizing knowledge learned from PV systems with sufficient historical data to assist PV systems with limited historical data in establishing a digital twin of power generation forecasting model, not only can obtain accurate prediction results but also save training time for the model. In this study, the PV historical data of three distinct sites from the open source websites of Queensland University and Shanxi Jinneng Clean Energy Company are used to validate the validity of the suggested technique.

Uncertain power flow calculation and global sensitivity analysis considering parametric probability-boxes

Probabilistic power flow is one of the fundamental tools for assessing the impacts of uncertainties on the operating states of power systems. However, this analysis requires sufficient historical data to obtain precise probability distributions of input variables, which may not be met in practical engineering problems. In this paper, input variables with insufficient data are represented by parametric probability boxes (p-boxes), i.e., probability distributions with imprecise parameters. In order to facilitate the uncertain power flow calculation with p-boxes, a polynomial chaos expansion-based method is developed. Moreover, the interval-valued Borgonovo index is proposed for global sensitivity analysis and to identify the input variables that have critical impacts on systems. The simulations in IEEE 14-bus and 118-bus systems verify the accuracy and efficiency of the proposed method by comparing it with the conventional double-loop sampling method.

A Smart City Air Quality Data Imputation Method Using Markov Weights-Based Fuzzy Transfer Learning

An intelligent air quality monitoring system (IAQMS) is one of the key aspects of any smart city. The success of these monitoring systems largely depends on the quality of data. Missing air quality data is one of the crucial issues in any IAQMS, especially in small cities where sufficient historical air quality data is not available. In this study, a fuzzy transfer learning-based imputation (FTLI) method is proposed for the smart imputation of missing air quality data. The central concept of this proposed method is to acquire knowledge through fuzzy inference systems from other air quality monitoring systems called source domains where sufficient data is available. Later that knowledge is applied to impute the missing values of the target IAQMS (target domain) through some knowledge adaptation techniques. In this proposed method, Markov weights (transition probability matrix) are used to impute the missing values more accurately. The proposed imputation method is tested on the various missing data situations, which include random missing values, continuous missing values, and high missing rates. In this study, the performance of the proposed method is compared with other well-established methods, and the comparison results exhibit that the FTLI method outruns other methods under different missing rates.

A Multi-Party Agent for Privacy Preference Elicitation

In today’s world, the decisions that individuals make online often include their surroundings and social circles. For example, Alice posts on TikTok to celebrate her friend Bob’s birthday and reminisce about their best memories together. She, then, proceeds to create a campaign to fund her local place of worship and tags members of her community who share her religious belief. Alice might equally like to take initiative at work as she plans her team-building trip and excitedly shares the programme on Facebook. While doing all of this, she is involving family members, close friends, co-workers, acquaintances, and others from her social circle, all of whom might have different opinions about their privacy. While she sees no issue with her actions, her friend Bob, for one, might not agree, hence, the issue of multi-party privacy. Many researchers have focused on conflict resolution, which occurs when the sharer’s privacy preferences do not align with the other parties involved. However, one key point in this approach is eliciting the preferences of these individuals. Oftentimes, there is an underlying assumption that the system has sufficient historical data to represent the perspective of the multi-party members. The problem is that this is not always the case in real life and the cold start problem might be unavoidable. The system that is meant to nudge the sharer to reduce the multi-party disclosure might not even be capable of representing the preferences of everyone involved at the beginning. Hence, this paper addresses this issue through the use of the Classification and Regression Tree (CART) combined with the Rasch model. Study participants (N = 800) responded to realistic scenarios showcasing multi-party disclosure, which is used to construct and test the multi-party agent. The results suggest that the system performs well in overcoming the cold start problem as reported by the accuracy, precision, and recall.

Fundamental challenge and solution methods in prescriptive analytics for freight transportation

Prescriptive analytics, in which some parameters are predicted using statistical or machine learning models and then input into an optimization model, is often used to prescribe recommended solutions to freight transportation problems. The effectiveness of the optimal decision prescribed by prescriptive analytics is typically evaluated through a comparison with the results of the current decision model using predicted data. However, such comparisons are often flawed because of insufficient and uncertain data. We use four freight transport examples to illustrate this fundamental challenge in prescriptive analytics modeling. Furthermore, we propose three solutions to fully or partially overcome this challenge and fairly compare the optimal decisions generated by prescriptive analytics and the current approach. The three solutions involve using sufficient historical data, constructing new test sets, and generating synthetic data. We show how these solutions address the challenges in the four examples and are suitable for different problems considering data availability. The proposed solutions allow for a more comprehensive, accurate, and fair comparison of the optimal decisions to validate those generated by prescriptive analytics. This improves the effectiveness of the prescriptive analytics paradigm and can promote its application in freight transport and other disciplines.