Linear Gaussian Process Research Articles

Improving Aviation Incidents using Association Rule Mining Algorithm and Time Series Analysis

Advance technology helps to forecast and show the different hazards associated in the world of transportation systems. Despite the rapid advancement of technology, there are still huge number of aviation accidents and incidents happened. This paper concerns application of various data analytics algorithm to derive a time series forecasting method and frequent accident and incident pattern specifically for Philippine aviation setting for predicting conditions that would increase the likelihood of aviation accidents and incidents involving fatal and non-fatal activities. Data Analytics algorithms are Association Rule Mining using FP-Growth algorithms and Time Series Forecasting methods using Linear Regression, Gaussian Processes, Multilayer Perceptron, and SMOreg are applied to datasets derived from original data obtained from Department of Transportation (DOTr) with its attached agency, Civil Aviation Authority of the Philippines (CAAP) and both incident and accident records stored in “Philippine Aviation Incident Reporting System (PAIRS)” from 2008-2017. The aviation accident data is based on flight information such as aircraft attributes, aviation accident factors, type of occurrences, geographical location, weather conditions and phases of operation. The results lead to prescriptive analytics for development of business rules and help identify pattern that can define aircraft accidents/incidents occurrences. The ability to predict accidents has the main objective of saving lives and will be impactful in cost saving in terms of any aircraft damages. The result created important points in improving the aviation operation and assists in better decision-making to help create policy and future improvement for aviation-safety in the country.

Comparative Analysis of Regression Learning Methods for Estimation of Energy Performance of Residential Structures

Energy efficiency is a top priority for private and commercial buildings. This study evaluates the performance of six regression learning methods, including Linear Regressor, MLP Regressor, RBF Regressor, SVM Regressor, Gaussian Processes, and ANFIS Regressor to predict the heating and cooling loads of residential buildings. 768 buildings were considered and analyzed based on the influential parameters, such as relative density, surface area, wall area, roof area, overall height, orientation, glazing area, and glazing area distribution for predicting heating load and cooling load. Three statistical criteria such as correlation coefficient (R), mean absolute error (MAE) and root mean square error (RMSE) were used to assess the potential of the regression methods used in this study. The best estimation results were obtained with the ANFIS regression model, with R of 0.998, MAE of 0.46 and RMSE of 0.68 for HL; and with R of 0.990, MAE of 1.26 and RMSE of 1.60 for CL.

Forecasting vegetation condition for drought early warning systems in pastoral communities in Kenya

Droughts are a recurring hazard in sub-Saharan Africa, that can wreak huge socioeconomic costs. Acting early based on alerts provided by early warning systems (EWS) can potentially provide substantial mitigation, reducing the financial and human cost. However, existing EWS tend only to monitor current, rather than forecast future, environmental and socioeconomic indicators of drought, and hence are not always sufficiently timely to be effective in practice. Here we present a novel method for forecasting satellite-based indicators of vegetation condition. Specifically, we focused on the 3-month Vegetation Condition Index (VCI3M) over pastoral livelihood zones in Kenya, which is the indicator used by the Kenyan National Drought Management Authority (NDMA). Using data from MODIS and Landsat, we apply linear autoregression and Gaussian process modelling methods and demonstrate high forecasting skill several weeks ahead. As a bench mark we predicted the drought alert marker used by NDMA (VCI3M<35). Both of our models were able to predict this alert marker four weeks ahead with a hit rate of around 89% and a false alarm rate of around 4%, or 81% and 6% respectively six weeks ahead. The methods developed here can thus identify a deteriorating vegetation condition well and sufficiently in advance to help disaster risk managers act early to support vulnerable communities and limit the impact of a drought hazard.

Forecasting the Concentration of Particulate Matter in the Seoul Metropolitan Area Using a Gaussian Process Model.

Recently, the population of Seoul has been affected by particulate matter in the atmosphere. This problem can be addressed by developing an elaborate forecasting model to estimate the concentration of fine dust in the metropolitan area. We present a forecasting model of the fine dust concentration with an extended range of input variables, compared to existing models. The model takes inputs from holistic perspectives such as topographical features on the surface, chemical sources of the fine dusts, traffic and the human activities in sub-areas, and meteorological data such as wind, temperature, and humidity, of fine dust. Our model was evaluated by the index-of-agreement (IOA) and the root mean-squared error (RMSE) in predicting PM2.5 and PM10 over three subsequent days. Our model variations consist of linear regressions, ARIMA, and Gaussian process regressions (GPR). The GPR showed the best performance in terms of IOA that is over 0.6 in the three-day predictions.

Appraisal of spatial and temporal behavior in monsoon precipitation series of Punjab-Pakistan using hierarchical Bayesian Models

Precipitation shows that structured variation across space and time and the pattern of this variation is changing round the world due to global warming. These changes affect the water resources and hydrological cycle. A spatio-temporal dataset of precipitation records together with data on temperature, wind speed and humidity is used to analyze changes in spatiotemporal pattern of precipitation across 17 major meteorological stations of Punjab-Pakistan for monsoon period during 2004–2012. A distinguishable trend in precipitation structure along spatial and temporal domains is noticed. We consider a model-based bayesian spatio-temporal analysis using dynamic linear model and gaussian process model. The meteorological variables (covariates) temperature, wind speed and humidity show varying effect on the precipitation at different locations and time points. Additionally, we construct the spatial and temporal boxplots of all meteorological stations and time points to observe the distribution of precipitation and covariates at micro level. Further to this, the interval plots also reveal the spatially varying and temporally dynamic pattern of covariates. The prediction results through predictive contour plots show high and low precipitation zones in the study area and are helpful for policy makers to identify the homogeneous climate.

Active Learning for Gaussian Process Considering Uncertainties With Application to Shape Control of Composite Fuselage

In the machine learning domain, active learning is an iterative data selection algorithm for maximizing information acquisition and improving model performance with limited training samples. It is very useful, especially for industrial applications where training samples are expensive, time-consuming, or difficult to obtain. Existing methods mainly focus on active learning for classification, and a few methods are designed for regression, such as linear regression or Gaussian process (GP). Uncertainties from measurement errors and intrinsic input noise inevitably exist in the experimental data, which further affects the modeling performance. The existing active learning methods do not incorporate these uncertainties for GP. In this article, we propose two new active learning algorithms for the GP with uncertainties, which are variance-based weighted active learning algorithm and D-optimal weighted active learning algorithm. Through numerical study, we show that the proposed approach can incorporate the impact of uncertainties and realize better prediction performance. This approach has been applied to improving the predictive modeling for automatic shape control of composite fuselage.

Almost sure limit theorems for the maxima of stochastic volatility models

In this paper, we proved the almost sure limit theorems for the maxima of stochastic volatility models with both light-tailed and heavy-tailed noises, where the volatility sequence is a log-Gaussian linear process. For the light-tailed noise case, we assume that the autocorrelation function of the Gaussian linear process satisfies for some . For the heavy-tailed noise case, we assume that the Gaussian linear process is strong mixing with mixing rate satisfying for some .

Regional September Sea Ice Forecasting with Complex Networks and Gaussian Processes

Abstract Reliable predictions of the Arctic sea ice cover are becoming of paramount importance for Arctic communities and industry stakeholders. In this study pan-Arctic and regional September mean sea ice extents are forecast with lead times of up to 3 months using a complex network statistical approach. This method exploits relationships within climate time series data by constructing regions of spatiotemporal homogeneity (i.e., nodes), and subsequently deriving teleconnection links between them. Here the nodes and links of the networks are generated from monthly mean sea ice concentration fields in June, July, and August; hence, individual networks are constructed for each respective month. Network information is then utilized within a linear Gaussian process regression forecast model, a Bayesian inference technique, in order to generate predictions of sea ice extent. Pan-Arctic forecasts capture a significant amount of the variability in the satellite observations of September sea ice extent, with detrended predictive skills of 0.53, 0.62, and 0.81 at 3-, 2-, and 1-month lead times, respectively. Regional forecasts are also performed for nine Arctic regions. On average, the highest predictive skill is achieved in the Canadian Archipelago, Beaufort, Chukchi, East Siberian, Laptev, and Kara Seas, although the ability to accurately predict many of these regions appears to be changing over time.

Forecasting and Pattern Analysis of Philippine Regions’ Palay and Corn Production

Inadequate food production has been an immediate concern that the country is trying to address. Where farmers used fertilizers in their farms in the hope of increasing their agricultural production. Though, if fertilizer application is poorly managed, instead of having an increased production it will result in the contrary. Therefore, the efficient use of fertilizer is critical and has a great impact on crop production. Thus this study aims to show the pattern of Philippine’ regions crop production, specifically in rainfed and irrigated palay; white and yellow corn. This will also show which fertilizer will maximize their crop production and to seek for the most applicable association model in forecasting future crop production. Three predictive techniques were used namely, canopy clustering, Apriori association rule mining and time series forecasting models. Results reveal that all regions have a low volume of production for rainfed rice. The canopy clustering shows the pattern leading to the high production of irrigated rice for Region III. Also, Region II, Region X, and Region XII have a high volume of production of yellow corn and lastly clustering results on white corn shows Region VII has a Mid area harvested but shows Low volume of production, while Region X though have a low area harvested managed to have a Mid volume of production. The association of fertilizers to the volume of production shows that low Ammophos leads to a lower volume of production and the low Ammosul is not associated with a low volume of production hence a combination of low Ammosul and low Ammophos leads to a low volume of production. The forecasting methods’ linear regression, Gaussian processes, and SMOreg are all applicable in predicting the regions’ volume of production, whereas the SMOreg has the least MAE of 8.90% for Region VI.

Physics-Informed Neural Networks for Cardiac Activation Mapping

A critical procedure in diagnosing atrial fibrillation is the creation of electro-anatomic activation maps. Current methods generate these mappings from interpolation using a few sparse data points recorded inside the atria; they neither include prior knowledge of the underlying physics nor uncertainty of these recordings. Here we propose a physics-informed neural network for cardiac activation mapping that accounts for the underlying wave propagation dynamics and we quantify the epistemic uncertainty associated with these predictions. These uncertainty estimates not only allow us to quantify the predictive error of the neural network, but also help to reduce it by judiciously selecting new informative measurement locations via active learning. We illustrate the potential of our approach using a synthetic benchmark problem and a personalized electrophysiology model of the left atrium. We show that our new method outperforms linear interpolation and Gaussian process regression for the benchmark problem and linear interpolation at clinical densities for the left atrium. In both cases, the active learning algorithm achieves lower error levels than random allocation. Our findings open the door towards physics-based electro-anatomic mapping with the ultimate goals to reduce procedural time and improve diagnostic predictability for patients affected by atrial fibrillation. Open source code is available at https://github.com/fsahli/EikonalNet.

Intelligent system based on data mining techniques for prediction of preterm birth for women with cervical cerclage

Preterm birth, defined as a delivery before 37 weeks’ gestation, continues to affect 8–15% of all pregnancies and is associated with significant neonatal morbidity and mortality. Effective prediction of timing of delivery among women identified to be at significant risk for preterm birth would allow proper implementation of prophylactic therapeutic interventions. This paper aims first to develop a model that acts as a decision support system for pregnant women at high risk of delivering prematurely before having cervical cerclage. The model will predict whether the pregnancy will continue beyond 26 weeks’ gestation and the potential value of adding the cerclage in prolonging the pregnancy. The second aim is to develop a model that predicts the timing of spontaneous delivery in this high risk cohort after cerclage. The model will help treating physicians to define the chronology of management in relation to the risk of preterm birth, reducing the neonatal complications associated with it. Data from 274 pregnancies managed with cervical cerclage were included. 29 of the procedures involved multiple pregnancies. To build the first model, a data balancing technique called SMOTE was applied to overcome the problem of highly imbalanced class distribution in the dataset. After that, four classification models, namely Decision Tree, Random Forest, K-Nearest Neighbors (K-NN), and Neural Network (NN) were used to build the prediction model. The results showed that Random Forest classifier gave the best results in terms of G-mean and sensitivity with values of 0.96 and 1.00, respectively. These results were achieved at an oversampling ratio of 200%. For the second prediction model, five classification models were used to predict the time of spontaneous delivery; linear regression, Gaussian process, Random Forest, K-star, and LWL classifier. The Random Forest classifier performed best, with 0.752 correlation value. In conclusion, computational models can be developed to predict the need for cerclage and the gestation of delivery after this procedure. These models have moderate/high sensitivity for clinical application.

高斯线性过程和多邻域优化的高光谱图像分类

高斯线性过程和多邻域优化的高光谱图像分类

Super resolution and recognition of unconstrained ear image

In this paper, a framework is proposed to super-resolve low resolution ear images and to recognise these images, without external dataset. This frame uses linear kernel co-variance function-based Gaussian process regression to super-resolve the ear images. The performance of the proposed framework is evaluated on UERC database by comparing and analysing the peak signal to noise ratio, structural similarity index matrix and visual information fidelity in pixel domain. The results are compared with the state-of-the-art-algorithms. The results demonstrate that the proposed approach outperforms the state-of-the-art super resolution approaches.

Assessing prediction error at interpolation and extrapolation points

Common model selection criteria, such as $AIC$ and its variants, are based on in-sample prediction error estimators. However, in many applications involving predicting at interpolation and extrapolation points, in-sample error does not represent the relevant prediction error. In this paper new prediction error estimators, $tAI$ and $Loss(w_{t})$ are introduced. These estimators generalize previous error estimators, however are also applicable for assessing prediction error in cases involving interpolation and extrapolation. Based on these prediction error estimators, two model selection criteria with the same spirit as $AIC$ and Mallow’s $C_{p}$ are suggested. The advantages of our suggested methods are demonstrated in a simulation and a real data analysis of studies involving interpolation and extrapolation in linear mixed model and Gaussian process regression.

Forecasting Stock Market Indices Using Machine Learning Algorithms

In recent years machine learning algorithms have become a very popular tool for analysing financial data and forecasting stock prices. The goal of this article is to forecast five major stock market indexes (DAX, Dow Jones, NASDAQ, Nikkei 225 and S&P 500) using machine learning algorithms (Linear regression, Gaussian Processes, SMOreg and neural network Multilayer Perceptron) on historical data covering the period February 1, 2010, to January 31, 2020. The forecasts were made by using historical data in different base period lengths and forecasting horizons. The precision of machine learning algorithms was evaluated with the help of error metrics. The results of the analysis have shown that machine learning algorithms achieved highly accurate forecasting performance. The overall precision of all algorithms was better for shorter base period lengths and forecast horizons. The results obtained from this analysis could help investors in determining their optimal investment strategy. Stock price prediction remains, however, one of the most complex issues in the field of finance.

Super resolution and recognition of unconstrained ear image

In this paper, a framework is proposed to super-resolve low resolution ear images and to recognise these images, without external dataset. This frame uses linear kernel co-variance function-based Gaussian process regression to super-resolve the ear images. The performance of the proposed framework is evaluated on UERC database by comparing and analysing the peak signal to noise ratio, structural similarity index matrix and visual information fidelity in pixel domain. The results are compared with the state-of-the-art-algorithms. The results demonstrate that the proposed approach outperforms the state-of-the-art super resolution approaches.

Common Crossing Structural Health Analysis with Track-Side Monitoring

Track-side inertial measurements on common crossings are the object of the present study. The paper deals with the problem of measurement's interpretation for the estimation of the crossing structural health. The problem is manifested by the weak relation of measured acceleration components and impact lateral distribution to the lifecycle of common crossing rolling surface. The popular signal processing and machine learning methods are explored to solve the problem.&#x0D; The Hilbert-Huang Transform (HHT) method is used to extract the time-frequency features of acceleration components. The method is based on Ensemble Empirical Mode Decomposition (EEMD) that is advantageous to the conventional spectral analysis methods with higher frequency resolution and managing nonstationary nonlinear signals. Linear regression and Gaussian Process Regression are used to fuse the extracted features in one structural health (SH) indicator and study its relation to the crossing lifetime. The results have shown the significant relation of the derived with GPR indicator to the lifetime.

Image classification on IoT edge devices: profiling and modeling

With the advent of powerful, low-cost IoT systems, processing data closer to where the data originates, known as edge computing, has become an increasingly viable option. In addition to lowering the cost of networking infrastructures, edge computing reduces edge-cloud delay, which is essential for mission-critical applications. In this paper, we show the feasibility and study the performance of image classification using IoT devices. Specifically, we explore the relationships between various factors of image classification algorithms that may affect energy consumption, such as dataset size, image resolution, algorithm type, algorithm phase, and device hardware. In order to provide a means of predicting the energy consumption of an edge device performing image classification, we investigate the usage of three machine learning algorithms using the data generated from our experiments. The performance as well as the trade-offs for using linear regression, Gaussian process, and random forests are discussed and validated. Our results indicate that the random forest model outperforms the two former algorithms, with an R-squared value of 0.95 and 0.79 for two different validation datasets. The random forest also served as a feature extraction mechanism which enabled us to identify which predictor variables influenced our model the most.

Security Against False Data-Injection Attack in Cyber-Physical Systems

In this paper, secure, remote estimation of a linear Gaussian process via observations at multiple sensors is considered. Such a framework is relevant to many cyber-physical systems and Internet-of-things applications. Sensors make sequential measurements that are shared with a fusion center; the fusion center applies a filtering algorithm to make its estimates. The challenge is the presence of a few unknown malicious sensors which can inject anomalous observations to skew the estimates at the fusion center. The set of malicious sensors may be time varying. The problems of malicious sensor detection and secure estimation are considered. First, an algorithm for secure estimation is proposed. The proposed estimation scheme uses a novel filtering and learning algorithm, where an optimal filter is learned over time by using the sensor observations in order to filter out malicious sensor observations while retaining other sensor measurements. Next, a novel detector to detect injection attacks on an unknown sensor subset is developed. Numerical results demonstrate up to 3-dB gain in the mean-squared error and up to $\text{75} \%$ higher attack detection probability under a small false alarm rate constraint, against a competing algorithm that requires additional side information.

Gaussian Process-Based Hybrid Model for Predicting Oxygen Consumption in the Converter Steelmaking Process

Oxygen is one of the most important energies used in converter steelmaking processes of integrated iron and steel works. Precisely forecasting oxygen consumption before processing can benefit process control and energy optimization. This paper assumes there is a linear relationship between the oxygen consumption and input materials, and random noises are caused by other unmeasurable materials and unobserved reactions. Then, a novel hybrid prediction model integrating multiple linear regression (MLR) and Gaussian process regression (GPR) is introduced. In the hybrid model, the MLR method is developed to figure the global trend of the oxygen consumption, and the GPR method is applied to explore the local fluctuation caused by noise. Additionally, to accelerate the computational speed on the practical data set, a K-means clustering method is devised to respectively train a number of GPR models. The proposed hybrid model is validated with the actual data collected from an integrated iron and steel work in China, and compared with benchmark prediction models including MLR, artificial neural network, support vector machine and standard GPR. The forecasting results indicate that the suggested model is able to not only produce satisfactory point forecasts, but also estimate accurate probabilistic intervals.