Generated Data Points Research Articles

A criterion for selecting the appropriate one from the trained models for model‐based offline policy evaluation

AbstractOffline policy evaluation, evaluating and selecting complex policies for decision‐making by only using offline datasets is important in reinforcement learning. At present, the model‐based offline policy evaluation (MBOPE) is widely welcomed because of its easy to implement and good performance. MBOPE directly approximates the unknown value of a given policy using the Monte Carlo method given the estimated transition and reward functions of the environment. Usually, multiple models are trained, and then one of them is selected to be used. However, a challenge remains in selecting an appropriate model from those trained for further use. The authors first analyse the upper bound of the difference between the approximated value and the unknown true value. Theoretical results show that this difference is related to the trajectories generated by the given policy on the learnt model and the prediction error of the transition and reward functions at these generated data points. Based on the theoretical results, a new criterion is proposed to tell which trained model is better suited for evaluating the given policy. At last, the effectiveness of the proposed criterion is demonstrated on both benchmark and synthetic offline datasets.

HERALD: A domain-specific query language for longitudinal health data analytics

BackgroundLarge-scale health data has significant potential for research and innovation, especially with longitudinal data offering insights into prevention, disease progression, and treatment effects. Yet, analyzing this data type is complex, as data points are repeatedly documented along the timeline. As a consequence, extracting cross-sectional tabular data suitable for statistical analysis and machine learning can be challenging for medical researchers and data scientists alike, with existing tools lacking balance between ease-of-use and comprehensiveness. ObjectiveThis paper introduces HERALD, a novel domain-specific query language designed to support the transformation of longitudinal health data into cross-sectional tables. We describe the basic concepts, the query syntax, a graphical user interface for constructing and executing HERALD queries, as well as an integration into Informatics for Integrating Biology and the Bedside (i2b2). MethodsThe syntax of HERALD mimics natural language and supports different query types for selection, aggregation, analysis of relationships, and searching for data points based on filter expressions and temporal constraints. Using a hierarchical concept model, queries are executed individually for the data of each patient, while constructing tabular output. HERALD is closed, meaning that queries process data points and generate data points. Queries can refer to data points that have been produced by previous queries, providing a simple, but powerful nesting mechanism. ResultsThe open-source implementation consists of a HERALD query parser, an execution engine, as well as a web-based user interface for query construction and statistical analysis. The implementation can be deployed as a standalone component and integrated into self-service data analytics environments like i2b2 as a plugin. HERALD can be valuable tool for data scientists and machine learning experts, as it simplifies the process of transforming longitudinal health data into tables and data matrices. ConclusionThe construction of cross-sectional tables from longitudinal data can be supported through dedicated query languages that strike a reasonable balance between language complexity and transformation capabilities.

Feature selection via robust weighted score for high dimensional binary class-imbalanced gene expression data

In this paper, a robust weighted score for unbalanced data (ROWSU) is proposed for selecting the most discriminative features for high dimensional gene expression binary classification with class-imbalance problem. The method addresses one of the most challenging problems of highly skewed class distributions in gene expression datasets that adversely affect the performance of classification algorithms. First, the training dataset is balanced by synthetically generating data points from minority class observations. Second, a minimum subset of genes is selected using a greedy search approach. Third, a novel weighted robust score, where the weights are computed by support vectors, is introduced to obtain a refined set of genes. The highest scoring genes based on this approach are combined with the minimum subset of genes selected by the greedy search approach to form the final set of genes. The novel method ensures the selection of the most discriminative genes, even in the presence of skewed class distribution, thereby improving the performance of the classifiers. The performance of the proposed ROWSU method is evaluated on 7 gene expression datasets. Classification accuracy, sensitivity and F1-score are used as performance metrics to compare the proposed ROWSU algorithm with several other state-of-the-art methods. Boxplots and stability plots are also constructed for a better understanding of the results. The results show that the proposed method outperforms the existing feature selection procedures based on classification performance from k nearest neighbors (kNN) and random forest (RF) classifiers.

Learning Deep Generative Clustering via Mutual Information Maximization.

Deep clustering refers to joint representation learning and clustering using deep neural networks. Existing methods can be mainly categorized into two types: discriminative and generative methods. The former learns representations for clustering with discriminative mechanisms directly, and the latter estimate the latent distribution of each cluster for generating data points and then infers cluster assignments. Although generative methods have the advantage of estimating the latent distributions of clusters, their performances still significantly fall behind discriminative methods. In this work, we argue that this performance gap might be partly due to the overlap of data distribution of different clusters. In fact, there is little guarantee of generative methods to separate the distributions of different clusters in the data space. To tackle these problems, we theoretically prove that mutual information maximization promotes the separation of different clusters in the data space, which provides a theoretical justification for deep generative clustering with mutual information maximization. Our theoretical analysis directly leads to a model which integrates a hierarchical generative adversarial network and mutual information maximization. Moreover, we further propose three techniques and empirically show their effects to stabilize and enhance the model. The proposed approach notably outperforms other generative models for deep clustering on public benchmarks.

Online Symbolic Regression with Informative Query

Symbolic regression, the task of extracting mathematical expressions from the observed data, plays a crucial role in scientific discovery. Despite the promising performance of existing methods, most of them conduct symbolic regression in an offline setting. That is, they treat the observed data points as given ones that are simply sampled from uniform distributions without exploring the expressive potential of data. However, for real-world scientific problems, the data used for symbolic regression are usually actively obtained by doing experiments, which is an online setting. Thus, how to obtain informative data that can facilitate the symbolic regression process is an important problem that remains challenging. In this paper, we propose QUOSR, a query-based framework for online symbolic regression that can automatically obtain informative data in an iterative manner. Specifically, at each step, QUOSR receives historical data points, generates new x, and then queries the symbolic expression to get the corresponding y, where the (x, y) serves as new data points. This process repeats until the maximum number of query steps is reached. To make the generated data points informative, we implement the framework with a neural network and train it by maximizing the mutual information between generated data points and the target expression. Through comprehensive experiments, we show that QUOSR can facilitate modern symbolic regression methods by generating informative data.

Adversarial Perturbation Elimination with GAN Based Defense in Continuous-Variable Quantum Key Distribution Systems

Machine learning is being applied to continuous-variable quantum key distribution (CVQKD) systems as defense countermeasures for attack classification. However, recent studies have demonstrated that most of these detection networks are not immune to adversarial attacks. In this paper, we propose to implement typical adversarial attack strategies against the CVQKD system and introduce a generalized defense scheme. Adversarial attacks essentially generate data points located near decision boundaries that are linearized based on iterations of the classifier to lead to misclassification. Using the DeepFool attack as an example, we test it on four different CVQKD detection networks and demonstrate that an adversarial attack can fool most CVQKD detection networks. To solve this problem, we propose an improved adversarial perturbation elimination with a generative adversarial network (APE-GAN) scheme to generate samples with similar distribution to the original samples to defend against adversarial attacks. The results show that the proposed scheme can effectively defend against adversarial attacks including DeepFool and other adversarial attacks and significantly improve the security of communication systems.

TLMOTE: A Topic-based Language Modelling Approach for Text Oversampling

Training machine learning and deep learning models on unbalanced datasets can lead to a bias portrayed by the models towards the majority classes. To tackle the problem of bias towards majority classes, researchers have presented various techniques to oversample the minority class data points. Most of the available state-of-the-art oversampling techniques generate artificial data points which cannot be comprehensibly understood by the reader, despite the synthetic data points generated being similar to the original minority class data points. In this work, we present Topic-based Language Modelling Approach for Text Oversampling (TLMOTE), a novel text oversampling technique for supervised learning from unbalanced datasets. TLMOTE improves upon previous approaches by generating data points which can be intelligibly understood by the reader, can relate to the main topics of the minority class, and introduces more variations to the synthetic data generated. We evaluate the efficacy of our approach on various tasks like Suggestion Mining SemEval 2019 Task 9 Subtasks A and B, SMS Spam Detection, and Sentiment Analysis. Experimental results verify that oversampling unbalanced datasets using TLMOTE yields a higher macro F1 score than with other oversampling techniques.

Augmentation of Human Action Datasets with Suboptimal Warping and Representative Data Samples.

The popularity of action recognition (AR) approaches and the need for improvement of their effectiveness require the generation of artificial samples addressing the nonlinearity of the time-space, scarcity of data points, or their variability. Therefore, in this paper, a novel approach to time series augmentation is proposed. The method improves the suboptimal warped time series generator algorithm (SPAWNER), introducing constraints based on identified AR-related problems with generated data points. Specifically, the proposed ARSPAWNER removes potential new time series that do not offer additional knowledge to the examples of a class or are created far from the occupied area. The constraints are based on statistics of time series of AR classes and their representative examples inferred with dynamic time warping barycentric averaging technique (DBA). The extensive experiments performed on eight AR datasets using three popular time series classifiers reveal the superiority of the introduced method over related approaches.

A Critical Evaluation and Modification of the Padé-Laplace Method for Deconvolution of Viscoelastic Spectra.

This paper shows that using the Padé–Laplace (PL) method for deconvolution of multi-exponential functions (stress relaxation of polymers) can produce ill-conditioned systems of equations. Analysis of different sets of generated data points from known multi-exponential functions indicates that by increasing the level of Padé approximants, the condition number of a matrix whose entries are coefficients of a Taylor series in the Laplace space grows rapidly. When higher levels of Padé approximants need to be computed to achieve stable modes for separation of exponentials, the problem of generating matrices with large condition numbers becomes more pronounced. The analysis in this paper discusses the origin of ill-posedness of the PL method and it was shown that ill-posedness may be regularized by reconstructing the system of equations and using singular value decomposition (SVD) for computation of the Padé table. Moreover, it is shown that after regularization, the PL method can deconvolute the exponential decays even when the input parameter of the method is out of its optimal range.

Class imbalance learning using fuzzy ART and intuitionistic fuzzy twin support vector machines

The classification in imbalanced datasets is one of the main problems for machine learning techniques. Support vector machine (SVM) is biased to the majority class samples, and the minority class samples may incorrectly be considered as noise. Therefore, SVM has poor predictive accuracy for imbalanced datasets and generates inaccurate classification models. Existing class imbalance learning (CIL) techniques can make SVM less sensitive to class imbalance, but these methods suffer from issues related to noise and outliers. Moreover, despite the solid theoretical basis and good classification performance, SVM is not appropriate for the classification of large-scale datasets because the training complexity of SVM is closely related to the dataset size. Class imbalance learning (CIL) using Fuzzy adaptive resonance theory (ART) and intuitionistic fuzzy twin SVM (CIL-FART-IFTSVM), which can be applied to address the class imbalance issue in the presence of noise and outliers and large scale datasets, is proposed to overcome these substantial difficulties. In this method, we modify the distribution of the datasets using fuzzy adaptive resonance theory (Fuzzy ART) as a clustering method to overcome the imbalance problem. Then, after data reduction, IFTSVM is utilized to find excellent non-parallel hyperplanes in the generated data points. Finally, a coordinate descent system with shrinking by an active set is applied to reduce the computational complexity. Forty-five imbalanced datasets are considered to validate the performance of the proposed CIL-FART-IFTSVM method. The Friedman test and the bootstrap technique with 95% confidence intervals are applied to quantify the results statistically. The experimental results indicate that the method proposed in this paper has a better performance compared with other methods, and the training time is significantly better than that of other classifiers for large-scale datasets.

Improving Commonsense Causal Reasoning by Adversarial Training and Data Augmentation

Determining the plausibility of causal relations between clauses is a commonsense reasoning task that requires complex inference ability. The general approach to this task is to train a large pretrained language model on a specific dataset. However, the available training data for the task is often scarce, which leads to instability of model training or reliance on the shallow features of the dataset. This paper presents a number of techniques for making models more robust in the domain of causal reasoning. Firstly, we perform adversarial training by generating perturbed inputs through synonym substitution. Secondly, based on a linguistic theory of discourse connectives, we perform data augmentation using a discourse parser for detecting causally linked clauses in large text, and a generative language model for generating distractors. Both methods boost model performance on the Choice of Plausible Alternatives (COPA) dataset, as well as on a Balanced COPA dataset, which is a modified version of the original data that has been developed to avoid superficial cues, leading to a more challenging benchmark. We show a statistically significant improvement in performance and robustness on both datasets, even with only a small number of additionally generated data points.

Experimental and numerical study of hot-rolled duplex stainless steel CHS columns

Stainless steel circular hollow section (CHS) members have been widely used in building structures because of their comparable strength to structural carbon steels, attractive appearance and excellent corrosion resistance. However, research on stainless steel CHS members has been limited to cold-formed members, whereas hot-rolled CHS columns have received less attention. To this end, the flexural buckling performance of hot-rolled duplex stainless steel CHS columns was investigated by a comprehensive experimental and numerical programme. A total of seven hot-rolled duplex stainless steel CHS columns covering two cross-sectional sizes with varying lengths have been tested under compression. Based on the test results, finite element (FE) models were developed and used in parametric studies to generate data points with a wider range of geometric dimensions. Comparing the test and FE results with current design guidance, it was found that the Chinese standard, CECS 410, and North American design guidance, AISC DG 27, underestimate significantly the flexural buckling resistance of such columns, while the design method in Eurocode 3: Part 1–4 might be unsafe in predicting the resistances for low slenderness values. New column buckling curves based on the Eurocode 3 and AISC DG 27 design formulae with improved accuracy and reduced scatters were also proposed and validated through careful reliability analysis.

Data driven robust optimization of grinding process under uncertainty

ABSTRACT Uncertain process parameters present in industrial grinding circuits (IGC) increase the difficulty in modeling IGC. Conventionally, researchers have resorted to box approach for sampling in the uncertain parameter space, mimicking the uncertain parameter realizations, to observe their effects in objective functions and constraints. In case data are scattered in the uncertain parameter space, sampling in the entire range, as done in the box approach, might lead to erroneous results. To mitigate this problem, a sampling technique to generate data points inside the admissible regions is proposed leading to accurate identification of uncertain space. The proposed technique uses neuro-fuzzy c means clustering to create optimal number of clusters in the uncertain parameter space. Data points are generated using SOBOL sampling technique within each cluster boundary obtained by Delaunay triangulations. Using the proposed sampling technique in robust optimization setting and comparing with the box sampling for various sample sizes (500, 1000, 2000, 3000, 4000 and 5000), the efficacy of the proposed method has been established.

Global temperature calibration of the Long chain Diol Index in marine surface sediments

The Long chain Diol Index (LDI) is a relatively new organic geochemical proxy for sea surface temperature (SST), based on the abundance of the C30 1,15-diol relative to the summed abundance of the C28 1,13-, C30 1,13- and C30 1,15-diols. Here we substantially extend and re-evaluate the initial core top calibration by combining the original dataset with 172 data points derived from previously published studies and 262 newly generated data points. In total, we considered 595 globally distributed surface sediments with an enhanced geographical coverage compared to the original calibration. The relationship with SST is similar to that of the original calibration but with considerably increased scatter. The effects of freshwater input (e.g., river runoff) and long-chain diol contribution from Proboscia diatoms on the LDI were evaluated. Exclusion of core-tops deposited at a salinity < 32 ppt, as well as core-tops with high Proboscia-derived C28 1,12-diol abundance, resulted in a substantial improvement of the relationship between LDI and annual mean SST. This implies that the LDI cannot be directly applied in regions with a strong freshwater influence or high C28 1,12-diol abundance, limiting the applicability of the LDI. The final LDI calibration (LDI = 0.0325 × SST + 0.1082; R2 = 0.88; n = 514) is not statistically different from the original calibration of Rampen et al. (2012) (https://doi.org/10.1016/j.gca.2012.01.024), although with a larger calibration error of 3 °C. This larger calibration error results from several regions where the LDI does not seem to have a strong temperature dependence with annual mean SST, posing a limitation on the application of the LDI.

Modeling of overall heat transfer coefficient of a concentric double pipe heat exchanger with limited experimental data by using curve fitting and ANN combination

The modeling accuracy of artificial neural networks (ANN) was evaluated by using limited heat exchanger data acquired experimentally. The artificial neural networks were used for predicting the overall heat transfer coefficient of a concentric double pipe heat exchanger where oil flowed inside the inner tube while the water flowed in the outer tube. In the cases of parallel and counter flows, the experimental data were collected by testing heat exchanger in wide range of operating conditions. Curve fitting and artificial neural network combination was used for the estimation of the overall heat transfer coefficient to compensate the experimental errors in the data. The curve fitting was used to detect the trend and generate data points between the experimentally collected points. The artificial neural network was trained better from the generated data set. The feed forward type artificial neural network was trained by using the Levenberg-Marquardt algorithm. Two backpropagation network type artificial neural network algorithms were also used, and their performance were compared with the estimation of the Levenberg-Marquardt algorithm. The average estimation error between the predictions and the experimental data were in the range of 1.31e?4 to 4.35e?2%. The study confirmed that curve fitting and artificial neural network combination could be used effectively to estimate the overall heat transfer coefficient of heat exchanger.

Streaming Synthetic Time Series for Simulated Condition Monitoring

The transformation of the common production plant to a cyber-physical system is one of the most recent developments in manufacturing industry. The analysis of data streams produced by such sensor-equipped plants promises potential for process optimization. However, for competitive reasons and due to the novelty of the development, the access to publicly available real-world data is quite limited. Hence, businesses and researchers new to this area are often confronted with the cumbersome and complex task of synthesizing time series in order to gain first experiences. This work presents a novel concept and a corresponding software implementation for the stream-wise generation and publication of sensor data, to simulate condition monitoring of industrial production plants. The resulting configuration file-driven tool is capable of acting like a sensor-equipped plant by generating data points from various Gaussian Process-based models and mathematical expressions, or by replaying data sets. Moreover, it can be seamlessly connected with existing surrounding systems by using the MQTT protocol. In this context, the software aims at laying the foundation for real-world applications and improving them by providing a simulation tool to prototype and test with.

Improve the robustness of data mining algorithm against adversarial evasion attack

Conventional data mining theories developed for general-purpose applications commonly focus on the reducing the bias and variance on the ideal i.i.d. datasets, but neglecting its potential failure on maliciously generated data points by observing the system's behaviours. Therefore, dealing with these adversarial samples is an essential part of a security system to handle the data that are intentionally made to deceive the system. Due to this concern, this paper proposes a novel approach that introduces uncertainty to the model behaviour, in order to obfuscate the decision process of the attacking strategy and improve the robustness of security system against attacks that try to evade the detection. Our approach addresses three problems. First, we build a pool of mining models to improve robustness of a variety of mining algorithms, similar to ensemble learning but focusing on the optimisation the trade-off between off-line accuracy and robustness. Second, we randomly select a subset of models at run time (when the model is used for detection) to further boost the robustness. Third, we propose a theoretical framework that bounds the minimal number of features an attacker needs to modify given a set of selected models.

Analysis of adsorption kinetics at solid/solution interface using a hyperbolic tangent model

Abstract In this study, a new hyperbolic tangent model has been proposed for analyzing adsorption kinetic at the solid-solution interface. The most important advantage of the proposed model is its ability to predicate the equilibrium time of adsorption, which is very vital to optimize the cost of the adsorption system design. The ability of the new model for modeling of adsorption kinetic and predication of the equilibrium time was tested by using of the generated data points based on the mixed 1,2-order equation (MOE). The new model was applied for fitting of the reported experimental data. The obtained results show that the new model can analyze the experimental data very well.

Imbalanced data classification via support vector machines and genetic algorithms

Many real data sets are imbalanced and contain a large number of a certain type of patterns, but a very small number of another type of patterns. Normal classification methods, such as support vector machine (SVM), do not work well for these imbalanced data sets (IDS). It is difficult for SVMs to get the optimal separation hyperplane when they are trained with imbalanced data. In this paper, we propose a genetic algorithm (GA)-based classification method. A draft hyperplane and support vectors are first generated by SVMs. Then, GA is applied to compensate the imbalanced data. Finally, SVM is used again to find the best hyperplane from the generated data points. Compared with the other popular classification algorithms, our method has better classification accuracy for several IDS.

False positives with visual analysis for nonconcurrent multiple baseline designs and ABAB designs: Preliminary findings

This study evaluated the probability of generating false positives with three-tier nonconcurrent multiple baseline (NMBL) designs and ABAB designs. For Experiment 1, we generated four sets of three-tier NMBL design graphs. The first, second, and third sets consisted of fixed A-phase data points for all three tiers at 0%, 25% and 50%, respectively, and randomly generated data points in the B phases. The fourth set consisted of randomly generated data points in the A and B phases for all three tiers. Across all four sets (N=1000), results show that false positives were produced with 7.5% of three-tier NMBL design graphs and were most probable when baseline levels were set at 0% or 25%. For Experiment 2, we generated 3000 ABAB design graphs consisting of three to five data points per phase. Results indicate that no false positives were produced, regardless of the number of data points included in each phase. Results of this study support specific guidelines for the use of NMBL designs and ABAB designs.