Simulated Datasets Research Articles

Data-driven approaches for generating probabilistic short-term renewable energy scenarios

Renewable energy sources (RES) are becoming increasingly prevalent in power systems, but their intermittent and unpredictable nature challenges deterministic optimal generation scheduling. Stochastic planning or operating methodologies offer superior performance compared to deterministic approaches, making renewable energy generation scenarios increasingly valuable inputs for multistage decision-making problems. In this paper, we introduce and compare three data-driven approaches for generating probabilistic renewable energy scenarios. Numerical results from both simulated and real-world datasets demonstrate the accuracy and computational efficiency of these methods. Our proposed approaches provide a powerful tool for creating precise and efficient probabilistic renewable energy scenarios, which can enhance optimal generation scheduling in power systems with high RES penetration.

COPSA: a computation offloading strategy based on PPO algorithm and self-attention mechanism in MEC-empowered smart factories

With the dawn of Industry 5.0 upon us, the smart factory emerges as a pivotal element, playing a crucial role in the realm of intelligent manufacturing. Meanwhile, mobile edge computing is proposed to alleviate the computational burden presented by substantial workloads in smart factories. Nonetheless, it is very challenging to effectively incorporate edge computing resources to improve the efficiency of resource deployment in smart factories. Accordingly, we devise a novel approach based on Proximal Policy Optimization algorithm with the Self-Attention Mechanism to implement computing resource allocation in MEC-Empowered Smart Factories. More specifically, the self-attention mechanism is incorporated to enable dynamic focus on state information, accelerates convergence and facilitates global control. A great number of experiments conducted on both simulated and real datasets have verified the superiority of our proposed approach compared to the state-of-the-art baselines.

TINNiK: inference of the tree of blobs of a species network under the coalescent model.

The tree of blobs of a species network shows only the tree-like aspects of relationships of taxa on a network, omitting information on network substructures where hybridization or other types of lateral transfer of genetic information occur. By isolating such regions of a network, inference of the tree of blobs can serve as a starting point for a more detailed investigation, or indicate the limit of what may be inferrable without additional assumptions. Building on our theoretical work on the identifiability of the tree of blobs from gene quartet distributions under the Network Multispecies Coalescent model, we develop an algorithm, TINNiK, for statistically consistent tree of blobs inference. We provide examples of its application to both simulated and empirical datasets, utilizing an implementation in the MSCquartets 2.0 R package.

The Effect of Domain Terms on Password Security

The predominant authentication method still relies on usernames and passwords. To enhance memorability, domain terms may have been opted to include as part of passwords. However, there is little analysis of the extent to which such practice affects password security, so there is a lack of guidance on how users use domain terms on websites with different domain characteristics. To address the problem, we propose a novel approach to analyze the security effect of using domain terms in passwords. The methodology primarily consists of three stages. Firstly, we utilize Web crawlers to harvest domain vocabularies, subsequently leveraging the TextRank algorithm to rank their importance. Afterward, we propose an algorithm for constructing a simulated domain-specific password dataset by replacing password elements with domain terms. Finally, password guessing experiments are done on the dataset using PCFG and Markov model to evaluate the impact of domain terms on password security. The experimental results indicate that, for systems without clear domain, 20% domain terms replacement in the test set can reduce the cracking rate by up to 5.45%. In contrast, for domain-specific systems, 20% domain terms replacement in the training set can increase the cracking rate by 6.45%. These findings provide practical guidance on the application of domain knowledge in password creation for different types of systems. In summary, this study offers a novel perspective for exploring the security implications of passwords influenced by specific domains.

Fault Diagnosis of Power Components with Reliability Assessment in Extraterrestrial Microgrids

This research investigates the possible failures caused by aging and other environmental and external factors that could significantly impact the performance of extraterrestrial power systems. Additionally, it presents a reliability assessment model for the space microgrid based on fault tree analysis (FTA). The reliability assessment model developed in this paper represents a tool that can be used by engineers to harden the system design for operational and economic benefits. To improve the reliability of the system, this work provides a broad review of the different fault detection and diagnosis (FDD) algorithms used for power microgrids and space applications. Using data sets from the Habitat Simulator developed through the NASA-funded Resilient Extraterrestrial Habitat Institute, this paper compares the applicability and accuracy of the different FDD methods. The primary FDD approach proposed and assessed in this work is based on the Markov reliability model. It predicts and detects future faults in the space microgrids by using past data samples and categorizing them into different classes. Data-driven-based models such as artificial neural networks are also investigated, tested, and evaluated using simulation data sets. According to the simulation results and the broad FDD algorithm comparison, this study provides the crew or maintenance engineers with a clear methodology to detect and localize power system failures.

Optimal proposal particle filters for detecting anomalies and manoeuvres from two line element data

Detecting the manoeuvres of satellites is an important goal within the broader task of space situational awareness. Moreover, it is advantageous to be able to detect other anomalous behaviour from satellites arising from, for instance, collisions or malfunction. The Two Line Element (TLE) data published by NORAD is the only publicy-available comprehensive source of data for satellite orbits. This paper presents a filtering approach for detecting anomalies in satellite orbits from TLE data. Particle filters are deployed to track the state of the satellites’ orbits. New TLEs that are deemed sufficiently unlikely given one’s belief of the current orbital state are designated as anomalies. The change in the orbits over time is modelled using the SGP4 model. However, it is shown that this model can have large errors in certain elements in some instances. The structure of these errors is analysed and a suitable model uncertainty is derived. The optimal proposal filter is used to further reduce the impact of the errors in the SGP4 model. This strategy improves the proposal distribution from which the particles are drawn before re-weighting, so that the resulting weights are less degenerate. The proposed techniques are evaluated on a benchmark set of 15 satellites for which an externally-obtained ground truth is available, along with simulated orbital data with inserted manoeuvres. Although this benchmark is larger than the evaluation sets used in other published work, it cannot be representative of the full population of satellites and so more emphasis is placed on the simulated data. The proposed techniques are compared against a baseline, which is similar to many previously-proposed techniques for satellite manoeuvre detection. The particle filters are shown to be superior at detecting the subtle in-track and cross-track manoeuvres in the simulated dataset. However, on the benchmark dataset, a simple baseline that focusses on the mean motion can perform equivalently to the best filters. This simple technique is likely overfit to the benchmark.

Performance Evaluation of QT-RR Adaptation Time Lag Estimation in Exercise Stress Testing.

Slower adaptation of the QT interval to sudden changes in heart rate has been identified as a risk marker of ventricular arrhythmia. The gradual changes observed in exercise stress testing facilitates the estimation of the QT-RR adaptation time lag. The time lag estimation is based on the delay between the observed QT intervals and the QT intervals derived from the observed RR intervals using a memoryless transformation. Assuming that the two types of QT interval are corrupted with either Gaussian or Laplacian noise, the respective maximum likelihood time lag estimators are derived. Estimation performance is evaluated using an ECG simulator which models change in RR and QT intervals with a known time lag, muscle noise level, respiratory rate, and more. The accuracy of T-wave end delineation and the influence of the learning window positioning for model parameter estimation are also investigated. Using simulated datasets, the results show that the proposed approach to estimation can be applied to any changes in heart rate trend as long as the frequency content of the trend is below a certain frequency. Moreover, using a proper position of the learning window for exercise so that data compensation reduces the effect of nonstationarity, a lower mean estimation error results for a wide range of time lags. Using a clinical dataset, the Laplacian-based estimator shows a better discrimination between patients grouped according to the risk of suffering from coronary artery disease. Using simulated ECGs, the performance evaluation of the proposed method shows that the estimated time lag agrees well with the true time lag.

Balancing accuracy and Interpretability: An R package assessing complex relationships beyond the Cox model and applications to clinical prediction

BackgroundAccurate and interpretable models are essential for clinical decision-making, where predictions can directly impact patient care. Machine learning (ML) survival methods can handle complex multidimensional data and achieve high accuracy but require post-hoc explanations. Traditional models such as the Cox Proportional Hazards Model (Cox-PH) are less flexible, but fast, stable, and intrinsically transparent. Moreover, ML does not always outperform Cox-PH in clinical settings, warranting a diligent model validation. We aimed to develop a set of R functions to help explore the limits of Cox-PH compared to the tree-based and deep learning survival models for clinical prediction modelling, employing ensemble learning and nested cross-validation. MethodsWe developed a set of R functions, publicly available as the package “survcompare”. It supports Cox-PH and Cox-Lasso, and Survival Random Forest (SRF) and DeepHit are the ML alternatives, along with the ensemble methods integrating Cox-PH with SRF or DeepHit designed to isolate the marginal value of ML. The package performs a repeated nested cross-validation and tests for statistical significance of the ML’s superiority using the survival-specific performance metrics, the concordance index, time-dependent AUC-ROC and calibration slope.To get practical insights, we applied this methodology to clinical and simulated datasets with varying complexities and sizes. ResultsIn simulated data with non-linearities or interactions, ML models outperformed Cox-PH at sample sizes ≥ 500. ML superiority was also observed in imaging and high-dimensional clinical data. However, for tabular clinical data, the performance gains of ML were minimal; in some cases, regularised Cox-Lasso recovered much of the ML’s performance advantage with significantly faster computations. Ensemble methods combining Cox-PH and ML predictions were instrumental in quantifying Cox-PH’s limits and improving ML calibration. Traditional models like Cox-PH or Cox-Lasso should not be overlooked while developing clinical predictive models from tabular data or data of limited size. ConclusionOur package offers researchers a framework and practical tool for evaluating the accuracy-interpretability trade-off, helping make informed decisions about model selection.

Identifying the Host Galaxies of Supermassive Black Hole Binaries Found by Pulsar Timing Arrays

Supermassive black hole binaries (SMBHBs) are thought to form in galaxy mergers, possessing the potential to produce electromagnetic (EM) radiation as well as gravitational waves (GWs) detectable with pulsar timing arrays (PTAs). Once GWs from individually resolved SMBHBs are detected, the identification of the host galaxy will be a major challenge due to the ambiguity in possible EM signatures and the poor localization capability of PTAs. To aid EM observations in choosing follow-up sources, we use NANOGrav’s galaxy catalog to quantify the number of plausible hosts in both realistic and idealistic scenarios. We outline a host identification pipeline that injects a single-source GW signal into a simulated PTA data set, recovers the signal using production-level techniques, quantifies the localization region and number of galaxies contained therein, and finally imposes cuts on the galaxies using parameter estimates from the GW search. In an ideal case, the 90% credible areas span 29–241 deg2, containing about 14–341 galaxies. After cuts, the number of galaxies remaining ranges from 22 at worst to one true host at best. In a realistic case, these areas range from 287 to 530 deg2 and enclose about 285–1238 galaxies. After cuts, the number of galaxies is 397 at worst and 27 at best. While the signal-to-noise ratio is the primary determinant of the localization area of a given source, we find that the area is also influenced by the proximity to nearby pulsars on the sky and the binary chirp mass.

A Next Generation of Hierarchical Bayesian Analyses of Hybrid Zones Enables Model-Based Quantification of Variation in Introgression in R.

Hybrid zones, where genetically distinct groups of organisms meet and interbreed, offer valuable insights into the nature of species and speciation. Here, we present a new R package, bgchm, for population genomic analyses of hybrid zones. This R package extends and updates the existing bgc software and combines Bayesian analyses of hierarchical genomic clines with Bayesian methods for estimating hybrid indexes, interpopulation ancestry proportions, and geographic clines. Compared to existing software, bgchm offers enhanced efficiency through Hamiltonian Monte Carlo sampling and the ability to work with genotype likelihoods combined with a hierarchical Bayesian approach, enabling inference for diverse types of genetic data sets. The package also facilitates the quantification of introgression patterns across genomes, which is crucial for understanding reproductive isolation and speciation genetics. We first describe the models underlying bgchm and then provide an overview of the R package and illustrate its use through the analysis of simulated and empirical data sets. We show that bgchm generates accurate estimates of model parameters under a variety of conditions, especially when the genetic loci analyzed are highly ancestry informative. This includes relatively robust estimates of genome-wide variability in clines, which has not been the focus of previous models and methods. We also illustrate how both selection and genetic drift contribute to variability in introgression among loci and how additional information can be used to help distinguish these contributions. We conclude by describing the promises and limitations of bgchm, comparing bgchm to other software for genomic cline analyses, and identifying areas for fruitful future development.

Combining Low-Rank and Deep Plug-and-Play Priors for Snapshot Compressive Imaging.

Snapshot compressive imaging (SCI) is a promising technique that captures a 3-D hyperspectral image (HSI) by a 2-D detector in a compressed manner. The ill-posed inverse process of reconstructing the HSI from their corresponding 2-D measurements is challenging. However, current approaches either neglect the underlying characteristics, such as high spectral correlation, or demand abundant training datasets, resulting in an inadequate balance among performance, generalizability, and interpretability. To address these challenges, in this article, we propose a novel approach called LR2DP that integrates the model-driven low-rank prior and data-driven deep priors for SCI reconstruction. This approach not only captures the spectral correlation and deep spatial features of HSI but also takes advantage of both model-based and learning-based methods without requiring any extra training datasets. Specifically, to preserve the strong spectral correlation of the HSI effectively, we propose that the HSI lies in a low-rank subspace, thereby transforming the problem of reconstructing the HSI into estimating the spectral basis and spatial representation coefficient. Inspired by the mutual promotion of unsupervised deep image prior (DIP) and trained deep denoising prior (DDP), we integrate the unsupervised network and pre-trained deep denoiser into the plug-and-play (PnP) regime to estimate the representation coefficient together, aiming to explore the internal target image prior (learned by DIP) and the external training image prior (depicted by pre-trained DDP) of the HSI. An effective half-quadratic splitting (HQS) technique is employed to optimize the proposed HSI reconstruction model. Extensive experiments on both simulated and real datasets demonstrate the superiority of the proposed method over the state-of-the-art approaches.

VAE-SIMCA — Data-driven method for building one class classifiers with variational autoencoders

The paper proposes a new method for building one class classifiers based on variational autoencoders (VAE). The classification decision is built on a linear combination of two squared distances: computed for the original and the reconstructed image as well as for the representation of the original image inside the latent space formed by VAE. Because both distances are well approximated by scaled chi-square distribution, the decision boundary is computed using the theoretical quantile function for this distribution and the predefined probability for Type I error, ⍺. Thereby the boundary does not require any specific optimization and is solely based on the model outcomes computed for the training set.The original idea of the proposed method is inherited from another OCC approach, Data Driven Soft Independent Method for Class Analogies, where singular value decomposition is employed for building the latent space. In this paper we show how this idea can be adopted to be used with VAE for detection of anomalies on images. The paper describes the theoretical background, introduces the main outcomes as well as tools for visual exploration of the classification results, and shows how the method works on several simulated and real datasets.

Osaka Feedback Model. III. Cosmological Simulation CROCODILE

We introduce our new cosmological simulation data set CROCODILE, executed using the GADGET4-Osaka smoothed particle hydrodynamics code. This simulation incorporates an updated supernova (SN) feedback model of Y. Oku et al. and an active galactic nuclei (AGN) feedback model. A key innovation in our SN feedback model is the integration of a metallicity- and redshift-dependent, top-heavy initial mass function. Our SN model introduces a new consideration that results in an order of magnitude difference in the energy injection rate per unit stellar mass formed at high redshift. The CROCODILE data set is comprehensive, encompassing a variety of runs with diverse feedback parameters. This allows for an in-depth exploration of the relative impacts of different feedback processes in galactic evolution. Our initial comparisons with observational data, spanning the galaxy stellar mass function, the star formation main sequence, and the mass–metallicity relation, show promising agreement, especially for the Fiducial run. These results establish a solid foundation for our future work. We find that SN feedback is a key driver in the chemical enrichment of the intergalactic medium (IGM). Additionally, the AGN feedback creates metal-rich, bipolar outflows that extend and enrich the circumgalactic medium and IGM over a few Mpc scales.

HRRPGraphNet: Make HRRPs to be graphs for efficient target recognition

AbstractHigh Resolution Range Profiles (HRRPs) have become a key area of focus in the domain of Radar Automatic Target Recognition (RATR). Despite the success of deep learning based HRRP recognition, these methods needs a large amount of training samples to generate good performance, which could be a severe challenge under non‐cooperative circumstances. Currently, deep learning based models treat HRRPs as sequences, which may lead to ignorance of the internal relationship of range cells. This letter proposes HRRPGraphNet, a novel graph‐theoretic approach, whose primary innovation is the use of the graph‐theory of HRRP which models the spatial relationships among range cells through a range cell amplitude‐based node vector and a range‐relative adjacency matrix, enabling efficient extraction of both local and global features in noneuclidean space. Experiments on the aircraft electromagnetic simulation dataset confirmed HRRPGraphNet's superior accuracy and robustness compared with existing methods, particularly in limited training sample condition. This underscores the potential of graph‐driven innovations in enhancing HRRP‐based RATR, offering a significant advancement over sequence‐based methods.

Population-Level Cell Trajectory Inference Based on Gaussian Distributions.

In the past decade, inferring developmental trajectories from single-cell data has become a significant challenge in bioinformatics. RNA velocity, with its incorporation of directional dynamics, has significantly advanced the study of single-cell trajectories. However, as single-cell RNA sequencing technology evolves, it generates complex, high-dimensional data with high noise levels. Existing trajectory inference methods, which overlook cell distribution characteristics, may perform inadequately under such conditions. To address this, we introduce CPvGTI, a Gaussian distribution-based trajectory inference method. CPvGTI utilizes a Gaussian mixture model, optimized by the Expectation-Maximization algorithm, to construct new cell populations in the original data space. By integrating RNA velocity, CPvGTI employs Gaussian Process Regression to analyze the differentiation trajectories of these cell populations. To evaluate the performance of CPvGTI, we assess CPvGTI's performance against several state-of-the-art methods using four structurally diverse simulated datasets and four real datasets. The simulation studies indicate that CPvGTI excels in pseudo-time prediction and structural reconstruction compared to existing methods. Furthermore, the discovery of new branch trajectories in human forebrain and mouse hematopoiesis datasets confirms CPvGTI's superior performance.

Detectability of runs of homozygosity is influenced by analysis parameters and population-specific demographic history.

Wild populations are increasingly threatened by human-mediated climate change and land use changes. As populations decline, the probability of inbreeding increases, along with the potential for negative effects on individual fitness. Detecting and characterizing runs of homozygosity (ROHs) is a popular strategy for assessing the extent of individual inbreeding present in a population and can also shed light on the genetic mechanisms contributing to inbreeding depression. Here, we analyze simulated and empirical datasets to demonstrate the downstream effects of program selection and long-term demographic history on ROH inference, leading to context-dependent biases in the results. Through a sensitivity analysis we evaluate how various parameter values impact ROH-calling results, highlighting its utility as a tool for parameter exploration. Our results indicate that ROH inferences are sensitive to factors such as sequencing depth and ROH length distribution, with bias direction and magnitude varying with demographic history and the programs used. Estimation biases are particularly pronounced at lower sequencing depths, potentially leading to either underestimation or overestimation of inbreeding. These results are particularly important for the management of endangered species, as underestimating inbreeding signals in the genome can substantially undermine conservation initiatives. We also found that small true ROHs can be incorrectly lumped together and called as longer ROHs, leading to erroneous inference of recent inbreeding. To address these challenges, we suggest using a combination of ROH detection tools and ROH length-specific inferences, along with sensitivity analysis, to generate robust and context-appropriate population inferences regarding inbreeding history. We outline these recommendations for ROH estimation at multiple levels of sequencing effort, which are typical of conservation genomics studies.

A modified gamma process for RUL prediction based on data with time-varying heavy-tailed distribution

Predicting remaining useful life (RUL) plays a critical role in condition-based maintenance (CBM). However, this task remains challenging as the collected data often has time-varying, non-homogenous properties (trend, variance) and exhibits non-Gaussian distributions. These aspects pose significant challenges to the classical approaches. To address these issues, we propose a modification of the gamma process (suitable for non-Gaussian data) to predict RUL for the degradation process with time-varying characteristics. Specifically, we use the Gompertz cumulative hazard function instead of the linear function used in the classical approach. The modified gamma process aims to outperform the classical one and other variants already used by exploiting its probabilistic nature to effectively manage the uncertainty in the degradation curve. In order to evaluate the effectiveness of the proposed approach, extensive experiments are performed on simulated data with both Gaussian and non-Gaussian distributions. In addition, the performance of the model is validated in real-world scenarios using two benchmark datasets. The results consistently demonstrate the effectiveness of the proposed approach for both simulated and real datasets.

Restoration of X-ray phase-contrast imaging based on generative adversarial networks

For light-element materials, X-ray phase contrast imaging provides better contrast compared to absorption imaging. While the Fourier transform method has a shorter imaging time, it typically results in lower image quality; in contrast, the phase-shifting method offers higher image quality but is more time-consuming and involves a higher radiation dose. To rapidly reconstruct low-dose X-ray phase contrast images, this study developed a model based on Generative Adversarial Networks (GAN), incorporating custom layers and self-attention mechanisms to recover high-quality phase contrast images. We generated a simulated dataset using Kaggle’s X-ray data to train the GAN, and in simulated experiments, we achieved significant improvements in Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM). To further validate our method, we applied it to fringe images acquired from three phase contrast systems: a single-grating phase contrast system, a Talbot-Lau system, and a cascaded grating system. The current results demonstrate that our method successfully restored high-quality phase contrast images from fringe images collected in experimental settings, though it should be noted that these results were achieved using relatively simple sample configurations.

Alleviating the resolution limit problem in spatial community detection: a local network structure-based method

Spatial community detection plays a crucial role in the analysis of spatially embedded networks. However, most existing methods adopt modularity as an objective function, which may fail to detect small spatial communities (i.e. the well-known resolution-limit problem). To alleviate this problem, a local network structure-based spatially constrained Leiden method was developed. First, the weights of the edges were reset based on the local network structure, which facilitated a clearer delineation between distinct communities. Second, we extended Leiden, an effective community detection method, to spatial community detection content by adding spatial constraints. Experiments on simulated datasets demonstrated that the proposed method is superior to four state-of-the-art methods for detecting spatial communities of different sizes. A case study conducted using the Shenzhen taxi dataset demonstrated that the proposed method outperformed four state-of-the-art methods in revealing urban spatial structures. Notably, the modularity of the spatial communities detected using the proposed method exhibited a marked improvement, nearly doubling that of the four comparative methods in the case study. This study presents a novel and promising framework for detecting spatial communities using modularity.

Covariate balancing for high-dimensional samples in controlled experiments

In controlled experiments, achieving covariate balancing across all groups is crucial as it ensures that the estimated treatment effects are not confounded by the effects of covariates. This study proposes a mixed-integer nonlinear programming model to address the covariate balancing problem. Specifically, we introduce a new covariate imbalance measure, which is the maximum discrepancy in both the first and second central moments between any two groups. The second central moment can effectively capture the correlation of covariates in a physical sense, which is crucial for partitioning high-dimensional samples. A mixed-integer nonlinear programming model is constructed to minimize the proposed measure to obtain the optimal partitioning results. The nonlinear model is then linearized to accelerate the optimization process. We conduct computational experiments based on simulated datasets, including one-dimensional, two-dimensional, and three-dimensional Gaussian distributed samples, and a real clinic trial dataset. Compared to the conventional discrepancy-based method, our method achieves a 54.81% and a 40.6% reduction in the maximum discrepancy of partitioning results in the two-dimensional simulated Gaussian samples and the real clinic trial dataset, respectively. These results demonstrate the superiority of the proposed model in partitioning high-dimensional samples with correlated covariates compared with the conventional discrepancy-based method.