Real Datasets Research Articles

Stress–strength reliability estimation of s-out-of-k multicomponent systems based on copula function for dependent strength elements under progressively censored sample

The main objective of this research is to determine the reliability of multicomponent stress–strength under progressive Type II censoring. The model comprises k independent strength components, and each component has two elements that are statistically dependent and exposed to the same random stress. To assess the correlation between their elements, we are utilizing the Gumbel copula and assuming that its parameter is unknown. The system is deemed operational only if at least certain number of strength variables s ( 1 ≤ s ≤ k ) surpass the random stress. We are using the Chen distribution to model the stress and strength variables, with varying shape parameters. Different methods are employed to estimate the unknown parameters, including maximum likelihood, maximum product of spacings, Cramér–von-Mises, and bootstrap confidence intervals. The effectiveness of these estimation techniques is evaluated through Monte Carlo simulation, and a real data set is analyzed to illustrate our procedures.

New alpha power transformed beta distribution with its properties and applications

The main purpose of this paper is to introduce a new alpha power transformed beta probability distribution that reveals interesting properties. The studuy provide a comprehensive explanation of the statistical characteristics of this innovative model. Various properties of the new distribution were derived, using the baseline beta distribution, statistical techniques, and probabilistic axioms. These include the probability density, cumulative distribution, survival function, hazard function, moments about the origin, moment generating function, and order statistics. For parameter estimation, the maximum likelihood estimation method using Newton Raphson numerical technique is employed. To evaluate the performance of our estimation method, the mean squared errors of the estimated parameters for different simulated sample sizes are used. In addition simulation studies of the new distribution are conducted to demonstrate the behavior of the probability model. To demonstrate the practical utility and flexibility of the alpha power transformed beta distribution, it is fitted to two real-life datasets and compared to commonly known probability distributions such as the Weibull, exponential Weibull, Beta, and Kumaraswamy beta distributions. It offers a superior fit to the data considered. The distribution reviales of the microbes reveald a wide range of shapes of probability density functions and flexible hazard rates. The distribution is a new contribution to the field of statistical and probability theory. The findings of the study can be used as a basis for future research in the area of statistical science and health.

The Impact of the Opportunity Zone Program on Residential Real Estate

Problem definition: Opportunity zones (OZs) are designated census tracts in which real estate investments can gain tax benefits. Introduced by the U.S. Tax Cuts and Jobs Act of 2017, the goal of the OZ program is to foster economic development in distressed neighborhoods. In this paper, we investigate and optimize the OZ selection process and examine the impact of OZs by exploiting two data sets: a proprietary real estate data set that includes 36.1 million residential transactions spanning all 50 U.S. states and census-tract demographics data between 2010 and 2019. Methodology/results: We show that census tracts with higher poverty and unemployment rates were more likely to be selected. Counterintuitively, however, tracts with a higher average real estate price were also more likely to be selected. We then apply difference-in-differences, synthetic control, and matching techniques to rigorously assess the impact of the OZ program on two key real estate metrics: price and transaction volume. We find that the OZ program increased real estate prices by 4.03%–6.13% but do not observe a significant effect on the transaction volume. We also find that investors primarily targeted the high-end real estate market, namely, exhibiting a cherry-picking behavior. To better fulfill its intended societal and economic goals, we propose an optimization framework with fairness considerations for OZ assignment decisions. We show that the OZs assigned from our fairness-aware optimization formulation can better serve distressed communities and mitigate investors’ cherry-picking behavior. Managerial implications: Our paper underscores the importance of incorporating fairness in OZ designation to achieve a desirable real estate market reaction. Our large-scale empirical analysis provides a comprehensive assessment of the current government OZ assignment, and our fairness-aware optimization framework provides concrete recommendations for policy makers. Supplemental Material: The online appendix is available at https://doi.org/10.1287/msom.2024.0746 .

Statistical inference on the exponentiated moment exponential distribution and its discretization

Hasnain and Ahmad (2013) propose a two-parameter Exponentiated Moment Exponential (EME) Distribution and investigate its some characteristics. In this paper we study its additional properties in the context of its applications. The study classifies the EME distribution to homogeneous subfamilies with respect to hazard rate. A simulation study is conducted for EME distribution to observe the parameters for maximum likelihood and quartile estimates based on bias and mean square error. Based on EME distribution, we develop a discretized model that gives its importance in the industry. However, some important structural properties are studied for discrete Exponentiated moment exponential distribution with useful characterizations. Shapes of density and failure rate function of new discrete model are studied. Moreover, we provide justifications for the usefulness of the model and its enhanced scope as compared to the existing discretized model with the help of real-life data sets.

A Maneuver in the Trade-Off Space of Federated Learning Aggregation Frameworks Secured with Polymorphic Encryption: PolyFLAM and PolyFLAP Frameworks

Maintaining user privacy in machine learning is a critical concern due to the implications of data collection. Federated learning (FL) has emerged as a promising solution by sharing trained models rather than user data. However, FL still faces several challenges, particularly in terms of security and privacy, such as vulnerability to inference attacks. There is an inherent trade-off between communication traffic across the network and computational costs on the server or client, which this paper aims to address by maneuvering between these performance parameters. To tackle these issues, this paper proposes two complementary frameworks: PolyFLAM (“Polymorphic Federated Learning Aggregation of Models”) and PolyFLAP (“Polymorphic Federated Learning Aggregation of Parameters”). These frameworks provide two options to suit the needs of users, depending on whether they prioritize reducing communication across the network or lowering computational costs on the server or client. PolyFLAM reduces computational costs by exchanging entire models, eliminating the need to rebuild models from parameters. In contrast, PolyFLAP reduces communication costs by transmitting only model parameters, which are smaller in size compared to entire models. Both frameworks are supported by polymorphic encryption, ensuring privacy is maintained even in cases of key leakage. Furthermore, these frameworks offer five different machine learning models, including support vector machines, logistic regression, Gaussian naïve Bayes, stochastic gradient descent, and multi-layer perceptron, to cover as many real-life problems as possible. The evaluation of these frameworks with simulated and real-life datasets demonstrated that they can effectively withstand various attacks, including inference attacks that aim to compromise user privacy by capturing exchanged models or parameters.

Gastrointestinal image stitching based on improved unsupervised algorithm.

Image stitching is a traditional but challenging computer vision task. The goal is to stitch together multiple images with overlapping areas into a single, natural-looking, high-resolution image without ghosts or seams. This article aims to increase the field of view of gastroenteroscopy and reduce the missed detection rate. To this end, an improved depth framework based on unsupervised panoramic image stitching of the gastrointestinal tract is proposed. In addition, preprocessing for aberration correction of monocular endoscope images is introduced, and a C2f module is added to the image reconstruction network to improve the network's ability to extract features. A comprehensive real image data set, GASE-Dataset, is proposed to establish an evaluation benchmark and training learning framework for unsupervised deep gastrointestinal image splicing. Experimental results show that the MSE, RMSE, PSNR, SSIM and RMSE_SW indicators are improved, while the splicing time remains within an acceptable range. Compared with traditional image stitching methods, the performance of this method is enhanced. In addition, improvements are proposed to address the problems of lack of annotated data, insufficient generalization ability and insufficient comprehensive performance in image stitching schemes based on supervised learning. These improvements provide valuable aids in gastrointestinal examination.

Exploring mixture estimators in stratified random sampling.

Advancements in sensor technology have brought a revolution in data generation. Therefore, the study variable and several linearly related auxiliary variables are recorded due to cost-effectiveness and ease of recording. These auxiliary variables are commonly observed as quantitative and qualitative (attributes) variables and are jointly used to estimate the study variable's population mean using a mixture estimator. For this purpose, this work proposes a family of generalized mixture estimators under stratified sampling to increase efficiency under symmetrical and asymmetrical distributions and study the estimator's behavior for different sample sizes for its convergence to the Normal distribution. It is found that the proposed estimator estimates the population mean of the study variable with more precision than the competitor estimators under Normal, Uniform, Weibull, and Gamma distributions. It is also revealed that the proposed estimator follows the Cauchy distribution when the sample size is less than 35; otherwise, it converges to normality. Furthermore, the implementation of two real-life datasets related to the health and finance sectors is also presented to support the proposed estimator's significance.

Feature extraction via 3-D homogeneous attribute decomposition for hyperspectral imagery classification

ABSTRACT Feature extraction is a core aspect in hyperspectral image classification, which can extract key information closely related to ground cover from complex scene, thus improving classification accuracy. Therefore, designing an effective feature extraction network is a hotspot and a challenge in the current research. In this paper, a feature extraction framework based on 3-D homogeneous attribute decomposition (3D-HAD) is proposed for HSI classification, which consists of the following key technologies. First, the principal component analysis algorithm is applied to the raw HSI to extract the principal components (PCs), and the raw HSI is clustered into many 3D superpixel blocks according to the first three PCs-based over-segmentation strategy. Then, a superpixel intrinsic attribute decomposition (SIAD) is designed to capture reflectance feature and suppress shading feature. Meanwhile, a metric entropy is introduced into the decomposition process to overcome the spectral-spatial weak assumption among pixels. Next, superpixel-guided recursive filtering is employed to preserve global details of HSI to enhance accuracy in HSI classification. Finally, the support vector machine classifier is used to obtain classification results of HSI. Experiments performed on several real hyperspectral data sets with limited training samples indicate that the proposed 3D-HAD method outperforms the classic, advance, and deep learning classification methods.

Data Synthesis is Going Mobile – On Community-driven Dataset Generation for Android Devices

Personal electronic devices such as smartphones and smartwatches have become indispensable daily companions, collecting a multitude of personal and sensitive data. As a result, they are of paramount importance in digital forensic examinations. However, there is a lack of publicly available and ready-to-use digital forensic datasets, especially in mobile forensics. This work presents a concept and an open-source proof-of-concept implementation, which simplifies and automates the creation of mobile forensic datasets within the scope of the Android operating system. In contrast to previous approaches, which populate the most common databases of an Android device, our concept is based on community-driven playbooks and makes use of interaction with the actual smartphone GUI. Hence, we are able to generate coherent and realistic traces as they occur in real-world human usage. Our proof-of-concept implementation is based on the standard Android emulation environment and borrows tools from the user interface testing community. Our evaluation shows that our approach actually generates realistic Android datasets. For instance, we can generate traces that cannot be simulated by gestures (e.g., changing the GPS position or triggering incoming phone calls). Recording the actual data synthesis process allows users to either create and share their own playbooks (i.e., the exact instructions for the data synthesis process rather than having to share the full image) or reproduce Android images with different scenarios using playbooks previously created and shared by the community.

Analyzing symmetric distributions by utilizing extropy measures based on order statistics

Quantification of the uncertainty of distribution functions, by using entropy and extropy, is important in many statistical analyses. Inspired by this, our study uses extropy and several related measures (including the cumulative residual extropy, cumulative past extropy, and extropy-inaccuracy measure) of order statistics (OSs) to offer multiple characterizations of symmetric continuous distributions. We demonstrate that a defining feature of symmetric distributions is the equality of these measures of upper and lower OSs. Using concomitants of OSs based on the bivariate distributions belonging to the Farlie-Gumbel-Morgenstern (FGM) family, the same characteristic is demonstrated for these measures. Finally, a real data set is used to illustrate the applicability of the suggested test.

Bayesian inference for Laplace distribution based on complete and censored samples with illustrations

In this paper, Bayesian estimates are derived for the location and scale parameters of the Laplace distribution based on complete, Type-I, and Type-II censored samples under different prior settings. Subsequently, Bayesian point and interval estimates, as well as the associated statistical inference, are discussed in detail. The developed methods are then applied to two real data sets for illustrative purposes. Moreover, a detailed Monte Carlo simulation study is carried out for evaluating the performance of the inferential methods developed here. Finally, we provide a brief discussion of the established results to demonstrate their practical utility and present some associated problems of further interest. Overall, this study fills an existing gap in the development of Bayesian inferential techniques for the parameters of the two-parameter Laplace distribution, making this research innovative and offering more investigative implications. It showcases the potential for broader methodological applications of Bayesian inference for complex real-world data sets, especially in scenarios involving different forms of censoring. This research provides a critical tool for statistical analysis in different fields such as engineering and finance, where the Laplace distribution is frequently adopted as a fundamental model.

An automated convergence diagnostic for phylogenetic MCMC analyses.

Assessing convergence of Markov chain Monte Carlo (MCMC) based analyses is crucial but challenging, especially so in high dimensional and complex spaces such as the space of phylogenetic trees (treespace). In practice, it is assumed that the target distribution is the unique stationary distribution of the MCMC and convergence is achieved when samples appear to be stationary. Here we leverage recent advances in computational geometry of the treespace and introduce a method that combines classical statistical techniques and algorithms with geometric properties of the treespace to automatically evaluate and assess practical convergence of phylogenetic MCMC analyses. Our method monitors convergence across multiple MCMC chains and achieves high accuracy in detecting both practical convergence and convergence issues within treespace. Furthermore, our approach is developed to allow for real-time evaluation during the MCMC algorithm run, eliminating any of the chain post-processing steps that are currently required. Our tool therefore improves reliability and efficiency of MCMC based phylogenetic inference methods and makes analyses easier to reproduce and compare. We demonstrate the efficacy of our diagnostic via a well-calibrated simulation study and provide examples of its performance on real data sets. Although our method performs well in practice, a significant part of the underlying treespace probability theory is still missing, which creates an excellent opportunity for future mathematical research in this area. The open source package for the phylogenetic inference framework BEAST2, called ASM, that implements these methods, making them accessible through a user-friendly GUI, is available from https://github.com/rbouckaert/asm/. The open source Python package, called tetres, that provides an interface for these methods enabling their applications beyond BEAST2 can be accessed at https://github.com/bioDS/tetres/.

On the Fairness of Internet Congestion Control over WiFi with Deep Reinforcement Learning

For over forty years, TCP has been the main protocol for transporting data on the Internet. To improve congestion control algorithms (CCAs), delay bounding algorithms such as Vegas, FAST, BBR, PCC, and Copa have been developed. However, despite being designed to ensure fairness between data flows, these CCAs can still lead to unfairness and, in some cases, even cause data flow starvation in WiFi networks under certain conditions. We propose a new CCA switching solution that works with existing TCP and WiFi standards. This solution is offline and uses Deep Reinforcement Learning (DRL) trained on features such as noncongestive delay variations to predict and prevent extreme unfairness and starvation. Our DRL-driven approach allows for dynamic and efficient CCA switching. We have tested our design preliminarily in realistic datasets, ensuring that they support both fairness and efficiency over WiFi networks, which requires further investigation and extensive evaluation before online deployment.

A Practical Solver for Scalar Data Topological Simplification.

This paper presents a practical approach for the optimization of topological simplification, a central pre-processing step for the analysis and visualization of scalar data. Given an input scalar field f and a set of "signal" persistence pairs to maintain, our approaches produces an output field g that is close to f and which optimizes (i) the cancellation of "non-signal" pairs, while (ii) preserving the "signal" pairs. In contrast to pre-existing simplification algorithms, our approach is not restricted to persistence pairs involving extrema and can thus address a larger class of topological features, in particular saddle pairs in three-dimensional scalar data. Our approach leverages recent generic persistence optimization frameworks and extends them with tailored accelerations specific to the problem of topological simplification. Extensive experiments report substantial accelerations over these frameworks, thereby making topological simplification optimization practical for real-life datasets. Our approach enables a direct visualization and analysis of the topologically simplified data, e.g., via isosurfaces of simplified topology (fewer components and handles). We apply our approach to the extraction of prominent filament structures in three-dimensional data. Specifically, we show that our pre-simplification of the data leads to practical improvements over standard topological techniques for removing filament loops. We also show how our approach can be used to repair genus defects in surface processing. Finally, we provide a C++ implementation for reproducibility purposes.

Sub-sampling graph neural networks for genomic prediction of quantitative phenotypes.

In genomics, use of deep learning (DL) is rapidly growing and DL has successfully demonstrated its ability to uncover complex relationships in large biological and biomedical data sets. With the development of high-throughput sequencing techniques, genomic markers can now be allocated to large sections of a genome. By analysing allele sharing between individuals, one may calculate realized genomic relationships from single nucleotide polymorphisms (SNPs) data rather than relying on known pedigree relationships under polygenic model. The traditional approaches in genome-wide prediction (GWP) of quantitative phenotypes utilise genomic relationships in fixed global covariance modelling, possibly with some non-linear kernel mapping (for example Gaussian processes). On the other hand, the DL approaches proposed so far for GWP fail to take into account the non-Euclidean graph structure of relationships between individuals over several generations. In this paper, we propose one global convolutional neural network (GCN) and one local sub-sampling architecture (GCN-RS) that are specifically designed to perform regression analysis based on genomic relationship information. A GCN is tailored to non-Euclidean spaces and consists of several layers of graph convolutions. The GCN-RS architecture is designed to further improve the GCN's performance by sub-sampling the graph to reduce the dimensionality of the input data. Through these graph convolutional layers, the GCN maps input genomic markers to their quantitative phenotype values. The graphs are constructed using an iterative nearest neighbour approach. Comparisons show that the GCN-RS outperforms the popular Genomic Best Linear Unbiased Predictor (GBLUP) method on one simulated and three real data sets from wheat, mice and pig with a predictive improvement of 4.4% to 49.4% in terms of test mean squared error (MSE). This indicates that GCN-RS is a promising tool for genomic predictions in plants and animals. Furthermore, GCN-RS is computationally efficient, making it a viable option for large-scale applications.

Properties, estimation, and applications of the extended log-logistic distribution

This paper presents the exponentiated alpha-power log-logistic (EAPLL) distribution, which extends the log-logistic distribution. The EAPLL distribution emphasizes its suitability for survival data modeling by providing analytical simplicity and accommodating both monotone and non-monotone failure rates. We derive some of its mathematical properties and test eight estimation methods using an extensive simulation study. To determine the best estimation approach, we rank mean estimates, mean square errors, and average absolute biases on a partial and overall ranking. Furthermore, we use the EAPLL distribution to examine three real-life survival data sets, demonstrating its superior performance over competing log-logistic distributions. This study adds vital insights to survival analysis methodology and provides a solid framework for modeling various survival data scenarios.

Review on synergizing the Metaverse and AI-driven synthetic data: enhancing virtual realms and activity recognition in computer vision

The Metaverse’s emergence is redefining digital interaction, enabling seamless engagement in immersive virtual realms. This trend’s integration with AI and virtual reality (VR) is gaining momentum, albeit with challenges in acquiring extensive human action datasets. Real-world activities involve complex intricate behaviors, making accurate capture and annotation difficult. VR compounds this difficulty by requiring meticulous simulation of natural movements and interactions. As the Metaverse bridges the physical and digital realms, the demand for diverse human action data escalates, requiring innovative solutions to enrich AI and VR capabilities. This need is underscored by state-of-the-art models that excel but are hampered by limited real-world data. The overshadowing of synthetic data benefits further complicates the issue. This paper systematically examines both real-world and synthetic datasets for activity detection and recognition in computer vision. Introducing Metaverse-enabled advancements, we unveil SynDa’s novel streamlined pipeline using photorealistic rendering and AI pose estimation. By fusing real-life video datasets, large-scale synthetic datasets are generated to augment training and mitigate real data scarcity and costs. Our preliminary experiments reveal promising results in terms of mean average precision (mAP), where combining real data and synthetic video data generated using this pipeline to train models presents an improvement in mAP (32.35%), compared to the mAP of the same model when trained on real data (29.95%). This demonstrates the transformative synergy between Metaverse and AI-driven synthetic data augmentation.

On Dagum-Pareto mixture distribution: properties, simulation and application in insurance

The most principal subject in actuarial science is the modeling of claims amount. There are two types of distributions associated with two types of risk. We speak of light-tailed distributions as opposed to heavy-tailed distributions or extreme value distribution. Among the distributions of the amount of heavy tail claim is: Weibull, Log-Normal, Pareto and Burr “Also named DAGUM”. Many attempts were implemented in expanding the classes of mixed and compound distributions. In this article, we present a new distribution by mixing Dagum distribution (α, λ, r) and Pareto distribution (α, λ) named Dagum-Pareto mixture distribution. Some of statistical properties of this new distribution are obtained which include reliability function, hazard function, skewness, kurtosis and quantile function. Also we use the maximum likelihood method to estimate the model parameters. A Simulation study was carried out to examine the performance and accuracy of the maximum likelihood estimates of the proposed distribution. An application of the new model to no-life insurance real data set is presented to illustrate its usefulness and applicability. In our case, it is proven that Dagum-Pareto mixture distribution provides better fit compared to others existing probability distributions using some goodness of fit measures.

Different estimation techniques and data analysis for constant-partially accelerated life tests for power half-logistic distribution

Partial accelerated life tests (PALTs) are employed when the results of accelerated life testing cannot be extended to usage circumstances. This work discusses the challenge of different estimating strategies in constant PALT with complete data. The lifetime distribution of the test item is assumed to follow the power half-logistic distribution. Several classical and Bayesian estimation techniques are presented to estimate the distribution parameters and the acceleration factor of the power half-logistic distribution. These techniques include Anderson–Darling, maximum likelihood, Cramér von-Mises, ordinary least squares, weighted least squares, maximum product of spacing and Bayesian. Additionally, the Bayesian credible intervals and approximate confidence intervals are constructed. A simulation study is provided to compare the outcomes of various estimation methods that have been provided based on mean squared error, absolute average bias, length of intervals, and coverage probabilities. This study shows that the maximum product of spacing estimation is the most effective strategy among the options in most circumstances when adopting the minimum values for MSE and average bias. In the majority of situations, Bayesian method outperforms other methods when taking into account both MSE and average bias values. When comparing approximation confidence intervals to Bayesian credible intervals, the latter have a higher coverage probability and smaller average length. Two authentic data sets are examined for illustrative purposes. Examining the two real data sets shows that the value methods are workable and applicable to certain engineering-related problems.

On autoregressive processes with Lindley-distributed innovations: modeling and simulation

In this paper, we develop an autoregressive process of order one, assuming that the innovation random variable has a Lindley distribution. The key properties of the process are investigated. Five distinct estimation techniques are used to estimate the parameters and simulation studies are conducted. The stationarity of the process is tested using a unit root test. The application of the proposed process to the analysis of time series data is demonstrated using real data sets. Based on some important statistical measures, the analysis of the data sets reveals that the proposed model fits well, and the errors are independent and Lindley-distributed.