Uncensored Data Research Articles

Nonparametric Estimation of Customer Segments from Censored Sales Panel Data

Specifically addressing different customer segments, e.g., via revenue management or customer relationship management, lets firms optimise their market response. Identifying such segments requires analysing large amounts of transactional data. To that end, we present a nonparametric approach to estimate the number of customer segments from censored panel data recording sales. As the approach models customer segments and choices via a general finite mixture model, it applies to a diverse range of settings. We evaluate several model selection criteria and imputation methods to compensate for censored observations under different demand scenarios. We measure estimation performance in a controlled environment via simulated data samples, benchmark it to common clustering indices and imputation methods, and analyse empirical data sample to validate practical applicability. The proposed algorithm outperforms all benchmarked cluster segmentation indices. In terms of model selection criteria, we find that the Baysian information criterion works best for uncensored panel data, whereas sequential hypothesis testing is better suited for censored data. For imputation methods, we find that simple rules outperform the nonparametric k-nearest neighbours algorithm and a multiple imputation method with additive regression and bootstrapping while being computationally much faster.

The Gompertz Length Biased Exponential Distribution and its application to Uncensored Data

The Gompertz Length Biased Exponential Distribution and its application to Uncensored Data

Lindley Dağılımında Sansürlü ve Sansürsüz Verilerde Parametre ve Sağ Kalım Fonksiyonları için Nokta ve Aralık Tahmini

Objective: Survival analysis is a statistical method used to compare the life expectancy of patients in health studies and to investigate the effectiveness of the treatments. The Lindley distribution is a widely used distribution in survival analysis in recent years and is used to describe the life of a process or device. The Lindley distribution is a two-parameter continuous distribution that is widely used in a wide range of fields including biology, engineering and medicine. Material and Method: In this study, point and interval estimates of the Lindley distribution were examined for censored and uncensored data. Parameter inferences and confidence intervals are shown. In the application section, real-time data (uncensored) and a simulation data (uncensored and censored) are compared with other distributions of the exponential distribution family. Results: According to the results obtained from the analysis, it was observed that Lindley distribution gave better results in special data structures compared to other distributions of exponential family. Lindley distribution; For uncensored real-life data and censored and uncensored simulation data, it has a lower model selection criterion. Conclusion: The exact distribution of the data to be used in survival analysis is one of the prerequisites for the success of the analysis. The Lindley distribution, with its increasing popularity in recent years, is a distribution that gives successful results in survival analysis.

Comparing Strategies for Modeling Competing Risks in Discrete-Event Simulations: A Simulation Study and Illustration in Colorectal Cancer.

Background. Different strategies toward implementing competing risks in discrete-event simulation (DES) models are available. This study aims to provide recommendations regarding modeling approaches that can be defined based on these strategies by performing a quantitative comparison of alternative modeling approaches. Methods. Four modeling approaches were defined: 1) event-specific distribution (ESD), 2) event-specific probability and distribution (ESPD), 3) unimodal joint distribution and regression model (UDR), and 4) multimodal joint distribution and regression model (MDR). Each modeling approach was applied to uncensored individual patient data in a simulation study and a case study in colorectal cancer. Their performance was assessed in terms of relative event incidence difference, relative absolute event incidence difference, and relative entropy of time-to-event distributions. Differences in health economic outcomes were also illustrated for the case study. Results. In the simulation study, the ESPD and MDR approaches outperformed the ESD and UDR approaches, in terms of both event incidence differences and relative entropy. Disease pathway and data characteristics, such as the number of competing risks and overlap between competing time-to-event distributions, substantially affected the approaches’ performance. Although no considerable differences in health economic outcomes were observed, the case study showed that the ESPD approach was most sensitive to low event rates, which negatively affected performance. Conclusions. Based on overall performance, the recommended modeling approach for implementing competing risks in DES models is the MDR approach, which is defined according to the general strategy of selecting the time-to-event first and the corresponding event second. The ESPD approach is a less complex and equally performing alternative if sufficient observations are available for each competing event (i.e., the internal validity shows appropriate data representation).

Function-based hypothesis testing in censored two-sample location-scale models.

Function-based hypothesis testing in two-sample location-scale models has been addressed for uncensored data using the empirical characteristic function. A test of adequacy in censored two-sample location-scale models is lacking, however. A plug-in empirical likelihood approach is used to introduce a test statistic, which, asymptotically, is not distribution free. Hence for practical situations bootstrap is necessary for performing the test. A multiplier bootstrap and a model appropriate resampling procedure are given to approximate critical values from the null asymptotic distribution. Although minimum distance estimators of the location and scale are deployed for the plug-in, any consistent estimators can be used. Numerical studies are carried out that validate the proposed testing method, and real example illustrations are given.

The type II Topp-Leone generated family of distributions : Properties and applications

In this study, a new family of distributions called the type II Topp-Leone is introduced and three new special models are presented. Some mathematical properties of the type II Topp-Leone family are studied. Explicit expressions for the moments, probability weighted, quantile function, mean deviation, order statistics, Rènyi and Shannon entropies are investigated. Parameter estimates of the family are obtained based on the maximum likelihood procedure for both uncensored and censored data. A simulation study is conducted to show the performance of the new family. Then, two real data sets based on censored and uncensored data sets are employed to show the usefulness of the new family.

A survival model with Birnbaum–Saunders frailty for uncensored and censored cancer data

Survival models with frailty are used when additional data are non-available to explain the occurrence time of an event of interest. This non-availability may be considered as a random effect related to unobserved explanatory variables, or that cannot be measured, often attributed to environmental or genetic factors. We propose a survival model with frailty based on the Birnbaum–Saunders distribution. This distribution has been widely applied to lifetime data. The random effect is the frailty, which is assumed to follow the Birnbaum–Saunders distribution and introduced on the baseline hazard rate to control the unobservable heterogeneity of the patients. We use the maximum likelihood method to estimate the model parameters and evaluate its performance under different censoring proportions by a Monte Carlo simulation study. Two types of residuals are considered to assess the adequacy of the proposed model. Examples with uncensored and censored real-world data sets illustrate the potential applications of the proposed model.

The Weibull Marshall–Olkin family: Regression model and application to censored data

We introduce a new class of distributions called the Weibull Marshall–Olkin-G family. We obtain some of its mathematical properties. The special models of this family provide bathtub-shaped, decreasing-increasing, increasing-decreasing-increasing, decreasing-increasing-decreasing, monotone, unimodal and bimodal hazard functions. The maximum likelihood method is adopted for estimating the model parameters. We assess the performance of the maximum likelihood estimators by means of two simulation studies. We also propose a new family of linear regression models for censored and uncensored data. The flexibility and importance of the proposed models are illustrated by means of three real data sets.

Impact of censoring types on the two-stage method for analyzing reliability experiments with random effects

Many reliability experiments are not completely randomized. Instead they involve subsamples, blocks, split-plot structures, etc. A common analysis often uses random effects to account for the impact of the experimental protocol. The two-stage method is an easy way for practitioners to incorporate random effects in the analysis. This article compares performance of the two-stage method under Type I, Type II censored, and uncensored data from a Weibull distribution. We evaluate the effects of censoring type, censoring rate, sample size, and shape parameter on the two-stage method. Then, we apply the two-stage method to a real experiment. Finally, we give practitioners some recommendations for designing and analyzing reliability experiments.

Simulation shows undesirable results for competing risks analysis with time-dependent covariates for clinical outcomes.

BackgroundWe evaluate three methods for competing risks analysis with time-dependent covariates in comparison with the corresponding methods with time-independent covariates.MethodsWe used cause-specific hazard analysis and two summary approaches for in-hospital death: logistic regression and regression of the subdistribution hazard. We analysed real hospital data (n=1864) and considered pneumonia on admission / hospital-acquired pneumonia as time-independent / time-dependent covariates for the competing events ’discharge alive’ and ’in-hospital death’. Several simulation studies with time-constant hazards were conducted.ResultsAll approaches capture the effect of time-independent covariates, whereas the approaches perform differently with time-dependent covariates. The subdistribution approach for time-dependent covariates detected effects in a simulated no-effects setting and provided counter-intuitive effects in other settings.ConclusionsThe extension of the Fine and Gray model to time-dependent covariates is in general not a helpful synthesis of the cause-specific hazards. Cause-specific hazard analysis and, for uncensored data, the odds ratio are capable of handling competing risks data with time-dependent covariates but the use of the subdistribution approach should be neglected until the problems can be resolved. For general right-censored data, cause-specific hazard analysis is the method of choice.

Big Groundwater Data Sets Reveal Possible Rare Contamination Amid Otherwise Improved Water Quality for Some Analytes in a Region of Marcellus Shale Development.

Eleven thousand groundwater samples collected in the 2010s in an area of Marcellus shale-gas development are analyzed to assess spatial and temporal patterns of water quality. Using a new data mining technique, we confirm previous observations that methane concentrations in groundwater tend to be naturally elevated in valleys and near faults, but we also show that methane is also more concentrated near an anticline. Data mining also highlights waters with elevated methane that are not otherwise explained by geologic features. These slightly elevated concentrations occur near 7 out of the 1,385 shale-gas wells and near some conventional gas wells in the study area. For ten analytes for which uncensored data are abundant in this 3,000 km2 rural region, concentrations are unchanged or improved as compared to samples analyzed prior to 1990. Specifically, TDS, Fe, Mn, sulfate, and pH show small but statistically significant improvement, and As, Pb, Ba, Cl, and Na show no change. Evidence from this rural area could document improved groundwater quality caused by decreased acid rain (pH, sulfate) since the imposition of the Clean Air Act or decreased steel production (Fe, Mn). Such improvements have not been reported in groundwater in more developed areas of the U.S.

Adaptive Queuing Censoring for Big Data Processing

In the era of big data, adaptive censoring (AC) provides us a natural option of trimming data by only keeping the statistical informative data. However, the data chosen by AC may arrive in clusters, which do not relieve the computational resource requirement as expected. In this letter, we exploit queuing theory to model a single sink node with abundant sensor nodes. By adding a buffer to censored distributed wireless sensor networks (WSNs), the uncensored data can be modeled as a queue. With the buffer, the new algorithm entails simple, closed-form updates, and has no loss in terms of estimation accuracy comparing to the original AC method. The proposed model can further reduce the communication cost of distributed WSNs. The proposed model is illustrated in a linear regression setting. Numerical results validate the effectiveness of the proposed model in dealing with data congestion problem.

Long-Term Trends in the Concentrations of Legacy Persistent Organochlorine Pollutants in Whole-Body Samples of Bottomfeeding Fish from Six Large River Monitoring Stations in Kansas

Although persistent organic pollutants (POPs) in bottom-feeding fish have been monitored for decades in Kansas, no information regarding long term trends has been published. Therefore, in this study we examined fish tissue data collected from six long-term, large-river, fish tissue monitoring stations in Kansas for trends in POP concentrations. Based on the findings we also forecasted future persistence and discussed the practical implications of our findings. The data we examined consisted of whole-body composited multi-fish samples collected over periods spanning 21–33 years. Common carp was the target species, but occasionally other bottom-feeding species were substituted when necessary. The sample constituents examined were dieldrin, technical chlordane, dichlorodiphenyltrichloroethanes (DDTs), and polychlorinated biphenyls (PCBs). Constituent data were normalized by lipid content prior to analysis, however, because lipid content was not measured during certain time periods, multiple imputation (MI) and analyses techniques (SAS 2014) were employed to overcome the data gaps. The lipidnormalized data were analyzed for time trends using robust non-parametric techniques, which were capable of handling both censored and uncensored data. Because of expected high sample variability, correlations between time and constituent concentrations were considered statistically probable (significant) at p-values < 0.1. Where correlations were significant, the trend line equations were extrapolated to current method reporting limit (RLs) concentrations and 90% prediction intervals (PIs) were constructed around the forecasted RL/year intercepts. Fifteen of the 24 analyses resulted in statistical significance, but no monitoring station experienced significantly declining trends among all four constituents, nor did any constituent significantly decrease across all six stations. The average forecasted RL/year intercept was 2050 (PI = /- 23 years), however some upper PIs suggested some POPS could remain detectable in fish for at least another century.

A new useful four-parameter extension of the Gumbel distribution: Properties, regression model and applications using the GAMLSS framework

ABSTRACTIn this paper we introduce a flexible extension of the Gumbel distribution called the odd log-logistic exponentiated Gumbel distribution. The new model was implemented in GAMLSS package of R software and a brief tutorial on how to use this package is presented throughout the paper. We provide a comprehensive treatment of its general mathematical properties. Further, we propose a new extended regression model considering four regression structures. We discuss estimation methods based on censored and uncensored data. Two simulation studies are presented and four real data sets are applied to illustrating the usefulness of the new model.

Estimation of ultimate bond strength for soil nails in clayey soils using maximum likelihood method

ABSTRACTThis paper presents a maximum likelihood estimation of the ultimate bond strength for soil nails in clays. Both uncensored and censored ultimate bond strength data for soil nails are collected from the literature. Based on the concept of maximum likelihood, a log-likelihood function is constructed for estimating the mean and coefficient of variation (COV) of the ultimate bond strength jointly using the two types of data. The mean and COV are determined as the pair that maximises the log-likelihood function. Two distribution models (normal and lognormal) are used for the estimation. A comparison of the relative competence between the two candidate distribution models that are adopted for describing the collected uncensored and censored data is performed using the Bayesian Information Criterion. Example designs of soil nail walls against internal pullout limit state of nails and overall stability limit state are provided to demonstrate the benefit of taking censored data into account for estimation of the ultimate bond strength of soil nails.

The power-Cauchy negative-binomial: properties and regression

We propose and study a new compounded model to extend the half-Cauchy and power-Cauchy distributions, which offers more flexibility in modeling lifetime data. The proposed model is analytically tractable and can be used effectively to analyze censored and uncensored data sets. Its density function can have various shapes such as reversed-J and right-skewed. It can accommodate different hazard shapes such as decreasing, upside-down bathtub and decreasing-increasing-decreasing. Some mathematical properties of the new distribution can be determined from a linear combination for its density function such as ordinary and incomplete moments. The performance of the maximum likelihood method to estimate the model parameters is investigated by a simulation study. Further, we introduce the new log-power-Cauchy negative-binomial regression model for censored data, which includes as sub-models some widely known regression models that can be applied to censored data. Four real life data sets, of which one is censored, have been analyzed and the new models provide adequate fits.

A new family of distributions to analyze lifetime data

In this paper, a new family of distributions is proposed by using quantile functions of known distributions. Some general properties of this family are studied. A special case of the proposed family is studied in detail, namely the Lomax-Weibull distribution. Some structural properties of the special model are established. This distribution has been applied to several censored and uncensored data sets with various shapes.

Consumer Choice Models with Endogenous Network Effects

Network externality arises when the utility of a product depends not only on its attributes but also on the number of consumers who purchase the same product. In this paper, we study consumer choice models that endogenize such network externality. We first characterize the choice probabilities under such models and conduct studies on comparative statics. Then we investigate the assortment optimization problem under such choice models. Although the problem is generally NP-hard, we show that a new class of assortments, called quasi-revenue-ordered assortments, which consist of a revenue-ordered assortment plus at most one additional item, are optimal under mild conditions. We also propose an iterative estimation method to calibrate such choice models, for both uncensored and censored data cases. An empirical study on a mobile game data set shows that our proposed model can provide better fits for the data, increase the prediction accuracy for consumer choices, and potentially increase revenue.This paper was accepted by Noah Gans, stochastic models and simulation.

The Adjusted Log-logistic Generalized Exponential Distribution with Application to Lifetime Data

This paper introduces a new generator of probability distribution-the adjusted log-logistic generalized (ALLoG) distribution and a new extension of the standard one parameter exponential distribution called the adjusted log-logistic generalized exponential (ALLoGExp) distribution. The ALLoGExp distribution is a special case of the ALLoG distribution and we have provided some of its statistical and reliability properties. Notably, the failure rate could be monotonically decreasing, increasing or upside-down bathtub shaped depending on the value of the parameters $\delta$ and $\theta$. The method of maximum likelihood estimation was proposed to estimate the model parameters. The importance and flexibility of he ALLoGExp distribution was demonstrated with a real and uncensored lifetime data set and its fit was compared with five other exponential related distributions. The results obtained from the model fittings shows that the ALLoGExp distribution provides a reasonably better fit than the one based on the other fitted distributions. The ALLoGExp distribution is therefore ecommended for effective modelling of lifetime data sets.

The Extended Erlang-Truncated Exponential distribution: Properties and application to rainfall data.

The Erlang-Truncated Exponential ETE distribution is modified and the new lifetime distribution is called the Extended Erlang-Truncated Exponential EETE distribution. Some statistical and reliability properties of the new distribution are given and the method of maximum likelihood estimate was proposed for estimating the model parameters. The usefulness and flexibility of the EETE distribution was illustrated with an uncensored data set and its fit was compared with that of the ETE and three other three-parameter distributions. Results based on the minimized log-likelihood (), Akaike information criterion (AIC), Bayesian information criterion (BIC) and the generalized Cramér–von Mises statistics shows that the EETE distribution provides a more reasonable fit than the one based on the other competing distributions.