Recency Of Data Research Articles

Staggered HLL: Near-continuous-time cardinality estimation with no overhead

Most of existing cardinality estimation algorithms do not support natively interval queries under a sliding window model and are thereby insensitive to data recency. We present Staggered-HyperLogLog (ST-HLL), a probabilistic data structure that takes inspiration from HyperLogLog (HLL) and provides nearly continuous-time estimation of cardinality rates, rather than absolute counts. Our solution has zero-bit overhead with respect to vanilla HLL and negligible additional computational complexity. It is based on a periodic staggered reset of HLL registers and a register equalization operation at query times to compensate for staggered counting. We tested ST-HLL over both synthetic and real Internet traffic traces, showing its ability to track variations of the flow cardinality, quickly adapting to variations under non-stationary flow arrival processes. We show that for the same amount of memory footprint, our algorithm improves the accuracy up to a factor 2x with respect to the state-of-the-art solution, Sliding HLL.

Systematic Development of Short-Term Load Forecasting Models for the Electric Power Utilities: The Case of Pakistan

Load forecasts are fundamental inputs for the reliable and resilient operation of a power system. Globally, researchers endeavor to improve the accuracy of their forecast models. However, lack of studies detailing standardized model development procedures remains a major issue. In this regard, this study advances the knowledge of the systematic development of short-term load forecast (STLF) models for electric power utilities. The proposed model has been developed by using hourly load (time series) of five years of an electric power utility in Pakistan. Following the investigation of previously developed load forecast models, this study addresses the challenges of STLF by utilizing multiple linear regression, bootstrap aggregated decision trees, and artificial neural networks (ANNs) as mutually competitive forecasting techniques. The study also highlights both rudimentary and advanced elements of data extraction, synthetic weather station development, and the use of elastic nets for feature space development to upscale its reproducibility at global level. Simulations showed the superior forecasting prowess of ANNs over other techniques in terms of mean absolute percentage error (MAPE), root mean squared error (RMSE) and R2 score. Furthermore, an empirical approach has been taken to underline the effects of data recency, climatic events, power cuts, human activities, and public holidays on the model’s overall performance. Further analysis of the results showed how climatic variations, causing floods and heavy rainfalls, could prove detrimental for a utility’s ability to forecast its load demand in future.

1508. Hospitalizations and Emergency Department Visits for Respiratory Syncytial Virus Among Infants Aged < 1 year in the United States: An Analysis of Nationwide Inpatient and Emergency Room Data

BackgroundRespiratory syncytial virus (RSV) is a common cause of illness and hospitalization for infants and children globally. The objective of this study was to characterize the burden of RSV and all-cause bronchiolitis (ACB) inpatient hospitalizations and emergency room department (ED) visits in U.S. infants aged < 1 year with the most recent years of data available.Figure 1. RSV and ACB hospitalizations among US infants <1 year old (NIS 2016) Figure 2. ED visits due to RSV and ACB among US infants < 1 year old (NEDS 2016) MethodsThe National (Nationwide) Inpatient Sample (NIS) and the Nationwide Emergency Department Sample (NEDS), which are large national publicly available all-payer databases in the U.S., were used to estimate the burden of RSV in 2016 based on ICD-9 and 10 codes. The proportions of hospitalizations or ED visits due to RSV in infants aged < 1 year were quantified. Due to the potential of missing RSV encounters based on coding practices, ACB was also evaluated. Based on availability of variables and data recency, 2011 NIS data were used to describe RSV burden by age in months. Sensitivity analyses were conducted with NIS and NEDS data from other years (2011-2015).ResultsA clear seasonal pattern was observed for RSV hospitalizations and RSV ED visits with a peak in December-February and a trough in June-August in 2016. During the RSV season and peak months, RSV was a leading cause of hospitalization (12065/45490=27% in Jan and 12050/45080=27% in Feb) and ED visits (26423/316709=8% in Jan and 24721/306397=8% in Feb)) among U.S. infants under 1 year of age. Similar patterns were seen for ACB in 2016 (38% hospitalization and 17-18% ED visits in Jan and Feb) and for RSV and ACB in the other years. For the inpatient setting in 2011, RSV hospitalizations were the highest among the youngest patients (except those aged <1 month) and decreased with age during the RSV season and peak months.ConclusionThese results show that during the RSV season, RSV and ACB were a leading cause of hospitalization and ED visit among US infants under 1 year of age. Current policy does not support routine RSV testing of clinical lower respiratory tract infections (LRTIs) among infants. In that context, as an approximation of RSV LRTI visits in each setting, ACB can be considered an upper bound and RSV can be considered a lower bound of the true proportion of hospital encounters associated with RSV in these settings..DisclosuresMina Suh, MPH, International Health, EpidStrategies (Employee) Xiaohui Jiang, MS, EpidStrategies (Employee)

Synchronization-based clustering on evolving data stream

Clustering streams of data is of increasing importance in many applications. In this paper, we propose a new synchronization-based clustering approach for evolving data streams, called SyncTree, which maintains all micro-clusters at different levels of granularity depending upon the data recency. Instead of using a sliding window or decay function to focus on recent data, SyncTree summarizes all continuously-arriving objects as synchronized micro-clusters sequentially in a batch fashion. Owing to the powerful concept of synchronization, the derived micro-clusters truly reflect the intrinsic cluster structure rather than summarize statistics of data, and old micro-clusters can be intuitively summarized at a higher level by iterative clustering to fit memory constraints. Building upon the hierarchical micro-clusters, SyncTree allows investigating the cluster structure of the data stream between any two time stamps in the past, and also provides a principled way to analyze the cluster evolution. Empirical results demonstrate that our method has good performance compared to state-of-the-art algorithms.

PCN5 - Comparison of Structured Versus Abstracted Comorbidities Using Oncology EHR Data from Cancer Patients in the Flatiron Health Network

The Charlson Comorbidity Index (CCI) is used to characterize risk in populations, and is typically calculated using ICD code algorithms from real-world data (RWD) sources, mainly claims data. Electronic health record (EHR) data are becoming a more common source of RWD for research, and are themselves a source of ICD codes. Using data captured in a specialty Oncology EHR, this study compared the accuracy of the CCI calculated using ICD codes versus abstracted data. We included patients with advanced non-small cell lung cancer from the Flatiron Health EHR-derived Database who received platinum doublet induction and pemetrexed maintenance therapies in the community oncology setting. Data recency was through April 30, 2016. We identified comorbidities used in the calculation of the modified CCI, not including cancer or metastasis, using structured data (ICD-9/10-CM codes) and unstructured data (technology-enabled chart abstraction) from an Oncology EHR. We assessed sensitivity, specificity, and negative predictive value (NPV) of presence of any comorbidity (CCI >= 1) as well as each individual comorbidity, considering the unstructured data to be the gold standard for purposes of calculation. In 715 patients, the sensitivity of CCI >=1 from structured data was low (9%, 95% confidence interval: 5-14%), whereas specificity and NPV were high (99%, 98-100% and 72%, 69-75%, respectively). A similar pattern was observed across individual comorbidities: sensitivity < 30%, specificity 99-100%, and NPV 75-100%. Using data from an Oncology EHR, ICD codes were not sufficient for identification of comorbidities as compared to abstracted information from clinician notes, and may not capture CCI-relevant comorbidities that are documented elsewhere in the EHR. Comorbidity data abstracted from an Oncology EHR may be incomplete and not an ideal gold standard. Next steps include combining Oncology EHR data with additional data sources to increase the completeness of data for use in CCI calculation.

An efficient and highly available framework of data recency enhancement for eventually consistent data stores

Data items are usually replicated in modern distributed data stores to obtain high performance and availability. However, the availability-consistency and latencyconsistency trade-offs exist in data replication, thus system designers intend to choose weak consistency models, such as eventual consistency, which may result in stale reads. Since stale data items may lead to serious application semantic problems, we consider how to increase the probability of data recency which provides a uniform view on recent versions of data items for all clients. In this work, we propose HARP, a framework that can enhance data recency of eventually consistent distributed data stores in an efficient and highly available way. Through detecting possible stale reads under failures or not, HARP can perform reread operations to eliminate stale results only when needed based on our analysis on write/read processes. We also present solutions on how to deal with some practical anomalies in HARP, including delayed, reordered and dropped messages and clock drift, and show how to extend HARP to multiple datacenters. Finally we implement HARP based on Cassandra, and the experiments show that HARP can effectively eliminate stale reads, with a low overhead (less than 6.9%) compared with original eventually consistent Cassandra.

Estimating the Annual Number of Strokes and the Issue of Imperfect Data: An Example from Australia

Estimates of strokes in Australia are typically obtained using 1996-1997 age-specific attack rates from the pilot North East Melbourne Stroke Incidence (NEMESIS) Study (eight postcode regions). Declining hospitalizations for stroke indicate the potential to overestimate cases. To illustrate how current methods may potentially overestimate the number of strokes in Australia. Hospital separations data (primary discharge ICD10 codes I60 to I64) and three stroke projection models were compared. Each model had age- and gender-specific attack rates from the NEMESIS study applied to the 2003 population. One model used the 2003 Burden of Disease approach where the ratio of the 1996-1997 NEMESIS study incidence to hospital separation rate in the same year was adjusted by the 2002/2003 hospital separation rate within the same geographic region using relevant ICD-primary diagnosis codes. Hospital separations data were inflated by 12·1% to account for nonhospitalized stroke, while the Burden of Disease model was inflated by 27·6% to account for recurrent stroke events in that year. The third model used 1997-1999 attack rates from the larger 22-postcode NEMESIS study region. In 2003, Australian hospitalizations for stroke (I60 to I64) were 33,022, and extrapolation to all stroke (hospitalized and nonhospitalized) was 37,568. Applying NEMESIS study attack rates to the 2003 Australian population, 50,731 strokes were projected. Fewer cases for 2003 were estimated with the Burden of Disease model (28,364) and 22-postcode NEMESIS study rates (41,332). Estimating the number of strokes in a country can be highly variable depending on the recency of data, the type of data available, and the methods used.

Probabilistically bounded staleness for practical partial quorums

Data store replication results in a fundamental trade-off between operation latency and data consistency. In this paper, we examine this trade-off in the context of quorum-replicated data stores. Under partial, or non-strict quorum replication, a data store waits for responses from a subset of replicas before answering a query, without guaranteeing that read and write replica sets intersect. As deployed in practice, these configurations provide only basic eventual consistency guarantees, with no limit to the recency of data returned. However, anecdotally, partial quorums are often "good enough" for practitioners given their latency benefits. In this work, we explain why partial quorums are regularly acceptable in practice, analyzing both the staleness of data they return and the latency benefits they offer. We introduce Probabilistically Bounded Staleness (PBS) consistency, which provides expected bounds on staleness with respect to both versions and wall clock time. We derive a closed-form solution for versioned staleness as well as model real-time staleness for representative Dynamo-style systems under internet-scale production workloads. Using PBS, we measure the latency-consistency trade-off for partial quorum systems. We quantitatively demonstrate how eventually consistent systems frequently return consistent data within tens of milliseconds while offering significant latency benefits.

The control of computerized information relating to branch manpower management follow-up systems for banks

The content of information control in complex organizations continues to be a source of interest for both researchers and business managers. This study attempts to demonstrate the application of model-sampling and its effects in relation to accuracy, frequency and recency of data under scrutiny. The study uses a linear model to measure and forecast. The findings of this study are applicable not only to banks but also to Accounting Information Systems (AIS) in general. Moreover, they are as applicable to a computerized follow-up survey as to the initial manual system survey. In fact, this study presents an approach for conducting a follow-up program using an on-line interactive evaluation system.