Subsample Of Data Research Articles

Landmark estimation of transition probabilities in non-Markov multi-state models with covariates.

In non-Markov multi-state models, the traditional Aalen-Johansen (AJ) estimator for state transition probabilities is generally not valid. An alternative, suggested by Putter and Spitioni, is to analyse a subsample of the full data, consisting of the individuals present in a specific state at a given landmark time-point. The AJ estimator of occupation probabilities is then applied to the landmark subsample. Exploiting the result by Datta and Satten, that the AJ estimator is consistent for state occupation probabilities even in non-Markov models given that censoring is independent of state occupancy and times of transition between states, the landmark Aalen-Johansen (LMAJ) estimator provides consistent estimates of transition probabilities. So far, this approach has only been studied for non-parametric estimation without covariates. In this paper, we show how semi-parametric regression models and inverse probability weights can be used in combination with the LMAJ estimator to perform covariate adjusted analyses. The methods are illustrated by a simulation study and an application to population-wide registry data on work, education and health-related absence in Norway. Results using the traditional AJ estimator and the LMAJ estimator are compared, and show large differences in estimated transition probabilities for highly non-Markov multi-state models.

Compressed Sensing with Gaussian Sampling Kernel for Ultrasound Imaging

Recently, compressed sensing (CS) has been applied to ultrasound imaging for either data reduction or frame rate improvement. However, there are no detailed reports yet on strategies for lateral undersampling of channel data in conventional focused beamforming (CFB) and its recovery exploiting the CS approach. We propose a strategic lateral undersampling approach for channel data using the Gaussian sampling scheme and compare it with a direct extension of the often-used uniform undersampling reported for axial undersampling to the lateral direction and 2-D random sampling reported in the literature. As opposed to the reported 2-D random undersampling, we explore undersampling of channel data in the lateral direction by acquiring radiofrequency data from only a reduced number of chosen receive elements and subjecting these data to further undersampling in the axial direction. The effect of the sampling schemes on CS recovery was studied using data from simulations and experiments for various lateral and axial undersampling rates. The results suggest that CS-recovered data from the Gaussian distribution-based channel data subsampling yielded better recovery and contrast in comparison to those obtained from the often-used uniform distribution-based undersampling. Although 90% of the samples from the original data using the proposed sampling scheme were discarded, the contrast of the CS-recovered image was comparable to that of the reference image. Thus, CS with the proposed Gaussian sampling scheme for channel data subsampling not only reduces the data size significantly, but also strategically uses only a few active receive elements in the process; thus, it can provide an attractive option for the affordable point-of-care ultrasound system.

Fast and Accurate Uncertainty Estimation in Chemical Machine Learning.

We present a scheme to obtain an inexpensive and reliable estimate of the uncertainty associated with the predictions of a machine-learning model of atomic and molecular properties. The scheme is based on resampling, with multiple models being generated based on subsampling of the same training data. The accuracy of the uncertainty prediction can be benchmarked by maximum likelihood estimation, which can also be used to correct for correlations between resampled models and to improve the performance of the uncertainty estimation by a cross-validation procedure. In the case of sparse Gaussian Process Regression models, this resampled estimator can be evaluated at negligible cost. We demonstrate the reliability of these estimates for the prediction of molecular and materials energetics and for the estimation of nuclear chemical shieldings in molecular crystals. Extension to estimate the uncertainty in energy differences, forces, or other correlated predictions is straightforward. This method can be easily applied to other machine-learning schemes and will be beneficial to make data-driven predictions more reliable and to facilitate training-set optimization and active-learning strategies.

Stability in GRN Inference.

Reconstructing a gene regulatory network from one or more sets of omics measurements has been a major task of computational biology in the last 20 years. Despite an overwhelming number of algorithms proposed to solve the network inference problem either in the general scenario or in an ad-hoc tailored situation, assessing the stability of reconstruction is still an uncharted territory and exploratory studies mainly tackled theoretical aspects. We introduce here empirical stability, which is induced by variability of reconstruction as a function of data subsampling. By evaluating differences between networks that are inferred using different subsets of the same data we obtain quantitative indicators of the robustness of the algorithm, of the noise level affecting the data, and, overall, of the reliability of the reconstructed graph. We show that empirical stability can be used whenever no ground truth is available to compute a direct measure of the similarity between the inferred structure and the true network. The main ingredient here is a suite of indicators, called NetSI, providing statistics of distances between graphs generated by a given algorithm fed with different data subsets, where the chosen metric is the Hamming-Ipsen-Mikhailov (HIM) distance evaluating dissimilarity of graph topologies with shared nodes. Operatively, the NetSI family is demonstrated here on synthetic and high-throughput datasets, inferring graphs at different resolution levels (topology, direction, weight), showing how the stability indicators can be effectively used for the quantitative comparison of the stability of different reconstruction algorithms.

Subsampling MCMC - an Introduction for the Survey Statistician

The rapid development of computing power and efficient Markov Chain Monte Carlo (MCMC) simulation algorithms have revolutionized Bayesian statistics, making it a highly practical inference method in applied work. However, MCMC algorithms tend to be computationally demanding, and are particularly slow for large datasets. Data subsampling has recently been suggested as a way to make MCMC methods scalable on massively large data, utilizing efficient sampling schemes and estimators from the survey sampling literature. These developments tend to be unknown by many survey statisticians who traditionally work with non-Bayesian methods, and rarely use MCMC. Our article explains the idea of data subsampling in MCMC by reviewing one strand of work, Subsampling MCMC, a so called pseudo-marginal MCMC approach to speeding up MCMC through data subsampling. The review is written for a survey statistician without previous knowledge of MCMC methods since our aim is to motivate survey sampling experts to contribute to the growing Subsampling MCMC literature.

What drives credit risk in the Indian banking industry? An empirical investigation

Using a two-step system GMM approach on a unique bank-level dataset for the period 1998/99–2013/14, this paper tries to explore the key determinants of credit risk in the Indian banking industry. The main premise of this paper is that, along with regulatory and institutional factors, both macroeconomic and bank-specific variables influence the formation of credit risk in a banking system, and their influences vary across ownership groups. The empirical findings suggest that lower profitability, more diversification in the banking business, the large size of banks and a higher concentration of banks in lending increase the probability of defaults in India. We find a significant degree of persistence in credit risk, and the observed persistence is higher in the gross non-performing loans (NPLs) specification relative to what has been observed in the net NPLs specification. In the case of public sector banks, NPLs are more sensitive to internal bank-specific factors, while for private and foreign banks, macroeconomic and industry-related factors play a significant role in determining credit risk. Our results are robust for different panel data estimation models and sub-samples of ownership groups. The findings of this paper provide important insights into the formation of default risk in the banking system of an emerging market economy.

Removal of batch effects using stratified subsampling of metabolomic data for in vitro endocrine disruptors screening

The human adrenal cell line H295R constitutes a well-established model to evaluate potential alterations of steroidogenic pathways as a result of chemical exposure. However, to date most assays are based on the targeted investigation of a limited number of steroid hormones, thus preventing in-depth mechanistic interpretation with respect to steroidogenesis. In that context, analytical strategies coupling liquid chromatography and high-resolution mass spectrometry (LC-HRMS) have been reported as promising methods for an extended monitoring of steroid metabolites. However, unwanted sources of variability occurring during the acquisition process, including batch effects, may prevent relevant biochemical information to be properly highlighted. Dedicated data mining strategies are therefore needed to overcome these limitations, and extract relevant extended steroidomic profiles. The present study combines an untargeted LC-HRMS acquisition strategy with automated steroid metabolite annotation based on accurate mass and isotopic patterns, and a chemometric tool allowing the different sources of variability to be decomposed based on experimental design. This workflow was applied to the extended monitoring of steroidogenic dysregulations due to endocrine disrupting chemicals (EDCs) exposure in H295R cell cultures. A series of six chemicals, including acetyl tributylcitrate, octyl methoxycinnamate, torcetrapib, forskolin, linuron, and octocrylene, and dimethylsulfoxide as solvent control, were investigated through the simultaneous monitoring of 130 potential steroid metabolites, repeating the whole experiment independently three times. A stratified subsampling strategy was carried out to remove efficiently systematic batch variations and highlight subgroups of chemicals with similar steroid patterns. The proposed approach was reported as a potent screening strategy, as it allowed specific alterations of the steroid hormone biosynthesis and metabolism related to distinct mechanisms of action to be distinguished.

Assay sensitivity in “Hybrid thorough QT/QTc (TQT)” study

ABSTRACT A concurrent positive control should be included in a thorough QTc clinical trial to validate the study according to ICH E14 guidance. Some pharmaceutical companies have started to use “hybrid TQT” study to meet ICH E14 regulatory requirements since the release of ICH E14 Q&A (R3). The “hybrid TQT” study includes the same treatment arms (therapeutic and/or supratherapeutic dose of investigational drug, placebo, and positive control) with sample size less than traditional TQT studies, but use concentration-QTc (C-QTc) analysis as primary analysis and assay sensitivity analysis. To better understand the statistical characteristics of assay sensitivity with a commonly used positive control – Moxifloxacin - in “hybrid TQT” studies, we examined the original and subsampled moxifloxacin and placebo data from more than a hundred of TQT studies submitted to FDA. The assay sensitivity results are quite consistent between classical E14 analysis and C-QTc analysis using the original datasets. Performance of assay sensitivity in “hybrid TQT” studies using subsampled data depends on number of moxifloxacin subjects, study design (crossover design and parallel design), and C-QTc model. The results presented here can aid the design of future “hybrid TQT” studies.

Do we need to count everything? Monitoring conspicuous epibenthic communities using seabed imagery

The purpose of this paper is to identify whether sub-sampling methods such as the Generalised Visual Fast Count can provide adequate inferences from video data. This is done by comparing the results from sub-sampling video data on species counts from the Fladen Grounds, U.K. with those results from the full data set. We show that the GVFC method can provide complete information on species richness and that estimates of key species’ densities are adequate compared to those from the full data. This is achieved from counting only 10% of the individuals present – thus saving time and money.

Spatial domain morphological filtering for interpolation of the Fourier domain

We establish here a method to partially recover the missing frequencies in data acquired through sub-sampling in the Fourier domain. Non-linear open and close operations are applied recursively to the raw spatial image estimated from its sparse Fourier samples. The mean of the open and closed images updates the estimated image after applying each structuring element (SE). The updated Fourier spectra are required at each stage to remain consistent with the known sample values. The SE size is incremented until the values estimated for the missing Fourier coefficients converge. The average image obtained from applying multiple sets of randomly selected SEs forms the final result. A denoising scheme is finally used to supplement the interpolated result. The denoising estimates the alias artefacts that arise from the initial sub-sampling by assuming that a narrow region around the borders of the reconstructed image has constant grey level. Convergence occurs in around 30 iterations that usually requires, in total a little less than one minute on a 64GB RAM, i7 machine. Since the computational complexity of the method is low, it can serve as a reliable initialization for more sophisticated iterative reconstruction schemes. Results for several simulated and real sub-sampled image data show an improvement of about 0.1 in Structural SIMilarity index (SSIM, whose maximum possible value is 1) in comparison to the raw inverse FFT image estimate. This method is more likely to underestimate the missing frequency values.

Efficient B-Mode Ultrasound Image Reconstruction From Sub-Sampled RF Data Using Deep Learning

In portable, 3-D, and ultra-fast ultrasound imaging systems, there is an increasing demand for the reconstruction of high-quality images from a limited number of radio-frequency (RF) measurements due to receiver (Rx) or transmit (Xmit) event sub-sampling. However, due to the presence of side lobe artifacts from RF sub-sampling, the standard beamformer often produces blurry images with less contrast, which are unsuitable for diagnostic purposes. Existing compressed sensing approaches often require either hardware changes or computationally expensive algorithms, but their quality improvements are limited. To address this problem, in this paper, we propose a novel deep learning approach that directly interpolates the missing RF data by utilizing redundancy in the Rx-Xmit plane. Our extensive experimental results using sub-sampled RF data from a multi-line acquisition B-mode system confirm that the proposed method can effectively reduce the data rate without sacrificing the image quality.

Generalized Mean Apparent Propagator MRI to Measure and Image Advective and Dispersive Flows in Medicine and Biology.

Water transport in biological systems spans different regimes with distinct physical behaviors: diffusion, advection, and dispersion. Identifying these regimes is of paramount importance in many in vivo applications, among them, measuring microcirculation of blood in capillary networks and cerebrospinal fluid transport in the glymphatic system. Diffusion magnetic resonance imaging (dMRI) can be used to encode water displacements, and a Fourier transform of the acquired signal furnishes a displacement probability density function known as the propagator. This transformation normally requires the use of a fast Fourier transform (FFT), which presents major feasibility challenges when scanning in vivo, mainly because of dense signal sampling, resulting in long acquisition times. A second approach to reconstruct the propagator is by using analytical representation of the signal, overcoming many of the FFT's limitations. In all analytical implementations of dMRI to date, the translational motion of water has been assumed to be exclusively diffusive, which is the case only in the absence of flow. However, retaining the phase information from the diffusion signal provides the ability to measure both mean coherent velocity and random diffusion from a single experiment. We implement and extend an analytical framework, mean apparent propagator (MAP), which can account for non-zero flow conditions. We call this method generalized MAP or GMAP. We describe a numerical optimization scheme and implement it on data from an MRI flow phantom constructed from a pack of 10- [Formula: see text] beads. The advantages of GMAP over the FFT-based method in the context of sampling density and low-flow detection were demonstrated, and analytically derived propagator moments were shown to agree with theoretical values even after data subsampling. GMAP would enable the detection of microflow in vivo that could help elucidate many important biological processes.

Sampling sufficiency for determining hydraulic habitat diversity

The geometry and hydrodynamics of river reaches are key ecohydraulic descriptors. Statistics of water depth and velocity measurements are usually taken as proxies for habitat suitability in rivers. However, little is known about the sufficiency of data to produce effective and representative results. In this research, 19 reaches with differences in terms of discharge, river width, substrate, reach length, cross-section spacing and geomorphology are investigated. Measurements of flow depth and velocity were taken at multiple, equally spaced cross-sections along each reach. Data were sub-sampled using different methodologies and analysed each time. The sets of sub-sampled data were then compared with those calculated with the full data set from a reach. The focus was put towards the hydro-morphological index of diversity (HMID), a combination of the classical ecohydraulic variables flow depth and velocity. It represents the spatial variability of hydraulic habitats in a reach. The results point out that, with a well-defined sampling strategy, 100 measurement points lead to a good estimation of the HMID value in a reach, if more than eight measurement points are taken per cross-section. For geomorphologies with small complexity or when the analysis only includes the estimation of mean flow depth or mean flow velocity, this number can be decreased according to the results presented here. These findings help both, aquatic ecologists and engineers to estimate their data reliability for hydraulic field measurements in a river reach and are herein discussed taking into account the different studied morphologies.

Optimized Interpolation and Cached Data Access in LUT-Based RGB-to-RGBW Conversion

In recent display panels, the RGBW color domain becomes popular because it improves the brightness of the display panel compared to the conventional RGB domain. An RGBW display system, in general, uses a look-up table (LUT) for fast RGB-to-RGBW conversion. However, a large-sized LUT is implemented in external memory, which slows down system speed and increases power consumption. For robust LUT-based RGB-to-RGBW conversion, this brief proposes two schemes. First, the data mapping of the LUT is sub-sampled. For fast conversion, the sub-sampled data are approximately interpolated. Moreover, the interpolation performance is improved by using a refinement scheme. Second, a small internal buffer that caches the information from the LUT is used for the conversion. When the information stored in the buffer is used for conversion, the access to the LUT of the external memory is skipped, thereby reducing the execution time during RGB-to-RGBW conversion. Experimental results show that the average of mean squared errors by the proposed sub-sampling and interpolation schemes varies from 0.00 to 21.72 when the quantization step is changed from 1 to 5, respectively. Compared to the trilinear interpolation, the proposed interpolation scheme reduces the number of accesses to the LUT from 8 to 2 and that of linear interpolation executions from 7 to 1. The proposed caching scheme with a 2.2 KB buffer reduces the execution time by 81.93%.

Technical Note: Interleaved bipolar acquisition and low-rank reconstruction for water-fat separation in MRI.

To acquire interleaved bipolar data and reconstruct the full data using low-rank property for water-fat separation. Bipolar acquisition suffers from issues related to gradient switching, the opposite gradient polarities, and other system imperfections, which prevent accurate water-fat separation. In this study, an interleaved bipolar acquisition scheme and a low-rank reconstruction method were proposed to reduce issues from the bipolar gradients while achieving a short imaging time. The proposed interleaved bipolar acquisition scheme collects echo-time signals from both gradient polarities; however, the sequence increases the imaging time. To reduce the imaging time, the signals were subsampled at every dimension of k-space. The low-rank property of the bipolar acquisition was defined and exploited to estimate the full data from the acquired subsampled data. To eliminate the bipolar issues, in the proposed method, the water-fat separation was performed separately for each gradient polarity, and the results for the positive and negative gradient polarities were combined after the water-fat separation. A phantom study and invivo experiments were conducted on a 3T Siemens Verio system. The results for the proposed method were compared with the results of the fully sampled interleaved bipolar acquisition and the Soliman's method, which was the previous water-fat separation approach for reducing the issues of bipolar gradients and accelerating the interleaved bipolar acquisition. The proposed method provided accurate water and fat images without the issues of bipolar gradients and demonstrated a better performance compared with the results of the previous methods. The water-fat separation using the bipolar acquisition has several benefits including a short echo-spacing time. However, it suffers from bipolar-gradient issues such as strong gradient switching, system imperfection, and eddy current effects. This study demonstrated that accurate water-fat separated images can be obtained using the proposed interleaved bipolar acquisition and low-rank reconstruction by using the benefits of the bipolar acquisition while reducing the bipolar-gradient issues with a short imaging time.

Copula based generalized additive models for location, scale and shape with non-random sample selection

Non-random sample selection is a commonplace amongst many empirical studies and it appears when an output variable of interest is available only for a restricted non-random sub-sample of data. An extension of the generalized additive models for location, scale and shape which accounts for non-random sample selection by introducing a selection equation is discussed. The proposed approach allows for potentially any parametric distribution for the outcome variable, any parametric link function for the selection equation, several dependence structures between the (outcome and selection) equations through the use of copulae, and various types of covariate effects. Using a special case of the proposed model, it is shown how the score equations are corrected for the bias deriving from non-random sample selection. Parameter estimation is carried out within a penalized likelihood based framework. The empirical effectiveness of the approach is demonstrated through a simulation study and a case study. The models can be easily employed via the gjrm() function in the R package GJRM.

Invited review: Reproducible research from noisy data: Revisiting key statistical principles for the animal sciences

Reproducible results define the very core of scientific integrity in modern research. Yet, legitimate concerns have been raised about the reproducibility of research findings, with important implications for the advancement of science and for public support. With statistical practice increasingly becoming an essential component of research efforts across the sciences, this review article highlights the compelling role of statistics in ensuring that research findings in the animal sciences are reproducible-in other words, able to withstand close interrogation and independent validation. Statistics set a formal framework and a practical toolbox that, when properly implemented, can recover signal from noisy data. Yet, misconceptions and misuse of statistics are recognized as top contributing factors to the reproducibility crisis. In this article, we revisit foundational statistical concepts relevant to reproducible research in the context of the animal sciences, raise awareness on common statistical misuse undermining it, and outline recommendations for statistical practice. Specifically, we emphasize a keen understanding of the data generation process throughout the research endeavor, from thoughtful experimental design and randomization, through rigorous data analysis and inference, to careful wording in communicating research results to peer scientists and society in general. We provide a detailed discussion of core concepts in experimental design, including data architecture, experimental replication, and subsampling, and elaborate on practical implications for proper elicitation of the scope of reach of research findings. For data analysis, we emphasize proper implementation of mixed models, in terms of both distributional assumptions and specification of fixed and random effects to explicitly recognize multilevel data architecture. This is critical to ensure that experimental error for treatments of interest is properly recognized and inference is correctly calibrated. Inferential misinterpretations associated with use of P-values, both significant and not, are clarified, and problems associated with error inflation due to multiple comparisons and selective reporting are illustrated. Overall, we advocate for a responsible practice of statistics in the animal sciences, with an emphasis on continuing quantitative education and interdisciplinary collaboration between animal scientists and statisticians to maximize reproducibility of research findings.

Efficient Bayesian Model Selection in PARAFAC via Stochastic Thermodynamic Integration

Parallel factor analysis (PARAFAC) is one of the most popular tensor factorization models. Even though it has proven successful in diverse application fields, the performance of PARAFAC usually hinges up on the rank of the factorization, which is typically specified manually by the practitioner. In this study, we develop a novel parallel and distributed Bayesian model selection technique for rank estimation in large-scale PARAFAC models. The proposed approach integrates ideas from the emerging field of stochastic gradient Markov Chain Monte Carlo, statistical physics, and distributed stochastic optimization. As opposed to the existing methods, which are based on some heuristics, our method has a clear mathematical interpretation, and has significantly lower computational requirements, thanks to data subsampling and parallelization. We provide formal theoretical analysis on the bias induced by the proposed approach. Our experiments on synthetic and large-scale real datasets show that our method is able to find the optimal model order while being significantly faster than the state-of-the-art.

Household purchasing of cheap alcohol: Who would be most affected by minimum unit pricing?

Minimum unit pricing of alcohol has been proposed as a more effective policy for reducing alcohol-related harms than general taxation. Growing international evidence points to its effectiveness in targeting high risk drinkers, but concerns about the impact on low risk drinkers and those on low incomes remain. This paper uniquely models the potential effect on both low risk and low income purchasers in a Scottish subsample of UK panel data from introducing a minimum unit price and addresses the impact of missing income data on the results. Weekly household purchases of cheap off-sales alcohol from Kantar Worldpanel (2008-2010) are analysed using Hausman-Taylor, Tobit and OLS models. Explanatory variables include economic and demographic characteristics of the households, average alcohol purchasing level for all off-sales alcohol, and characteristics of alcohol products purchased. The number of cheap alcohol units purchased are predicted by income group and purchasing level. Results indicate that the amount of cheap alcohol purchased is positively associated with the average purchasing level for all off-sales alcohol. Lower occupational group is associated with purchasing more units of cheap alcohol. Type of alcohol and purchasing alcohol on promotion are also highly significant. Predicted quantities of cheap alcohol purchased are not higher for the low-income group. Households in the top decile of alcohol purchasing level are predicted to purchase between 18 units of cheap alcohol weekly (low income) and 24 units (intermediate income). There was no evidence of bias due to missing income data. Minimum unit pricing for alcohol will have most impact on households purchasing the most alcohol, at all income levels. Restrictions on promotions should be considered as these are also associated with increased purchasing of cheap alcohol. Published: Online April 2018.

Sparse Power-Law Network Model for Reliable Statistical Predictions Based on Sampled Data.

A projective network model is a model that enables predictions to be made based on a subsample of the network data, with the predictions remaining unchanged if a larger sample is taken into consideration. An exchangeable model is a model that does not depend on the order in which nodes are sampled. Despite a large variety of non-equilibrium (growing) and equilibrium (static) sparse complex network models that are widely used in network science, how to reconcile sparseness (constant average degree) with the desired statistical properties of projectivity and exchangeability is currently an outstanding scientific problem. Here we propose a network process with hidden variables which is projective and can generate sparse power-law networks. Despite the model not being exchangeable, it can be closely related to exchangeable uncorrelated networks as indicated by its information theory characterization and its network entropy. The use of the proposed network process as a null model is here tested on real data, indicating that the model offers a promising avenue for statistical network modelling.