Data Setting Research Articles

Inference on heterogeneous treatment effects in high‐dimensional dynamic panels under weak dependence

This paper provides estimation and inference methods for conditional average treatment effects (CATE) characterized by a high‐dimensional parameter in both homogeneous cross‐sectional and unit‐heterogeneous dynamic panel data settings. In our leading example, we model CATE by interacting the base treatment variable with explanatory variables. The first step of our procedure is orthogonalization, where we partial out the controls and unit effects from the outcome and the base treatment and take the cross‐fitted residuals. This step uses a novel generic cross‐fitting method that we design for weakly dependent time series and panel data. This method “leaves out the neighbors” when fitting nuisance components, and we theoretically power it by using Strassen's coupling. As a result, we can rely on any modern machine learning method in the first step, provided it learns the residuals well enough. Second, we construct an orthogonal (or residual) learner of CATE—the lasso CATE—that regresses the outcome residual on the vector of interactions of the residualized treatment with explanatory variables. If the complexity of CATE function is simpler than that of the first‐stage regression, the orthogonal learner converges faster than the single‐stage regression‐based learner. Third, we perform simultaneous inference on parameters of the CATE function using debiasing. We also can use ordinary least squares in the last two steps when CATE is low‐dimensional. In heterogeneous panel data settings, we model the unobserved unit heterogeneity as a weakly sparse deviation from Mundlak's (1978) model of correlated unit effects as a linear function of time‐invariant covariates and make use of L1‐penalization to estimate these models. We demonstrate our methods by estimating price elasticities of groceries based on scanner data. We note that our results are new even for the cross‐sectional (i.i.d.) case.

A Monte Carlo study on the size and power of panel unit root tests

The aim of this paper is to explore the properties of various panel unit root tests in terms of their power and size regarding different panel data structures, with a special focus on small data samples. In particular, different values of the cross-sectional and time dimensions for both heterogeneous and homogeneous data settings with intercept and with and without time trend are observed. We examine and compare the results of five commonly used first-generation panel unit root tests: Levin, Lin and Chu test, Im Pesaran and Shin test, Harris–Tzavalis test, Breitung test and a Fischer type test. The results are derived using Monte Carlo simulations and show that all the observed panel unit root tests suffer from a serious lack of power and tend to either over or under reject the null hypothesis when the time dimensions are small. It is evident that the results of conducting panel unit root tests for data with T<30 are erroneous and unreliable, and it is therefore concluded that panel unit root tests should not be conducted for such samples in the first place. This is even more pronounced when there is a time trend or heterogeneity.

Let's enhance: A deep learning approach to extreme deblurring of text images

This work presents a novel deep-learning-based pipeline for the inverse problem of image deblurring, leveraging augmentation and pre-training with synthetic data. Our results build on our winning submission to the recent Helsinki Deblur Challenge 2021, whose goal was to explore the limits of state-of-the-art deblurring algorithms in a real-world data setting. The task of the challenge was to deblur out-of-focus images of random text, thereby in a downstream task, maximizing an optical-character-recognition-based score function. A key step of our solution is the data-driven estimation of the physical forward model describing the blur process. This enables a stream of synthetic data, generating pairs of ground-truth and blurry images on-the-fly, which is used for an extensive augmentation of the small amount of challenge data provided. The actual deblurring pipeline consists of an approximate inversion of the radial lens distortion (determined by the estimated forward model) and a U-Net architecture, which is trained end-to-end. Our algorithm was the only one passing the hardest challenge level, achieving over $ 70\% $ character recognition accuracy. Our findings are well in line with the paradigm of data-centric machine learning, and we demonstrate its effectiveness in the context of inverse problems. Apart from a detailed presentation of our methodology, we also analyze the importance of several design choices in a series of ablation studies. The code of our challenge submission is available under https://github.com/theophil-trippe/HDC_TUBerlin_version_1.

Divide-and-conquer Bayesian inference in hidden Markov models

Divide-and-conquer Bayesian methods consist of three steps: dividing the data into smaller computationally manageable subsets, running a sampling algorithm in parallel on all the subsets, and combining parameter draws from all the subsets. These methods use the combined parameter draws for efficient posterior inference in massive data settings. Existing divide-and-conquer methods have a major limitation in that their first two steps assume that the observations are independent. We address this problem by developing a divide-and-conquer method for Bayesian inference in parametric hidden Markov models, where the state space is known and finite. Our main contributions are two-fold. First, we partition the data into smaller blocks of consecutive observations and modify the likelihood on every time block. For any time block, the posterior distribution computed using the modified likelihood is such that its variance has the same asymptotic order as that of the true posterior. Second, suppose the number of subsets is chosen appropriately depending on the mixing properties of the hidden Markov chain. In that case, we show that the subset posterior distributions defined using the modified likelihood are asymptotically normal as the subset sample size tends to infinity. This result facilitates using any existing combination algorithm in the third step. We show that the combined posterior distribution obtained using one such algorithm is close to the true posterior distribution in 1-Wasserstein distance under widely used regularity assumptions. Our numerical results show that the proposed method provides an accurate approximation of the true posterior distribution than its competitors in simulation studies and a real data analysis.

Long Short-Term Memory-Based Deep Learning Models for Screening Parkinson's Disease Using Sequential Diagnostic Codes.

It is challenging to detect Parkinson's disease (PD) in its early stages, which has prompted researchers to develop techniques based on machine learning methods for detecting PD. However, previous studies did not fully incorporate the slow progression of PD over a long period of time nor consider that its symptoms occur in a time-sequential manner. Contributing to the literature on PD, which has relied heavily on cross-sectional data, this study aimed to develop a method for detecting PD early that can process time-series information using the long short-term memory (LSTM) algorithm. We sampled 926 patients with PD and 9,260 subjects without PD using medical-claims data. The LSTM algorithm was tested using diagnostic histories, which contained the diagnostic codes and their respective time information. We compared the prediction power of the 12-month diagnostic codes under two different settings over the 4 years prior to the first PD diagnosis. The model that was trained using the most-recent 12-month diagnostic codes had the best performance, with an accuracy of 94.25%, a sensitivity of 82.91%, and a specificity of 95.26%. The other three models (12-month codes from 2, 3, and 4 years prior) were found to have comparable performances, with accuracies of 92.27%, 91.86%, and 91.81%, respectively. The areas under the curve from our data settings ranged from 0.839 to 0.923. We explored the possibility that PD specialists could benefit from our proposed machine learning method as an early detection method for PD.

Accounting Information and Value Relevance Nexus: Evidence from Pakistan

This study aims at studying the value relevance of accounting information produced by large companies in Pakistan as there is a dearth of research in linking the returns on securities with accounting information produced by the companies in the Pakistan economy. For this purpose, the data was collected from 170 companies in the non-financial sector for 11 years (2006-2016). Panel data settings were used to regress the accounting information i.e., Earnings with Returns which are calculated using the Returns model. Empirical results prove that Earning figures are consistently value relevant throughout the period but the increase in value relevance of earnings is not significant. It is also shown that investors in Pakistan are giving more importance to Changes in Earnings over a sample period than to Earnings. This study is likely to highlight the importance of accounting information to investors in making their investment decisions.

Ridge Regression for Functional Form Identification of Continuous Predictors of Clinical Outcomes in Glomerular Disease

Introduction: Penalized regression models can be used to identify and rank risk factors for poor quality of life or other outcomes. They often assume linear covariate associations, but the true associations may be nonlinear. There is no standard, automated method for determining optimal functional forms (shapes of relationships) between predictors and the outcome in high-dimensional data settings. Methods: We propose a novel algorithm, ridge regression for functional form identification of continuous predictors (RIPR) that models each continuous covariate with linear, quadratic, quartile, and cubic spline basis components in a ridge regression model to capture potential nonlinear relationships between continuous predictors and outcomes. We used a simulation study to test the performance of RIPR compared to standard and spline ridge regression models. Then, we applied RIPR to identify top predictors of Patient-Reported Outcomes Measurement Information System (PROMIS) adult global mental and physical health scores using demographic and clinical characteristics among N = 107 glomerular disease patients enrolled in the Nephrotic Syndrome Study Network (NEPTUNE). Results: RIPR resulted in better predictive accuracy than the standard and spline ridge regression methods in 56–80% of simulation repetitions under a variety of data characteristics. When applied to PROMIS scores in NEPTUNE, RIPR resulted in the lowest error for predicting physical scores, and the second-lowest error for mental scores. Further, RIPR identified hemoglobin quartiles as an important predictor of physical health that was missed by the other models. Conclusion: The RIPR algorithm can capture nonlinear functional forms of predictors that are missed by standard ridge regression models. The top predictors of PROMIS scores vary greatly across methods. RIPR should be considered alongside other machine learning models in the prediction of patient-reported outcomes and other continuous outcomes.

Invariance encoding in sliced-Wasserstein space for image classification with limited training data

Deep convolutional neural networks (CNNs) are broadly considered to be state-of-the-art generic end-to-end image classification systems. However, they are known to underperform when training data are limited and thus require data augmentation strategies that render the method computationally expensive and not always effective. Rather than using a data augmentation strategy to encode invariances as typically done in machine learning, here we propose to mathematically augment a nearest subspace classification model in sliced-Wasserstein space by exploiting certain mathematical properties of the Radon Cumulative Distribution Transform (R-CDT), a recently introduced image transform. We demonstrate that for a particular type of learning problem, our mathematical solution has advantages over data augmentation with deep CNNs in terms of classification accuracy and computational complexity, and is particularly effective under a limited training data setting. The method is simple, effective, computationally efficient, non-iterative, and requires no parameters to be tuned. Python code implementing our method is available at https://github.com/rohdelab/mathematical_augmentation. Our method is integrated as a part of the software package PyTransKit, which is available at https://github.com/rohdelab/PyTransKit.

Research on Vehicle Active Steering Stability Control Based on Variable Time Domain Input and State Information Prediction

The controller design of vehicle systems depends on accurate reference index input. Considering information fusion and feature extraction based on existing data settings in the time domain, if reasonable input is selected for prediction to obtain accurate information of future state, it is of great significance for control decision-making, system response, and driver’s active intervention. In this paper, the nonlinear dynamic model of the four-wheel steering vehicle system was built, and the Long Short-Term Memory (LSTM) network architecture was established. On this basis, according to the real-time data under different working conditions, the information correction calculation of variable time-domain length was carried out to obtain the real-time state input length. At the same time, the historical state data of coupled road information was adopted to train the LSTM network offline, and the acquired real-time data state satisfying the accuracy was used as the LSTM network input to carry out online prediction of future confidence information. In order to solve the problem of mixed sensitivity of the system, a robust controller for vehicle active steering was designed with the sideslip angle of the centroid of 0, and the predicted results were used as reference inputs for corresponding numerical calculation verification. Finally, according to the calculated results, the robust controller with information prediction can realize the system stability control under coupling conditions on the premise of knowing the vehicle state information in advance, which provides an effective reference for controller response and driver active manipulation.

Guided conditional functional dependency discovery

Conditional functional dependencies (CFDs) generalize functional dependencies and lend themselves to wide applicability. CFDs on data are usually unknown and too costly to be designed manually. To this end, CFD discovery methods are studied for discovering hidden CFDs from data. In the setting of data cleaning, only a small number of CFDs are used to detect and repair errors, while common CFD discovery methods find all CFDs (approximately) holding on data, and an expensive post-processing step is further required for selecting those relevant ones. In this paper, we present an approach to discover CFDs that can detect errors in data, guided by a small set of erroneous attribute values labeled by users. (1) We present a method that consists of several modules of data sampling, CFD discovery and refinement guided by the user labeling and data re-sampling guided by the discovered CFDs, working in an iterative way. (2) We present novel efficient techniques to facilitate our approach, aiming at identifying CFDs useful for cleaning and reducing user interactions. (3) We conduct extensive experimental evaluations to verify our approach, against the state-of-the-art CFD discovery algorithms with or without user interactions.

Robust Testing of Paired Outcomes Incorporating Covariate Effects in Clustered Data with Informative Cluster Size

Paired outcomes are common in correlated clustered data where the main aim is to compare the distributions of the outcomes in a pair. In such clustered paired data, informative cluster sizes can occur when the number of pairs in a cluster (i.e., a cluster size) is correlated to the paired outcomes or the paired differences. There have been some attempts to develop robust rank-based tests for comparing paired outcomes in such complex clustered data. Most of these existing rank tests developed for paired outcomes in clustered data compare the marginal distributions in a pair and ignore any covariate effect on the outcomes. However, when potentially important covariate data is available in observational studies, ignoring these covariate effects on the outcomes can result in a flawed inference. In this article, using rank based weighted estimating equations, we propose a robust procedure for covariate effect adjusted comparison of paired outcomes in a clustered data that can also address the issue of informative cluster size. Through simulated scenarios and real-life neuroimaging data, we demonstrate the importance of considering covariate effects during paired testing and robust performances of our proposed method in covariate adjusted paired comparisons in complex clustered data settings.

Intrusion Detection in Big Data Era: A Multi- Objective Approach for Longer Model Lifespans

Despite highly accurate intrusion detection schemes based on machine learning (ML) reported in the literature, changes in network traffic behaviour quickly yield low accuracy rates. An intrusion detection model update is not easily feasible due to the enormous amount of network traffic to be processed in near real-time for high- speed networks, in particular, under big data settings. In this paper, we propose a new scalable long-lasting intrusion detection architecture for the processing of network content and the building of a reliable ML-based intrusion detection model. Experiments performed through the analysis of five years of network traffic, about 20 TB of data, have shown that our approach extends the lifespan of our model by up to six weeks. That occurs because the average accuracy rate of our proposal lasted eight weeks after the training phase, and traditional ones reached only two weeks after the model building. Additionally, our proposal achieves up to 10 Gbps of detection throughput in a 20-core big data processing cluster. KEYWORDS: Machine Learning, Intrusion Detection, Big Data, High-Speed Networks.

A semi-supervised Teacher-Student framework for surgical tool detection and localization

ABSTRACT Surgical tool detection in minimally invasive surgery is an essential part of computer-assisted interventions. Current approaches are mostly based on supervised methods requiring large annotated datasets. However, labelled datasets are often scarce. Semi-supervised learning (SSL) has recently emerged as a viable alternative showing promise in producing models retaining competitive performance to supervised methods. Therefore, this paper introduces an SSL framework in the surgical tool detection paradigm, which aims to mitigate training data scarcity and data imbalance problems through a knowledge distillation approach. In the proposed work, we train a model with labelled data which initialises the Teacher-Student joint learning, where the Student is trained on Teacher-generated pseudo-labels from unlabelled data. We also propose a multi-class distance with a margin-based classification loss function in the region-of-interest head of the detector to segregate the foreground-background region effectively. Our results on m2cai16-tool-locations dataset indicates the superiority of our approach on different supervised data settings (1%, 2%, 5% and 10% of annotated data) where our model achieves overall improvements of 8%, 12%, and 27% in mean average precision on 1% labelled data over the state-of-the-art SSL methods and the supervised baseline, respectively. The code is available at https://github.com/Mansoor-at/Semi-supervised-surgical-tool-detection.

Trends in body mass index, overweight and obesity among adults in the USA, the NHANES from 2003 to 2018: a repeat cross-sectional survey

ObjectivesTo analyse detailed trends in adult obesity from 2003 to 2018 in the USA, and provide the latest national estimates of adult obesity in 2017–2018.Design, setting and participantsAnalysis of data,...

Chest Disease Detection and Classification

Chest diseases & conditions such as Atelectasis, Cardiomegaly, Lung consolidation, Hernia, and Fibrosis becoming increasingly prevalent in the Asia-Pacific region. The Asia-Pacific Burden of Respiratory Diseases study examined the disease and economic burden of lung diseases across the Asia-Pacific and more specifically India. The objective is to use a deep learning model to diagnose pathologies from Chest X-Rays. ML approaches on CT and Xray images aided incorrectly in identifying lung diseases. Respiratory diseases range from mild and self-limiting, such as the common cold, influenza, and pharyngitis to life-threatening diseases such as bacterial pneumonia, pleural thickening, hernia, and severe acute respiratory syndromes, such as COVID-19. Authorities & Doctors will be able to deal with the effects more efficiently if such illnesses can be detected speedily and accurately with little human intervention in the future. In addition, various additional elements, such as environmental influences and commonalities among the most afflicted places, should be considered to slow the spread of lung diseases and precautions should be taken. Chest X-ray exam is one of the most frequent and cost-effective medical imaging examination. However clinical diagnosis of chest X-ray can be challenging, and sometimes believed to be harder than diagnosis via chest CT imaging. Even some promising work have been reported in the past, and especially in recent deep learning work on Tuberculosis (TB) classification. To achieve clinically relevant computer-aided detection and diagnosis (CAD) in real world medical sites on all data settings of chest X-rays is still very difficult, if not impossible when only several thousands of images are employed for study. This is evident from [2] where the performance deep neural networks for thorax disease recognition is severely limited by the availability of only 4143 frontal view images [3].

The population-attributable fraction for time-to-event data.

Even though the population-attributable fraction (PAF) is a well-established metric, it is often incorrectly estimated or interpreted not only in clinical application, but also in statistical research articles. The risk of bias is especially high in more complex time-to-event data settings. We explain how the PAF can be defined, identified and estimated in time-to-event settings with competing risks and time-dependent exposures. By using multi-state methodology and inverse probability weighting, we demonstrate how to reduce or completely avoid severe types of biases including competing risks bias, immortal time bias and confounding due to both baseline and time-varying patient characteristics. The method is exemplarily applied to a real data set. Moreover, we estimate the number of deaths that were attributable to ventilator-associated pneumonia in France in the year 2016. The example demonstrates how, under certain simplifying assumptions, PAF estimates can be extrapolated to a target population of interest. Defining and estimating the PAF in advanced time-to-event settings within a framework that unifies causal and multi-state modelling enables to tackle common sources of bias and allows straightforward implementation with standard software packages.

Racial Inequities in Adolescent Contraceptive Care Delivery: A Reproductive Justice Issue

The objective of this study was to examine racial/ethnic disparities in contraceptive delivery for adolescent patients within an adolescent medicine subspecialty clinical system before and during the COVID-19 pandemic. Secondarily, we aimed to assess the relationship between race and contraceptive delivery by telehealth. Retrospective cohort study using electronic health record data SETTING: Three adolescent medicine subspecialty clinics in a large academic hospital system, including an urban location and 2 suburban locations PARTICIPANTS: Patients assigned female sex at birth prescribed hormonal contraception between January 1st, 2018, and May 31st, 2021. Method and type of contraceptive prescribed (short-acting, medium-acting, long-acting reversible contraception [LARC]) RESULTS: There were 2453 patients in the study; 47.5% were White, 36.0% were Black, and 8.1% identified as Hispanic. After controlling for insurance and age, Black patients, compared with non-Black patients, had twofold higher odds of receiving LARC compared with a short-acting method across the study period (aOR=2.0; 95% CI, 1.52-2.62). We identified effect modification with significant interaction between Black race and the pandemic period, with evidence of a higher marginal probability of Black patients receiving LARCs during the pandemic. Additionally, during the pandemic, patients receiving new contraceptive prescriptions via telehealth were less likely to be Black (aOR=0.63; 95% CI, 0.41-0.94) or publicly insured (aOR=0.56; 95% CI, 0.38-0.81). Our data show significantly higher prescribing of LARCs to Black adolescents by clinicians, which could suggest differences in physician contraceptive counseling with a bias toward preferentially counseling Black patients toward LARCs. Our data also show that Black and publicly insured patients had decreased utilization of contraceptive care by telehealth during the pandemic.

Gradient Tree Boosting for Hierarchical Data

Gradient tree boosting is a powerful machine learning technique that has shown good performance in predicting a variety of outcomes. However, when applied to hierarchical (e.g., longitudinal or clustered) data, the predictive performance of gradient tree boosting may be harmed by ignoring the hierarchical structure, and may be improved by accounting for it. Tree-based methods such as regression trees and random forests have already been extended to hierarchical data settings by combining them with the linear mixed effects model (MEM). In the present article, we add to this literature by proposing two algorithms to estimate a combination of the MEM and gradient tree boosting. We report on two simulation studies that (i) investigate the predictive performance of the two MEM boosting algorithms and (ii) compare them to standard gradient tree boosting, standard random forest, and other existing methods for hierarchical data (MEM, MEM random forests, model-based boosting, Bayesian additive regression trees [BART]). We found substantial improvements in the predictive performance of our MEM boosting algorithms over standard boosting when the random effects were non-negligible. MEM boosting as well as BART showed a predictive performance similar to the correctly specified MEM (i.e., the benchmark model), and overall outperformed the model-based boosting and random forest approaches.

Asynchronous and Distributed Data Augmentation for Massive Data Settings

Data augmentation (DA) algorithms are slow in massive data settings due to multiple passes through the entire data. We address this problem by developing a DA extension that exploits asynchronous and distributed computing. The extended DA algorithm is called Asynchronous and Distributed (AD) DA with the original DA as its parent. Any ADDA is indexed by a parameter r ∈ ( 0 , 1 ) and starts by dividing the entire data into k disjoint subsets and storing them on k processes. Every iteration of ADDA augments only an r-fraction of the k data subsets with some positive probability and leaves the remaining ( 1 − r ) -fraction of the augmented data unchanged. The parameter draws are obtained using the r-fraction of new and ( 1 − r ) -fraction of old augmented data. We show that the ADDA Markov chain is Harris ergodic with the desired stationary distribution under mild conditions on the parent DA algorithm. We demonstrate that ADDA is significantly faster than its parent for many (k, r) choices in three representative models. We also establish the geometric ergodicity of the ADDA Markov chain for all the three models, which yields asymptotically valid standard errors for estimates of desired posterior quantities. Supplementary materials for this article are available online.

Distributed inference for two‐sample U‐statistics in massive data analysis

AbstractThis paper considers distributed inference for two‐sample U‐statistics under the massive data setting. In order to reduce the computational complexity, this paper proposes distributed two‐sample U‐statistics and blockwise linear two‐sample U‐statistics. The blockwise linear two‐sample U‐statistic, which requires less communication cost, is more computationally efficient especially when the data are stored in different locations. The asymptotic properties of both types of distributed two‐sample U‐statistics are established. In addition, this paper proposes bootstrap algorithms to approximate the distributions of distributed two‐sample U‐statistics and blockwise linear two‐sample U‐statistics for both nondegenerate and degenerate cases. The distributed weighted bootstrap for the distributed two‐sample U‐statistic is new in the literature. The proposed bootstrap procedures are computationally efficient and are suitable for distributed computing platforms with theoretical guarantees. Extensive numerical studies illustrate that the proposed distributed approaches are feasible and effective.