Longitudinal Clinical Trial Data Research Articles

A clinical site workload prediction model with machine learning lifecycle

In clinical trial monitoring, substantial resources are allocated to perform source data verification (SDV). SDV ensures accurate and reliable transcription of trial participant information. Clinical site visits are typically scheduled at a fixed frequency for SDV, without objectively factoring in individual site workload. This often results in wasted resources and directly influences clinical trial cost. We leveraged historical data from several hundred clinical trials to predict SDV workload at trial sites using machine learning. Specifically, we adopted cross industry standard process for data mining (CRISP-DM) process model and devised a novel deep learning pipeline for longitudinal clinical trial data. The deep learning pipeline, which comprises recurrent neural network-based encoder and decoder, ingests multivariate sequence data from study sites and predicts SDV workload for future months. We also developed an efficient model enhancement workflow, in a data science platform that facilitates machine learning operations best practices, for timely adaptation of new features and changes. Several enhancement iterations have been performed since the launch of first SDV workload prediction model, resulting in a more accurate latest model compared to previous versions. We discuss these enhancements in the context of CRISP-DM phases. In conclusion, the SDV workload prediction model has enabled informed planning and optimization of resources within clinical trial operations.

The Prader-Willi Syndrome Anxiousness and Distress Behaviors Questionnaire: Development and Psychometric Validation

ObjectivesTo facilitate the development of new therapies for Prader-Willi syndrome (PWS), we sought to develop a reliable and valid assessment of anxiousness and distress, common characteristics that have a significant negative impact on individuals with PWS and their families. MethodsThe PWS Anxiousness and Distress Behaviors Questionnaire (PADQ) was developed with extensive input from clinical experts, as well as caregivers of individuals with PWS, who participated in iterative sets of qualitative interviews. The psychometric properties of the PADQ were subsequently demonstrated in a cross-sectional evaluation using data from the Global PWS Registry provided by > 400 caregivers and confirmed using data from a phase 3 clinical trial of an oxytocin analogue (intranasal carbetocin, LV-101). ResultsQualitative interview participants consistently endorsed the content of the PADQ and were confident they could accurately respond to each item based on their observations of their child’s behavior. Analysis of cross-sectional data supported the computation of a total PADQ score, as well as the reliability and validity of the measure. The results of analyses using longitudinal clinical trial data confirmed these properties and provided evidence for the responsiveness of the PADQ, further supporting its appropriateness for the evaluation of new treatments targeting anxiousness and distress in PWS. ConclusionsThe current body of evidence supports the conclusion that the PADQ measures observable behaviors that are meaningful to patients and their families and provides a valid and reliable method to assess beneficial treatment effects for some of the most challenging behaviors associated with PWS.

Predictors of severe CRS in longitudinal CAR T-cell clinical trial data.

2540 Background: Cytokine release syndrome (CRS) is a life-threatening toxicity of chimeric antigen receptor (CAR) T-cell therapy and there is a limited understanding of its risk factors. Known markers of severe CRS lack specificity or require central lab facilities, making them unsuitable for safety surveillance during trials. Using data from anti-CD19 CAR-T trials from the Medidata Enterprise Data Store (MEDS) we analyzed laboratory tests with repeated measurements to identify differences in trends that persist across a variety of study designs, sponsors and indications. Our aim was to capture the dynamic response to lymphodepletion (LD) and CAR-T infusion events in the time course values of laboratory markers and compare these in the severe CRS (grade 3 and higher) and non severe CRS groups. Methods: Subjects in the anti-CD19 CAR-T trials obtained from MEDS had relapsed or refractory disease in one of the following: B Acute Lymphoblastic Leukemia (B-ALL), Mantle Cell Lymphoma (MCL), Non-Hodgkin Lymphoma (NHL) and Diffuse Large B-cell Lymphoma (DLBCL). Laboratory markers with repeated measurements across all studies were included in the analysis. Response to LD, CAR-T infusion and the onset of hematologic recovery was modeled with piecewise linear splines with knots at the boundaries of the pre LD (-18d to LD), the LD - infusion and post infusion (1d to 6d and 6d to 18d) intervals using linear mixed effects regression. Results: The dataset consisted of 542 patients: 24.1% B-ALL, 44.4% NHL, 16.8% MCL, 14.6% DLBCL; mean age 54.8 (SE = 0.021) years, 67.7% males. 27.3% had CRS 3+ with a median time-to-event of 4.5 days. Average rates of change (units per day) for the severe CRS subjects in the pre LD, LD-infusion and post infusion intervals relative to the rest are shown (table). None of the markers had a significant relative rate of change prior to LD. The average change in Albumin, Platelets and Hematocrit was consistently lower in severe CRS patients. Conclusions: Longitudinal laboratory marker data may be used to derive predictors of severe CRS in patients undergoing anti-CD19 CAR-T therapy. While some of the laboratory markers discussed above corroborate earlier findings, modeling the pre and post infusion kinetics can yield novel markers with improved precision in severe CRS prediction and better patient safety. [Table: see text]

Comparison of Methods Dealing with Missing Data in a Longitudinal Rheumatologic Study

Missing data are unavoidable in longitudinal studies and can lead toserious problems, such as loss of power and biased estimates, which should be solved in the statistical analysis of clinical studies. In this paper, three different techniques for handling missing data are shown using an example from a rheumatologic study. It is also shown how sensitive the conclusions of the study can be in terms of how the incomplete data are analyzed. The missing data process is studied in the framework of longitudinal data. The common approaches to handling missing longitudinal clinical trial data because of dropout are complete case (CC) and last observation carried forward (LOCF) analyses. These methods, while intuitively appealing, require tough assumptions to reach valid statistical conclusions. A relatively new and up to date statistical method for analyzing data with incomplete repeated measures is “likelihoodbased ignorable method” which has less constraints and fewer tough assumptions than those required for CC and LOCF. We apply these three methods to data set of a rheumatologic trial comparing disease groups in terms of the joint pain scores using a mixed model. No significant differences were found between the methods of analysis. It can be concluded that attention to the mechanisms of missing data should be very important part of the analysis of rheumatologic data.

Missing Data Imputation With Baseline Information in Longitudinal Clinical Trials

In longitudinal clinical trials, missing data are inevitable despite every effort made to retain patients in the trial. Missing data cause difficulty in the estimation and interpretation of the treatment effect. When the primary objective is to assess the treatment effect in a realistic setting, it is necessary to take into consideration the impact of noncompliance to the treatment regimen. For estimation following the intention-to-treat principle, a return-to-baseline (RTB) approach may be used for continuous endpoints in some longitudinal clinical trials. The RTB approach is based on the assumption that the unobserved outcomes at the end of the trial represent a return to the baseline value, that is, any change observed while on treatment can be expected to wash out after patients drop out. This article describes a statistical approach for RTB analyses. The method for calculating the sample size using RTB approach is presented. A detailed illustration using this RTB approach based on publicly available longitudinal antidepressant clinical trial data is provided. Extensive simulations are presented to evaluate the performance of this RTB approach under various missing data mechanisms. Important limitations regarding the appropriateness of the underlying assumptions of RTB are discussed.

An imputation approach for fitting two-part mixed effects models for longitudinal semi-continuous data.

Two-part mixed effects models are often used for analyzing longitudinal data with many zeros. Typically, these models are formulated with binary and continuous components separately with random effects that are correlated between the two components. Researchers have developed maximum-likelihood and Bayesian approaches for fitting these models that often require using particular software packages or very specialized software. We propose an imputation approach that will allow practitioners to separately use standard linear and generalized linear mixed models to estimate the fixed effects for two-part mixed effects models with complex random effects structures. An approximation to the conditional distribution of positive measurements given an individual's pattern of non-zero measurements is proposed that can be easily estimated and then imputed from. We show that for a wide range of parameter values, the imputation approach results in nearly unbiased estimation and can be implemented with standard software. We illustrate the proposed imputation approach for the analysis of longitudinal clinical trial data with many zeros.

Fixed Size Confidence Regions for the Parameters of the Mixed Effects Logistic Regression Model

We develop fixed size confidence regions for estimating the fixed and random effects parameters of the mixed effects logistic regression model. This model applies to, among others, the study of the effects of covariates on a dichotomous response variable when subjects are sampled in clusters. Two sequential procedures are developed to estimate with a prescribed accuracy (confidence level) and fixed precision the set of fixed and random effects parameters and linear transformations of these parameters, respectively. We show that the two procedures are asymptotically consistent (i.e., the coverage probability converges to the nominal confidence level) and asymptotically efficient (i.e., the ratio of the expected random sample size to the unknown best fixed sample size converges to 1) as the width of the confidence region converges to 0. Suggestions to improve the performance of the procedures are provided based on Monte Carlo simulation and illustrated through a longitudinal clinical trial data.

On model selections for repeated measurement data in clinical studies.

Repeated measurement designs have been widely used in various randomized controlled trials for evaluating long-term intervention efficacies. For some clinical trials, the primary research question is how to compare two treatments at a fixed time, using a t-test. Although simple, robust, and convenient, this type of analysis fails to utilize a large amount of collected information. Alternatively, the mixed-effects model is commonly used for repeated measurement data. It models all available data jointly and allows explicit assessment of the overall treatment effects across the entire time spectrum. In this paper, we propose an analytic strategy for longitudinal clinical trial data where the mixed-effects model is coupled with a model selection scheme. The proposed test statistics not only make full use of all available data but also utilize the information from the optimal model deemed for the data. The performance of the proposed method under various setups, including different data missing mechanisms, is evaluated via extensive Monte Carlo simulations. Our numerical results demonstrate that the proposed analytic procedure is more powerful than the t-test when the primary interest is to test for the treatment effect at the last time point. Simulations also reveal that the proposed method outperforms the usual mixed-effects model for testing the overall treatment effects across time. In addition, the proposed framework is more robust and flexible in dealing with missing data compared with several competing methods. The utility of the proposed method is demonstrated by analyzing a clinical trial on the cognitive effect of testosterone in geriatric men with low baseline testosterone levels.

Handling Dropouts in Longitudinal Clinical Trials Aata: Likelihood-based Analysis Versus Inverse Probability Weighting

Dropout is a pervasive problem in longitudinal clinical trials, and it is the result mainly of non-responses due to individuals who leave the study and are therefore lost to follow-up. The current paper deals with incomplete longitudinal clinical trials data when there are dropout. Statistical methods that ignore the mechanism for dropouts are susceptible to biased inference. This article focuses on dropouts missing at random (MAR). The study demonstrates application and the performance of likelihood-based and inverse probability weighting (IPW) in handling dropout in longitudinal continuous responses. The main objective of this paper is to compare the performance of these methods under different dropout rates. Data from a study with individual heart rate as the outcome is used to investigate the performance of the considered methods. Based on this longitudinal clinical trial data, results from IPW will be compared with those obtained from likelihood-based analysis. The performance of these methods are compared in terms of bias and efficiency.

A structured framework for assessing sensitivity to missing data assumptions in longitudinal clinical trials

The objective of this research was to demonstrate a framework for drawing inference from sensitivity analyses of incomplete longitudinal clinical trial data via a re-analysis of data from a confirmatory clinical trial in depression. A likelihood-based approach that assumed missing at random (MAR) was the primary analysis. Robustness to departure from MAR was assessed by comparing the primary result to those from a series of analyses that employed varying missing not at random (MNAR) assumptions (selection models, pattern mixture models and shared parameter models) and to MAR methods that used inclusive models. The key sensitivity analysis used multiple imputation assuming that after dropout the trajectory of drug-treated patients was that of placebo treated patients with a similar outcome history (placebo multiple imputation). This result was used as the worst reasonable case to define the lower limit of plausible values for the treatment contrast. The endpoint contrast from the primary analysis was - 2.79 (p = .013). In placebo multiple imputation, the result was - 2.17. Results from the other sensitivity analyses ranged from - 2.21 to - 3.87 and were symmetrically distributed around the primary result. Hence, no clear evidence of bias from missing not at random data was found. In the worst reasonable case scenario, the treatment effect was 80% of the magnitude of the primary result. Therefore, it was concluded that a treatment effect existed. The structured sensitivity framework of using a worst reasonable case result based on a controlled imputation approach with transparent and debatable assumptions supplemented a series of plausible alternative models under varying assumptions was useful in this specific situation and holds promise as a generally useful framework.

Threshold for success in RA drug development

Determining the potential success of investigational therapies for rheumatoid arthritis early in the development process would enable better allocation of increasingly limited resources. Such ‘go or no-go’ decision-making could be improved by a method of analysing longitudinal clinical trial data to establish a threshold for success of a new drug.

Analyzing longitudinal clinical trial data with nonignorable missingness and unknown missingness reasons

Longitudinal clinical trials are often plagued by nonmonotone missingness due to both patient dropout and intermittent missingness. Standard analysis assumes that missingness is ignorable. Because the assumption can be questionable, the sensitivity of inferences to alternative assumptions about missingness needs to be evaluated. This need arises in the analysis of a longitudinal prostate cancer quality-of-life (QoL) clinical trial dataset, in which nonmonotone missingness occurs. The choice of the missing data model is studied in the analysis. A local sensitivity analysis method is then applied to analyze the dataset and to investigate the changes in parameter estimates in the neighborhood of the ignorable model. One advantage of the method is that it surmounts computational difficulty and completely avoids evaluating the high-dimensional integrals in the likelihood due to nonmonotone missingness. Another is that it can be implemented using the standard software without excessive additional computation. The method is especially advantageous for large clinical datasets for which alternative approaches can become computationally prohibitive. In addition, the analysis demonstrates the importance of exploiting information on reasons for missingness. When such information is unavailable for some missingness and therefore the missingness types (i.e., dropout versus intermittent missingness) are unknown, a bound analysis is proposed, combined with genetic algorithms, to account for unknown missingness types. The analysis demonstrates the usefulness of the method as a general approach to evaluating the sensitivity of standard analysis to nonignorable nonmonotone missingness in clinical trials.

Handling missing data issues in clinical trials for rheumatic diseases

Missing data are ubiquitous in clinical trials for rheumatic diseases, and it is important to accommodate them using appropriate statistical techniques. We review some of the basic considerations for missing data and survey a range of statistical techniques for analysis of longitudinal clinical trial data with missingness. Using clinical trial data from patients with diffuse systemic sclerosis, we show that different approaches to handling missing data can lead to different conclusions on the efficacy of the treatment. We then suggest how such discrepancies might be addressed. In particular, we emphasize that the commonly used method in rheumatic clinical trials of carrying the last observation forward to impute missing values should not be the primary analysis. We review software for analyzing different types of missing data and discuss our freely available software library for analyzing the more difficult but more realistic situation when the probability of dropout or missing data may depend on the unobserved missing value.

Optimal Partitioning for Linear Mixed Effects Models: Applications to Identifying Placebo Responders

A longstanding problem in clinical research is distinguishing drug-treated subjects that respond due to specific effects of the drug from those that respond to nonspecific (or placebo) effects of the treatment. Linear mixed effect models are commonly used to model longitudinal clinical trial data. In this paper we present a solution to the problem of identifying placebo responders using an optimal partitioning methodology for linear mixed effects models. Since individual outcomes in a longitudinal study correspond to curves, the optimal partitioning methodology produces a set of prototypical outcome profiles. The optimal partitioning methodology can accommodate both continuous and discrete covariates. The proposed partitioning strategy is compared and contrasted with the growth mixture modeling approach. The methodology is applied to a two-phase depression clinical trial where subjects in a first phase were treated openly for 12 weeks with fluoxetine followed by a double blind discontinuation phase where responders to treatment in the first phase were randomized to either stay on fluoxetine or switched to a placebo. The optimal partitioning methodology is applied to the first phase to identify prototypical outcome profiles. Using time to relapse in the second phase of the study, a survival analysis is performed on the partitioned data. The optimal partitioning results identify prototypical profiles that distinguish whether subjects relapse depending on whether or not they stay on the drug or are randomized to a placebo.

Understanding the relationships between health outcomes in generalized anxiety disorder clinical trials

The Wilson-Cleary health outcomes model is a hypothesized pathway linking traditional clinical variables to health-related quality of life (HRQL). This study tested the application of the Wilson-Cleary model to a patient population with generalized anxiety disorder (GAD) using longitudinal clinical trial data. These secondary analyses pooled data from three similar 8-week, placebo-controlled, double-blind, randomized, multicenter trials of quetiapine XR therapy in GAD. Relevant health assessments for the model concepts included the Clinical Global Impression-Severity of Illness, Hamilton Rating Scale for Anxiety, the Pittsburgh Sleep Quality Index and Quality of Life Enjoyment and Satisfaction Questionnaire-Short Form. A lagged path model tested whether the hypothesized relationships at baseline and week 8 between the concepts of the adapted Wilson-Cleary model were consistent with the observed data. The resulting model fit the data (RMSEA = 0.077; CFI = 0.98; NFI = 0.96) and explained 56% of the variance in overall quality of life assessment at baseline and 69% of the variance in this assessment at week 8. Moderate to strong relationships between the adjacent hypothesized concepts support the specified model. This adapted Wilson-Cleary model for health outcomes validated in GAD should improve the understanding and usefulness of health status measurements in this condition and increase the applications of this model to other clinical trial data.

Conceptual Considerations regarding Endpoints, Hypotheses, and Analyses for Incomplete Longitudinal Clinical Trial Data

Missing data is an ever-present problem in longitudinal clinical trials. Considerable advances in statistical methodology and in our ability to implement those methods have been made in recent years, leading to more choices in analytic approaches for longitudinal data. However, more analytic choice has led to greater complexity in understanding the strengths and limitations of the various approaches, especially since analytic choices are often inextricably linked to choices of endpoints and hypotheses. In drug development, assessing efficacy and effectiveness are both essential. The key is to put the greatest emphasis on the one most appropriate for the given stage of development. Efficacy hypotheses are more closely aligned with the design and goals of confirmatory trials, whereas effectiveness is best tested in more naturalistic settings. Analyses that include dropout as part of the outcome or use data obtained after discontinuation of the study drug may play a role in analysis plans, but they do not provide meaningful solutions to the problem of missing data in the primary analysis of confirmatory trials. Consequently, broad consensus across industry and academia has emerged indicating that the primary analyses of data from confirmatory longitudinal clinical trials should move away from single imputation methods such as last and baseline observation carried forward to analyses such as likelihood-based mixed-effects analyses and multiple imputation. Sensitivity analyses should also be incorporated into analysis plans to specifically assess the robustness of results to key assumptions of the primary analysis.

LONGITUDINAL AND INCOMPLETE CLINICAL STUDIES

Repeated measures are obtained whenever an outcome is measured repeatedly within a set of units. The fact that observations from the same unit, in general, will not be independent poses particular challenges to the statistical procedures used for the analysis of such data. The current paper is dedicated to an overview of frequently used statistical models for the analysis of repeated measurements, with emphasis on model formulation and parameter interpretation. Missing data frequently occur in repeated measures studies, especially in humans. An important source for missing data is patients who leave the study prematurely, so-called dropouts. When patients are evaluated only once under treatment, then the presence of dropouts makes it hard to comply with the intention-to-treat (ITT) principle. However, when repeated measurements are taken then one can make use of the observed portion of the data to retrieve information on dropouts. Generally, commonly used methods to analyze incomplete longitudinal clinical trial data include complete-case (CC) analysis and an analysis using the last observation carried forward (LOCF). However, these methods rest on strong and unverifiable assumptions about the dropout mechanism. Over the last decades, a number of longitudinal data analysis methods have been suggested, providing a valid estimate for, e.g., the treatment effect under less restrictive assumptions. We will argue that direct likelihood methods, using all available data, require the relatively weak missing at random assumption only. Likewise, weighted generalized estimating equations and multiple imputation are discussed. Finally, because it is impossible to verify that the dropout mechanism is MAR, we argue that, to evaluate the robustness of the conclusion, a sensitivity analysis thereby varying the assumption on the dropout mechanism should become a standard procedure when analyzing the results of a clinical trial.

Generalizability in NonGaussian Longitudinal Clinical Trial Data Based on Generalized Linear Mixed Models

This work investigates how generalizability, an extension of reliability, can be defined and estimated based on longitudinal data sequences resulting from, for example, clinical studies. Useful and intuitive approximate expressions are derived based on generalized linear mixed models. Data from four double-blind, randomized clinical trials into schizophrenia motivate the research and are used to estimate generalizability for a binary response parameter.

A Latent‐Class Mixture Model for Incomplete Longitudinal Gaussian Data

In the analyses of incomplete longitudinal clinical trial data, there has been a shift, away from simple methods that are valid only if the data are missing completely at random, to more principled ignorable analyses, which are valid under the less restrictive missing at random assumption. The availability of the necessary standard statistical software nowadays allows for such analyses in practice. While the possibility of data missing not at random (MNAR) cannot be ruled out, it is argued that analyses valid under MNAR are not well suited for the primary analysis in clinical trials. Rather than either forgetting about or blindly shifting to an MNAR framework, the optimal place for MNAR analyses is within a sensitivity-analysis context. One such route for sensitivity analysis is to consider, next to selection models, pattern-mixture models or shared-parameter models. The latter can also be extended to a latent-class mixture model, the approach taken in this article. The performance of the so-obtained flexible model is assessed through simulations and the model is applied to data from a depression trial.

A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia.