Cross-validation Design Research Articles

Depth video data-enabled predictions of longitudinal dairy cow body weight using thresholding and Mask R-CNN algorithms

Monitoring cow body weight is crucial to support farm management decisions due to its direct relationship with the growth, nutritional status, and health of dairy cows. Cow body weight is a repeated trait, however, the majority of previous body weight prediction research only used data collected at a single point in time. Furthermore, the utility of deep learning-based segmentation for body weight prediction using videos remains unanswered. Therefore, the objectives of this study were to predict cow body weight from repeatedly measured video data, to compare the performance of the thresholding and Mask R-CNN deep learning approaches, to evaluate the predictive ability of body weight regression models, and to promote open science in the animal science community by releasing the source code for video-based body weight prediction. An Intel RealSense D435 camera was installed on Virginia Tech dairy complex to collect top-view videos of 10 lactating Holstein cows twice a day for 28 days after milking sessions and 2 non-lactating, pregnant Jersey cows once a week upon calving. At the same time, the ground truth body weight records were collected using a walk-over weighing system. A total of 40,405 depth images and depth map files were obtained. Three approaches were investigated to segment the cow's body from the background, including single thresholding, adaptive thresholding, and Mask R-CNN. Four image-derived biometric features, such as dorsal length, abdominal width, height, and volume, were estimated from the segmented images and fitted using ordinary least squares, ridge regression, least absolute shrinkage and selection operator, and linear mixed models. Two cross-validation designs, forecasting and leave-three-cows-out, were used to evaluate prediction performance. The Pearson correlation between image-derived biometric features and scale-based body weight ranged from 0.74 to 0.95. On average, the Mask-RCNN approach combined with a linear mixed model resulted in the best prediction coefficient of determination and mean absolute percentage error of 0.98 and 2.03%, respectively, in the forecasting cross-validation. The Mask-RCNN approach was also the best in the leave-three-cows-out cross-validation, followed by adaptive and single thresholding. The prediction coefficients of determination and mean absolute percentage error of the Mask-RCNN coupled with the linear mixed model were 0.90 and 4.70%, respectively. Our results suggest that deep learning-based segmentation improves the prediction performance of cow body weight from longitudinal depth video data.

151 Depth Video Data-Enabled Predictions of Longitudinal Dairy Cow Body Weight Using Thresholding and Mask R-Cnn Algorithms

Abstract Cow body weight monitoring is critical in farm management because it is associated with the growth, nutritional status, and health of dairy cows. Regularly collecting body weight records often requires human labor or expensive equipment. Recently, a low-cost 3D depth sensor camera-enabled computer vision analysis has been proposed to address the body weight prediction challenges. Cow body weight is a repeated trait; however, past body weight prediction research only used data collected at a single time point. Furthermore, the utility of deep learning on body weight predictions remains unanswered. Therefore, the objectives of this study were to predict cow body weight from repeatedly measured video data and compare the performance of thresholding and Mask R-CNN deep learning approaches. We set up an Intel Realsense D435 camera at the Virginia Tech Kentland farm to collect top-view videos for 10 Holstein and 2 Jersey cows for 28 days, twice per day. Real body weight records were also collected by a walk-over-weighing system simultaneously. A total of 40,405 depth images and depth map files were obtained. We explored three approaches to segment the cow body from the background, including single thresholding, adaptive thresholding, and Mask R-CNN. Four image-derived biometric measurements, length, width, height, and volume, were estimated from segmented images and fitted in ordinary least squares. Two cross-validation designs, forecasting and leave-several-cow-out, were used to evaluate prediction performance. The correlation between image-derived biometric measurements and body weight ranged from 0.84 to 0.95. On average, the Mask-RCNN approach resulted in the best prediction coefficient of determination and mean absolute percentage error of 0.96 and 3.54% in the forecasting cross-validation, respectively. The predictive performance of the single and adaptive thresholding approaches was similar. The Mask-RCNN approach was also the best in the leave-several-cow-out cross-validation, followed by adaptive and single thresholding. The prediction coefficients of determination of Mask-RCNN for leave-one-cow-out, leave-two-cow-out, leave-three-cow-out, leave-four-cow-out were 0.05, 0.69, 0.85, and 0.90, respectively, while mean absolute percentage errors were 4.51%, 4.51%, and 4.59%, 4.70% respectively. Our results suggest predicting cow body weight from depth video data using deep learning is feasible.

Including genomic information in the genetic evaluation of production and reproduction traits in South African Merino sheep.

Genomic selection (GS) has become common in sheep breeding programmes in Australia, New Zealand, France and Ireland but requires validation in South Africa (SA). This study aimed to compare the predictive ability, bias and dispersion of pedigree BLUP (ABLUP) and single-step genomic BLUP (ssGBLUP) for production and reproduction traits in South African Merinos. Animals in this study originated from five research and five commercial Merino flocks and included between 54,072 and 79,100 production records for weaning weight (WW), yearling weight (YW), fibre diameter (FD), clean fleece weight (CFW) and staple length (SL). For reproduction traits, the dataset included 58,744 repeated records from 17,268 ewes for the number of lambs born (NLB), number of lambs weaned (NLW) and the total weight weaned (TWW). The single-step relationship matrix, H, was calculated using PreGS90 software combining 2811 animals with medium density (~50 K) genotypes and the pedigree of 88,600 animals. Assessment was based on single-trait analysis using ASREML V4.2 software. The accuracy of prediction was evaluated according to the 'LR-method' by a cross-validation design. Validation candidates were assigned according to Scenario I: born after a certain time point; and Scenario II: born in a particular flock. In Scenario I, the genotyping rate for the reference population between 2006 and the 2013 cut-off point approached 7% of animals with phenotypes and 20% of their sires. For reproduction traits, about 20% of ewes born between 2006 and 2011 cut-off were genotyped, along with 15% of their sires. Genotyping rates in the validation set of Scenario I were 3.7% (production) and 11.4% (reproduction) for candidates and 35% of their sires. Sires were allowed to have progeny in both the reference and validation set. In Scenario I, ssGBLUP increased the accuracy of prediction for all traits except NLB, ranging between +8% (0.62-0.67) for FD and +44% (0.36-0.52) for WW. This showed a promising gain in accuracy despite a modestly sized reference population. In the 'across flock validation' (Scenario II), overall accuracy was lower, but with greater differences between ABLUP and ssGBLUP ranging between +17% (0.12-0.14) for TWW and +117% (0.18-0.39) for WW. There was little indication of severe bias, but some traits were prone to over dispersion and the use of genomic information did not improve this. These results were the first to validate the benefit of genomic information in South African Merinos. However, because production traits are moderately heritable and easy to measure at an early age, future research should be aimed at best exploiting GS methods towards genetic prediction of sex-limited and/or lowly heritable traits such as NLW. GS methods should be combined with dedicated efforts to increase genetic links between sectors and improved phenotyping by commercial flocks.

Predictive modeling of antibiotic eradication therapy success for new-onset Pseudomonas aeruginosa pulmonary infections in children with cystic fibrosis.

Chronic Pseudomonas aeruginosa (Pa) lung infections are the leading cause of mortality among cystic fibrosis (CF) patients; therefore, the eradication of new-onset Pa lung infections is an important therapeutic goal that can have long-term health benefits. The use of early antibiotic eradication therapy (AET) has been shown to clear the majority of new-onset Pa infections, and it is hoped that identifying the underlying basis for AET failure will further improve treatment outcomes. Here we generated machine learning models to predict AET outcomes based on pathogen genomic data. We used a nested cross validation design, population structure control, and recursive feature selection to improve model performance and showed that incorporating population structure control was crucial for improving model interpretation and generalizability. Our best model, controlling for population structure and using only 30 recursively selected features, had an area under the curve of 0.87 for a holdout test dataset. The top-ranked features were generally associated with motility, adhesion, and biofilm formation.

Exploring the predictive validity of personality disorder criteria.

We tested the predictive validity of personality disorder (PD) indicators at different levels of aggregation, ranging from general PD severity to PD syndrome scales to individual PD criteria. We compared the predictive validity of models on these levels based on interview data on all 78 DSM-IV PD criteria, by using 19 outcome scales in three different samples (N = 651, N = 552, and N = 1,277). We hypothesized that criteria of personality pathology yield a significant increase in predictive validity compared with scales that are aggregated at the syndrome- or general severity-level. We assessed out of sample performance of predictive models in a repeated cross-validation design using regularized linear regression and regression forest algorithms. We observed no significant difference in predictive performance between models trained at the item-level and models trained on scale-level data. We further tested the predictive performance of the trained linear models across samples on outcome measures shared between samples and inspected models for criteria-level information they relied on to make predictions. Our results suggest that little predictive variance is lost when interview items assessing DSM-IV PD criteria are aggregated to dimensional PD scales. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

The Affect Intolerance Scale (AIS), German version: A validation study in a student and clinical sample of patients with trauma-related disorders

Background and objectivesThe Affect Intolerance Scale (AIS) assesses two core concepts of emotion regulation: appraisals of negative emotions as threatening and proneness to emotional avoidance. Maladaptive emotion regulation is associated with various psychopathologies. We translated and validated the AIS in a German student and clinical sample of patients with trauma-related disorders. Methods340 patients, 161 with post-traumatic stress disorder and 179 with adjustment disorder, and 322 students were enrolled. We employed exploratory and confirmatory factor analyses in a cross-validation design to investigate construct validity, convergent and discriminant validity, and reliability. ResultsWe replicated the originally described two-factor structure in both samples. Cronbach's α was 0.947 in the student and 0.950 in the clinical sample. AIS subscales showed moderate to high correlations with convergent and low correlations with discriminant measures. AIS total scores were significantly larger in the clinical sample, controlled for gender and age. LimitationsThis study provides a unified cross-validation model in a clinical and a student sample at the cost of reduced sample sizes. ConclusionsThe AIS is a valid measure of affect intolerance with the discriminative ability to distinguish between patients with trauma-related disorders and students. Test redundancy within both sub-constructs of the AIS might lead to biased estimates but allows for increased test precision, rendering the AIS a tool suitable for individual treatment monitoring.

Optimal deep brain stimulation sites and networks for stimulation of the fornix in Alzheimer’s disease

Deep brain stimulation (DBS) to the fornix is an investigational treatment for patients with mild Alzheimer’s Disease. Outcomes from randomized clinical trials have shown that cognitive function improved in some patients but deteriorated in others. This could be explained by variance in electrode placement leading to differential engagement of neural circuits. To investigate this, we performed a post-hoc analysis on a multi-center cohort of 46 patients with DBS to the fornix (NCT00658125, NCT01608061). Using normative structural and functional connectivity data, we found that stimulation of the circuit of Papez and stria terminalis robustly associated with cognitive improvement (R = 0.53, p < 0.001). On a local level, the optimal stimulation site resided at the direct interface between these structures (R = 0.48, p < 0.001). Finally, modulating specific distributed brain networks related to memory accounted for optimal outcomes (R = 0.48, p < 0.001). Findings were robust to multiple cross-validation designs and may define an optimal network target that could refine DBS surgery and programming.

Sample size requirement for achieving multisite harmonization using structural brain MRI features

When data is pooled across multiple sites, the extracted features are confounded by site effects. Harmonization methods attempt to correct these site effects while preserving the biological variability within the features. However, little is known about the sample size requirement for effectively learning the harmonization parameters and their relationship with the increasing number of sites. In this study, we performed experiments to find the minimum sample size required to achieve multisite harmonization (using neuroHarmonize) using volumetric and surface features by leveraging the concept of learning curves. Our first two experiments show that site-effects are effectively removed in a univariate and multivariate manner; however, it is essential to regress the effect of covariates from the harmonized data additionally. Our following two experiments with actual and simulated data showed that the minimum sample size required for achieving harmonization grows with the increasing average Mahalanobis distances between the sites and their reference distribution. We conclude by positing a general framework to understand the site effects using the Mahalanobis distance. Further, we provide insights on the various factors in a cross-validation design to achieve optimal inter-site harmonization.

Use of gradient boosting machine for exposome-wide analyses: application for pharmaceutical risk factors for amyotrophic lateral sclerosis

Background Accumulating evidence suggests that human health is affected by a complex set of exposures, including to environmental toxicants, dietary constituents, psychosocial stressors, and physical factors. Capturing the complexity of the whole exposome is pivotal for advancing etiological knowledge, yet standard statistical methods, as well as recent developments for mixtures analysis, are limited to considering only a modest number of exposures and interactions while often also making the implicit assumption that the exposures are continuous. Machine learning approaches could offer benefits in these very high-dimensional settings. Method Using gradient boosted decision trees coupled with Bayesian model optimization and a nested cross-validation design, we describe an approach for outcome prediction based on a mixture of exposures, and for identification of specific culprit factors within the mixture that consistently drive the association while allowing for synergistic or antagonistic interactions between the predictors. While this flexible approach is applicable to many settings, we used it to evaluate the association between patients’ history of medication use and risk of amyotrophic lateral sclerosis (ALS), as a steppingstone for integrating additional exposures and in a setting that largely avoids exposure measurement errors that further complicate many toxicant mixtures. Results Of nearly 800 binary predictors, we identified 7 medication classes that were consistently associated with ALS risk across independently trained models. Interactions between medication groups did not substantially affect the risk. Prediction accuracy was consistent, but low due to the lack of information on other etiological risk factors for ALS. Summary The described methodology allows to predict the overall effect of a mixture and to identify specific culprit factors in very high dimensional settings and when both continuous and categorical exposures are of interest. While causal interpretation of purely predictive models should generally be avoided, the repeated sampling of the dataset has interesting causal inference implications.

Comparative study on the performance of different classification algorithms, combined with pre- and post-processing techniques to handle imbalanced data, in the diagnosis of adult patients with familial hypercholesterolemia.

Familial Hypercholesterolemia (FH) is an inherited disorder of cholesterol metabolism. Current criteria for FH diagnosis, like Simon Broome (SB) criteria, lead to high false positive rates. The aim of this work was to explore alternative classification procedures for FH diagnosis, based on different biological and biochemical indicators. For this purpose, logistic regression (LR), naive Bayes classifier (NB), random forest (RF) and extreme gradient boosting (XGB) algorithms were combined with Synthetic Minority Oversampling Technique (SMOTE), or threshold adjustment by maximizing Youden index (YI), and compared. Data was tested through a 10 × 10 repeated k-fold cross validation design. The LR model presented an overall better performance, as assessed by the areas under the receiver operating characteristics (AUROC) and precision-recall (AUPRC) curves, and several operating characteristics (OC), regardless of the strategy to cope with class imbalance. When adopting either data processing technique, significantly higher accuracy (Acc), G-mean and F1 score values were found for all classification algorithms, compared to SB criteria (p < 0.01), revealing a more balanced predictive ability for both classes, and higher effectiveness in classifying FH patients. Adjustment of the cut-off values through pre or post-processing methods revealed a considerable gain in sensitivity (Sens) values (p < 0.01). Although the performance of pre and post-processing strategies was similar, SMOTE does not cause model’s parameters to loose interpretability. These results suggest a LR model combined with SMOTE can be an optimal approach to be used as a widespread screening tool.

Spatiotemporal distribution of labeled data can bias the validation and selection of supervised learning algorithms: A marine remote sensing example

Supervised learning allows the prediction of variables measured in-situ from variables that can be measured from satellites. A labeled data set for this purpose is typically created by matching in-situ and satellite data and split into subsets for model training and initial validation. However, the available data are often not randomly distributed in space and time. In theory, this can bias estimates of prediction errors. Here, remote sensing of chlorophyll a in the Baltic Sea serves as an example to demonstrate the importance of this problem in marine remote sensing, and to test how well different statistical designs for validation mitigate it. Semi-synthetic data sets were created by combining daily chlorophyll a fields from a biogeochemical model hindcast with real-world locations and times of in-situ measurements, generated by sampling 2,000 combinations of cruises from an oceanographic database. These data sets were matched with co-located satellite data and used to train and validate four algorithms using remote sensing reflectances as input. The algorithms were validated using different methods including random hold-out sets and various block cross-validation designs based on geographical location, time, or location in predictor space. The resulting error estimates were compared to true errors calculated from differences to the biogeochemical model outputs serving as response variable. All validation methods underestimated prediction errors, in many cases by >30%. While a simple band-ratio algorithm had the smallest true errors (e.g., absolute percentage difference: APD = 50%), estimated errors were smallest for more complicated and in fact less accurate machine learning algorithms. For example, ten-fold cross-validation led to selection of the truly best algorithm among four candidates for <10% of data sets. The biases were smallest, but not absent, for spatial block cross-validation, which selected the truly best algorithm for 21–46% of data sets, depending on the error measure. When the analyses were repeated with data that were randomly distributed in space and time, the biases of error estimates based on random splits became much smaller (e.g., 10-fold cross-validation estimated errors within 2% of their true values and selected the truly best algorithm for >99% of data sets), spatial block cross-validation overestimated prediction errors (often by >40%), all algorithms achieved lower true errors, and a random forest made the most accurate predictions overall (APD = 27%). These results show that more attention should be paid to statistical methods for estimating the errors of supervised learning algorithms, e.g., by using multiple validation methods in combination and critically discussing error estimates considering questions of dependence, representativeness, and stationarity. Furthermore, non-random spatiotemporal distribution of labeled data can be a barrier to harnessing the full potential of machine learning algorithms in marine remote sensing.

Prediction of clinical trial enrollment rates.

Clinical trials represent a critical milestone of translational and clinical sciences. However, poor recruitment to clinical trials has been a long standing problem affecting institutions all over the world. One way to reduce the cost incurred by insufficient enrollment is to minimize initiating trials that are most likely to fall short of their enrollment goal. Hence, the ability to predict which proposed trials will meet enrollment goals prior to the start of the trial is highly beneficial. In the current study, we leveraged a data set extracted from ClinicalTrials.gov that consists of 46,724 U.S. based clinical trials from 1990 to 2020. We constructed 4,636 candidate predictors based on data collected by ClinicalTrials.gov and external sources for enrollment rate prediction using various state-of-the-art machine learning methods. Taking advantage of a nested time series cross-validation design, our models resulted in good predictive performance that is generalizable to future data and stable over time. Moreover, information content analysis revealed the study design related features to be the most informative feature type regarding enrollment. Compared to the performance of models built with all features, the performance of models built with study design related features is only marginally worse (AUC = 0.78 ± 0.03 vs. AUC = 0.76 ± 0.02). The results presented can form the basis for data-driven decision support systems to assess whether proposed clinical trials would likely meet their enrollment goal.

Develop and Apply Electrocardiography-Based Risk Score to Identify Community-Based Elderly Individuals at High-Risk of Mortality.

With an aging world population, risk stratification of community-based, elderly population is required for primary prevention. This study proposes a combined score developed using electrocardiographic (ECG) parameters and determines its long-term prognostic value for predicting risk of cardiovascular mortality. A cohort-study, conducted from December 2008 to April 2019, enrolled 5,380 subjects in Taiwan, who were examined, using three-serial-12-lead ECGs, and their health/demographic information were recorded. To understand the predictive effects of ECG parameters on overall-survival, Cox hazard regression analysis were performed. The mean age at enrollment was 69.04 ± 8.14 years, and 47.4% were males. ECG abnormalities, LVH [hazard ratio (HR) = 1.39, 95% confidence intervals (CI) = (1.16–1.67), P = 0.0003], QTc [HR = 1.31, CI = (1.07–1.61), P = 0.007] and PR interval [HR = 1.40, CI = (1.01–1.95), P = 0.04], were significantly associated with primary outcome all-cause death. Furthermore, LVH [HR = 2.37, CI = (1.48–3.79), P = 0.0003] was significantly associated with cardiovascular death, while PR interval [HR = 2.63, CI = (1.24– 5.57), P = 0.01] with unexplained death. ECG abnormality (EA) score was defined based on the number of abnormal ECG parameters for each patient, which was used to divide all patients into sub-groups. Competing risk survival analysis using EA score were performed by using the Gray's test, which reported that high-risk EA groups showed significantly higher cumulative incidence for all three outcomes. Prognostic models using the EA score as predictor were developed and a 10-fold cross validation design was adopted to conduct calibration and discrimination analysis, to establish the efficacy of the proposed models. Overall, ECG model could successfully predict people, susceptible to all three death outcomes (P < 0.05), with high efficacy. Statistically significant (P < 0.001) improvement of the c-indices further demonstrated the robustness of the prediction model with ECG parameters, as opposed to a traditional model with no EA predictor. The EA score is highly associated with increased risk of mortality in elderly population and may be successfully used in clinical practice.

Individualized prediction of three- and six-year outcomes of psychosis in a longitudinal multicenter study: a machine learning approach

Schizophrenia and related disorders have heterogeneous outcomes. Individualized prediction of long-term outcomes may be helpful in improving treatment decisions. Utilizing extensive baseline data of 523 patients with a psychotic disorder and variable illness duration, we predicted symptomatic and global outcomes at 3-year and 6-year follow-ups. We classified outcomes as (1) symptomatic: in remission or not in remission, and (2) global outcome, using the Global Assessment of Functioning (GAF) scale, divided into good (GAF ≥ 65) and poor (GAF < 65). Aiming for a robust and interpretable prediction model, we employed a linear support vector machine and recursive feature elimination within a nested cross-validation design to obtain a lean set of predictors. Generalization to out-of-study samples was estimated using leave-one-site-out cross-validation. Prediction accuracies were above chance and ranged from 62.2% to 64.7% (symptomatic outcome), and 63.5–67.6% (global outcome). Leave-one-site-out cross-validation demonstrated the robustness of our models, with a minor drop in predictive accuracies of 2.3% on average. Important predictors included GAF scores, psychotic symptoms, quality of life, antipsychotics use, psychosocial needs, and depressive symptoms. These robust, albeit modestly accurate, long-term prognostic predictions based on lean predictor sets indicate the potential of machine learning models complementing clinical judgment and decision-making. Future model development may benefit from studies scoping patient’s and clinicians' needs in prognostication.

Synthetic microbleeds generation for classifier training without ground truth

Background and ObjectiveCerebral microbleeds (CMB) are important biomarkers of cerebrovascular diseases and cognitive dysfunctions. Susceptibility weighted imaging (SWI) is a common MRI sequence where CMB appear as small hypointense blobs. The prevalence of CMB in the population and in each scan is low, resulting in tedious and time-consuming visual assessment. Automated detection methods would be of value but are challenged by the CMB low prevalence, the presence of mimics such as blood vessels, and the difficulty to obtain sufficient ground truth for training and testing. In this paper, synthetic CMB (sCMB) generation using an analytical model is proposed for training and testing machine learning methods. The main aim is creating perfect synthetic ground truth as similar as reals, in high number, with a high diversity of shape, volume, intensity, and location to improve training of supervised methods. MethodsCMB were modelled with a random Gaussian shape and added to healthy brain locations. We compared training on our synthetic data to standard augmentation techniques. We performed a validation experiment using sCMB and report result for whole brain detection using a 10-fold cross validation design with an ensemble of 10 neural networks. ResultsPerformance was close to state of the art (~9 false positives per scan), when random forest was trained on synthetic only and tested on real lesion. Other experiments showed that top detection performance could be achieved when training on synthetic CMB only. Our dataset is made available, including a version with 37,000 synthetic lesions, that could be used for benchmarking and training. ConclusionOur proposed synthetic microbleeds model is a powerful data augmentation approach for CMB classification with and should be considered for training automated lesion detection system from MRI SWI.

Estimating Energy Expenditure Using Wearable Sensors During Locomotion

Introduction. Measures of metabolic energy expenditure can provide valuable insight into healthy and impaired gait, the design and control of assistive devices, and rehabilitation progress. The gold standard for estimating energy expenditure during locomotion is indirect calorimetry, where oxygen use is captured at the mouth. Although accurate, indirect calorimetry systems are expensive, cumbersome, and often limited to lab settings.&#x0D; Objective. The aim of our research is to develop a lightweight, portable, and low-cost method for accurately estimating energy expenditure using wearable sensors. Our method must meet the following design criteria: i. estimate walking and running energy expenditure within 5% error of gold standard measures, ii. maintain accuracy given changes to terrain and external loads, iii. provide a continuous estimate with estimate intervals a maximum of one minute apart, and iv. cost under $1000.&#x0D; Methods. In pilot testing, we instrumented two participants (male, 21-22 years, 84-90 kg, 1.88-1.90m) with indirect calorimetry to measure gold standard energy expenditure, as well as the following wearable sensors: an accelerometer at the pelvis and foot, a heart rate monitor, and a respiratory belt. The participants walked and ran on a predefined outdoor route on Queen’s campus, including sections with distinct average inclines (0% and 5%). Participants also wore ankle weights (3% body weight) for particular sections of the route. We will use a multiple regression analysis, with cross-validation design, to predict energy expenditure using custom metrics derived from the wearable sensors.

How often were you traumatized? Reconceptualizing adverse childhood experiences for sexual and gender minorities

Introduction: The manifold consequences of adverse childhood experiences (ACEs) are well-documented. Recent research has demonstrated that sexual and gender minorities (SGMs) typically encounter ACEs more often than cisgender heterosexual individuals. Given the higher exposure rate, the measurement of frequency of exposure to traumatic events may be relevant for SGMs. Methods: We changed the response options of the ACEs index from dichotomous to a five-point Likert scale that described frequency of exposure. As part of a larger community-based participatory research study, the Likert ACEs measure was distributed to a large and diverse sample of SGM participants in San Antonio. Results: A cross-validation design demonstrated that the Likert ACEs scores outperformed the traditional ACEs index in predicting self-reported anxiety and post-traumatic stress disorder. Half of the SGMs in this sample experienced 3 or more ACEs, compared to only 10% of Americans in a nationally representative sample. Limitations: These analyses were based on retrospective self-report data instead of structured clinical interviews. Since only the Likert ACEs was administered, we had to assume that any response other than “never” on Likert ACEs corresponded to “yes” on the ACEs Index. Conclusions: Future research may assess the utility of the Likert ACEs approach with other minoritized or intersectional populations. For clinical practitioners, these results suggest that a better way to measure ACEs for SGMs is to ask them how often they were exposed, rather than asking whether they were exposed.

Genomic prediction of agronomic traits in wheat using different models and cross-validation designs.

Genomic predictions across environments and within populations resulted in moderate to high accuracies but across-population genomic prediction should not be considered in wheat for small population size. Genomic selection (GS) is a marker-based selection suggested to improve the genetic gain of quantitative traits in plant breeding programs. We evaluated the effects of training population (TP)composition, cross-validation design, and genetic relationship between the training and breeding populations on the accuracy of GS in spring wheat (Triticum aestivum L.). Two populations of 231 and 304 spring hexaploid wheat lines that were phenotyped for six agronomic traits and genotyped with the wheat 90K array were used to assess the accuracy of seven GS models (RR-BLUP, G-BLUP, BayesB, BL, RKHS, GS + de novo GWAS, and reaction norm) using different cross-validation designs. BayesB outperformed the other models for within-population genomic predictions in the presence of few quantitative trait loci (QTL) with large effects. However, including fixed-effect marker covariates gave better performance for an across-population prediction when the same QTL underlie traits in both populations. The accuracy of prediction was highly variable based on the cross-validation design, which suggests the importance to use a design that resembles the variation within a breeding program. Moderate to high accuracies were obtained when predictions were made within populations. In contrast, across-population genomic prediction accuracies were very low, suggesting that the evaluated models are not suitable for prediction across independent populations. On the other hand, across-environment prediction and forward prediction designs using the reaction norm model resulted in moderate to high accuracies, suggesting that GS can be applied in wheat to predict the performance of newly developed lines and lines in incomplete field trials.

The T Index: Measuring the Reliability of Accuracy Estimates Obtained from Non-Probability Samples

In remote sensing, the term accuracy typically expresses the degree of correctness of a map. Best practices in accuracy assessment have been widely researched and include guidelines on how to select validation data using probability sampling designs. In practice, however, probability samples may be lacking and, instead, cross-validation using non-probability samples is common. This practice is risky because the resulting accuracy estimates can easily be mistaken for map accuracy. The following question arises: to what extent are accuracy estimates obtained from non-probability samples representative of map accuracy? This letter introduces the T index to answer this question. Certain cross-validation designs (such as the common single-split or hold-out validation) provide representative accuracy estimates when hold-out sets are simple random samples of the map population. The T index essentially measures the probability of a hold-out set of unknown sampling design to be a simple random sample. To that aim, we compare its spread in the feature space against the spread of random unlabelled samples of the same size. Data spread is measured by a variant of Moran’s I autocorrelation index. Consistent interpretation of the T index is proposed through the prism of significance testing, with T values &lt; 0.05 indicating unreliable accuracy estimates. Its relevance and interpretation guidelines are also illustrated in a case study on crop-type mapping. Uptake of the T index by the remote-sensing community will help inform about—and sometimes caution against—the representativeness of accuracy estimates obtained by cross-validation, so that users can better decide whether a map is fit for their purpose or how its accuracy impacts their application. Subsequently, the T index will build trust and improve the transparency of accuracy assessment in conditions which deviate from best practices.

0386 Identification of Sleep Complaints Using Social Media: Effect of the Daylight Savings Time to Standard Time Transition

Abstract Introduction Transitions to and from daylight savings time (DST) are natural experiments of circadian disruption and are associated with negative health consequences. Yet, the majority of the United States and several other countries still adopt these changes. Large observational studies focused on understanding the impact of DST transitions on sleep are difficult to conduct. Social media platforms, like Twitter, are powerful sources of human behavior data. We used machine learning to identify tweets reporting sleep complaints (TRSC) during the week of the standard time (ST)-DST transition. Next, we evaluated the circadian patterns of TRSC and compared their prevalence before and after the transition. Methods Using data publicly available via the Twitter API, we collected 500 tweets with evidence of sleep complaints, and manually annotated each tweet to validate true sleep complaints. Next, we calculated term frequency-inverse document frequency of each word in each tweet and trained a random forest to classify TRSC using a 3-fold cross-validation design. The trained model was then used to annotate a collection of tweets captured between Oct. 30, 2019-Nov. 6, 2019, overlapping with the DST-ST transition, which occurred on Nov. 3, 2019. Results Random forest demonstrated good performance in classifying TRSC (AUC[95%CI]=0.85[0.82-0.89]). This model was applied to 3,738,383 tweets collected around the DST-ST transition, and identified 11,044 TRSC. Posting of these tweets had a circadian pattern, with peak during nighttime. We found a higher frequency of TRSC after the DST-ST transition (0.33% vs. 0.27%, p&amp;lt;0.00001), corresponding to a ~20% increase in the odds of reporting sleep complaints (OR[95%CI]=1.21[1.16-1.25]). Conclusion Using machine learning and Twitter data, we identified tweets reporting sleep complaints, described their circadian patterns and demonstrated that the prevalence of these types of tweets is significantly increased after the transition from DST to ST. These results demonstrate the applicability of social media data mining for public health in sleep medicine. Support NIH (K01LM012870); AASM Foundation (194-SR-18)